Celery Worker Daemon - celery.bin.celeryd

celeryd

-c, --concurrency

Number of child processes processing the queue. The default is the number of CPUs available on your system.

-f, --logfile

Path to log file. If no logfile is specified, stderr is used.

-l, --loglevel

Logging level, choose between DEBUG, INFO, WARNING, ERROR, CRITICAL, or FATAL.

-n, --hostname

Set custom hostname.

-B, --beat

Also run the celerybeat periodic task scheduler. Please note that there must only be one instance of this service.

-Q, queues

List of queues to enable for this worker separated by comma. By default all configured queues are enabled. Example: -Q video,image

-s, --schedule

Path to the schedule database if running with the -B option. Defaults to celerybeat-schedule. The extension “.db” will be appended to the filename.

-E, --events

Send events that can be captured by monitors like celerymon.

--purge, --discard

Discard all waiting tasks before the daemon is started. WARNING: This is unrecoverable, and the tasks will be deleted from the messaging server.

--time-limit

Enables a hard time limit (in seconds) for tasks.

--soft-time-limit

Enables a soft time limit (in seconds) for tasks.

--maxtasksperchild

Maximum number of tasks a pool worker can execute before it’s terminated and replaced by a new worker.

class celery.bin.celeryd.Worker(concurrency=0, loglevel=30, logfile=None, hostname=None, discard=False, run_clockservice=False, schedule='celerybeat-schedule', task_time_limit=None, task_soft_time_limit=None, max_tasks_per_child=None, queues=None, events=False, db=None, **kwargs)

class WorkController(concurrency=None, logfile=None, loglevel=None, send_events=False, hostname=None, ready_callback=<function noop at 0x103921e60>, embed_clockservice=False, pool_cls='celery.concurrency.processes.TaskPool', listener_cls='celery.worker.listener.CarrotListener', mediator_cls='celery.worker.controllers.Mediator', eta_scheduler_cls='celery.worker.controllers.ScheduleController', schedule_filename='celerybeat-schedule', task_time_limit=None, task_soft_time_limit=None, max_tasks_per_child=None, pool_putlocks=True, db=None)

Executes tasks waiting in the task queue.

Parameters:

• concurrency – see concurrency.
• logfile – see logfile.
• loglevel – see loglevel.
• embed_clockservice – see run_clockservice.
• send_events – see send_events.

concurrency

The number of simultaneous processes doing work (default: conf.CELERYD_CONCURRENCY)

loglevel

The loglevel used (default: logging.INFO)

logfile

The logfile used, if no logfile is specified it uses stderr (default: celery.conf.CELERYD_LOG_FILE).

embed_clockservice

If True, celerybeat is embedded, running in the main worker process as a thread.

send_events

Enable the sending of monitoring events, these events can be captured by monitors (celerymon).

logger

The logging.Logger instance used for logging.

pool

The multiprocessing.Pool instance used.

ready_queue

The Queue.Queue that holds tasks ready for immediate processing.

schedule_controller

Instance of celery.worker.controllers.ScheduleController.

mediator

Instance of celery.worker.controllers.Mediator.

listener

Instance of CarrotListener.

process_task(wrapper)

Process task by sending it to the pool of workers.

start()

Starts the workers main loop.

stop()

Graceful shutdown of the worker server.

terminate()

Not so graceful shutdown of the worker server.

Worker.init_loader()

Worker.init_queues()

Worker.on_listener_ready(listener)

Worker.purge_messages()

Worker.redirect_stdouts_to_logger()

Worker.run()

Worker.run_worker()

Worker.startup_info()

Worker.tasklist(include_builtins=True)

Worker.worker_init()

celery.bin.celeryd.dump_version(*args)

celery.bin.celeryd.install_worker_int_again_handler(worker)

celery.bin.celeryd.install_worker_int_handler(worker)

celery.bin.celeryd.install_worker_restart_handler(worker)

celery.bin.celeryd.install_worker_term_handler(worker)

celery.bin.celeryd.main()

celery.bin.celeryd.parse_options(arguments)

Parse the available options to celeryd.

celery.bin.celeryd.run_worker(**options)

celery.bin.celeryd.set_process_status(info)