Geek asks and answers

Running multiple celery instances on the same server

I want to run two copies of celery on the same machine. One for version "A" of my application, the other for version "B".

I have two instances that I start as follows:

(env1)/home/me/firstapp$ celery -A app.tasks worker --config celeryconfig
(env2)/home/me/secondapp$ celery -A app.tasks worker -n Carrot --config celeryconfig

In tasks.py in each application, I create an instance of celery, like this:

 celery = Celery('tasks', backend='amqp', broker='amqp://guest@127.0.0..1.5672//')
 @celery.task
 def run_a_task():
     do_stuff()

In env2 task.py, how can I indicate that I want to use the second instance of celery from secondapp (named Carrot), and not the first from firstapp? I suspect that I need to change something in the constructor for celery on the first line, but I do not know what to add.

+5
source share
2 answers

I solved this using a virtual host for celery.

Once the rabbitmq server is running, I issue the following commands:

rabbitmqctl add_user user password
rabbitmqctl add_vhost app2
rabbitmqctl set_permissions -p app2 user ".*" ".*" ".*"

:

celery -A tasks worker --broker=amqp://user:password@localhost/app2

:

celery = Celery('tasks', backend='amqp', broker='amqp://user:password@localhost:5672/app2
+10

, AMQP, , .

blogposts AMQP: http://blogs.digitar.com/jjww/?s=rabbits&x=0&y=0

, : http://docs.celeryproject.org/en/latest/userguide/routing.html

, , - :

from kombu import Exchange, Queue

CELERY_DEFAULT_QUEUE = 'app1'
CELERY_QUEUES = (
    Queue('app1', Exchange('app1'), routing_key='app1'),
)
+5

More articles:


All Articles