AMQP Messaging, Task, and Queue handling in Python using Celery. ()

headers

James Brady | 1 Jun 2012 21:02

Tasks from Django not executing when using RabbitMQ

Hi all,

I'm a new user to Celery - trying to deploy it with Django and RabbitMQ to handle long-running tasks.

When I use the django broker, tasks are delivered to workers as expected.

However, when using RabbitMQ as the broker, I get an AMQP connection entry in RabbitMQ's log when celeryd starts:

=INFO REPORT==== 1-Jun-2012::18:55:47 ===

accepting AMQP connection <0.4123.0> (127.0.0.1:41801 -> 127.0.0.1:5672)

celeryd starts with no errors, and the last log lines are:

[2012-06-01 19:55:47,372: DEBUG/MainProcess] Start from server, version: 8.0, properties: {u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2012 VMware, Inc.', u'capabilities': {}, u'platform': u'Erlang/OTP', u'version': u'2.8.2'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']

[2012-06-01 19:55:47,373: DEBUG/MainProcess] Open OK! known_hosts []

[2012-06-01 19:55:47,373: DEBUG/MainProcess] Consumer: Connection established.

[2012-06-01 19:55:47,374: DEBUG/MainProcess] using channel_id: 1

[2012-06-01 19:55:47,375: DEBUG/MainProcess] Channel open

[2012-06-01 19:55:47,378: DEBUG/MainProcess] basic.qos: prefetch_count->8

[2012-06-01 19:55:47,378: DEBUG/MainProcess] using channel_id: 2

[2012-06-01 19:55:47,379: DEBUG/MainProcess] Channel open

[2012-06-01 19:55:47,448: DEBUG/MainProcess] Consumer: Starting message consumer...

[2012-06-01 19:55:47,449: DEBUG/MainProcess] Consumer: Ready to accept tasks!

From here, I create tasks like so in Django:

task = package_task.delay(**data)

_log.info("created task {}".format(task))

And I see in my wsgi log:

Jun 1 18:56:12 localhost [ INFO] 2012-06-01 19:56:12,494 standalone.views -- created task 966816f7-06ea-4a59-8c01-52544b6250fa

However, that task never appears in the database:

mysql> select id, task_id, status, date_done from celery_taskmeta;

+----+--------------------------------------+---------+---------------------+

+----+--------------------------------------+---------+---------------------+

| 1 | 47d71f31-6967-4758-bd90-82bb2f792a8e | FAILURE | 2012-05-31 20:00:56 |

| 2 | 3a05776a-247f-4d63-a0fb-8235aa3a1f5f | FAILURE | 2012-05-31 20:02:50 |

| 3 | 54057efb-a010-44e2-8543-56227a053952 | FAILURE | 2012-06-01 13:23:38 |

| 4 | e3230bc2-1b10-4d48-969c-66b0414ab3db | FAILURE | 2012-06-01 13:24:48 |

| 5 | 223501da-30b7-41e0-b934-c3ca807fe8d4 | FAILURE | 2012-06-01 19:21:01 |

| 6 | 518f12c3-bb87-4fd5-bb35-96bb80ae4d79 | FAILURE | 2012-06-01 19:31:34 |

+----+--------------------------------------+---------+---------------------+

I'm not too familiar with the RabbitMQ management console, but as far as I can tell, there are no messages going through; the message rates show no publishes and no deliveries on the celery queue.

Does anyone have suggestions for where I can look next to narrow down the problem?

Thanks!

James

--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/celery-users/-/Qbf-iJhv020J.
To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to celery-users+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

headers

Ask Solem | 3 Jun 2012 23:00

Re: Tasks from Django not executing when using RabbitMQ

On 1 Jun 2012, at 20:02, James Brady wrote:

> Hi all,
> I'm a new user to Celery - trying to deploy it with Django and RabbitMQ to handle long-running tasks.
> 
> When I use the django broker, tasks are delivered to workers as expected.
> 
> However, when using RabbitMQ as the broker, I get an AMQP connection entry in RabbitMQ's log when celeryd starts:
> =INFO REPORT==== 1-Jun-2012::18:55:47 ===
> accepting AMQP connection <0.4123.0> (127.0.0.1:41801 -> 127.0.0.1:5672)
> 
> 
> From here, I create tasks like so in Django:
> 	task = package_task.delay(**data)
> 	_log.info("created task {}".format(task))
> 
> And I see in my wsgi log:
> Jun  1 18:56:12 localhost [   INFO] 2012-06-01 19:56:12,494 standalone.views -- created task 966816f7-06ea-4a59-8c01-52544b6250fa
> 
> However, that task never appears in the database:
> 
> mysql> select id, task_id, status, date_done from celery_taskmeta;
> 

Hey James,

Celery does not write anything to the results table when a task is sent.
The results are only written by the worker.

You can try to see how many workers are consuming from your queue,
the default queue is named "celery", so if you haven't changed that you can
use this command:

    $ sudo rabbitmqctl list_queues name messages consumers | grep celery

If you are using a custom vhost you have to specify that too:

    $ sudo rabbitmqctl list_queues -p vhost name messages consumers | grep celery

(note that the leading slash in vhosts is optional, but it must be used consistently, e.g.
 "/foo" is not the same vhost as "foo").

The reason why I tell you to look at this is because a common mistake it having
old workers from an experimentation running.  These workers will then hijack the tasks,
you can kill them off by doing:

    ps auxww | grep celeryd | awk '{print $2}' | xargs kill -9

To make sure that this never happens you can use the --pidfile argument to celeryd:

    celeryd --pidfile=/var/pid/celeryd

--

-- 
Ask Solem
twitter.com/asksol | +44 (0)7713357179