AMQP Messaging, Task, and Queue handling in Python using Celery. ()

bastian | 13 Aug 2012 14:08

Daemonizing Celery for Django

Hi,

I am new to Celery and followed the instructions in the doc. I can execute basic tasks through it without issue launching celery in a terminal and Django through wsgi and virtualenv. I can see the tasks being executed and the results. Now I'd like to install it as a daemon on an Ubuntu server. Again I followed the instructions and used the celeryd init scripts and config files. I can see the daemon running ok but when it receives the very same tasks they don't get executed and the log says: 'Received unregistered task'. What could have gone wrong? Where can I look to try to understand why the tasks are registered when launched via command line and not when launched via daemon mode?

Thanks!

--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/celery-users/-/_d2z4F1MSuoJ.
To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to celery-users+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

headers

Marcin Tustin | 14 Aug 2012 02:29

Re: Daemonizing Celery for Django

Probably the easiest thing is to get the django-supervisor extension (and supervisor), and daemonize supervisor instead.

On Mon, Aug 13, 2012 at 8:08 AM, bastian <bastien.rocheron <at> gmail.com> wrote:

Hi,

I am new to Celery and followed the instructions in the doc. I can execute basic tasks through it without issue launching celery in a terminal and Django through wsgi and virtualenv. I can see the tasks being executed and the results. Now I'd like to install it as a daemon on an Ubuntu server. Again I followed the instructions and used the celeryd init scripts and config files. I can see the daemon running ok but when it receives the very same tasks they don't get executed and the log says: 'Received unregistered task'. What could have gone wrong? Where can I look to try to understand why the tasks are registered when launched via command line and not when launched via daemon mode?

Thanks!
--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/celery-users/-/_d2z4F1MSuoJ.
To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To unsubscribe from this group, send email to celery-users+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

--
Marcin Tustin
Tel: 07773 787 105

--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to celery-users+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

headers

bastian | 14 Aug 2012 10:37

Re: Daemonizing Celery for Django

Ok I'm going to try the supervisor way today :)

On Tuesday, August 14, 2012 2:29:19 AM UTC+2, Marcin wrote:

Probably the easiest thing is to get the django-supervisor extension (and supervisor), and daemonize supervisor instead.

On Mon, Aug 13, 2012 at 8:08 AM, bastian <bastien....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Hi,

I am new to Celery and followed the instructions in the doc. I can execute basic tasks through it without issue launching celery in a terminal and Django through wsgi and virtualenv. I can see the tasks being executed and the results. Now I'd like to install it as a daemon on an Ubuntu server. Again I followed the instructions and used the celeryd init scripts and config files. I can see the daemon running ok but when it receives the very same tasks they don't get executed and the log says: 'Received unregistered task'. What could have gone wrong? Where can I look to try to understand why the tasks are registered when launched via command line and not when launched via daemon mode?

Thanks!
--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/celery-users/-/_d2z4F1MSuoJ.
To post to this group, send email to celery...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To unsubscribe from this group, send email to celery-users... <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

--
Marcin Tustin
Tel: 07773 787 105

--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/celery-users/-/3-7JtxVQ1uIJ.
To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to celery-users+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

headers

Matt C | 7 Sep 2012 13:27

Re: Daemonizing Celery for Django

I will be doing the same thing later today using init.d and will post my setup for you.

--Matt

On Mon, Aug 13, 2012 at 8:08 AM, bastian <bastien.rocheron-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

Hi,

I am new to Celery and followed the instructions in the doc. I can execute basic tasks through it without issue launching celery in a terminal and Django through wsgi and virtualenv. I can see the tasks being executed and the results. Now I'd like to install it as a daemon on an Ubuntu server. Again I followed the instructions and used the celeryd init scripts and config files. I can see the daemon running ok but when it receives the very same tasks they don't get executed and the log says: 'Received unregistered task'. What could have gone wrong? Where can I look to try to understand why the tasks are registered when launched via command line and not when launched via daemon mode?

Thanks!
--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/celery-users/-/_d2z4F1MSuoJ.
To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To unsubscribe from this group, send email to celery-users+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

headers

bastian | 7 Sep 2012 13:34

Re: Daemonizing Celery for Django

Thanks Matt! I can't wait to read that. Meanwhile I installed supervisord and I'm now running celery through it.

Cheers.

On Friday, September 7, 2012 1:27:07 PM UTC+2, Matt wrote:

I will be doing the same thing later today using init.d and will post my setup for you.

--Matt

On Mon, Aug 13, 2012 at 8:08 AM, bastian <bastien.... <at> gmail.com> wrote:
Hi,

I am new to Celery and followed the instructions in the doc. I can execute basic tasks through it without issue launching celery in a terminal and Django through wsgi and virtualenv. I can see the tasks being executed and the results. Now I'd like to install it as a daemon on an Ubuntu server. Again I followed the instructions and used the celeryd init scripts and config files. I can see the daemon running ok but when it receives the very same tasks they don't get executed and the log says: 'Received unregistered task'. What could have gone wrong? Where can I look to try to understand why the tasks are registered when launched via command line and not when launched via daemon mode?

Thanks!
--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/celery-users/-/_d2z4F1MSuoJ.
To post to this group, send email to celery...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To unsubscribe from this group, send email to celery-users... <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/celery-users/-/gFQDFi7xwhkJ.
To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to celery-users+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

headers

Matt Long | 7 Sep 2012 20:28

Re: Daemonizing Celery for Django

We've been daemonizing django-celery pretty successfully with a modified (and slightly outdated) version of the generic init.d script (https://github.com/celery/celery/blob/master/extra/generic-init.d/celeryd). The trick is making sure you're pointing the init.d script at the correct CELERYD_MULTI, CELERYD, etc since you're trying to run them via your Django project's manage.py. This happens in /etc/default/celeryd.

Here's our init.d script and default file:

init.d: http://pastebin.com/UPL1NsRh

default: http://pastebin.com/i3QkKR8Z

The check_celeryd_script function was a custom addition to the init.d script as an extra sanity check that manage.py actually exists where expected. For it to work, CELERYD_SCRIPT must be defined via your default file. It's possible there were other customizations we made, but they were probably minor.

Our environment is Ubuntu 11.10 with v2.5.5 of django-celery and celery.

Cheers,

Matt

On Fri, Sep 7, 2012 at 4:34 AM, bastian <bastien.rocheron-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

Thanks Matt! I can't wait to read that. Meanwhile I installed supervisord and I'm now running celery through it.

Cheers.

On Friday, September 7, 2012 1:27:07 PM UTC+2, Matt wrote:

I will be doing the same thing later today using init.d and will post my setup for you.

--Matt

On Mon, Aug 13, 2012 at 8:08 AM, bastian <bastien....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

Hi,

I am new to Celery and followed the instructions in the doc. I can execute basic tasks through it without issue launching celery in a terminal and Django through wsgi and virtualenv. I can see the tasks being executed and the results. Now I'd like to install it as a daemon on an Ubuntu server. Again I followed the instructions and used the celeryd init scripts and config files. I can see the daemon running ok but when it receives the very same tasks they don't get executed and the log says: 'Received unregistered task'. What could have gone wrong? Where can I look to try to understand why the tasks are registered when launched via command line and not when launched via daemon mode?

Thanks!

--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/celery-users/-/_d2z4F1MSuoJ.

To post to this group, send email to celery...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To unsubscribe from this group, send email to celery-users... <at> googlegroups.com.

For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "celery-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/celery-users/-/gFQDFi7xwhkJ.

To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To unsubscribe from this group, send email to celery-users+unsubscribe <at> googlegroups.com.
For more options, visit this group at http://groups.google.com/group/celery-users?hl=en.

--
Matt Long

Cofounder and Lead Developer

crocodoc.com