Configuration of asynchronous tasks in heroku using python with django and celery

This is aimed at those people who already have a python project with django in active heroku, the steps to follow to upload a project will be ignored.

Previous Requirements
First we need have install in our project celery and django-celery so let’s go see:

  • installl celery and django-celery in your virtualenv:
    “pip install Celery django-celery”
  • Note: remember each new dependency install have to added in the file requiremenst.txt for when you upload the changes to heroku he install this dependencies. you can see all dependencies instaled runing the comad pip freeze

     2. After they are installed all dependency, in our have to added:


    besides importing

    import djcelery

    After performing this configuration execute the command:

    “python migrate djcelery”

    with this command have to create the tables neceary to configure the djcelery in django admin. If you want to expand more you can visit the following link y

    Note: For to realize local test is necesary have to install a handler for the message queue between django and celery which can be Redis Rabbitmq
    • If you decide utilice redis have to install one aditional dependency in our virutalenv “pip install -U “celery[redis]”, as well as adding it to the requirements.txt file. affter added in the file CELERY_BROKER_URL = ‘redis://localhost:6379/0’
    • if you decide utilice rabbitmq addd in the file CELERY_BROKER_URL = ‘amqp://localhost’

    Remember that the redis or the rabbitmq must be turned on by the console to work

    Configuration of celery
    we have to create a new file whose name is in the configuration folder in our project

    • Navigate to the root project configuration module (where are the URL modules and settings):





      **       **


      **       **

      **       **

    • Inside of the file write this:
      from __future__ import absolute_import, unicode_literals
      import os
      from celery import Celery
      from django.conf import settings
      # set the default Django settings module for the 'celery' program.
      os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'yourproject.settings')
      app = Celery('yourproject')
      # Using a string here means the worker don't have to serialize
      # the configuration object to child processes.
      # - namespace='CELERY' means all celery-related configuration keys
      # should have a `CELERY_` prefix.

    Load task modules from all registered Django app configs.

    app.autodiscovertasks(lambda: settings.INSTALLEDAPPS)
    • then update the file in the configuration folder in our project:
      from __future__ import absolute_import, unicode_literals
      # This will make sure the app is always imported when
      # Django starts so that shared_task will use this app.
      from .celery import app as celery_app
      __all__ = ['celery_app']

    with this have the celery correctily configured in our project. now we will proceed to realice the tasks

    Creating asynchronous and periodic tasks

    • we need create a file in the Django aplication you want (remember this aplication it must be valid in INSTALLED_APPS):
      from __future__ import absolute_import, unicode_literals
      from celery import shared_task
      from account import models as account_models
    @sharedtask def disableCodeRecoveryPassword(instance): instance = accountmodels.User.objects.get(id=instance) instance.recovery = ” return True
    • to finish we will activated the worker with the next command: “celery -A yourproject worker -l info”. but before having your server running in another terminal staying like this:

    now we have all configuration to need to iniciate a tasks asynchronous in django, now we show to you how to run in heroku

    Project in heroku

    Note: To be able to use celery in heroku we have need a pay account, because when we try to install the redis or rabbitmq addons we will request a credit card

    • in the procfile file we must added two new line being that way:
      web: gunicorn yourproject.wsgi --log-file -
      worker: python celery worker --without-gossip --without-mingle --loglevel=info
      beat: python celery beat –loglevel=info
    • then we must added in the file
      BROKER_URL = os.environ.get("CLOUDAMQP_URL", "django://")

    After to upload all changes in heroku we must added a new addons in this example used the addons cloudAMQP writting the next command in console

    “$ heroku addons:add cloudamqp

    or you can go to heroku in the resource part and look for the addons and install it:

    Note: the previus configuration only work with cloudamqp I invite you to do your tests with redis install the addons

    Then we have all ready upload the project to heroku, now to activate the worker in heroku we must execute next command “$ heroku ps:scale worker=1”

    if you need to activate the beat, then do it using the same command, only changing the worker for the beat “$ heroku ps:scale beat=1”

    to simplify things you can activate both at the same time
    “$ heroku ps:scale worker=1 beat=1”

    only in the case that you are using the asynchronous and periodic tasks

    In doing so, this will be shown in the console
    “heroku-cli: update available from 6.14.36-15f8a25 to 6.15.25-635e5c6
    Scaling dynos… done, now running beat at 1:Hobby, worker at 1:Hobby”
    for all cases if only active the worker will exit: worker at 1:Hobby

    to off the worker execute the next command:
    $ heroku ps:scale worker=0 beat=0

    to verify that the worker is working well execute the command: “heroku ps” 

    inally show the logs the heroku executing the command
    heroku logs -t -p worker

    heroku logs -t -p beat

    to see if we have an error when uploading it or it was done correctly

    to finist execute the command:
    “heroku restart”

    if you followed all the steps to the letter you can perform leave asynchronous tasks in heroku

    I hope this guide has been useful, share and leave your comments 🙂




    Contact Us

    Adress: 66 W Flagler St, Miami, Florida
    Phone: +1 (786) 991-3467

    © 2020 Cobuild Lab All Rights Reserved.