DEV Community

Văn Hiếu Lê
Văn Hiếu Lê

Posted on • Originally published at heterl0.is-a.dev on

Set up Celery production for Django project

#experience #celery #django #production

🚀 Background

Celery is a cron job service commonly used in Django projects. In my case, I used Celery to reset user streaks at midnight (00:00 AM) for my application.

Set up the Celery for django project

In development, I had to run three terminal windows:

  • One for the Django server: python manage.py runserver

  • One for the Celery worker: celery -A proj worker

  • One for the Celery beat: celery -A proj beat

That setup was a bit complex. So in this blog, I’ll share how I set up Celery as a production-ready service using systemd.

🛠️ Approach

When I first moved my app to production, I used Gunicorn to serve Django, but forgot about Celery. As a result, scheduled tasks didn’t run because both Celery Worker and Celery Beat need to run in parallel.

✅ Option 1: Using tmux

Initially, I used tmux:

  1. SSH into the server.

  2. Start a tmux session and split the window with Ctrl + b → %.

  3. Run the worker and beat processes in separate panes.

Even after closing SSH, the processes stayed alive (confirmed using htop). This works, but it’s not ideal for long-term use.

✅ Option 2: Using systemd Services

When I got a new VPS, I wanted a better solution. After some research (and help from ChatGPT 😄), I found a reliable approach using systemd services to run Celery in the background.

Reference: Django with Celery in Production

⚙️ 1. Create the Celery Worker Service

Create a new service file:

sudo nano /etc/systemd/system/celery.service
Enter fullscreen mode Exit fullscreen mode

Paste the following (replace user, paths, and [celery_app] accordingly):

[Unit] Description=Celery Worker Service After=network.target [Service] Type=simple User=your_username Group=your_username WorkingDirectory=/home/your_username/your_project ExecStart=/home/your_username/.local/share/virtualenvs/your-venv/bin/celery -A [celery_app] worker --loglevel=INFO Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

👉 Replace:

  • your_username with your actual Linux username.

  • [celery_app] with the value from your celery.py file.

Then run:

sudo systemctl daemon-reload sudo systemctl enable celery sudo systemctl start celery
Enter fullscreen mode Exit fullscreen mode

🕒 2. Create the Celery Beat Service

Now create the Beat service:

sudo nano /etc/systemd/system/celery-beat.service
Enter fullscreen mode Exit fullscreen mode

Paste the following:

[Unit] Description=Celery Beat Service After=network.target [Service] Type=simple User=your_username Group=your_username WorkingDirectory=/home/your_username/your_project ExecStart=/home/your_username/.local/share/virtualenvs/your-venv/bin/celery -A [celery_app] beat --loglevel=INFO Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

Then run:

sudo systemctl daemon-reload sudo systemctl enable celery-beat sudo systemctl start celery-beat
Enter fullscreen mode Exit fullscreen mode

✅ Check status:

sudo systemctl status celery-beat
Enter fullscreen mode Exit fullscreen mode

If you see active (running), it means everything is set up correctly.

🧾 Conclusion

That’s how I set up Celery and Celery Beat in production using systemd. It’s a clean, reliable, and maintainable way to manage background tasks in Django.

📚 References:

Thanks for reading! 🙏 Hope this helps you in your deployment journey.

Top comments (0)