Background Jobs in Flask: Celery and Flask-Scheduler

Introduction

Background jobs allow you to offload time-consuming tasks from the main application, improving performance and user experience. Flask can integrate with tools like Celery for task queues and Flask-Scheduler for periodic jobs.

This article explains step-by-step how to implement background jobs in Flask using Celery and Flask-Scheduler.

Part 1: Task Queues with Celery

Step 1: Install Celery and a Message Broker

Install Celery and a message broker like Redis:

            pip install celery redis

Ensure Redis is installed and running:

            sudo apt update
            sudo apt install redis
            sudo systemctl start redis

Step 2: Configure Celery in Your Flask App

Create a celery_app.py file to configure Celery:

    from celery import Celery

    def make_celery(app):
        celery = Celery(
            app.import_name,
            backend=app.config['CELERY_RESULT_BACKEND'],
            broker=app.config['CELERY_BROKER_URL']
        )
        celery.conf.update(app.config)
        return celery

Update your Flask application (app.py):

    from flask import Flask
    from celery_app import make_celery

    app = Flask(__name__)
    app.config['CELERY_BROKER_URL'] = 'redis://localhost:6379/0'
    app.config['CELERY_RESULT_BACKEND'] = 'redis://localhost:6379/0'

    celery = make_celery(app)

     @app.route('/long-task')
     def long_task():
        task = background_task.apply_async()
        return f"Task ID: {task.id}"

    @celery.task
    def background_task():
        import time
        time.sleep(10)  # Simulate a long task
        return "Task completed!"

Step 3: Start Celery and Test

Start the Celery worker:

            celery -A app.celery worker --loglevel=info

Run the Flask application and access /long-task. Celery processes the task in the background.

Part 2: Periodic Jobs with Flask-Scheduler

Step 1: Install Flask-Scheduler

Install Flask-Scheduler for scheduling cron-like tasks:

            pip install flask-apscheduler

Step 2: Configure Flask-Scheduler

Add Flask-Scheduler to your Flask app:

    from flask import Flask
    from flask_apscheduler import APScheduler

    app = Flask(__name__)
    scheduler = APScheduler()

    def scheduled_task():
         print("Running scheduled task")

    if __name__ == "__main__":
         app.config['SCHEDULER_API_ENABLED'] = True
        scheduler.init_app(app)
        scheduler.start()

        scheduler.add_job(
            id='Scheduled Task',
            func=scheduled_task,
            trigger='interval',
            seconds=10
        )
        app.run(debug=True)

Step 3: Test the Scheduled Task

Run the Flask application. The task logs "Running scheduled task" every 10 seconds to the console.

Conclusion

Using Celery and Flask-Scheduler, you can implement both ad-hoc and periodic background jobs in Flask. This improves the scalability and functionality of your application by efficiently handling tasks.




Advertisement