Flask-APScheduler multiple jobs running at the same time when having multiple workers

[denisvitez]

Hi,

I'm using the Flask-APScheduler for a simple job inside my application that checks for pending notification messages and sends them. I moved the application onto the production server where I'm running it using uWSGI and multiple workers. Now the side-effect is that every worker is executing the same job, so now I have 8 workers trying to send the same email. Is there an option to configure that the job is only executed on one worker?

My job is currently configured as:

{
            'id': 'email_notification_service',
            'func': 'application.main.jobs:send_new_notifications',
            'args': (),
            'trigger': 'interval',
            'seconds': int(os.getenv('SCHEDULER_INTERVAL', 60))
}

[christopherpickering]

Hey Denis, that is not possible with Apscheduler v3: https://viniciuschiele.github.io/flask-apscheduler/rst/tips.html.
If you want to keep many workers in your primary app, you could run the scheduler as an independent process from your primary app and make api calls to it. Also, if your runners are doing intense jobs you could also run them in a second api process with more workers and call to it from the scheduler's runner function.

Basically:
web app (8 workers) >>>> scheduler api (1 worker) >>>> runner api (many workers)