Skip to content

Commit

Permalink
Fixed(?): Multiple celery workers.
Browse files Browse the repository at this point in the history
These changes should start up the workers to only take tasks from the correct queue, and should inform the django apps as to which queue to send tasks too.

Obviously I won't know if this works until we try running it on the live server, but it seems to work locally (I can start multiple celery instances connected to the project, and only the one pointed at the correct queue will take the tasks)

#210
  • Loading branch information
agpar committed Mar 1, 2016
1 parent c0c2cb0 commit e7e3c7d
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 4 deletions.
2 changes: 1 addition & 1 deletion celery_start_dev.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ source ${VIRTUAL_ENV}/bin/activate
# Move to project directory
cd ${PROJECT_PATH}
# Run your worker... old: exec celery worker -A elvis -l DEBUG --loglevel=INFO
exec celery worker -A elvis -l info --pidfile="/run/celery/%n-dev.pid"
exec celery worker -A elvis -l info --pidfile="/run/celery/%n-dev.pid" -Q elvis-dev
2 changes: 1 addition & 1 deletion celery_start_prod.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ source ${VIRTUAL_ENV}/bin/activate
# Move to project directory
cd ${PROJECT_PATH}
# Run your worker... old: exec celery worker -A elvis -l DEBUG --loglevel=INFO
exec celery worker -A elvis -l info --pidfile="/run/celery/%n-prod.pid"
exec celery worker -A elvis -l info --pidfile="/run/celery/%n-prod.pid" -Q elvis-prod
10 changes: 8 additions & 2 deletions elvis/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
import os
import re
from kombu import Exchange, Queue

BASE_DIR = os.path.abspath('./')

Expand Down Expand Up @@ -259,14 +260,19 @@
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ACCEPT_CONTENT = ['json']

CELERY_QUEUE_DICT = {'queue': 'elvis-{}'.format(SETTING_TYPE)}
CELERY_ROUTES = {'elvis.zip_files': CELERY_QUEUE_DICT,
'elvis.delete_zip_file': CELERY_QUEUE_DICT,
'elvis.rebuild_suggesters': CELERY_QUEUE_DICT}

# Elvis Web App Settings
# ======================
ELVIS_EXTENSIONS = ['.xml', '.mxl', '.krn', '.md', '.nwc', '.tntxt', '.capx',
'.abc', '.mid', '.midi', '.pdf', '.mei', '.ma', '.md2', '.json']
ELVIS_BAD_PREFIX = ['.', '..', '_', '__']
SUGGEST_DICTS = ['composerSuggest', 'pieceSuggest', 'collectionSuggest', 'languageSuggest', 'genreSuggest', 'locationSuggest', 'sourceSuggest', 'instrumentSuggest', 'tagSuggest']
SUGGEST_DICTS = ['composerSuggest', 'pieceSuggest', 'collectionSuggest',
'languageSuggest', 'genreSuggest', 'locationSuggest',
'sourceSuggest', 'instrumentSuggest', 'tagSuggest']


LOGGING = {
Expand Down

0 comments on commit e7e3c7d

Please sign in to comment.