Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding advanced role management #195

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Configurable options, shown here with defaults:
:sidekiq_options => nil
:sidekiq_require => nil
:sidekiq_tag => nil
:sidekiq_config => nil # if you have a config/sidekiq.yml, do not forget to set this.
:sidekiq_config => nil # if you have a config/sidekiq.yml, do not forget to set this.
:sidekiq_queue => nil
:sidekiq_timeout => 10
:sidekiq_roles => :app
Expand All @@ -52,7 +52,7 @@ Configurable options, shown here with defaults:
:monit_bin => '/usr/bin/monit'
:sidekiq_monit_default_hooks => true
:sidekiq_monit_group => nil
:sidekiq_service_name => "sidekiq_#{fetch(:application)}_#{fetch(:sidekiq_env)}" + (index ? "_#{index}" : '')
:sidekiq_service_name => "sidekiq_#{fetch(:application)}_#{fetch(:sidekiq_env)}" + (index ? "_#{index}" : '')

:sidekiq_cmd => "#{fetch(:bundle_cmd, "bundle")} exec sidekiq" # Only for capistrano2.5
:sidekiqctl_cmd => "#{fetch(:bundle_cmd, "bundle")} exec sidekiqctl" # Only for capistrano2.5
Expand All @@ -77,6 +77,23 @@ and `low`:
set :sidekiq_options_per_process, ["--queue high", "--queue default --queue low"]
```

In addition, you can define a diferent schema per role:
```ruby
set :sidekiq_roles, [:web, :mailers, :async, :support]
set :sidekiq_config_files_per_role, { web: ["sidekiq_web"],
mailers: ["sidekiq_mailers", "sidekiq_mailers_collect"],
async: ["sidekiq_paperclip"],
support: ["sidekiq_support"] }

server '192.168.1.3', user: fetch(:user), roles: [:web, :mailers, :async]
server '192.168.1.4', user: fetch(:user), roles: [:support, :async]
```
In this case, we're going to define multiple config files: `sidekiq_web.yml`, `sidekiq_mailers`....
All of them are defined inside of `config` directory (i.ex: `config/sidekiq_web.yml`)

each role can define diferent process, for example `mailers` role.
`Mailers` role should execute a process with config file `sidekiq_mailers.yml` and other process with config file `sidekiq_mailers_collect`

## Different number of processes per role

You can configure how many processes you want to run on each host next way:
Expand Down
2 changes: 1 addition & 1 deletion lib/capistrano/sidekiq/version.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module Capistrano
SidekiqVERSION = '1.0.0'
SidekiqVERSION = '1.0.1'
end
15 changes: 12 additions & 3 deletions lib/capistrano/tasks/sidekiq.rake
Original file line number Diff line number Diff line change
Expand Up @@ -140,11 +140,13 @@ namespace :sidekiq do
end

def pid_files
sidekiq_roles = Array(fetch(:sidekiq_roles))
sidekiq_roles = Array(fetch(:sidekiq_roles)).dup
sidekiq_roles.select! { |role| host.roles.include?(role) }
sidekiq_roles.flat_map do |role|
processes = fetch(:"#{ role }_processes") || fetch(:sidekiq_processes)
Array.new(processes) { |idx| fetch(:sidekiq_pid).gsub(/\.pid$/, "-#{idx}.pid") }
processes = fetch(:"#{ role }_processes") ||
fetch(:sidekiq_config_files_per_role)&.dig(role)&.size ||
fetch(:sidekiq_processes)
Array.new(processes) { |idx| fetch(:sidekiq_pid).gsub(/\.pid$/, "-#{role}-#{idx}.pid") }
end
end

Expand Down Expand Up @@ -180,11 +182,18 @@ namespace :sidekiq do
Array(fetch(:sidekiq_queue)).each do |queue|
args.push "--queue #{queue}"
end

args.push "--config #{fetch(:sidekiq_config)}" if fetch(:sidekiq_config)
args.push "--concurrency #{fetch(:sidekiq_concurrency)}" if fetch(:sidekiq_concurrency)
if process_options = fetch(:sidekiq_options_per_process)
args.push process_options[idx]
end

if process_config_file = fetch(:sidekiq_config_files_per_role)
process_role = pid_file.match(/(\w+)-(\d).pid$/)[1].to_s.to_sym
process_id = pid_file.match(/(\w+)-(\d).pid$/)[2].to_i
args.push "-C #{release_path}/config/#{process_config_file[process_role][process_id]}.yml"
end
# use sidekiq_options for special options
args.push fetch(:sidekiq_options) if fetch(:sidekiq_options)

Expand Down