Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup BWC repos, Install BWC and Setup RBAC #126

Merged
merged 40 commits into from
Mar 22, 2017
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
ab143a0
Initial commit
Mar 7, 2017
6d05514
First round of fixes after U16 testing
Mar 8, 2017
dca13a3
Remove Debug for GPG_URL
Mar 8, 2017
3098871
Fixes based on centos7 testing
Mar 8, 2017
b850d58
Fix typo in variable name
Mar 9, 2017
c287cc3
Remove support for user supplied RBAC roles&assignments
Mar 9, 2017
b3772fc
Let's support roles and assignments via dicts
Mar 9, 2017
8548acc
Rename files -> templates
Mar 9, 2017
a09cb88
use to_nice_yaml
Mar 9, 2017
ac4cd06
use yaml as file ext and not yml
Mar 9, 2017
69fd550
Move back RBAC to same role as BWC
Mar 10, 2017
1aa9d35
Remove separate RBAC role
Mar 10, 2017
ea34e3c
Coalesce bwc_repos into bwc
Mar 10, 2017
22f86a1
Fixes after testing
Mar 10, 2017
1c1ebd0
Fix variable names
Mar 10, 2017
bd198f2
Fix st2.conf file path
Mar 10, 2017
17d7045
@ me baby one more time
Mar 11, 2017
aaccdab
Setup CI
Mar 11, 2017
18da115
Lint fix
Mar 11, 2017
a5f2478
Variable name goofups (fix)
Mar 13, 2017
b7d84e7
Merge branch 'master' into bwc_install_role
Mar 13, 2017
a343cd6
Address review comments
Mar 13, 2017
6ff9e0b
Address code review comments
Mar 14, 2017
e1977f0
Fix README
Mar 14, 2017
5318182
Use no_log to hide tokens
Mar 14, 2017
9a5dee6
Use to_nice_json as opposed to to_nice_yaml
Mar 14, 2017
df65ee2
Setup default RBAC roles and assignments always
Mar 14, 2017
2fd65e8
YAML cleanup
Mar 14, 2017
56d0587
Fix 'no_logs' -> 'no_log' blocking the Ansible run
Mar 15, 2017
c1d98a9
Fix 'st2web' vs 'bwc-ui' conflict when enterprise is installed
Mar 15, 2017
8837fff
Fix st2web yaml
Mar 15, 2017
48dcd6b
Remove all safety checks because arma says so
Mar 16, 2017
b4843f3
Use to_nice_yaml
Mar 16, 2017
76084dd
Add smoke tests for BWC RBAC
Mar 16, 2017
15ac7e6
Add bwc_smoketests to stackstorm.yml
Mar 16, 2017
9891ed2
lets see if this makes ansible-lint happy
Mar 16, 2017
9b5ff73
Smoke tests need not be idempotent
Mar 20, 2017
a729179
Use hash to save license info and handle when license changed
Mar 21, 2017
51ff517
Use hash as opposed to password_hash
Mar 21, 2017
7baa909
Merge branch 'master' into bwc_install_role
Mar 22, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions roles/bwc/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
bwc_pkg_repo: "enterprise"
# 'latest' to get latest version or numeric like '2.1.1'
bwc_version: latest
# used only if 'bwc_version' is numeric
bwc_revision: 1

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please define all vars used in this role with comments, as well as include them in README.md.

st2_config_file_path: /etc/st2/st2.conf
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As noted before, don't forget to hardcode st2.conf path.
We don't need it in vars (#126 (comment) reasons why).

5 changes: 5 additions & 0 deletions roles/bwc/handlers/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---

- name: reload bwc_rbac
become: yes
command: st2-apply-rbac-definitions --config-file {{ st2_config_file_path }}
27 changes: 27 additions & 0 deletions roles/bwc/meta/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
galaxy_info:
description: Install BWC Entperprise components, setup RBAC and LDAP
author: lakshmi-kannan
company: StackStorm
license: Apache 2.0
min_ansible_version: 2.2
platforms:
- name: Ubuntu
versions:
- trusty
- xenial
- name: EL
versions:
- 6
- 7
categories:
- stackstorm
- BWC
- Brocade Workflow Composer
- repositories
- packagecloud
dependencies:
- role: st2repos
- role: st2
- role: st2web
- role: bwc_repos
22 changes: 22 additions & 0 deletions roles/bwc/tasks/ldap.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---

- name: Setup st2.conf auth backend to LDAP
become: yes
ini_file:
dest: "{{ st2_config_file_path }}"
section: auth
option: backend
value: ldap
backup: yes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notify st2 restart or reload handler (if enough) trigger when changing the st2.conf file.
Example: https://github.com/StackStorm/ansible-st2/blob/master/roles/st2/tasks/auth.yml#L34

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we also have to set the backend_kwargs before restarting. See next step. I did notify.

Copy link
Member

@arm4b arm4b Mar 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Include notify for every task where its needed logically (ex: changing the conf file), its less error prone since "next task" could be removed, adjusted or never ran because of some conditional.

It doesn't matter how many times we call Ansible handlers, - they will be executed in the end of entire play run in any way.

See "Handlers: Running Operations On Change": http://docs.ansible.com/ansible/playbooks_intro.html with better explanation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, good to know about that.


- name: Setup st2.conf auth backend_kwargs for LDAP
become: yes
ini_file:
dest: "{{ st2_config_file_path }}"
section: auth
option: backend_kwargs
value: "{{ ldap.config | to_nice_json }}"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This prints out a multi-line string in the ini file and our config parser pukes. I tried to_json. Still the same.

backup: yes
notify:
- restart st2api
- restart st2stream
29 changes: 29 additions & 0 deletions roles/bwc/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---

- name: Install latest bwc-enterprise package
become: yes
package:
name: bwc-enterprise
state: latest
when: bwc_version == "latest"
tags:
- bwc
- st2 enterprise

- name: Install pinned bwc-enterprise package
become: yes
package:
name: bwc-enterprise={{ bwc_version }}-{{ bwc_revision }}
state: present
when: bwc_version != "latest"
tags:
- bwc
- st2 enterprise

- name: Setup RBAC and setup roles and assignments if enable_rbac is defined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is enable_rbac?

include: "rbac.yml"
when: rbac is defined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please name vars accordingly to role, eg: bwc_rbac.


- name: Setup LDAP and set up LDAP configuration
include: "ldap.yml"
when: ldap is defined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please name vars accordingly to role, eg: bwc_ldap.

47 changes: 47 additions & 0 deletions roles/bwc/tasks/rbac.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---

- name: Create BWC RBAC directories
become: yes
file:
path: "{{ item }}"
mode: "u+rw,g-wx,o-rwx"
owner: st2
group: st2
state: directory
with_items:
- /opt/stackstorm/rbac/assignments
- /opt/stackstorm/rbac/roles

- name: Copy RBAC roles to /opt/stackstorm/rbac/roles directory
become: yes
template:
src: rbac_roles/roles.yml.j2
dest: /opt/stackstorm/rbac/roles/{{ item.name }}.yaml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing any file in /rbac/ dir should notify reload bwc_rbac trigger.

Make sure we run that thing for other tasks in this file.

owner: st2
group: st2
with_items: "{{ rbac.roles }}"
when: rbac_roles is defined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't find any occurrence of rbac_roles var. Is it used anywhere?


- name: Copy RBAC assignments to /opt/stackstorm/rbac/assignments directory
become: yes
template:
src: rbac_assignments/assignments.yml.j2
dest: /opt/stackstorm/rbac/assignments/{{ item.name }}.yaml
owner: st2
group: st2
with_items: "{{ rbac.assignments }}"
when: rbac_assignments is defined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't find any occurrence of rbac_assignments var. Is it used anywhere?


- name: Enable RBAC in st2 configuration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enable auth maybe?

become: yes
ini_file:
dest: /etc/st2/st2.conf
section: auth
option: enable
value: True
backup: yes
notify:
- restart st2
- reload bwc_rbac
- restart st2api

4 changes: 4 additions & 0 deletions roles/bwc/templates/rbac_assignments/assignments.yml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---

username: {{ item.name }}
roles: {{ item.roles | to_nice_yaml }}
5 changes: 5 additions & 0 deletions roles/bwc/templates/rbac_roles/roles.yml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---

name: {{ item.name }}
description: {{ item.description }}
permission_grants: {{ item.permission_grants | to_nice_yaml }}
19 changes: 19 additions & 0 deletions roles/bwc_repos/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
# BWC PackageCloud repository to install: enterprise, enterprise-unstable, staging-enterprise, staging-enterprise-unstable.
bwc_pkg_repo: enterprise

deb_os: "{{ ansible_distribution|lower }}"
deb_os_version: "{{ ansible_distribution_release|lower }}"

rpm_os: "el"
rpm_os_version: "{{ ansible_distribution_major_version }}"

master_token: "{{ license }}"

deb_gpg_key_url: "https://{{ master_token }}:@packagecloud.io/install/repositories/StackStorm/enterprise/gpg_key_url.list?os={{ deb_os }}&dist={{ deb_os_version }}&name={{ ansible_nodename }}"

deb_config_file_url: "https://{{ master_token }}:@packagecloud.io/install/repositories/StackStorm/{{ bwc_pkg_repo }}/config_file.list?os={{ deb_os }}&dist={{ deb_os_version }}&name={{ ansible_nodename }}"
deb_config_file_location: "/etc/apt/sources.list.d/packagecloud_io_StackStorm_{{ bwc_pkg_repo }}.list"

rpm_config_file_url: "https://{{ master_token }}:@packagecloud.io/install/repositories/StackStorm/{{ bwc_pkg_repo }}/config_file.repo?os={{ rpm_os }}&dist={{ rpm_os_version }}&name={{ ansible_nodename }}"
rpm_config_file_location: "/etc/yum.repos.d/packagecloud_io_StackStorm_{{ bwc_pkg_repo }}.repo"
22 changes: 22 additions & 0 deletions roles/bwc_repos/meta/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
galaxy_info:
description: Install BWC PackageCloud repository
author: lakshmi-kannan
company: StackStorm
license: Apache 2.0
min_ansible_version: 2.2
platforms:
- name: Ubuntu
versions:
- trusty
- xenial
- name: EL
versions:
- 6
- 7
categories:
- stackstorm
- BWC
- Brocade Workflow Composer
- repositories
- packagecloud
40 changes: 40 additions & 0 deletions roles/bwc_repos/tasks/bwc_repos_apt.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---

- name: Assert that master_token is specified
fail: msg="License key must be supplied for BWC enterprise installation."
when: license is not defined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please include all possible vars used in the roles in defaults.yml with comments and decent README.
With that, instead of relying on is not defined we can rely on is empty conditional check.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, vars should follow the {{ role_name}}_var_name naming.
I guess for bwc role ideal var could be {{ bwc_license }}. And yeah, I must admit that for bwc_repo or bwc_repos role var name {{ bwc_repo_license }} looks a bit ugly :/


- name: Install prereqs (Debian)
become: yes
apt:
name: "{{ item }}"
state: present
with_items:
- debian-archive-keyring
- apt-transport-https

- name: Get GPG key URL for private repo
become: yes
shell: 'curl "{{ deb_gpg_key_url }}"'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not idempotent

register: deb_gpg_key

# This is the exact key as the open source repo but this behavior might change. So just
# following what's in packagecloud docs for private repos.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, and it's not idempotent.

Please refactor bwcrepos role following the st2repos logic where possible.

Ex for this case:

- name: Add keys to keyring
become: yes
apt_key:
id: 418A7F2FB0E1E6E7EABF6FE8C2E73424D59097AB
url: https://packagecloud.io/StackStorm/{{ st2_pkg_repo }}/gpgkey
state: present

- name: Add GPG key to keyring
become: yes
shell: 'curl -L "{{ deb_gpg_key.stdout }}" | apt-key add -'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not idempotent

when: ansible_os_family == "Debian" and license is defined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We never reach this task if license is not defined, because of fail task in this playbook.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ansible_os_family == "Debian"

We're already in Debian play, according to file name bwc_repos_apt.yml.


- name: "Adding packagecloud.io repository: StackStorm/{{ bwc_pkg_repo }} with generated read token"
become: yes
shell: 'curl "{{ deb_config_file_url }}" > {{ deb_config_file_location }}'
args:
creates: "{{ deb_config_file_location }}"
register: added_bwc_deb_repository
when: ansible_os_family == "Debian" and license is defined

- name: Update APT package cache
become: yes
apt:
update_cache: true
when: ansible_os_family == "Debian" and added_bwc_deb_repository|success
43 changes: 43 additions & 0 deletions roles/bwc_repos/tasks/bwc_repos_yum.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---

- name: Assert that master_token is specified
fail: msg="License key must be supplied for BWC enterprise installation."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fail check is duplicated in both bwc_repos_yum.yml and bwc_repos_apt.yml files.
Sounds cool if we can juse move it to parent main.yml file.

when: license is not defined

# Fixes "Failure talking to yum: Cannot retrieve repository metadata (repomd.xml) for repository: StackStorm_stable. Please verify its path and try again" when installing st2
- name: Update ca-certificates package
become: yes
yum:
name: ca-certificates
state: latest
tags: skip_ansible_lint

# See: https://github.com/docker-library/docs/tree/master/centos#package-documentation
# We ship `nginx.conf` via `st2` package doc files, for example
- name: Enable shipping package documentation files for EL
become: yes
ini_file:
dest: /etc/yum.conf
section: main
option: tsflags
value: nodocs
state: absent
when: ansible_os_family == "RedHat"

- name: Add BWC enterprise repo
become: yes
shell: 'curl "{{ rpm_config_file_url }}" > {{ rpm_config_file_location }}'
args:
creates: "{{ rpm_config_file_location }}"
register: added_bwc_rpm_repository
when: ansible_os_family == "RedHat" and license is defined

- name: Update yum package cache
become: yes
shell: yum -q makecache -y --disablerepo='*' --enablerepo='StackStorm_{{ bwc_pkg_repo|replace("/", "_") }}'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any idea why do we need this?
Also it's not idempotent.

when: ansible_os_family == "RedHat" and added_bwc_rpm_repository|success

- name: Update yum package cache BWC enterprise source repo
become: yes
shell: yum -q makecache -y --disablerepo='*' --enablerepo='StackStorm_{{ bwc_pkg_repo|replace("/", "_") }}-source'
when: ansible_os_family == "RedHat" and added_bwc_rpm_repository|success
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when: ansible_os_family == "RedHat"

we're already in os-family specific yum task.

6 changes: 6 additions & 0 deletions roles/bwc_repos/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: Add BWC enterprise repos on {{ ansible_distribution }}
include: bwc_repos_{{ ansible_pkg_mgr }}.yml
tags:
- BWC repos
- StackStorm enterprise