Ansible initiative #60

saravanan30erd · 2021-09-02T16:03:27Z

Description

Ansible scripts for opensearch setup

Issues Resolved

Check List

Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

saravanan30erd · 2021-09-02T16:08:05Z

@peterzhuamazon

I have split the task into many phases, currently I have completed the single node opensearch setup.

I will work on below tasks in upcoming days,

opensearch multi-node cluster setup
opensearch dashbaords Installation
Performance analyzer plugin configuration

saravanan30erd · 2021-09-07T08:52:35Z

@peterzhuamazon

Completed the task for opensearch dashbaords Installation. I think we can start from here since pretty much I covered all which is necessary for single node installation.

peterzhuamazon · 2021-09-08T16:41:55Z

Thanks @saravanan30erd sorry for the late responses.

@DandyDeveloper @TheAlgo Please review and comment when you have time.

bbarani · 2021-09-08T16:40:39Z

ansible/inventories/opensearch/group_vars/all/all.yml

+xmx_value: 2
+
+## Admin user credentials will be used for User & Roles management and cluster operations
+admin_password: Test@123


@saravanan30erd can we encapsulate the credentials rather than adding it directly to yml?

bbarani · 2021-09-08T16:42:49Z

ansible/roles/centos7/dashboards/tasks/dashboards.yml

+- name: Dashboards Install | create systemd service
+  template:
+    src: dashboards.service
+    dest: /etc/systemd/system/dashboards.service


@saravanan30erd Can we avoid hardcoding the paths?

peterzhuamazon · 2021-09-08T19:15:00Z

Adding @hyandell to co-proofread for any changes in the PR.

hyandell

Hi Saravanan,

As Ansible is GPL licensed and has many plugins under different licenses, did you base your work here on a pre-existing codebase? The concern with code that touches on other projects being that we don't want to confuse the licensing with different licenses that affect the overall licensing.

I did see that you had https://github.com/saravanan30erd/opendistro_standalone_installation - which I assume this is based partly on; I'm just concerned about any other repos either this PR or that earlier repo is based on.

saravanan30erd · 2021-09-09T14:09:59Z

Hi @hyandell

I have created this opendistro repo from scratch, its not based on any external repo. Also this PR is using only few logics from opendistro one but major areas are created from scratch.

saravanan30erd · 2021-09-09T14:27:11Z

@bbarani I have made the changes based on your feedback.

peterzhuamazon · 2021-09-09T16:39:21Z

Thanks @saravanan30erd for the confirmation and the changes here.
Need more eyes to check this @smlx let me know if you can help with the approval.
I will test this in a few days as I am currently working on another OpenSearch releases.

hyandell · 2021-09-09T22:50:40Z

Hi @hyandell

I have created this opendistro repo from scratch, its not based on any external repo. Also this PR is using only few logics from opendistro one but major areas are created from scratch.

Thank you Saravanan :)

DandyDeveloper · 2021-09-10T03:54:18Z

@peterzhuamazon I think for this one, I don't see any problems, but I don't have the time to test this at this moment in time. So I'll pass on this specific PR for review.

peterzhuamazon · 2021-09-10T17:32:26Z

Thanks @hyandell @DandyDeveloper for your inputs on this.
@saravanan30erd I will start the tests next week.
@TheAlgo please also take a look when you have time.

Thanks.

TheAlgo · 2021-09-11T07:35:54Z

ansible/README.md

@@ -0,0 +1,45 @@
+Deploy OpenSearch with Dashboards using Ansible


Nit : OpenSearch with OpenSearch Dashboards

TheAlgo · 2021-09-11T07:37:18Z

ansible/roles/centos7/dashboards/templates/opensearch_dashboards.yml

+server.host: "{{ hostvars[inventory_hostname]['ip'] }}"
+opensearch.hosts: ["{{ os_nodes_dashboards }}"]
+opensearch.ssl.verificationMode: none
+opensearch.username: "kibanaserver"


Can we use a different name here , maybe dashboardserver?

@TheAlgo kibanaserver is the name used in default configuration file from Opensearch dashboards source file(Tar file). Refer: https://opensearch.org/docs/dashboards/install/tls/

I suggest we should follow the default configuration from installation source to avoid confusions. May be it should be changed first in source files and official documentations then we can change it.

@peterzhuamazon Just curious , why are we using kibanaserver till now?

@TheAlgo @peterzhuamazon kibanaserver is used in security plugin where it should be changed and then configuration. I raised the below PRs to fix this. Once this merged and released, then we can update here.

opensearch-project/security#1443
opensearch-project/opensearch-build#458

We are still in discussion with Security Team on this @saravanan30erd please hold on for some time before we make a decision on this.

Thanks.

@peterzhuamazon Sure, I think we can continue with kibanaserver here as of now.

We will keep this for now @TheAlgo.
@saravanan30erd please create an issue to track this.
Thanks.

TheAlgo · 2021-09-11T07:38:12Z

ansible/roles/centos7/dashboards/vars/main.yml

@@ -0,0 +1,7 @@
+---
+# vars file for elasticsearch


Can we name all instances of elasticsearch to opensearch and kibana to dashboards here?

@TheAlgo Done the changes.

TheAlgo · 2021-09-11T07:39:46Z

@saravanan30erd I took a quick pass. PR looks good to me except some nitpicks regarding naming. Good work !!

peterzhuamazon · 2021-09-13T20:25:19Z

Rename PR: opensearch-project/security#1443

saravanan30erd · 2021-09-13T21:39:40Z

@peterzhuamazon I think we will rename kibanaserver later in this project after its changed in security plugin and dashboard configuration, now we can continue the work here.

peterzhuamazon · 2021-09-13T22:37:37Z

@peterzhuamazon I think we will rename kibanaserver later in this project after its changed in security plugin and dashboard configuration, now we can continue the work here.

I agree let us keep this in mind for now.
Can you create an issue to track this?
Thanks.

saravanan30erd · 2021-09-14T04:38:57Z

@peterzhuamazon https://github.com/opensearch-project/opensearch-devops/issues/69

peterzhuamazon · 2021-09-15T17:05:35Z

Hi @saravanan30erd

We have recently created a ansible-playbook repo here: https://github.com/opensearch-project/ansible-playbook

To avoid migration issues like helm-charts, would you mind move this PR to there as well?
We can link this PR as reference.

This would be easier for people to access Ansible as a whole without mixing with other components.
You just need to add a pointer from devops repo to ansible repo once complete, so people knows where is ansible playbook.

This devops repo will serve as a central hub for all the OpenSearch Project related automations.

Let me know what you think.

Thanks.

saravanan30erd · 2021-09-15T18:07:18Z

@peterzhuamazon Great, I will move this PR to new repo. Its better to maintain the separate repo as it gets complex when we add other features.

peterzhuamazon · 2021-09-15T19:27:23Z

@peterzhuamazon Great, I will move this PR to new repo. Its better to maintain the separate repo as it gets complex when we add other features.

That is exactly the reason why we make the standalone repo.
Just take a bit time since Ansible itself is not Apache2 to start with, so we carefully review the use case and your PR content before we proceed with the repo creation.
Thanks.

peterzhuamazon · 2021-09-15T23:02:53Z

Please close this PR once you raise the other one and link this one there.

saravanan30erd · 2021-09-16T09:30:34Z

@peterzhuamazon I moved this PR to new ansible repo(opensearch-project/ansible-playbook#2).

peterzhuamazon · 2021-09-16T17:54:19Z

Thanks @saravanan30erd I will close this one and lets work on the other one, I migrate opensearch-project/ansible-playbook#3 to ansible playbook repo as well.
I also create an new issue here #71 so you can take action to create pointer to point people from devops to ansible repo.

Thanks.

saravanan30erd added 7 commits August 31, 2021 17:32

initial commit - copy from opendistro

9f7e44b

check cert for single noe

865c1f3

fix the naming convention

cbb337d

create OS installation tasks

339be65

create security configuration tasks

1356eb3

fix the syntax errors

815f73e

update readme

80c376b

TheAlgo requested a review from peterzhuamazon September 6, 2021 08:29

saravanan30erd added 3 commits September 6, 2021 20:39

add scripts for dashboards

6e817be

fix the errors in dashbaord installation

bc850a4

update readme

3c9a194

peterzhuamazon requested a review from TheAlgo September 8, 2021 16:40

bbarani reviewed Sep 8, 2021

View reviewed changes

peterzhuamazon requested a review from hyandell September 8, 2021 19:14

hyandell reviewed Sep 8, 2021

View reviewed changes

pass credentials in command

807949c

hyandell approved these changes Sep 9, 2021

View reviewed changes

TheAlgo requested changes Sep 11, 2021

View reviewed changes

fix the naming conventions

ff9abe5

saravanan30erd mentioned this pull request Sep 16, 2021

Ansible playbook Initiative opensearch-project/ansible-playbook#2

Merged

1 task

peterzhuamazon closed this Sep 16, 2021

		@@ -0,0 +1,45 @@
		Deploy OpenSearch with Dashboards using Ansible

Ansible initiative #60

Ansible initiative #60

Conversation

saravanan30erd commented Sep 2, 2021 • edited Loading

Description

Issues Resolved

Check List

saravanan30erd commented Sep 2, 2021

saravanan30erd commented Sep 7, 2021 • edited Loading

peterzhuamazon commented Sep 8, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peterzhuamazon commented Sep 8, 2021

hyandell left a comment

Choose a reason for hiding this comment

saravanan30erd commented Sep 9, 2021

saravanan30erd commented Sep 9, 2021

peterzhuamazon commented Sep 9, 2021 • edited Loading

hyandell commented Sep 9, 2021

DandyDeveloper commented Sep 10, 2021

peterzhuamazon commented Sep 10, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saravanan30erd Sep 11, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saravanan30erd Sep 11, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheAlgo commented Sep 11, 2021

peterzhuamazon commented Sep 13, 2021 • edited Loading

saravanan30erd commented Sep 13, 2021 • edited Loading

peterzhuamazon commented Sep 13, 2021

saravanan30erd commented Sep 14, 2021

peterzhuamazon commented Sep 15, 2021

saravanan30erd commented Sep 15, 2021

peterzhuamazon commented Sep 15, 2021

peterzhuamazon commented Sep 15, 2021

saravanan30erd commented Sep 16, 2021

peterzhuamazon commented Sep 16, 2021 • edited Loading

saravanan30erd commented Sep 2, 2021 •

edited

Loading

saravanan30erd commented Sep 7, 2021 •

edited

Loading

peterzhuamazon commented Sep 8, 2021 •

edited

Loading

peterzhuamazon commented Sep 9, 2021 •

edited

Loading

saravanan30erd Sep 11, 2021 •

edited

Loading

saravanan30erd Sep 11, 2021 •

edited

Loading

peterzhuamazon commented Sep 13, 2021 •

edited

Loading

saravanan30erd commented Sep 13, 2021 •

edited

Loading

peterzhuamazon commented Sep 16, 2021 •

edited

Loading