Deploy an Instance in MicroStack with Terraform

This section is intended to show how to deploy an instance in OpenStack using Terraform. Consider all the steps outlined. The test was done on an OpenStack based on MicroStack Canonical distribution. The deployment was performed in another machine named Deployment Machine with the following hardware requirements:

Feature	Value
CPU	2
RAM	4 GiB
Disk	50 GB
OS Used	Ubuntu 20.04 LTS

The basic connection architecture between OpenStack and Terraform is presented in the next figure:

The contents of this page are:

• Introduction
  • Install Terraform
  • Adding Additional Images to OpenStack
  • Configure the Integration between OpenStack and Terraform
  • Initialize Terraform + OpenStack
    • Cloud Init file
  • Build the Infrastructure

Introduction

Terraform is an open-source infrastructure as code software tool that provides a consistent CLI workflow to manage hundreds of cloud services. Terraform codifies cloud APIs into declarative configuration files. A complete and wide definition can be found in the official page.

Install Terraform

This procedure summarizes the Terraform installation. Go to this link and download the .zip file for your operating system (OS). In our case, the OS is Ubuntu 20.04 LTS. Next, unzip the file and copy the Terraform file in bin files. The steps (in our case) are next.

sudo apt update && sudo apt upgrade -y
sudo apt install -y zip
# At time of writing, Terraform latest stable version is 0.14.10
cd && wget https://releases.hashicorp.com/terraform/0.14.10/terraform_0.14.10_linux_amd64.zip
unzip terraform*.zip
sudo rm terraform*.zip
sudo mv terraform /usr/local/bin/

Next, for validating the installation. In CLI you should see Terraform version printed out.

terraform version
# Terraform v0.14.10

Now, before initialize Terraform, create the the next folders.

mkdir ~/terraform && cd ~/terraform
mkdir openstack
mkdir files

These folders may be used for provide OpenStack to terraform.

Reference: https://www.techrepublic.com/article/how-to-install-terraform-on-ubuntu-server/

Adding Additional Images to OpenStack

By default, MicroStack only includes the CirrOS image. While it is useful for testing that OpenStack can provision resources, it does not provide the ability to install packages. Many Linux distributions create OpenStack-ready images, a list of which can be found in: https://docs.openstack.org/image-guide/obtain-images.html.

For this experiment, we use a Debian Buster image. The next steps shows how to download the OpenStack compatible image and add to the cloud platform.

# (i) Download the image
wget https://cdimage.debian.org/cdimage/openstack/current/debian-10.6.0-openstack-amd64.qcow2
# (ii) Add the image to OpenStack
openstack image create --container-format bare --disk-format qcow2 --file debian-9-openstack-amd64.qcow2 debian10

Next, verify that the image is defined. Assume that you have alias openstack="microstack.openstack" configured.

openstack image list
# +--------------------------------------+---------------+--------+
# | ID                                   | Name          | Status |
# +--------------------------------------+---------------+--------+
# | cf5aa2d9-1ed6-4292-975c-ff0d7ae121ec | cirros        | active |
# | b4e47934-c000-467c-bced-bb7f17d31373 | debian10      | active |
# +--------------------------------------+---------------+--------+

The advantage of this image (and many others) is that it allow the use of cloud-config file, allowing you to specify first-time boot configuration, package installation and remote access credentials like SSH passwords.

Configure the Integration between OpenStack and Terraform

There are two possible ways to integrate Terraform with OpenStack. Both are based on Terraform's access to the OpenStack API, therefore, an Application Credential or a direct Log In can be used. In this case, I consider the second option that is insecure, but, for practical purposes, it is a good option to show the integration between Terraform and OpenStack. Please see this link to get more detail about Application Credential integration. Also, this section shows how to create an application credential on OpenStack.

First, consider download the RC file of your OpenStack. For this, go to UI and in your user option, download the RC file. See the next figure:

When you open this file, an structure as the next is shown:

#!/usr/bin/env bash
# To use an OpenStack cloud you need to authenticate against the Identity
# service named keystone, which returns a **Token** and **Service Catalog**.
# The catalog contains the endpoints for all services the user/tenant has
# access to - such as Compute, Image Service, Identity, Object Storage, Block
# Storage, and Networking (code-named nova, glance, keystone, swift,
# cinder, and neutron).
#
# *NOTE*: Using the 3 *Identity API* does not necessarily mean any other
# OpenStack API is version 3. For example, your cloud provider may implement
# Image API v1.1, Block Storage API v2, and Compute API v2.0. OS_AUTH_URL is
# only for the Identity API served through keystone.
export OS_AUTH_URL=http://localhost:5000/v3/
# With the addition of Keystone we have standardized on the term **project**
# as the entity that owns the resources.
export OS_PROJECT_ID=7625993ecf6d4b759984126f50d5ce4a
export OS_PROJECT_NAME="default"
export OS_USER_DOMAIN_NAME="Default"
if [ -z "$OS_USER_DOMAIN_NAME" ]; then unset OS_USER_DOMAIN_NAME; fi
export OS_PROJECT_DOMAIN_ID="default"
if [ -z "$OS_PROJECT_DOMAIN_ID" ]; then unset OS_PROJECT_DOMAIN_ID; fi
# unset v2.0 items in case set
unset OS_TENANT_ID
unset OS_TENANT_NAME
# In addition to the owning entity (tenant), OpenStack stores the entity
# performing the action as the **user**.
export OS_USERNAME="j.caviedes"
export OS_PASSWORD=password1234
# If your configuration has multiple regions, we set that information here.
# OS_REGION_NAME is optional and only valid in certain environments.
export OS_REGION_NAME="microstack"
# Don't leave a blank variable, unset it if it was empty
if [ -z "$OS_REGION_NAME" ]; then unset OS_REGION_NAME; fi
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3

Please change the OS_PASSWORD and directly use the variable instead of entering it via the CLI. You can execute the RC file for export the variables using sh <file_RC>.sh. However, I prefer to suggest you put the variables in /etc/environmentand ~/.bashrc files. All comments are unnecessary for this case. See the mext examples for both files.

/etc/environment

# Add to the end of file (/etc/environmet), the next OpenStack Credentials. Modify based on your RC values.
OS_AUTH_URL=http://localhost:5000/v3/
OS_PROJECT_ID=7625993ecf6d4b759984126f50d5ce4a
OS_PROJECT_NAME="default"
OS_USER_DOMAIN_NAME="Default"
OS_PROJECT_DOMAIN_ID="default"
OS_USERNAME="j.caviedes"
OS_PASSWORD=password1234 
OS_REGION_NAME="microstack"
OS_INTERFACE=public
OS_IDENTITY_API_VERSION=3

~/.bashrc

# Add to the end of file (~/.bashrc), the next OpenStack Credentials. Modify based on your RC values.
export OS_AUTH_URL=http://localhost:5000/v3/
export OS_PROJECT_ID=7625993ecf6d4b759984126f50d5ce4a
export OS_PROJECT_NAME="default"
export OS_USER_DOMAIN_NAME="Default"
export OS_PROJECT_DOMAIN_ID="default"
export OS_USERNAME="j.caviedes"
export OS_PASSWORD=password1234
export OS_REGION_NAME="microstack"
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3

Next, execute:

source ~/.bashrc

If any of the variables are wrongly assigned, please consider the following commands to fix them.

• To get OS_AUTH_URL: openstack endpoint list and look for keystone service.
• To get OS_PROJECT_ID and OS_PROJECT_NAME: openstack project list and look for your project,. For example, admin or default.
• To get OS_USER_DOMAIN_NAME: openstack domain list and look for Name column.
• To get OS_PROJECT_DOMAIN_ID: openstack domain list and look for ID column.
• To get OS_USERNAME: openstack user list and look for Name column.
• To get OS_PASSWORD: If you use MicroStack, please use sudo snap get microstack config.credentials.keystone-password and get the admin password.
• To get OS_REGION_NAME: openstack region list.
• To get OS_INTERFACE: In most cases, by default interface is public.
• To get OS_IDENTITY_API_VERSION: In most cases, by default identity API version is 3.

Deploy an Instance in OpenStack with Terraform

In the ~/terraform/openstack folder is necessary to create a main file. The content of this file match with the authentication method discussed in the previous section. Create a instance.tf file and put into them the next code (replace the values with your MicroStack current configuration). Please see the next considerations before execute:

• A Debian 10 image must already be uploaded to OpenStack with the name debian10.
• You need a flavor with 2 vCPU and 4 GiB RAM, named m1.medium.
• A cloud init file must be defined.
• A network named mgmt must be defined before in OpenStack.
• The floating IP 10.20.20.165 must be available.
• Consider that a security group is creating with this Terraform execution. If you have an already security group, please edit this file and use the already group.

provider "openstack" {
}

data "openstack_images_image_v2" "debian-buster" {
  name        = "debian10"
  most_recent = true
}

data "openstack_compute_flavor_v2" "m1-micro" {
  name  = "m1.medium"
}

resource "openstack_compute_instance_v2" "debian-buster" {
  name            = "debian-buster"
  image_id        = data.openstack_images_image_v2.debian-buster.id
  flavor_id       = data.openstack_compute_flavor_v2.m1-micro.id
  security_groups = [
    openstack_networking_secgroup_v2.buster.name
  ]
  user_data       = data.template_file.debian.template

  metadata = {
    prometheus = "true"
    node_exporter = "true"
  }

  network {
    name = "mgmt"
  }
}

data "openstack_networking_floatingip_v2" "debian-buster-fip" {
    address  = "10.20.20.165"
}


resource "openstack_compute_floatingip_associate_v2" "debian-buster-fip" {
    floating_ip = data.openstack_networking_floatingip_v2.debian-buster-fip.address
    instance_id = openstack_compute_instance_v2.debian-buster.id
}

resource "openstack_networking_secgroup_v2" "buster" {
    name        = "buster"
    description = "Buster Security Group"
}

resource "openstack_networking_secgroup_rule_v2" "node_exporter" {
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = 9100
  port_range_max    = 9100
  remote_ip_prefix  = "0.0.0.0/0"
  security_group_id = openstack_networking_secgroup_v2.buster.id
}

resource "openstack_networking_secgroup_rule_v2" "ssh" {
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = 22
  port_range_max    = 22
  remote_ip_prefix  = "0.0.0.0/0"
  security_group_id = openstack_networking_secgroup_v2.buster.id
}

resource "openstack_networking_secgroup_rule_v2" "http" {
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = 80
  port_range_max    = 80
  remote_ip_prefix  = "0.0.0.0/0"
  security_group_id = openstack_networking_secgroup_v2.buster.id
}

resource "openstack_networking_secgroup_rule_v2" "icmp_v4" {
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "icmp"
  remote_ip_prefix  = "0.0.0.0/0"
  security_group_id = openstack_networking_secgroup_v2.buster.id
}

Please see that the first part of file (named provider "openstack") is empty. The reason is that, if this part is empty, Terraform queries the environment variables (/etc/environment or ~/.bashrc) for the connection to OpenStack. You need to create an user_data.tf file in the folder ~/terraform/openstack/. The user_data.tf file is defined as shown.

data "template_file" "debian" {
  template = "${file("$HOME/terraform/files/debian.tpl")}"
}

data "template_cloudinit_config" "debian" {
  gzip          = false
  base64_encode = false

  part {
    filename     = "init.cfg"
    content_type = "text/cloud-config"
    content      = data.template_file.debian.rendered
  }
}

Cloud Init file

The contents of the template file (in this case, located in $HOME/terraform/files/debian.tpl) are:

#cloud-config
package_upgrade: true

password: orion
chpasswd: { expire: False }
ssh_pwauth: True

runcmd:
 - [ sh, -c, 'echo "nameserver 8.8.8.8" >> /etc/resolv.conf' ]
 - sudo apt update && sudo apt install -y apache2
 - sudo systemctl restart apache2
 - sudo apt install -y prometheus-node-exporter
 - sudo apt install -y stress-ng
 - sudo mkdir -p /var/www/example.com
 - sudo chown -R $USER:$USER /var/www/example.com
 - sudo chmod -R 755 /var/www/example.com
 - sudo echo -e "<html>\n\t<head>\n\t\t<title>Welcome to example.com\!</title>\n\t</head>\n\t<body>\n\t\t<h1>Success\!  The example.com virtual host is working!</h1>\n\t</body>\n</html>" >> /var/www/example.com/index.html
 - sudo sed -i 's/\\//g' /var/www/example.com/html/index.html
 - sudo echo -e "<VirtualHost *:80>\n\tServerAdmin webmaster@localhost\n\tServerName example.com\n\tServerAlias www.example.com\n\tDocumentRoot /var/www/example.com\n\tErrorLog /var/log/apache2/error.log\n\tCustomLog /var/log/apache2/access.log combined\n</VirtualHost>" >> /etc/apache2/sites-available/example.com.conf
 - sudo a2ensite example.com.conf
 - sudo a2dissite 000-default.conf
 - sudo apache2ctl configtest
 - sudo systemctl restart apache2

Note that in cloud init file we add some lines that define a SSH password (debian) to access the instance. In our case, we add this password to facilitate experiments. Also, note the initial installation of apache2, the prometheus-node-exporter and stress-ng packages. With these, it is possible to init a web server, export metrics for the monitoring with prometheus and grafana, and stress a web server if necesssary. The rest of the file is a configuration for a basic web server.

If you want to know more about prometheus and grafana monitoring for OpenStack instace resources, please see the section Using Prometheus and Grafana for Virtual Resources Monitoring.

Build the Infrastructure

We can now apply the configuration and see if it builds an OpenStack instance. For this, only execute the next lines in the ~/terraform/openstack directory. After the creation of the instance.tf, user_data.tf and debian.tpl files, perform the next step.

cd ~/terraform/openstack
terraform init

In CLI you should see a message like Terraform has been successfully initialized! in some part of log. To summarize the steps performed in this file, we:

• Use a data source to match the Debian Buster Openstack image that we define earlier.
• Define an OpenStack flavour, i.e., size of the instance (CPU, RAM and Disk).
  • Flavor m1.medium: 1 vCPU, 512 MB of RAM, 5 GB of Disk and Debian as OS based on Debian Buster image.
• Create an OpenStack instance called debian-buster using the flavour, image and SSH password in cloud init file. Also:
  • Associate a security group (i.e. firewall rules). Allow 22 and 9100 mainly.
  • Use a cloud-config template to the instance. Note that OpenStack support user_data in Terraform configuration.
  • Add metadata (i.e. tags) for prometheus and node_exporter.
  • Associate the network test which is predefined in MicroStack.
• Create a floating IP and associate it with this instance.
  • The instance can be destroyed and recreated, but retain the same floating IP.
• Create the security group and add rules for inbound SSH, ICMP (i.e. ping commands) and also allow the node_exporter port that is TCP9100.

Now, you can do:

terraform apply

When finish, the CLI shown Apply complete! Resources: 9 added, 0 changed, 0 destroyed in some part of log. For double check that Terraform is managing these resources execute the next command. You will see similar messages in your CLI.

terraform state list
# data.openstack_images_image_v2.debian-buster
# data.template_cloudinit_config.debian
# data.template_file.debian
# openstack_compute_flavor_v2.m1-micro
# openstack_compute_floatingip_associate_v2.debian-buster-fip
# openstack_compute_instance_v2.debian-buster
# openstack_compute_keypair_v2.symphonyx
# openstack_networking_floatingip_v2.debian-buster-fip
# openstack_networking_secgroup_rule_v2.icmp_v4
# openstack_networking_secgroup_rule_v2.node_exporter
# openstack_networking_secgroup_rule_v2.ssh
# openstack_networking_secgroup_v2.buster

Moreover, we can see if the instance was created in the OpenStack UI. For this, click on Admin page, next on Compute and next on Instances. In your screen you must see some like this.

Now, let's access via SSH to the instance using the password provided in the cloud-config file. Use the floating IP configured in OpenStack. In Our case 10.20.20.165. Be sure that the instance is on.

ssh [email protected]
[email protected] password:
# Linux debian-buster 4.19.0-11-cloud-amd64 #1 SMP Debian 4.19.146-1 (2020-09-17) x86_64
# 
# The programs included with the Debian GNU/Linux system are free software;
# the exact distribution terms for each program are described in the
# individual files in /usr/share/doc/*/copyright.
# 
# Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
# permitted by applicable law.
# Last login: Thu Oct  8 00:21:03 2020 from 10.20.20.1
# debian@debian-buster:~$

Finally, run the next command to see if the prometheus-node-exporter package is running into instance.

debian@debian-buster:~$ ps aux | grep -i prometh
prometh+   408  0.0  2.0 335540 10364 ?        Ssl  21:10   0:00 /usr/bin/prometheus-node-exporter
debian     856  0.0  0.1   6148   884 pts/0    S+   21:10   0:00 grep -i prometh

Also, you can see the web server that is initialized with the cloud init file using the IP of the instance (10.20.20.165). In your browser, type this IP and it displays the following message once Terraform configuration is applied.

Reference: https://yetiops.net/posts/prometheus-service-discovery-openstack/