Monday, 10 December 2018 17:11

Learning Ansible With Docker

Written by
Rate this item
(0 votes)

Ansible is an open source operations automation platform which is very simple to setup and use. It makes performing repetitive tasks on multiple computing resources far simpler and more consistent. Getting started with Ansible is simple and uncomplicated. Here is one way to start learning Ansible.

This is the first of a multi-part series on learning Ansible. In this article, we will learn the basic concepts and terms to establish a basic vocabulary. This will only be an overview of basic concepts. A more detailed explanation of the concepts will be covered in subsequent articles.

We will also set up a small, "virtual data" center with a few compute instances for Ansible to manage. The environment will contain one instance on which we will run our Ansible commands and scripts, and three others which will represent the resources we want to manage. This small lab environment can be used to try out any and all the features of Ansible.

Ansible Overview

Ansible is an open source application that automates software provisioning, configuration management, and application deployment. Ansible connects via SSH, remote PowerShell or via other remote APIs. It is a simple to use the tool in helping you with configuration management, application deployment, and the automation of tasks that involve servers, switches, and other network elements.

Tasks can be configured to run in a sequence on multiple different network devices. For example, a software update can be scheduled for a group of devices. Ansible can upgrade the devices one after the other to ensure the group is always accessible.

Ansible uses an agentless architecture, which does not require the installation and background execution of daemons - processes that handle requests for services and are inactive until called upon. Instead, Ansible uses OpenSSH to communicate with other devices on the network.

Ansible establishes connections with all the devices specified in its inventory file, executes commands on those devices, and the returns their results.

Modules

Commands are executed through the use of small programs called modules. They are Python scripts that get executed on the network devices that Ansible manages. Modules can have arguments assigned to them which modify their operation.

After being executed on a device, a module returns information to Ansible in JSON formatted responses. These responses are used to populate variables in the Ansible runtime context, allowing multiple modules to share data.

Single modules can be run using the ansible command. This is helpful for performing simple tasks on one or more network connected devices.

Multiple modules are executed through the use of "playbook" files which specifies multiple modules to be run in a specific order.

Playbooks

Playbooks are YAML files Ansible uses for the configuration, deployment, and orchestration of remote devices. They are a set of instructions that tell Ansible how to execute modules. For example, playbooks can describe the tasks that Ansible needs to perform a rolling update for multiple devices.

Playbooks can declare configurations. They can also be used to orchestrate the steps of any manually ordered process, even if different steps required to jump from one device to another in a specific sequence. Playbooks can launch tasks both synchronously and asynchronously.

Inventory

The inventory contains the list of hosts managed by Ansible. Hosts in the inventory are generally arranged into groups and the actions performed by Ansible are carried out on all the hosts in a group simultaneously. The Ansible host file is usually found at /etc/ansible/hosts.

It is possible to create multiple inventory files to better manage what devices are targeted for Ansible modules execution. Ansible can make use of multiple inventory files at the same time.

Playbooks are executed in the context of an inventory. This provides the playbook with the hosts against which the modules are to be executed. A playbook can be executed against only the web servers for example by specifying the inventory file which contains the web server hostnames.

Tasks

Tasks are used by playbooks to execute a module with a specific set of arguments. Every playbook contains a lists of tasks. Ansible goes through a playbook and executes tasks in sequential order. It runs a task on all devices that match the host pattern assigned to the task. After completing the task, Ansible moves to the next task in the playbook.

During the execution of a playbook, all hosts are going to get the same task directives. The purpose of a playbook is to map a selection of hosts to a selection of tasks, while the purpose of a task is to execute a module, usually with very specific arguments. Variables can be used in arguments assigned to modules.

Templates

Templates are tokenized strings which when processed by a template engine, produce rendered strings with values in place of the tokens. During the execution of a playbook, Ansible can read template files containing tokenized strings and substitutes the tokens in the template with variables and outputs a file that can be used in the configuration of a network connected device.

Ansible templates are realized in Ansible as a module. Templates in Ansible are implemented using Jinja2 - a popular Python template engine. For more information about Jinja2, see http://jinja.pocoo.org/.

Variables

Variables are placeholders for which their value may or not be known. The variable name is used to reference the value stored. This separation between the variable name and value allows the variable name to be used independently of the exact value it represents.

Although variable names can be composed of letters, numbers, and underscores, they should always start with a letter.

Variables can be assigned to hosts in the inventory file at either the host or group level. They can also be specified in a playbook and other files.

Create Our Environment

Now that we have established the basic Ansible vocabulary, we will create our virtual data center. It will be our "laboratory" to try out Ansible is a safe, self-contained environment.

The first compute instance will be the Ansible control node. We will SSH into this node and execute all our Ansible commands. The remaining nodes will represent the compute instances we want to configure and manage. One will be a web server, another will be a database server.

Why Docker?

Early into this tutorial, some may ask why use Ansible to configure Docker images? While we will be doing exactly this, the goal is not to create configured containers, but to create blank Linux environments. The use of Docker is to create very lightweight analog to virtual machines or compute instances. If you have an account with Amazon Web Services (AWS), Google Cloud Platform (GCP), Azure or any of the other cloud providers, you can use compute instances in the cloud. This tutorial uses Docker because it is an inexpensive alternative to running full virtual machines.

Docker Install

Installing docker is covered here in far better detail than this article will attempt. Each platform is different and using the official Docker instructions is simple and complete.

Once Docker is installed on your machine, you should be able to verify its operation by entering docker --version on the command line

Docker Images

We will be using two docker images for our lab environment. The first is an Ansible image which allows you to SSH into a command line and run Ansible commands. The second is a simple Linux node which allows you to connect via SSH and run Python.

There is a GitHub repository which contains the Dockerfiles for creating your own images for this lab.

1. Project Directory

Create a directory to hold your project. This will be referred to as <project> in subsequent steps.

Create a directory called control within your project directory. This will be <project>/control in subsequent steps.

Create a directory called node within your project directory. This will be <project>/node in subsequent steps.

2. Control Node

The control node is an Ubuntu image with SSH and Ansible installed. It allows you to set up a container which listens to SSH and use Ansible from the command line.

In this step we are going to create a Docker image from which we can create a container that has everything installed to run Ansible plays.

Change to the <project>/control directory and create a file named Dockerfile with the following contents:

FROM ubuntu:18.04

# Install Ansible
RUN apt-get update \
&& apt-get install -y software-properties-common \
&& apt-add-repository ppa:ansible/ansible \
&& apt-get update -y \
&& apt-get install ansible -y

# Install the SSH daemon so we can log into this container and run our playbooks
RUN apt-get install -y openssh-server \
&& mkdir /var/run/sshd

# Install some helper tools - git, vim, sudo and unzip
RUN apt-get install -y unzip git sudo vim

# Add a non privileged user that Ansible can use for connections
RUN useradd -m ansible -s /bin/bash \
&& echo 'ansible:ansible' | chpasswd

# Make sure the ansible user can sudo
RUN echo "ansible ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/ansible \
&& chmod 0440 /etc/sudoers.d/ansible

EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]

From the command line in the same directory containing the Dockerfile you created (<project>/control), run

docker build -t ansible .

This will create a Docker image from which containers can be run which will allow you to log in with SSH and run Ansible playbooks. For this image, you will SSH using a username of ansible with a password of ansible

3. Remote Node

The remote node image represents a blank Linux system. This image will only contain the base operating system, SSH and Python. There are no agents needed for Ansible to manage a system so this is a minimal install of Linux. We will use Ansible to SSH into this image and configure / provision it to meet our needs.

In this step we will create a Docker Image which contains a Linux Operating System, SSH daemon and Python, the minimum base image needed to be managed by Ansible. From this image we can create as many containers as we like and manage them all with our control node. It is possible to just spin up this container as a blank canvas and then call a playbook to make it into almost anything we need; a database, a web server, an application server, development workstation, just about anything. A blank remote node like this is a quick way to develop and test your playbooks.

Change into the <project>/node directory.

Create a file named Dockerfile with the following contents:

FROM ubuntu:18.04

# Add the minimal packages for an Ansible to connect and manage our configuration
RUN apt-get update \
&& apt-get install -y openssh-server sudo python2.7 python-pip python3-pip \
&& mkdir /var/run/sshd

# Add a non privileged user that Ansible can use for connections
RUN useradd -m ansible -s /bin/bash \
&& echo 'ansible:ansible' | chpasswd

# Make sure the ansible user can sudo
RUN echo "ansible ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/ansible \
&& chmod 0440 /etc/sudoers.d/ansible

# Expose ssh and common ports.
EXPOSE 22 80 443

# Run SSH daemon
CMD ["/usr/sbin/sshd", "-D"]

From the command line in the same directory containing the Dockerfile you created (<project>/node), run

docker build -t node .

This will create a Docker image from which containers can be run allowing you to log in with SSH. The node image has two users: root with no password  and the ansible user with a password of ansible.

4. Check Your Images

You can perform a quick check to ensure that both the control and node image exists in your docker repository with the following command:

docker images

You should see at least the node and control images in your repository. If not, it is safe to repeat the steps above for the missing images.

Setting up the Lab

In this section, we will set up an Ansible control node and 3 bare Linux hosts representing our mini data center using the Docker images we built above.

Open a command line and change to the <project> directory you created above. Enter the following commands to set up an Ansible control node and 3 remote nodes:

docker run -d -P -v ansible:/opt/ansible --name ansible --hostname ansible ansible
docker run -d -p :22 -p 80:80 --name nodea --hostname nodea node
docker run -d -p :22 --name nodeb --hostname nodeb node
docker run -d -p :22 --name nodec --hostname nodec node

The arguments above may seem complicated, so let's break them down:

  • -d instructs Docker to run the container in detached mode, no console output.
  • -v maps a volume within the contatiner (see Saving Our Data below)
  • -P instructs Docker to map all exposed ports to the local host
  • -p instructs Docker to map a specific port on the local host to a specific port in the container
  • --name is the name of the container on our workstation
  • --hostname also sets the hostname inside the container

You can now run docker ps and see four containers running.

running containers

Saving Our Data

Filespace in containers is ephemeral. When a container is deleted, all it files space is deleted as well. The work we do in the control node will be important. We would like it to persist our data even if the container is deleted. We can accomplish this through the use of Docker volumes which will persist our data when the container is gone.

A Docker volume is a special layer which can be applied to the container after it is started, providing a filespace for the container to use. When the container is removed, the volume layer remains to be reused by another container. We will start the control node with a layer which will be mapped to /opt/ansible so any files we place in that location will not be lost.

Node Addresses

The Ansible control node will need the IP addresses of the nodes in its inventory. Run the following commands to retrieve the internal IP addresses of the (remote) nodes:

docker inspect -f '{{ .NetworkSettings.IPAddress}}' nodea
docker inspect -f '{{ .NetworkSettings.IPAddress}}' nodeb
docker inspect -f '{{ .NetworkSettings.IPAddress}}' nodec

You will see the IP addresses for each of the nodes after each command. Make a note of each one, you will be creating an inventory file next. The following is an example:

Node Addresses

Log Into the Control Node

We should have four containers running within Docker: ansible, nodea, nodeb, and nodec. Confirm this by entering docker ps from your command line.

The ansible node is our control node from which we will be running all our ansible commands. We will need to SSH into that container, but we need to know what port that container has mapped to SSH port 22. This is available from the output of the docker ps command. Not everyone's port mapping will be the same as Docker uses the next available port for its mapping and depends on the order in which the containers were loaded.

You are looking on the row for the container with the name of ansible that also has the image set to ansible. We are looking for what port is mapped to port 22 on the container. On my instance, there is a row which looks like this:

CONTAINER ID  IMAGE    COMMAND            CREATED     STATUS         PORTS                  NAMES
1d56a7ba0462  ansible  "/usr/sbin/sshd -D"  6 days ago  Up 19 seconds  0.0.0.0:32768->22/tcp  ansible

The important part is 0.0.0.0:32768->22/tcp . This tells us on what ports the ansible node is listening. In this case, we see ansible is listening on our local host address with a port 32768 and forwarding traffic to port 22 on the container. This means to SSH into that container, we need to point our SSH client to port 32768 on the local host. Yours will probably be different.

Open an SSH session with the ansible container on the port specified on that line. You will be using a username of ansible and a password of ansible. If you are using Windows, you will probably be using PuTTY:

 Control Node SSH Session

Update Ansible Hosts File

Setup the main inventory file with the IP addresses of the nodes in the lab. SSH into the ansible node.

This container image has vim installed as the editor. you can install your own editor if you want, or just us vi. If you prefer another editor, you should be able to call apt-get to install your editor of choice. For example, apt-get install nano should install nano on the control node.

Using your editor of choice, update the /etc/ansible/hosts file by adding the IP addresses of the nodes you discovered in the Node Addresses section above to the bottom of the file, each on a separate line. You will need to sudo as root to edit this file. Here is what mine looks like although your IP addresses may be different:

[email protected]:~# tail /etc/ansible/hosts
## 10.25.1.57

# Here's another example of host ranges, this time there are no
# leading 0s:

## db-[99:101]-node.example.com
172.17.0.3
172.17.0.4
172.17.0.5

[email protected]:~#

The default inventory file is now created.

Best practice naming - Optional

If you want to use best practices and benefit from a more natural output, don't just put in the IP addresses, but use names. Here is a better way to specify the addresses and associate them with meaningful names in our lab:

nodea ansible_host=172.17.0.3
nodeb ansible_host=172.17.0.4
nodec ansible_host=172.17.0.5

Now the names of the remote nodes will be displayed as the playbooks execute instead of the IP addresses.

SSH Credentials

Ansible will not be able to SSH into the remote nodes unless SSH credentials are stored remotely. You will need to generate a key and copy it to each of the remote nodes.

Note that it is possible to use the username:password approach, but that requires you to include the password as a variable in playbooks which can complicate matters. There is a feature called Ansible Vault that allows you to keep sensitive data such as passwords or keys in encrypted files, but this will add a level of complexity we just don't need for this environment. We will use SSH key pairs instead.

Generate a Key

SSH into the Ansible control node and run ssh-keygen and accept the defaults.

ssh-keygen -t rsa -b 4096 -N ""

The above will generate a 4096 bit RSA key. It may be a bit much for the lab, but it won't hurt.

Copy Keys to Each Node

This can be tedious, but a necessary part of the process. Use the ssh-copy-id script to log into a host and copy the local SSH key (possibly generated above) to the remote host:

/usr/bin/ssh-copy-id [email protected]<ip_address>

This will be done for each of the 3 nodes you placed in your /etc/ansible/hosts file above.

You will be asked to accept the ECDSA key for the host (if this is the first time you are connecting to this address) and for the password to the account on the remote machine to authenticate. A session will look something like this:

SSH Copy ID Example

The ssh-copy-id script takes the ssh keys in the users id_rsa.pub file, logs into the remote machine (which is why you were prompted for a password), adds each of the public keys to the remote server and then logs out of the remote system. You should now be able to SSH into the remote system and not have to enter a password. This is what we want for Ansible to run smoothly.

Be sure to perform this for each of the three nodes (nodea, nodeb, and nodec) using their respective IP addresses.

Ping the Remote Nodes

You should now be able to ping all your nodes with Ansible:

Ansible Ping

Conclusion

We have covered some of the basics of Ansible and developed a common vocabulary. More will be covered later, but for now, we have enough to get us started.

We set up some Linux environments to be managed and one to act as our control node. From the control node we can run Ansible commands and playbooks to start practicing our Ansible skills.

In the next article of the series, we will cover Ansible basics and provisioning a Content Management System running in containers.

Last modified on Tuesday, 05 March 2019 21:45
Steve Cote

Steve has been a systems architect for the telecommunications and electric utility industries creating circuit provisioning systems and smart grid solutions. He is a hands-on architect working with teams to deliver large complex projects. More recently, Steve has been consulting agile teams from Fortune 15 organizations, start-ups, and everything in-between to be more productive with DevOps and agile practices.

Leave a comment

Make sure you enter all the required information, indicated by an asterisk (*). HTML code is not allowed.