You’ve been automating your repetitive sysadmin duties with Ansible for some time now. You’ve bounced troublesome companies, deployed objects to Kubernetes, up to date techniques, and carried out rolling restarts. Possibly you will have even deployed and configured companies utilizing roles you discovered on Ansible Galaxy. Nonetheless, it’s lastly occurred. You possibly can’t discover a position on Ansible Galaxy that does what you need it to (you checked, proper?), or you will have been tasked with writing a task on your group’s app. Regardless of the purpose, it’s time so that you can write your personal position. How do you try this in a method the place you could be assured your position works as meant? This submit will reply that query by offering steerage on find out how to finest start growing Ansible roles.
Infrastructure as Code and DevOps
Earlier than we dive into specifics, it will be useful to outline some terminology. Ansible falls below the infrastructure as code (IaC) device umbrella. IaC is the method of managing and provisioning IT assets by means of machine readable definition recordsdata. This method permits the storing of infrastructure configuration in git, gaining all the advantages of doing so, together with branching, historical past, assessment and approval insurance policies, and many others.
DevOps is the set of practices that mixes software program improvement (dev) and IT operations (ops) to shorten the event lifecycle and supply steady supply of software program. IaC instruments are one piece of the puzzle that permit organizations to undertake a DevOps method. The DevOps course of is often depicted as follows:
Determine 1: The DevOps Infinity Loop.
It’s useful to have this course of in thoughts whereas growing your Ansible roles, as a result of as software program they’ll additionally profit from a DevOps method.
Ansible Roles
In Ansible, roles are a technique of mechanically loading sure variables, duties, recordsdata, templates, and handlers primarily based on a identified file construction. Grouping content material by roles permits for straightforward sharing and reuse. The Ansible documentation on roles outlines the file construction and different concerns.
When growing roles, you’ll need to take care of varied considerations, together with what working system(s) and model(s) you’ll be supporting and whether or not you solely want a single node or if you must goal a cluster of machines. Additionally it is typically essential to start out from a contemporary state each time you rerun your position whereas growing it to make sure that (1) your roles full efficiently on their first run, and (2) adjustments carried out on earlier runs aren’t affecting the end result. You also needs to confirm that your position is idempotent to make sure that irrespective of what number of occasions it’s executed, you obtain the identical consequence. Additionally it is essential to confirm that issues ought to solely be modified in the event that they must be modified. For instance, a service ought to solely be restarted if configuration adjustments warrant a restart.
The ultimate factor to contemplate is find out how to examine that your position has carried out what you propose it to do. Logging in to a goal node and manually checking is actually one approach to do it. Nonetheless, it’s higher to write down checks that may be mechanically run after your Ansible position to confirm the precise state.
To automate some of these checks, you’ll want a goal host (or set of hosts) and also you’ll have to destroy and recreate that host (or set of hosts) always through the improvement course of. Additionally, you will be liable for managing connections to the host (or hosts) in one of many Ansible-supported strategies. As well as, it’s essential to hand off to your chosen testing device and deal with its connection to the node(s). Managing this improvement and testing infrastructure could be tedious and can eat up numerous the time that may very well be higher spent on position options.
Enter Molecule
Molecule is a undertaking for facilitating the event and testing of Ansible roles by dealing with the beforehand outlined set of considerations and streamlining your complete position improvement course of. Pairing Molecule with Docker because it’s provisioner lets you shortly and simply develop your roles in opposition to any variety of freshly deployed working techniques and variations concurrently. Molecule additionally has built-in idempotence checking and help for a wide range of verification testing strategies.
To get began, first set up Docker. Then assuming you will have pip3, set up Molecule and supporting dependencies:
pip3 set up yamllint ansible molecule[docker] docker pytest-testinfraWith Molecule and its dependencies put in, it’s time to start growing an Ansible position. Molecule can help right here by constructing out your position’s file construction and Molecule’s personal required config recordsdata for you:
$ molecule init position maheckathorn.instance -d docker
$ tree
.
├── README.md
├── defaults
│ └── important.yml
├── recordsdata
├── handlers
│ └── important.yml
├── meta
│ └── important.yml
├── molecule
│ └── default
│ ├── converge.yml
│ ├── molecule.yml
│ └── confirm.yml
├── duties
│ └── important.yml
├── templates
├── checks
│ ├── stock
│ └── take a look at.yml
└── vars
└── important.yml
10 directories, 11 recordsdataInside the Molecule folder, the next recordsdata have particular functions:
converge.ymlis the playbook file that comprises the decision on your position. Molecule will invoke this playbook withansible-playbookand run it in opposition to an occasion created by the driving force, which is Docker in our situation.molecule.ymlis the central configuration entrypoint for Molecule. With this file, you possibly can configure every device that Molecule will make use of when testing your position.confirm.ymlis the Ansible file used for testing as Ansible is the default verifier, which lets you write particular checks in opposition to the state of the container after your position has completed executing. Different verifier instruments can be found (Observe that TestInfra was the default verifier previous to Molecule model 3).
The molecule.yml file comprises completely different sections for configuring how molecule parts behave:
- The dependency supervisor—Molecule makes use of Galaxy by default to resolve your position dependencies.
- The driver supplier. Molecule makes use of Docker by default. Molecule makes use of the driving force to delegate the duty of making situations.
- The lint command—Molecule can name exterior instructions to make sure that finest practices are inspired. Observe: Ansible-lint isn’t included with molecule or molecule[lint].
- The platforms definitions—Molecule depends on this to know which situations to create and title and to determine which group every occasion belongs in. If you’ll want to take a look at your position in opposition to a number of fashionable distributions (CentOS, Fedora, Debian), you possibly can specify that on this part.
- The provisioner—Molecule solely supplies an Ansible provisioner. Ansible manages the lifecycle of the occasion primarily based on this configuration.
- The situation definition—Molecule depends on this configuration to manage the situation sequence order.
- The verifier framework—Molecule makes use of Ansible by default to supply a approach to write particular state checking checks (equivalent to deployment smoke checks) on the goal occasion.
There are lots of choices for configuring these sections to satisfy your wants. Nonetheless, sticking to a standard configuration file throughout initiatives helps to set normal expectations. The next relies on Jeff Geerling’s widespread molecule.yml file:
https://github.com/cmu-sei/ansible-role-silk/blob/grasp/molecule/default/molecule.yml
---
dependency:
title: galaxy
driver:
title: docker
platforms:
- title: occasion
picture: "geerlingguy/docker-${MOLECULE_DISTRO:-centos7}-ansible:newest"
command: ${MOLECULE_DOCKER_COMMAND:-""}
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
privileged: true
pre_build_image: true
env:
http_proxy: "${http_proxy}"
https_proxy: "${https_proxy}"
no_proxy: "${no_proxy} "
provisioner:
title: ansible
playbooks:
converge: ${MOLECULE_PLAYBOOK:-converge.yml}
verifier:
title: testinfra
choices:
v: 1The important thing factor to notice on this file is the utilization of Jeff Geerling’s Docker container(s) because the picture supply, the configuration of Testinfra because the verifier, and the MOLECULE_DISTRO atmosphere variable with centos7 because the default. The customized premade Docker photos already include Python, Ansible, and systemd. They assist to hurry up take a look at runs by not needing Molecule to do something to make use of the picture apart from fetch it. The MOLECULE_DISTRO atmosphere variable permits you to simply take a look at in opposition to different OS varieties and variations by:
$ MOLECULE_DISTRO=ubuntu1804 molecule take a look atDifferent prebuilt photos are listed right here.
Lastly, the Testinfra verifier configures Molecule to make use of Testinfra for verification testing, which was the default previous to molecule model 3. In the event you’ve been growing roles with verification checks for some time, it’s helpful to have the ability to configure this setting, which suggests the checks listing that molecule creates is unneeded. You may as well edit your converge.yml file to appear to be the next:
https://github.com/cmu-sei/ansible-role-silk/blob/grasp/molecule/default/converge.yml
---
- title: Converge
hosts: all
roles:
- position: "{ basename }"
atmosphere:
http_proxy: "{{ lookup('env', 'http_proxy') }}"
https_proxy: "{{ lookup('env', 'https_proxy') }}"
no_proxy: "{{ lookup('env', 'no_proxy') }}"This configuration helps to keep away from points on steady integration/steady deployment (CI/CD) techniques and likewise offers with Ansible undertaking namespace points.
With Molecule configured, we will run by means of the entire default Molecule situation, which is a take a look at suite on your new position:
$ molecule take a look at
INFO default situation take a look at matrix: dependency, lint, cleanup, destroy, syntax, create, put together, converge, idempotence, side_effect, confirm, cleanup, destroy
INFO Performing prerun...
INFO Set ANSIBLE_LIBRARY=/Customers/maheckathorn/.cache/ansible-compat/50d858/modules:/Customers/maheckathorn/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
INFO Set ANSIBLE_COLLECTIONS_PATH=/Customers/maheckathorn/.cache/ansible-compat/50d858/collections:/Customers/maheckathorn/.ansible/collections:/usr/share/ansible/collections
INFO Set ANSIBLE_ROLES_PATH=/Customers/maheckathorn/.cache/ansible-compat/50d858/roles:/Customers/maheckathorn/.ansible/roles:/usr/share/ansible/roles:/and many others/ansible/roles
INFO Utilizing /Customers/maheckathorn/.cache/ansible-compat/50d858/roles/maheckathorn.instance symlink to present repository with a view to allow Ansible to seek out the position utilizing its anticipated full title.
INFO Operating default > dependency
WARNING Skipping, lacking the necessities file.
WARNING Skipping, lacking the necessities file.
INFO Operating default > lint
INFO Lint is disabled.
INFO Operating default > cleanup
WARNING Skipping, cleanup playbook not configured.
INFO Operating default > destroy
INFO Sanity checks: 'docker'
PLAY [Destroy] *****************************************************************
TASK [Destroy molecule instance(s)] ********************************************
modified: [localhost] => (merchandise=occasion)
TASK [Wait for instance(s) deletion to complete] *******************************
FAILED - RETRYING: [localhost]: Wait as an example(s) deletion to finish (300 retries left).
okay: [localhost] => (merchandise=occasion)
TASK [Delete docker networks(s)] ***********************************************
PLAY RECAP *********************************************************************
localhost : okay=2 modified=1 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
INFO Operating default > syntax
playbook: /Customers/maheckathorn/take a look at/instance/molecule/default/converge.yml
INFO Operating default > create
PLAY [Create] ******************************************************************
TASK [Log into a Docker registry] **********************************************
skipping: [localhost] => (merchandise=None)
skipping: [localhost]
TASK [Check presence of custom Dockerfiles] ************************************
okay: [localhost] => (merchandise={'command': '', 'env': '', 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'title': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']})
TASK [Create Dockerfiles from image names] *************************************
skipping: [localhost] => (merchandise={'command': '', 'env': '', 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'title': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']})
TASK [Discover local Docker images] ********************************************
okay: [localhost] => (merchandise={'modified': False, 'skipped': True, 'skip_reason': 'Conditional consequence was False', 'merchandise': {'command': '', 'env': '', 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'title': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']}, 'ansible_loop_var': 'merchandise', 'i': 0, 'ansible_index_var': 'i'})
TASK [Build an Ansible compatible image (new)] *********************************
skipping: [localhost] => (merchandise=molecule_local/geerlingguy/docker-centos7-ansible:newest)
TASK [Create docker network(s)] ************************************************
TASK [Determine the CMD directives] ********************************************
okay: [localhost] => (merchandise={'command': '', 'env': '', 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'title': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']})
TASK [Create molecule instance(s)] *********************************************
modified: [localhost] => (merchandise=occasion)
TASK [Wait for instance(s) creation to complete] *******************************
FAILED - RETRYING: [localhost]: Wait as an example(s) creation to finish (300 retries left).
modified: [localhost] => (merchandise={'failed': 0, 'began': 1, 'completed': 0, 'ansible_job_id': '429858788464.21737', 'results_file': '/Customers/maheckathorn/.ansible_async/429858788464.21737', 'modified': True, 'merchandise': {'command': '', 'env': '', 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'title': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']}, 'ansible_loop_var': 'merchandise'})
PLAY RECAP *********************************************************************
localhost : okay=5 modified=2 unreachable=0 failed=0 skipped=4 rescued=0 ignored=0
INFO Operating default > put together
WARNING Skipping, put together playbook not configured.
INFO Operating default > converge
PLAY [Converge] ****************************************************************
TASK [Gathering Facts] *********************************************************
okay: [instance]
PLAY RECAP *********************************************************************
occasion : okay=1 modified=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
INFO Operating default > idempotence
PLAY [Converge] ****************************************************************
TASK [Gathering Facts] *********************************************************
okay: [instance]
PLAY RECAP *********************************************************************
occasion : okay=1 modified=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
INFO Idempotence accomplished efficiently.
INFO Operating default > side_effect
WARNING Skipping, aspect impact playbook not configured.
INFO Operating default > confirm
WARNING Skipping, no checks discovered.
INFO Operating default > cleanup
WARNING Skipping, cleanup playbook not configured.
INFO Operating default > destroy
PLAY [Destroy] *****************************************************************
TASK [Destroy molecule instance(s)] ********************************************
modified: [localhost] => (merchandise=occasion)
TASK [Wait for instance(s) deletion to complete] *******************************
FAILED - RETRYING: [localhost]: Wait as an example(s) deletion to finish (300 retries left).
modified: [localhost] => (merchandise=occasion)
TASK [Delete docker networks(s)] ***********************************************
PLAY RECAP *********************************************************************
localhost : okay=2 modified=2 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
INFO Pruning additional recordsdata from situation ephemeral listingSince we confirmed numerous output above let’s breakdown what fascinating issues are occurring right here. Instantly after operating the Molecule take a look at, we get some output telling us what steps within the take a look at course of are going to be run:
INFO default situation take a look at matrix: dependency, lint, cleanup, destroy, syntax, create, put together, converge, idempotence, side_effect, confirm, cleanup, destroyAs this output exhibits, by default Molecule runs these steps as a part of the take a look at matrix within the order proven. Any time we see a
INFO Operating default >line within the output, we’re taking a look at a distinct step within the matrix being run. In the event you dig by means of the output, you’ll see that most of the steps are literally skipped by default. For instance:
INFO Operating default > dependency
WARNING Skipping, lacking the necessities file.In our instance, step one within the matrix the place Molecule truly does one thing is the destroy step. At this step, Molecule interacts with the configured driver, in our case Docker, and makes an attempt to destroy any earlier take a look at environments to make sure a brand new clear testing atmosphere is used. Molecule interacts with the Docker daemon and destroys any operating container with our outlined title from our molecule.yml file:
platforms:
- title: occasionIf a operating container with that title doesn’t at present exist, as in our case, it merely strikes on. The subsequent step at which Molecule truly does one thing is the create step. At this level within the course of, molecule interacts with the driving force and makes an attempt to create a take a look at atmosphere utilizing the driving force we advised it to and configured within the method outlined within the platform part of molecule.yml:
driver:
title: docker
platforms:
- title: occasion
picture: "geerlingguy/docker-${MOLECULE_DISTRO:-centos7}-ansible:newest"
command: ${MOLECULE_DOCKER_COMMAND:-""}
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
privileged: true
pre_build_image: true
env:
http_proxy: "${http_proxy}"
https_proxy: "${https_proxy}"
no_proxy: "${no_proxy} "Since we’re utilizing Docker, Molecule handles pulling the container picture we outlined, setting any docker runtime choices, and runs the container within the background. The next traces exhibits the profitable creation of our desired take a look at atmosphere:
TASK [Wait for instance(s) creation to complete] *******************************
FAILED - RETRYING: [localhost]: Wait as an example(s) creation to finish (300 retries left).
modified: [localhost] => (merchandise={'failed': 0, 'began': 1, 'completed': 0, 'ansible_job_id': '429858788464.21737', 'results_file': '/Customers/maheckathorn/.ansible_async/429858788464.21737', 'modified': True, 'merchandise': {'command': '', 'env': '' '}, 'picture': 'geerlingguy/docker-centos7-ansible:newest', 'title': 'occasion', 'pre_build_image': True, 'privileged': True, 'volumes': ['/sys/fs/cgroup:/sys/fs/cgroup:ro']}, 'ansible_loop_var': 'merchandise'})With our take a look at atmosphere now in place, Molecule strikes on to operating the converge step of the method, which conveniently runs the playbook named converge.yml that we outlined earlier. This playbook runs our position. As of proper now, our position does nothing, as this output exhibits:
INFO Operating default > converge
PLAY [Converge] ****************************************************************
TASK [Gathering Facts] *********************************************************
okay: [instance]
PLAY RECAP *********************************************************************
occasion : okay=1 modified=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0What this does present, nonetheless, is that Molecule was in a position to efficiently connect with our take a look at atmosphere. Instantly after a profitable converge step, Molecule mechanically checks our position for idempotence. This consists of rerunning our converge.yml playbook and ensuring nothing was modified.
If we had outlined Testinfra checks, the confirm step would have run them:
INFO Operating default > confirm
WARNING Skipping, no checks discovered.An instance of some easy Testinfra code could be seen within the Ansible SiLK github repository:
import os
import testinfra.utils.ansible_runner
testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')
def test_silk_version(host):
model = "3.19.2"
command = """/usr/native/bin/silk_config --silk-version"""
cmd = host.run(command)
assert model in cmd.stdoutOn this case, Molecule would run this Python code on the Docker host and Testinfra would deal with connecting to the take a look at atmosphere, operating the checks, and offering output again to molecule. The take a look at we’re operating merely checks if the model of SiLK put in within the testing atmosphere is 3.19.2. If our checks go, Molecule considers the confirm step a hit and strikes on. The very last thing Molecule does throughout a take a look at run is to scrub up after itself by rerunning the destroy step.
Automating Confidence
As highlighted by our instance above, Molecule is able to streamlining your Ansible position improvement course of. It stands up and tears down configurable take a look at environments shortly and simply and handles idempotence and verification testing. This weblog posting solely scratches the floor of what Molecule is able to, nonetheless.
For instance, dealing with a clustered take a look at atmosphere is a breeze by merely including one other named occasion to the platform part of the molecule.yml file. Testing in opposition to completely different working techniques and completely different OS variations can be a easy command tweak away. Including preparations to the take a look at nodes that, for one purpose or one other, have to happen outdoors the position is easy by means of inclusion of the put together step (by including a put together.yml playbook). Additionally it is simple so as to add position dependencies by making a necessities.yml file within the Molecule listing.
Lastly, the entire course of is easy to maneuver to a CI/CD system. The ci.yml within the Ansible SiLK irepository exhibits how to do that with Github actions, however the course of is moveable sufficient to simply be recreated utilizing your CI/CD platform of selection. If you’re not utilizing Molecule for growing your Ansible roles, you might be severely slowing down your improvement cadence and decreasing the standard of your Ansible code. Writing Ansible roles with the assistance of Molecule makes it extremely seemingly which you could be assured your position does what you need it to, main to raised high quality code and lowered consumer frustration.
