How to test ansible roles? - docker

Question

Scenario
I want to develop ansible roles. Those roles should be validated through a CI/CD process with molecule and utilize docker as driver. This validation step should include multiple Linux flavours (e.g. centos/ubuntu/debian) times the supported ansible versions.
The tests should then be executed such that the role is verified with
centos:7 + ansible:2.5
centos:7 + ansible:2.6
centos:7 + ansible:2.7
...
ubuntu:1604 + ansible:2.5
ubuntu:1604 + ansible:2.6
ubuntu:1604 + ansible:2.7
...
The issues at hand
there is no official ansible image availabe
how to best test a role for ansible version compatibility?
Issue 1: no official ansible images
The official images by the ansible team have been deprecated for about 3 years now:
https://hub.docker.com/r/ansible/ubuntu14.04-ansible
https://hub.docker.com/r/ansible/centos7-ansible
In addition, the link the deprecated images refer to in order to find new images supporting ansible is quite useless due to the sheer amount of results
https://hub.docker.com/search/?q=ansible&page=1&isAutomated=0&isOfficial=0&pullCount=1&starCount=0
Is there a well-maintained ansible docker image by the community (or ansible) that fills the void?
Preferable with multiple versions that can be pulled and a CI process that builds and validates the created image regularly.
Why am I looking for ansible images? I do not want to reinvent the wheel (if possible). I would like to use the images to test ansible roles via molecule for version incompatibility.
I searched but could not find anything truly useful. What images are you using to run ansible in a container/orchestrator? Do you build and maintain the images yourself?
e.g. https://hub.docker.com/r/ansibleplaybookbundle/apb-base/tags
looked promising, however, the images in there are also over 7 months old (at least).
Issue 2: how to best test a role for ansible version compatibility?
Is creating docker images for each combination of OS and ansible version the best way to test via molecule and docker as driver? Or is there a smarter way to test backward compatibility of ansible roles with multiple OS times different ansible versions?
I already test my ansible roles with molecule and docker as driver. Those tests currently only testing the functionality of the role on various Linux distros, but not the ansible backward compatibility by running the role with older ansible versions.
Here an example role with travis tests for centos7/ubuntu1604/ubuntu1804 based on geerlingguy's ntp role: https://github.com/Gepardec/ansible-role-ntp

Answer 1

Solution
In order to test ansible roles with multiple versions of ansible, python and various Linux flavors we can use
molecule for our ansible role functionality
docker as our abstraction layer on which we run the target system for our ansible role
tox to setup generic virtualenvs and test our various combinations without side effects
travis to automate it all
This will be quite a long/detailed answer. You can check out an example ansible role with the whole setup here
https://github.com/ckaserer/ansible-role-example
https://travis-ci.com/ckaserer/ansible-role-example
Step 1: test ansible role with molecule
Molecule docs: https://molecule.readthedocs.io/en/stable/
Fixes Issue: 1) no official ansible images
I could create ansible images for every distro I would like to test as jeff geerling describes in his blog posts.
The clear downside of this approach: The images will need maintenance (eventually)
However, with molecule, we can combine base images with a Dockerfile.j2 (Jinja2 Template) to create the images with minimal requirements to run ansible. Through this approach, we are now able to use the official linux distro images from docker hub and do not need to maintain a docker repo for each linux distro and the various versions.
Here the important bits in molecule.yml
platforms:
- name: instance-${TOX_ENVNAME}
image: ${MOLECULE_DISTRO:-'centos:7'}
command: /sbin/init
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
privileged: true
The default dockerfile.j2 from molecule is already good, but I have a few additions
# Molecule managed
{% if item.registry is defined %}
FROM {{ item.registry.url }}/{{ item.image }}
{% else %}
FROM {{ item.image }}
{% endif %}
{% if item.env is defined %}
{% for var, value in item.env.items() %}
{% if value %}
ENV {{ var }} {{ value }}
{% endif %}
{% endfor %}
{% endif %}
RUN if [ $(command -v apt-get) ]; then apt-get update && apt-get install -y python sudo bash ca-certificates iproute2 && apt-get clean; \
elif [ $(command -v zypper) ]; then zypper refresh && zypper install -y python sudo bash python-xml iproute2 && zypper clean -a; \
elif [ $(command -v apk) ]; then apk update && apk add --no-cache python sudo bash ca-certificates; \
elif [ $(command -v xbps-install) ]; then xbps-install -Syu && xbps-install -y python sudo bash ca-certificates iproute2 && xbps-remove -O; \
elif [ $(command -v swupd) ]; then swupd bundle-add python3-basic sudo iproute2; \
elif [ $(command -v dnf) ] && cat /etc/os-release | grep -q '^NAME=Fedora'; then dnf makecache && dnf --assumeyes install python sudo python-devel python*-dnf bash iproute && dnf clean all; \
elif [ $(command -v dnf) ] && cat /etc/os-release | grep -q '^NAME="CentOS Linux"' ; then dnf makecache && dnf --assumeyes install python36 sudo platform-python-devel python*-dnf bash iproute && dnf clean all && ln -s /usr/bin/python3 /usr/bin/python; \
elif [ $(command -v yum) ]; then yum makecache fast && yum install -y python sudo yum-plugin-ovl bash iproute && sed -i 's/plugins=0/plugins=1/g' /etc/yum.conf && yum clean all; \
fi
# Centos:8 + ansible 2.7 failed with error: "The module failed to execute correctly, you probably need to set the interpreter"
# Solution: ln -s /usr/bin/python3 /usr/bin/python
By default, this will test the role with centos:7. However, we can set the environment variable MOLECULE_DISTRO to whichever image we would like to test and run it via
MOLECULE_DISTRO=ubuntu:18.04 molecule test
Summary
We use official distro images from docker hub to test our ansible role via molecule.
The files used in this step
molecule.yml https://github.com/ckaserer/ansible-role-example/blob/master/molecule/default/molecule.yml
Dockerfile.j2 https://github.com/ckaserer/ansible-role-example/blob/master/molecule/default/Dockerfile.j2
Source
https://molecule.readthedocs.io/en/stable/examples.html#systemd-container
https://www.jeffgeerling.com/blog/2019/how-i-test-ansible-configuration-on-7-different-oses-docker
https://www.jeffgeerling.com/blog/2018/testing-your-ansible-roles-molecule
Step 2: Check compatibility for your role ( python version X ansible version X linux distros )
Fixes Issue 2: how to best test a role for ansible version compatibility?
Let's use tox to create virtual environments to avoid side effects while testing various compatibility scenarios.
Here the important bits in tox.ini
[tox]
minversion = 3.7
envlist = py{3}-ansible{latest,29,28}-{ alpinelatest,alpine310,alpine39,alpine38, centoslatest,centos8,centos7, debianlatest,debian10,debian9,debian8, fedoralatest,fedora30,fedora29,fedora28, ubuntulatest,ubuntu2004,ubuntu1904,ubuntu1804,ubuntu1604 }
# only test currently supported ansible versions
# https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html#release-status
skipsdist = true
[base]
passenv = *
deps =
-rrequirements.txt
ansible25: ansible==2.5
...
ansiblelatest: ansible
commands =
molecule test
setenv =
TOX_ENVNAME={envname}
MOLECULE_EPHEMERAL_DIRECTORY=/tmp/{envname}
[testenv]
passenv =
{[base]passenv}
deps =
{[base]deps}
commands =
{[base]commands}
setenv =
...
centoslatest: MOLECULE_DISTRO="centos:latest"
centos8: MOLECULE_DISTRO="centos:8"
centos7: MOLECULE_DISTRO="centos:7"
...
{[base]setenv}
The entirety of requirements.txt
docker
molecule
by simply executing
tox
it will create virtual envs for each compatibility combination defined in tox.ini by
envlist = py{3}-ansible{latest,29,28}-{ alpinelatest,alpine310,alpine39,alpine38, centoslatest,centos8,centos7, debianlatest,debian10,debian9,debian8, fedoralatest,fedora30,fedora29,fedora28, ubuntulatest,ubuntu2004,ubuntu1904,ubuntu1804,ubuntu1604 }
which translates in this particular case to: python3 x ansible version x linux distro
Great! We have created tests for compatibility checks with the added benefit of always testing with ansible latest to notice breaking changes early on.
The files used in this step
tox.ini https://github.com/ckaserer/ansible-role-example/blob/master/tox.ini
requirements.txt https://github.com/ckaserer/ansible-role-example/blob/master/requirements.txt
Source
https://tox.readthedocs.io/en/latest/
https://molecule.readthedocs.io/en/stable/testing.html#tox
Step 3: CI with travis
Running the checks locally is good, running in a CI tool is great. So let's do that.
For this purpose following bits in the .travis.yml are important
---
version: ~> 1.0
os: linux
language: python
python:
- "3.8"
- "3.7"
- "3.6"
- "3.5"
services: docker
cache:
pip: true
directories:
- .tox
install:
- pip install tox-travis
env:
jobs:
# ansible:latest - check for breaking changes
...
- TOX_DISTRO="centoslatest" TOX_ANSIBLE="latest"
- TOX_DISTRO="centos8" TOX_ANSIBLE="latest"
- TOX_DISTRO="centos7" TOX_ANSIBLE="latest"
...
# ansible: check version compatibility
# only test currently supported ansible versions
#
https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html#release-status
- TOX_DISTRO="centoslatest" TOX_ANSIBLE="{29,28}"
- TOX_DISTRO="centos8" TOX_ANSIBLE="{29,28}"
- TOX_DISTRO="centos7" TOX_ANSIBLE="{29,28}"
...
script:
- tox -e $(echo py${TRAVIS_PYTHON_VERSION} | tr -d .)-ansible${TOX_ANSIBLE}-${TOX_DISTRO}
# remove logs/pycache before caching .tox folder
- |
rm -r .tox/py*/log/*
find . -type f -name '*.py[co]' -delete -o -type d -name __pycache__ -delete
First we have specified language: python to run builds with multiple versions of python as defined in the python: list.
We need docker, so we add it via services: docker.
The test will take quite some time, let's cache pip and our virtenv created by tox with
cache:
pip: true
directories:
- .tox
We need tox...
install:
- pip install tox-travis
And finally, we define all our test cases
env:
jobs:
# ansible:latest - check for breaking changes
...
- TOX_DISTRO="centoslatest" TOX_ANSIBLE="latest"
...
Note that I have separate jobs for latest and the distinct versions. That is on purpose. I would like to easily see what broke. Is it version compatibility or an upcoming change located in ansible's latest release.
The files used in this step
https://github.com/ckaserer/ansible-role-example/blob/master/.travis.yml
Source
https://docs.travis-ci.com/user/caching/
Bonus: run tox in parallel
You can run the tests in parallel (e.g. 3 test simultaneous) by executing
tox -p 3
However, this will not give the output from molecule. You can enable that with
tox -p 3 -o true
The obvious downside to this approach is the pain of figuring out which line belongs to which process in the parallel execution.
Source
https://tox.readthedocs.io/en/latest/example/basic.html#parallel-mode

Answer 2

No real answer here, but some ideas :
Ansible Silo might have fitted, but no commit for a year.
And it's not exactly what you're looking for, but Ansible Runner is meant to be a fit for the "run ansible" use case. And it's a part of Ansible Tower / AWS, so it should last.
Runner is intended to be most useful as part of automation and tooling that needs to invoke Ansible and consume its results.
They do mention executing from a container here
The design of Ansible Runner makes it especially suitable for controlling the execution of Ansible from within a container for single-purpose automation workflows
But the issue for you is that the official ansible/ansible-runner container is tagged after ansible-runner version, and ansible itself is installed through pip install ansible at container build time.