I am trying to build and run a docker container using ansible, but I am unable to pass the environment variable to the Dockerfile in the build state.
Below is my ansible file dev.yml
---
- name: setup docker
tasks:
- name: build dockerfile
community.docker.docker_container:
name: test
tag: v0
path: .
nocache: yes
env:
TEST_ENV: "SOME_TESTS_VARIABLE"
SSH_KEY: "{{LOCAL_SSH_KEY}}"
I am running ansible-playbook,
ansible-playbook -i hosts dev.yml -e "LOCAL_SSH_KEY='$(cat ~/.ssh/id_rsa)'"
I have figured out looking at the doc & the errors that community.docker.docker_container does not support nocache, path, tag and to build the container I should rather use docker_image which then does not supports env
Is there a way to build docker containers using the environment variables.
In other words how can I pass my ssh keys to the docker build step?
I have looked at other answers but those don't work for me, maybe because of ansible version 2.10 which I am using.
Related
I try to build a collabora Docker Image with ansible. Here is the code:
- name: Build collabora Docker image
docker_image:
source: build
name: '{{ collabora_docker.image_name }}'
tag: '{{ collabora_docker.version }}'
build:
path: '/tmp/collabora.git/docker/from-packages'
nocache: yes
dockerfile: Ubuntu
args:
type: cool
secret_key: '{{ collabora_docker.key }}'
environment:
DOCKER_BUILDKIT: 1
It fails with the following error msg:
Error building repositoryxy/collabora - code: None, message: the --mount option requires BuildKit. Refer to https://docs.docker.com/go/buildkit/ to learn how to build images with BuildKit enabled
As you can see with the code above I already try to pass the environment variable via ansible but to no avail. See also Use BuildKit for docker build within Ansible
I have also tried add the global setting to the docker daemon as described in https://docs.docker.com/build/buildkit/
Since I have BuildKit globally enabled in docker daemon.json, I can now manually build the image on the host with the same ansible user in question without setting the environment variable. Following command builds the docker image in question:
docker build --no-cache --secret id=secret_key,src=secret_key --build-arg type=cool -t repositoryxy/collabora:22.05.8-4 -f Ubuntu .
But when executed within the ansible playbook I'm still getting the above mention error message regarding buildkit not enabled.
We have used the technique detailed here to expose host environment variables to Docker build in a secured fashion.
# syntax=docker/dockerfile:1.2
FROM golang:1.18 AS builder
# move secrets out of the build process (and docker history)
RUN --mount=type=secret,id=github_token,dst=/app/secret_github_token,required=true,uid=10001 \
export GITHUB_TOKEN=$(cat /app/secret_github_token) && \
<nice command that uses $GITHUB_TOKEN>
And this command to build the image:
export DOCKER_BUILDKIT=1
docker build --secret id=github_token,env=GITHUB_TOKEN -t cool-image-bro .
The above works perfectly.
Now we also have a docker-compose file running in CI that needs to be modified. However, even if I confirmed that the ENV vars are present in that job, I do not know how to assign the environment variable to the github_token named secret ID.
In other words, what is the equivalent docker-compose command (up --build, or build) that can accept a mapping of an environment variable with a secret ID?
Turns out I was a bit ahead of the times. docker compose v.2.5.0 brings support for secrets.
After having modified the Dockerfile as explained above, we must then update the docker-compose to defined secrets.
docker-compose.yml
services:
my-cool-app:
build:
context: .
secrets:
- github_user
- github_token
...
secrets:
github_user:
file: secrets_github_user
github_token:
file: secrets_github_token
But where are those files secrets_github_user and secrets_github_token coming from? In your CI you also need to export the environment variable and save it to the default secrets file location. In our project we are using Tasks so we added these too lines.
Note that we are running this task from our CI, so you could do it differently without Tasks for example.
- printenv GITHUB_USER > /root/project/secrets_github_user
- printenv GITHUB_TOKEN > /root/project/secrets_github_token
We then update the CircleCI config and add two environment variable to our job:
.config.yml
name-of-our-job:
environment:
DOCKER_BUILDKIT: 1
COMPOSE_DOCKER_CLI_BUILD: 1
You might also need a more recent Docker version, I think they introduced it in a late 19 release or early 20. I have used this and it works:
steps:
- setup_remote_docker:
version: 20.10.11
Now when running your docker-compose based commands, the secrets should be successfully mounted through docker-compose and available to correctly build or run your Dockerfile instructions!
We want to use Paketo.io / CloudNativeBuildpacks (CNB) GitLab CI in the most simple way. Our GitLab setup uses an AWS EKS cluster with unprivileged GitLab CI Runners leveraging the Kubernetes executor. We also don't want to introduce security risks by using Docker in our builds. So we don't have our host’s /var/run/docker.sock exposed nor want to use docker:dind.
We found some guides on how to use Paketo with GitLab CI like this https://tanzu.vmware.com/developer/guides/gitlab-ci-cd-cnb/ . But as described beneath the headline Use Cloud Native Buildpacks with GitLab in GitLab Build Job WITHOUT Using the GitLab Build Template, the approach relies on Docker and pack CLI. We tried to resemble this in our .gitlab-ci.yml which looks like this:
image: docker:20.10.9
stages:
- build
before_script:
- |
echo "install pack CLI (see https://buildpacks.io/docs/tools/pack/)"
apk add --no-cache curl
(curl -sSL "https://github.com/buildpacks/pack/releases/download/v0.21.1/pack-v0.21.1-linux.tgz" | tar -C /usr/local/bin/ --no-same-owner -xzv pack)
build-image:
stage: build
script:
- pack --version
- >
pack build $REGISTRY_GROUP_PROJECT/$CI_PROJECT_NAME:latest
--builder paketobuildpacks/builder:base
--path .
But as outlined our setup does not support docker and we end up with the following error inside our logs:
...
$ echo "install pack CLI (see https://buildpacks.io/docs/tools/pack/)" # collapsed multi-line command
install pack CLI (see https://buildpacks.io/docs/tools/pack/)
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/4) Installing brotli-libs (1.0.9-r5)
(2/4) Installing nghttp2-libs (1.43.0-r0)
(3/4) Installing libcurl (7.79.1-r0)
(4/4) Installing curl (7.79.1-r0)
Executing busybox-1.33.1-r3.trigger
OK: 12 MiB in 26 packages
pack
$ pack --version
0.21.1+git-e09e397.build-2823
$ pack build $REGISTRY_GROUP_PROJECT/$CI_PROJECT_NAME:latest --builder paketobuildpacks/builder:base --path .
ERROR: failed to build: failed to fetch builder image 'index.docker.io/paketobuildpacks/builder:base': Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: command terminated with exit code 1
Any idea on how to use Paketo Buildpacks with GitLab CI without having Docker present inside our GitLab Kubernetes runners (which seems to be kind of a best practice)? We also don't want our setup to become to complex - e.g. by adding kpack.
TLDR;
Use the Buildpack's lifecycle directly inside your .gitlab-ci.yml here's a fully working example):
image: paketobuildpacks/builder
stages:
- build
# We somehow need to access GitLab Container Registry with the Paketo lifecycle
# So we simply create ~/.docker/config.json as stated in https://stackoverflow.com/a/41710291/4964553
before_script:
- mkdir ~/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_JOB_TOKEN\"}}}" >> ~/.docker/config.json
build-image:
stage: build
script:
- /cnb/lifecycle/creator -app=. $CI_REGISTRY_IMAGE:latest
The details: "using the lifecycle directly"
There are ongoing discussions about this topic. Especially have a look into https://github.com/buildpacks/pack/issues/564 and https://github.com/buildpacks/pack/issues/413#issuecomment-565165832. As stated there:
If you're looking to build images in CI (not locally), I'd encourage
you to use the lifecycle directly for that, so that you don't need
Docker. Here's an example:
The link to the example is broken, but it refers to the Tekton implementation on how to use buildpacks in a Kubernetes environment. Here we can get a first glue about what Stephen Levine referred to as "to use the lifecycle directly". Inside it the crucial point is the usage of command: ["/cnb/lifecycle/creator"]. So this is the lifecycle everyone is talking about! And there's good documentaion about this command that could be found in this CNB RFC.
Choosing a good image: paketobuildpacks/builder:base
So how to develop a working .gitlab-ci.yml? Let's start simple. Digging into the Tekton implementation you'll see that the lifecycle command is executed inside an environment defined in BUILDER_IMAGE, which itself is documented as The image on which builds will run (must include lifecycle and compatible buildpacks). That sound's familiar! Can't we simply pick the builder image paketobuildpacks/builder:base from our pack CLI command? Let's try this locally on our workstation before commiting to much noise into our GitLab. Choose a project you want to build (I created a example Spring Boot app if you'd like at gitlab.com/jonashackt/microservice-api-spring-boot you can clone) and run:
docker run --rm -it -v "$PWD":/usr/src/app -w /usr/src/app paketobuildpacks/builder bash
Now inside the paketobuildpacks/builder image powered container try to run the Paketo lifecycle directly with:
/cnb/lifecycle/creator -app=. microservice-api-spring-boot:latest
I only used the -app parameter of the many possible parameters for the creator command, since most of them have quite good defaults. But as the default app directory path is not the default /workspace - but the current directory, I configured it. Also we need to define an <image-name> at the end, which will simply be used as the resulting container image name.
The first .gitlab-ci.yml
Both commands did work at my local workstation, so let's finally create a .gitlab-ci.yml using this approach (here's a fully working example .gitlab-ci.yml):
image: paketobuildpacks/builder
stages:
- build
build-image:
stage: build
script:
- /cnb/lifecycle/creator -app=. $CI_REGISTRY_IMAGE:latest
docker login without docker
As we don't have docker available inside our Kubernetes Runners, we can't login into GitLab Container Registry as described in the docs. So the following error occured to me using this first approach:
===> ANALYZING
ERROR: failed to get previous image: connect to repo store "gitlab.yourcompanyhere.cloud:4567/yourgroup/microservice-api-spring-boot:latest": GET https://gitlab.yourcompanyhere.cloud/jwt/auth?scope=repository%3Ayourgroup%2Fmicroservice-api-spring-boot%3Apull&service=container_registry: DENIED: access forbidden
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: command terminated with exit code 1
Using the approach described in this so answer fixed the problem. We need to create a ~/.docker/config.json containing the GitLab Container Registry login information - and then the Paketo build will pick them up, as stated in the docs:
If CNB_REGISTRY_AUTH is unset and a docker config.json file is
present, the lifecycle SHOULD use the contents of this file to
authenticate with any matching registry.
Inside our .gitlab-ci.yml this could look like:
# We somehow need to access GitLab Container Registry with the Paketo lifecycle
# So we simply create ~/.docker/config.json as stated in https://stackoverflow.com/a/41710291/4964553
before_script:
- mkdir ~/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_JOB_TOKEN\"}}}" >> ~/.docker/config.json
Our final .gitlab-ci.yml
As we're using the image: paketobuildpacks/builder at the top of our .gitlab-ci.yml, we can now leverage the lifecycle directly. Which is what we wanted to do in the first place. Only remember to use the correct GitLab CI variables to describe your <image-name> like this:
/cnb/lifecycle/creator -app=. $CI_REGISTRY_IMAGE:latest
Otherwise the Buildpack process analyser step will break and it finally won't get pushed to the GitLab Container Registry. So finally our .gitlab-ci.yml looks like this (here's the fully working example):
image: paketobuildpacks/builder
stages:
- build
# We somehow need to access GitLab Container Registry with the Paketo lifecycle
# So we simply create ~/.docker/config.json as stated in https://stackoverflow.com/a/41710291/4964553
before_script:
- mkdir ~/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_JOB_TOKEN\"}}}" >> ~/.docker/config.json
build-image:
stage: build
script:
- /cnb/lifecycle/creator -app=. $CI_REGISTRY_IMAGE:latest
Our builds should now run successfully using Paketo/Buildpacks without pack CLI and Docker:
See the full log of the example project here.
If I use a environment variable the circle.yml bellow, fails, But if I statically type the machine name it will work.
How can I properly reference environment variables in CircleCI?
version: 2
executorType: machine
stages:
build:
workDir: ~/app
enviroment:
- IMAGE_NAME: "nginx-ks8-circleci-hello-world"
# - AWS_REGISTER: "096957576271.dkr.ecr.us-east-1.amazonaws.com"
steps:
- type: checkout
- type: shell
name: Build the Docker image
shell: /bin/bash
command: |
docker build --rm=false -t $IMAGE_NAME .
I check your syntax with this example of circleci docs https://circleci.com/docs/2.0/language-python/#config-walkthrough so you have to remove the hiphen
enviroment:
IMAGE_NAME: "nginx-ks8-circleci-hello-world"
Thats for the environment variable inside the docker image for CircleCi 2.0.
Circle runs each command in a subshell so there isn't a way to set environment variables for the CircleCi build from the build itself.
Instead use the actual CircleCi environment variables:
https://circleci.com/gh/{yourOrganization}/{yourRepo}/edit#env-vars
I'm new to ansible (and docker). I would like to test my ansible playbook before using it on any staging/production servers.
Since I don't have access to an empty remote server, I thought the easiest way to test would be to use Docker container and then just run my playbook with the Docker container as the host.
I have a basic DockerFile that creates a standard ubuntu container. How would I configure the ansible hosts in order to run it against the docker container? Also, I suspect I would need to "run" the docker container to allow ansible to connect to it.
Running the playbook in a docker container may not actually be the best approach unless your stage and production servers are also Docker containers. The Docker ubuntu image is stripped down and will have some differences from a full installation. A better option might be to run the playbook in an Ubuntu VM that matches your staging and production installations.
That said, in order to run the ansible playbook within the container you should write a Dockerfile that runs your playbook. Here's a sample Dockerfile:
# Start with the ubuntu image
FROM ubuntu
# Update apt cache
RUN apt-get -y update
# Install ansible dependencies
RUN apt-get install -y python-yaml python-jinja2 git
# Clone ansible repo (could also add the ansible PPA and do an apt-get install instead)
RUN git clone http://github.com/ansible/ansible.git /tmp/ansible
# Set variables for ansible
WORKDIR /tmp/ansible
ENV PATH /tmp/ansible/bin:/sbin:/usr/sbin:/usr/bin
ENV ANSIBLE_LIBRARY /tmp/ansible/library
ENV PYTHONPATH /tmp/ansible/lib:$PYTHON_PATH
# add playbooks to the image. This might be a git repo instead
ADD playbooks/ /etc/ansible/
ADD inventory /etc/ansible/hosts
WORKDIR /etc/ansible
# Run ansible using the site.yml playbook
RUN ansible-playbook /etc/ansible/site.yml -c local
The ansible inventory file would look like
[local]
localhost
Then you can just docker build . (where . is the root of the directory where your playbooks and Dockerfile live), then docker run on the resulting image.
Michael DeHaan, the CTO of Ansible, has an informative blog post on this topic.
There's a working example regarding this: https://github.com/William-Yeh/docker-ansible
First, choose the base image you'd like to begin with from the following list:
williamyeh/ansible:debian8-onbuild
williamyeh/ansible:debian7-onbuild
williamyeh/ansible:ubuntu14.04-onbuild
williamyeh/ansible:ubuntu12.04-onbuild
williamyeh/ansible:centos7-onbuild
williamyeh/ansible:centos6-onbuild
Second, put the following Dockerfile along with your playbook directory:
FROM williamyeh/ansible:ubuntu14.04-onbuild
# ==> Specify playbook filename; default = "playbook.yml"
#ENV PLAYBOOK playbook.yml
# ==> Specify inventory filename; default = "/etc/ansible/hosts"
#ENV INVENTORY inventory.ini
# ==> Executing Ansible...
RUN ansible-playbook-wrapper
Third, docker build .
For more advanced usage, the role in Ansible Galaxy williamyeh/nginx also demonstrates how to do a simple integration test for a variety of Linux distributions on Travis CI’s Ubuntu 12.04 worker instances.
Disclosure: I am the author of the docker-ansible and wiliamyeh/nginx projects.
I've created a role for this vary scenario: https://github.com/chrismeyersfsu/provision_docker. Easily start Docker containers and use them in your role or playbook, as inventory, to test.
Includes:
Curated Dockerfile for Ubuntu 12.04 & 14.04 as well as CentOS 6 & 7 that put back in the distro-removed init systems
start ssh
Also note the examples all have a .travis.yml file to form a CI pipeline using Travis CI.
Examples:
Simple: https://github.com/chrismeyersfsu/provision_docker/tree/master/test
Simple: https://github.com/chrismeyersfsu/role-iptables/tree/master/test
Advanced: https://github.com/chrismeyersfsu/role-install_mongod/tree/master/test
Apart from provisioning localhost (the machine where you have Ansible installed), you can also tell Ansible to:
create a new docker container,
provision that container,
destroy that container.
For this to work you need such a hosts.yaml file:
all:
hosts:
mycontainer:
ansible_connection: docker
localhost:
ansible_connection: local
such a playbook.yaml file:
---
- name: Create a container to be provisioned later
hosts: localhost
tasks:
- name: create docker container
docker_container:
name: mycontainer
image: python:2.7.16-slim-stretch
command: ["sleep", "1d"]
- name: Provision the container created above
hosts: mycontainer
roles:
- simple
and another playbook file: destroy.yaml used to destroy the container:
---
- name: Destroy a container
hosts: localhost
tasks:
- name: destroy docker container
docker_container:
name: mycontainer
state: absent
Create also a simple role: roles/simple/taksks/main.yaml
---
- name: Create a file
copy:
content: "hi!!"
dest: /tmp/hello
force: yes
mode: 0555
And now to create a container and provision it, run:
ansible-playbook -i ./hosts.yaml ./playbook.yml
Verify that container was provisioned (the file was created):
docker exec mycontainer cat /tmp/hello
To destroy the container run:
ansible-playbook -i ./hosts.yaml ./destroy.yml
There are of course disadvantages:
the container must have python installed
some Ansible modules might not work, because additional python packages have to be installed. E.g. if you wanted to deploy docker containers (in the docker container), you have to install docker python SDK (pip3 install docker)
I was inspired by this blog post: https://medium.com/#andreilhicas/provision-docker-containers-with-ansible-30cc5ee6d950