Launching Multiple Docker Containers using Ansible - docker

I am trying to build an image and trying to launch multiple docker containers using ansible playbook. I am not able to understand how do i publish the ports. The below playbook gives me an error which is quite obvious that the port is already allocated but then how do i achieve this because from outside the containers there will only be one port right to acces all the containers?
Playbook -
- name: Manage Docker instances via Ansible
hosts: shashank-VM
connection: local
become: yes
become_method: sudo
tasks:
- name: Building an image from Dockerfile
docker_image:
build:
path: .
pull: yes
name: web_new
source: build
- name: Creation of Docker Containers
docker_container:
name: my-app-{{ item }}
image: web_new
state: present
ports:
- "79:80"
with_sequence: count=3
- name: Starting Docker Containers
docker_container:
name: my-app-{{ item }}
image: web_new
state: started
with_sequence: count=3
Error -
changed: [shashank-VM]
TASK [Creation of Docker Containers] *********************************************************************************************************
changed: [shashank-VM] => (item=1)
changed: [shashank-VM] => (item=2)
changed: [shashank-VM] => (item=3)
TASK [Starting Docker Containers] ************************************************************************************************************
changed: [shashank-VM] => (item=1)
failed: [shashank-VM] (item=2) => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"}, "ansible_loop_var": "item", "changed": false, "item": "2", "msg": "Error starting container beb7f1d204f47862d16722f70b812df7193ddacf12d15350a9095cec2ebf4d85: 500 Server Error: Internal Server Error (\"driver failed programming external connectivity on endpoint my-app-2 (880c06fe9e2efa75537e350734be1d46d0cc76e7acf70733d19ad38706dde5ab): Bind for 0.0.0.0:78 failed: port is already allocated\")"}
failed: [shashank-VM] (item=3) => {"ansible_loop_var": "item", "changed": false, "item": "3", "msg": "Error starting container 048f2f3ea6fed5e094fdf59a4650b2b3f8164d804ee7dc8875e6e95bda1300d7: 500 Server Error: Internal Server Error (\"driver failed programming external connectivity on endpoint my-app-3 (8247f75384b240cb9bf1ee66cc9f0404df5465e6c08903304f14bd813c218fa1): Bind for 0.0.0.0:78 failed: port is already allocated\")"}
NOTE : I have an application for which I am building an image and there will be multiple containers running for that image. How do i accessible my application from outside? How do i work on the ports?
Any help is appreciated

The cause of the issue is here:
> Bind for 0.0.0.0:78 failed: port is already allocated
Check what application/container blocks port 78
You can do it with ss:
sudo ss -plunt | grep :78
Or with lsof:
lsof -i :78
Or with fuser:
fuser -v -n tcp 78

Related

How to access kind control plane port from another docker container?

I'm creating a kind cluster with kind create cluster --name kind and I want to access it from another docker container but when I try to apply a Kubernetes file from a container (kubectl apply -f deployment.yml) I got this error:
The connection to the server 127.0.0.1:6445 was refused - did you specify the right host or port?
Indeed when I try to curl kind control-plane from a container, it's unreachable.
> docker run --entrypoint curl curlimages/curl:latest 127.0.0.1:6445
curl: (7) Failed to connect to 127.0.0.1 port 6445 after 0 ms: Connection refused
However kind control-plane is publishing to the right port but only to the localhost.
> docker ps --format "table {{.Image}}\t{{.Ports}}"
IMAGE PORTS
kindest/node:v1.23.4 127.0.0.1:6445->6443/tcp
Currently the only solution I found is to set the host network mode.
> docker run --network host --entrypoint curl curlimages/curl:latest 127.0.0.1:6445
Client sent an HTTP request to an HTTPS server.
This solution don't look to be the most secure. Is there another way like connecting the kind network to my container or something like that that I missed ?
Don't have enough rep to comment on the other answer, but wanted to comment on what ultimately worked for me.
Takeaways
Kind cluster running in it own bridge network kind
Service with kubernetes client running in another container with a mounted kube config volume
As described above the containers need to be in the same network unless you want your service to run in the host network.
The server address for the kubeconfig is the container name + internal port e.g. kind-control-plane:6443. The port is NOT the exposed port in the example below 6443 NOT 38669
CONTAINER ID IMAGE PORTS
7f2ee0c1bd9a kindest/node:v1.25.3 127.0.0.1:38669->6443/tcp
Kube config for the container
# path/to/some/kube/config
apiVersion: v1
clusters:
- cluster:
insecure-skip-tls-verify: true # Don't use in Prod equivalent of --insecure on cli
server: https://<kind-control-plane container name>:6443 # NOTE port is internal container port
name: kind-kind # or whatever
contexts:
- context:
cluster: kind-kind
user: <some-service-account>
name: kind-kind # or whatever
current-context: kind-kind
kind: Config
preferences: {}
users:
- name: <some-service-account>
user:
token: <TOKEN>
Docker container stuff
If using docker-compose you can add the kind network to the container such as:
#docker-compose.yml
services:
foobar:
build:
context: ./.config
networks:
- kind # add this container to the kind network
volumes:
- path/to/some/kube/config:/somewhere/in/the/container
networks:
kind: # define the kind network
external: true # specifies that the network already exists in docker
If running a new container:
docker run --network kind -v path/to/some/kube/config:/somewhere/in/the/container <image>
Container already running?
docker network connect kind <container name>
I don't know exactly why you want to do this. but no problem I think this could help you:
first, lets pull your docker image:
❯ docker pull curlimages/curl
In my kind cluster I got 3 control plane nodes and 3 worker nodes. Here are the pod of my kind cluster:
❯ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
39dbbb8ca320 kindest/node:v1.23.5 "/usr/local/bin/entr…" 7 days ago Up 7 days 127.0.0.1:35327->6443/tcp so-cluster-1-control-plane
62b5538275e9 kindest/haproxy:v20220207-ca68f7d4 "haproxy -sf 7 -W -d…" 7 days ago Up 7 days 127.0.0.1:35625->6443/tcp so-cluster-1-external-load-balancer
9f189a1b6c52 kindest/node:v1.23.5 "/usr/local/bin/entr…" 7 days ago Up 7 days 127.0.0.1:40845->6443/tcp so-cluster-1-control-plane3
4c53f745a6ce kindest/node:v1.23.5 "/usr/local/bin/entr…" 7 days ago Up 7 days 127.0.0.1:36153->6443/tcp so-cluster-1-control-plane2
97e5613d2080 kindest/node:v1.23.5 "/usr/local/bin/entr…" 7 days ago Up 7 days 0.0.0.0:30081->30080/tcp so-cluster-1-worker2
0ca64a907707 kindest/node:v1.23.5 "/usr/local/bin/entr…" 7 days ago Up 7 days 0.0.0.0:30080->30080/tcp so-cluster-1-worker
9c5d26caee86 kindest/node:v1.23.5 "/usr/local/bin/entr…" 7 days ago Up 7 days 0.0.0.0:30082->30080/tcp so-cluster-1-worker3
The container that is interesting for us here is the haproxy one (kindest/haproxy:v20220207-ca68f7d4) which have the role of loadbalancing the enterring traffic to the nodes (and, in our example, especially the control plane nodes.) we can see that the port 35625 of our host machine is mapped to the port 6443 of the haproxy container. (127.0.0.1:35625->6443/tcp)
so, our cluster endpoint is https://127.0.0.1:35625, we can confirm this in our kubeconfig file (~/.kube/config):
❯ cat .kube/config
apiVersion: v1
kind: Config
preferences: {}
users:
- name: kind-so-cluster-1
user:
client-certificate-data: <base64data>
client-key-data: <base64data>
clusters:
- cluster:
certificate-authority-data: <certificate-authority-dataBase64data>
server: https://127.0.0.1:35625
name: kind-so-cluster-1
contexts:
- context:
cluster: kind-so-cluster-1
user: kind-so-cluster-1
namespace: so-tests
name: kind-so-cluster-1
current-context: kind-so-cluster-1
let's run the curl container in background:
❯ docker run -d --network host curlimages/curl sleep 3600
ba183fe2bb8d715ed1e503a9fe8096dba377f7482635eb12ce1322776b7e2366
as expected, we cant HTTP request the endpoint that listen on an HTTPS port:
❯ docker exec -it ba curl 127.0.0.1:35625
Client sent an HTTP request to an HTTPS server.
we can try to use the certificate that is in the field "certificate-authority-data" in our kubeconfig to check if that change something (it should):
Lets create a file named my-ca.crt that contain the stringData of the certificate:
base64 -d <<< <certificate-authority-dataBase64dataFromKubeConfig> > my-ca.crt
since the working directory of the curl docker image is "/" lets copy our cert to this location in the container and verify that it is actually there:
docker cp my-ca.crt ba183fe:/
❯ docker exec -it ba sh
/ $ ls my-ca.crt
my-ca.crt
Let's try again our curl request but with the certificate:
❯ docker exec -it ba curl --cacert my-ca.crt https://127.0.0.1:35625
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "forbidden: User \"system:anonymous\" cannot get path \"/\"",
"reason": "Forbidden",
"details": {},
"code": 403
}
YOU, can get the same result by adding the "--insecure" flag to your curl request:
❯ docker exec -it ba curl https://127.0.0.1:35625 --insecure
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "forbidden: User \"system:anonymous\" cannot get path \"/\"",
"reason": "Forbidden",
"details": {},
"code": 403
}
However, we can't access our cluster with anonymous user ! So lets get a token from kubernetes (cf https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-api/):
# Create a secret to hold a token for the default service account
kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: default-token
annotations:
kubernetes.io/service-account.name: default
type: kubernetes.io/service-account-token
EOF
Once the token controller has populated the secret with a token:
# Get the token value
❯ kubectl get secret default-token -o jsonpath='{.data.token}' | base64 --decode
eyJhbGciOiJSUzI1NiIsImtpZCI6InFSTThZZ05lWHFXMWExQlVSb1hTcHNxQ3F6Z2Z2aWpUaUYwd2F2TGdVZ0EifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJzby10ZXN0cyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIzYzY0OTg1OS0xNzkyLTQzYTQtOGJjOC0zMDEzZDgxNjRmY2IiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6c28tdGVzdHM6ZGVmYXVsdCJ9.VLfjuym0fohYTT_uoLPwM0A6u7dUt2ciWZF2K9LM_YvQ0UZT4VgkM8UBVOQpWjTmf9s2B5ZxaOkPu4cz_B4xyDLiiCgqiHCbUbjxE9mphtXGKQwAeKLvBlhbjYnHb9fCTRW19mL7VhqRgfz5qC_Tae7ysD3uf91FvqjjxsCyzqSKlsq0T7zXnzQ_YQYoUplGa79-LS_xDwG-2YFXe0RfS9hkpCILpGDqhLXci_gwP9DW0a6FM-L1R732OdGnb9eCPI6ReuTXQz7naQ4RQxZSIiNd_S7Vt0AYEg-HGvSkWDl0_DYIyHShMeFHu1CtfTZS5xExoY4-_LJD8mi
Now lets execute the curl command directly with the token !
❯ docker exec -it ba curl -X GET https://127.0.0.1:35625/api --header "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6InFSTThZZ05lWHFXMWExQlVSb1hTcHNxQ3F6Z2Z2aWpUaUYwd2F2TGdVZ0EifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJzby10ZXN0cyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIzYzY0OTg1OS0xNzkyLTQzYTQtOGJjOC0zMDEzZDgxNjRmY2IiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6c28tdGVzdHM6ZGVmYXVsdCJ9.VLfjuym0fohYTT_uoLPwM0A6u7dUt2ciWZF2K9LM_YvQ0UZT4VgkM8UBVOQpWjTmf9s2B5ZxaOkPu4cz_B4xyDLiiCgqiHCbUbjxE9mphtXGKQwAeKLvBlhbjYnHb9fCTRW19mL7VhqRgfz5qC_Tae7ysD3uf91FvqjjxsCyzqSKlsq0T7zXnzQ_YQYoUplGa79-LS_xDwG-2YFXe0RfS9hkpCILpGDqhLXci_gwP9DW0a6FM-L1R732OdGnb9eCPI6ReuTXQz7naQ4RQxZSIiNd_S7Vt0AYEg-HGvSkWDl0_DYIyHShMeFHu1CtfTZS5xExoY4-_LJD8mi" --insecure
{
"kind": "APIVersions",
"versions": [
"v1"
],
"serverAddressByClientCIDRs": [
{
"clientCIDR": "0.0.0.0/0",
"serverAddress": "172.18.0.5:6443"
}
]
}
It works !
I still don't know why you want to do this but I hope that this helped you.
Since It's not what you wanted because here I use host network, You can use this : How to communicate between Docker containers via "hostname" as proposed #SergioSantiago thanks for your comment !
bguess

Molecule : Testing roles : Failed to get Dbus Connection Operation not permitted

I facing an issue on my Molecule Test. I have begin to study this tool 2 days ago for information.
on a Ubuntu VM running with Vagrant,I have create a role and initialze Molecule's folder and create a testinfra test file ( with the docker provider ).
The error is when my task's role are running, at the step of checking service running, it failed.
fatal: [instance]: FAILED! => {"changed": false, "msg": "Could not find the requested service httpd: "}
I was design to simply install 2 packages including httpd on a Centos Image.
When im loggin directly to the Molecule VM ( so through docker ), when i simply type systemctl the error message is
Failed to get D-Bus connection: Operation not permitted
As adviced Geerlingguy, i have specify volume mapped on cgroup folder
platforms:
- name: instance
#image: docker.io/pycontribs/centos:7
image: geerlingguy/docker-${MOLECULE_DISTRO:-centos7}-ansible:latest
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
The error is not related to Testinfra but only the docker built image.
Could someone help me to understand why this error message ?
Is that because im on a VirtualBox ran by Vagrant ?
Thanks all for reading :-)
I have added that on my mocule.yml file config according molecule documentation ( https://molecule.readthedocs.io/en/latest/examples.html#docker ) :
platforms:
- name: instance
#image: docker.io/pycontribs/centos:7
image: geerlingguy/docker-centos7-ansible:latest
capabilities:
- SYS_ADMIN
command: /sbin/init
systemctl working fine now

how to make ansible get access to an sshd container?

I use an ansible script to load & start the https://hub.docker.com/r/rastasheep/ubuntu-sshd/ container.
so it starts well of course :
bash-4.4$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8bedbd3b7d88 rastasheep/ubuntu-sshd "/usr/sbin/sshd -D" 37 minutes ago Up 36 minutes 0.0.0.0:49154->22/tcp test
bash-4.4$
so after ansible failure on ssh access to it I tested manually from shell
this is also ok.
bash-4.4$ ssh root#172.17.0.2
The authenticity of host '172.17.0.2 (172.17.0.2)' can't be established.
ECDSA key fingerprint is SHA256:YtTfuoRRR5qStSVA5UuznGamA/dvf+djbIT6Y48IYD0.
ECDSA key fingerprint is MD5:43:3f:41:e9:89:45:06:6f:f6:42:c4:6a:70:37:f8:1d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.17.0.2' (ECDSA) to the list of known hosts.
root#172.17.0.2's password:
root#8bedbd3b7d88:~# logout
Connection to 172.17.0.2 closed.
bash-4.4$
so the step that failed is trying to get on it from ansible script & make access to ssh-copy-id
ansible error message is :
Fatal: [172.17.0.2]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '172.17.0.2' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey,password).\r\n", "unreachable": true}
---
- hosts: 127.0.0.1
tasks:
- name: start docker service
service:
name: docker
state: started
- name: load and start the container we wanna use
docker_container:
name: test
image: rastasheep/ubuntu-sshd
state: started
ports:
- "49154:22"
- name: Wait maximum of 300 seconds for ports to be available
wait_for:
host: 0.0.0.0
port: 49154
state: started
- hosts: 172.17.0.2
vars:
passwordadmin: $6$pbE6yznA$AeFIdI.....K0
passwordroot: $6$TMrxQUxT$I8.JIzR.....TV1
ansible_ssh_extra_args: "-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
tasks:
- name: Build test container root user rsa ssh-key
shell: docker exec test ssh-keygen -b 2048 -t rsa -f /root/.ssh/id_rsa -q -N ""
so I cannot even run the needed step to build ssh
how to do then ??
1st step (ansible task) : load docker container
2cd step (ansible task on only 172.17.0.2) : connect to it & setup it
there will be 3rd step to run application on it after that.
the problem occurs only when starting the 2cd step
Ok after many trys on a second container
conclusion is my procedure was bad
what I have done to solve that :
build a diroctory tree separating ./ ./inventory ./includes
build 1 yaml file by host (local, docker, labo)
build 1 main yaml file on ./
build 1 new host file in ./inventory
connect forced by sshpass to docker on default password
changed it
add the host key on authorized key to a login dedicated usage
installed pyhton (needed to answer ansible host else it makes
randomly module errors or refused connections depending on current
action)
setup a ssh login user in sudoers
then I can un the docker.yaml actions
then only at last I can run the labo.yaml actions.
Thanks for help
now I'm able to build the missing tools.

unable to load ansible playbook to a docker host (host unreachable)

im trying to : ansible-playbook install_docker.yml
and keep getting the following error:
TASK [setup] *******************************************************************
fatal: [172.17.0.2]: UNREACHABLE! => {"changed": false, "msg": "ERROR! SSH encountered an unknown error during the connection. We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue", "unreachable": true}
my playbook looks like this
---
- hosts: all
vars:
docker_opts: >
- "H unix:///var/run/docker.sock"
- "H tcp://0.0.0.0:2375"
remote_user: root
roles:
- angstwad.docker.ubuntu
im providing the docker host ip by copying the ip using:
docker inspect apacheweb1 | grep IPAddress
how can i reach the docker host?

using ansible with docker-compose

I am trying to deploy a docker setup using Ansible playbook. For this, I am using docker_service.
My Playbook looks like:
---
- name: Run Docker compose
hosts: all
gather_facts: no
tasks:
- debug: msg="Container - {{ inventory_hostname }}"
- docker_service:
project_src: "compose"
state: absent
- docker_service:
project_src: "compose"
state: present
Upon running this simple playbook as:
ansible-playbook -v playbook.yml --ask-sudo-pass
I added --ask-sudo-pass to ensure that it was not a permission issue.
OUTPUT
SUDO password:
PLAY [Run Docker compose] ******************************************************
TASK [debug] *******************************************************************
ok: [prolims-staging] => {
"msg": "Container - prolims-staging"
}
TASK [docker_service] **********************************************************
fatal: [prolims-staging]: FAILED! => {"changed": false, "msg": "Error connecting: Error while fetching server API version: ('Connection aborted.', error(13, 'Permission denied'))"}
to retry, use: --limit #/data/prolims-provision/provision-docker.retry
PLAY RECAP *********************************************************************
prolims-staging : ok=1 changed=0 unreachable=0 failed=1
I did try looking out for this issue on other forums as well ( and similar questions on this StackOverflow too), but those were not helpful.
Note: I am able to run docker-compose successfully in the target machine from its CLI (using sudo).
Also, I tried playing around with docker_container as well. I tried to execute a playbook with contents below:
...
- name: check container status
command: docker ps
register: result
- name: Create a container
docker_container:
name: db_pg
image: "postgres:latest"
state: present
recreate: yes
...
and running this playbook works perfectly fine.
I assume, posting my docker-compose file might not be relevant here.
I followed this example, but did not work. Maybe, I might be missing some stupid or really important thing here.
Any help on understanding and resolving this issue would be appreciated.
I am able to run docker-compose successfully in the target machine from its CLI (using sudo).
So you need to use become declaration for the task.
I added --ask-sudo-pass to ensure that it was not a permission issue.
Just adding --ask-sudo-pass to the ansible-playbook parameters doesn't have any effect unless the relevant tasks/plays have become declaration (and become_method is set to sudo, but this is by default).
Reference.

Resources