How to run ansible script inside a docker container - docker

I am using docker for testing my playbooks.
I created a container now when i am running below command inside container its giving me below error
ansible-playbook jenkins.yml
Error:-
[root#db1e9105692d jenkins-playbook]# ansible-playbook jenkins.yml -k -vvv
SSH password:
PLAY [localhost] **************************************************************
GATHERING FACTS ***************************************************************
<localhost> ESTABLISH CONNECTION FOR USER: root
<localhost> REMOTE_MODULE setup
<localhost> EXEC sshpass -d4 ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/root/.ansible/cp/ansible-ssh-%h-%p-%r" -o StrictHostKeyChecking=no -o GSSAPIAuthentication=no -o PubkeyAuthentication=no -o ConnectTimeout=10 localhost /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1454580537.38-114451000565344 && echo $HOME/.ansible/tmp/ansible-tmp-1454580537.38-114451000565344'
EXEC previous known host file not found for localhost
fatal: [localhost] => SSH Error: ssh: connect to host localhost port 22: Connection refused
while connecting to 127.0.0.1:22
It is sometimes useful to re-run the command using -vvvv, which prints SSH debug output to help diagnose the issue.
TASK: [jenkins | Include OS-Specific variables] *******************************
<localhost> ESTABLISH CONNECTION FOR USER: root
fatal: [localhost] => One or more undefined variables: 'ansible_os_family' is undefined
FATAL: all hosts have already failed -- aborting
PLAY RECAP ********************************************************************
to retry, use: --limit #/root/jenkins.retry
localhost : ok=0 changed=0 unreachable=2 failed=0
But if i run this command on host machine its running fine.Do i need to do anything so that connection do not gets refused on port 22.inside docker container
Please do not consider below line as reason for error.Its just that ansible has executed few more lines before throwing error. Actually its not able to run so thats why value of this variable is empty.
fatal: [localhost] => One or more undefined variables: 'ansible_os_family' is undefined

In your container start your playbook locally:
$ ansible-playbook jenkins.yml -c local -k -vvv

Do you have connection=local defined for localhost? It's trying to connect via ssh, which can not work because you probably do not have sshd running in your container.

Related

AWS local build (CodeBuild) pass credentials file with wsl2

I upgraded my docker desktop to the version 3.2.1 (61626), and choose to use wsl2, after that i cannot run Local builds of AWS CodeBuild because the AWS configuration is not being found, the command I use is (I run the command from a tab from Windows terminal using ubuntu 20 that I installed from the store):
./codebuild_build.sh -i aws/codebuild/standard:5.0 -a ./ -s ./ -b ./buildspec.yml -c ~/.aws
That command works with the version of docker that uses Hyper-V, after the upgrade to wsl2 i get the error:
agent_1 | [Container] 2021/03/05 21:04:05 Phase complete: DOWNLOAD_SOURCE State: FAILED
agent_1 | [Container] 2021/03/05 21:04:05 Phase context status code: Decrypted Variables Error Message: MissingRegion: could not find region configuration
The docker command that is generated is the following:
docker run -it -v /var/run/docker.sock:/var/run/docker.sock -e "IMAGE_NAME=aws/codebuild/standard:5.0" -e "ARTIFACTS=/mnt/c/[redacted]" -e "SOURCE=/mnt/c/[redacted]" -e "BUILDSPEC=/mnt/c/[redacted]" -e "AWS_CONFIGURATION=NONE" -e "INITIATOR=[redacted]" amazon/aws-codebuild-local:latest
edit:
running the command from git bash the generated command is:
winpty docker run -it -v //var/run/docker.sock:/var/run/docker.sock -e "IMAGE_NAME=aws/codebuild/standard:5.0" -e "ARTIFACTS=//C/[redacted]" -e "SOURCE=//C/[redacted]" -e "BUILDSPEC=//C/[redacted]" -e "AWS_CONFIGURATION=//C/Users/[redacted]/.aws" -e "INITIATOR=[redacted]" amazon/aws-codebuild-local:latest
But also fails with the error:
agent_1 | [Container] 2021/03/05 22:17:43 Phase complete: DOWNLOAD_SOURCE State: FAILED
agent_1 | [Container] 2021/03/05 22:17:43 Phase context status code: YAML_FILE_ERROR Message: stat /codebuild/output/srcDownload/src/buildspec.pr.yml: no such file or directory
With the previous command the variable AWS_CONFIGURATION had the path to my .aws folder, I had tried -c //c/Users/[myProfile]/.aws and /mnt/c/Users/[myProfile]/.aws but AWS_CONFIGURATION is always NONE
Is there a configuration that I'm missing? or I need add an extra step with wsl2?
Edit:
I installed Ubuntu 18 and failed in the same way.
I was having a similar problem. I realized that since I had to run docker as root using the sudo command, my home directory was now /root instead of /home/<username>.
There may be a better way around this, but I symlinked the folder /home/<username>/.aws to /root/.aws.
Also, you could pass the variables AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, and AWS_ACCESS_KEY_ID in through an environment file using the -e flag of the codebuild_build.shcommand.

How to kill/stop remote Docker container after disconnecting SSH

I have a remote docker container that I access over SSH. I start the container normally with the docker start command.
sudo docker start 1db52045d674
sudo docker exec -it 1db52045d674; bash
This starts an interactive tty in the container, which I access over ssh.
I'd like the container to kill itself if I close the SSH connection. Is there anyway to do this?
.bash_logout is executed every time you use exit command to end a terminal session.
So you can use this file to run the docker stop command when you exit the ssh connection on the remote server.
Create ~/.bash_logout file if not existing.
Add following command to stop the docker container in this file.
Example :
docker stop container_name
Note: If a user closes the terminal window instead of writing the exit command, this file is not executed.
I was hoping for a more elegant solution but in the end I launched a bash script over ssh to trap for a SIGHUP
something like:
trap 'docker stop CONTAINER_NAME' SIGHUP;
while sleep 5;
do echo "foo";
done;
so when the operator closes the SSH connection, the trap gets trigger and docker nicely stops
You can use the --init parameter for initializing. This way, your container will be able to take over the init process and you can send a kill signal to it: https://docs.docker.com/engine/reference/run/#specify-an-init-process
Start the server:
docker run --init \
-p 2222:2222 \
-e USER_NAME=user \
-e USER_PASSWORD=pass \
-e PASSWORD_ACCESS=true \
-e SUDO_ACCESS=true \
linuxserver/openssh-server
Just note the --init and -e SUDO_ACCESS=true parameters here.
In another (client) shell,
ssh into container:
$ ssh user#127.0.0.1 -p 2222 -oStrictHostKeyChecking=accept-new
Warning: Permanently added '[127.0.0.1]:2222' (ECDSA) to the list of known hosts.
user#127.0.0.1's password:
Welcome to OpenSSH Server
2a. send kill signal to PID1 (docker-init):
$ sudo kill -s SIGINT 1
[sudo] password for user:
$ Connection to 127.0.0.1 closed by remote host.
Connection to 127.0.0.1 closed.
Container is gone.
I hope this helps.

Invoking ansible-playbook not working from Jenkins

Attached images are the Ansible configuration screenshots on Jenkins.Trying to invoke an ansible-playbook from Jenkins I get the below error:
[test-ansible-on-remote] $ sshpass ******** /usr/bin/ansible-
playbook /var/jenkins_home/workspace/test-ansible-on-remote/test.yml
-i 40.68.3.120 -f 5 -u bmiadmin -k
FATAL: command execution failed
java.io.IOException: Cannot run program "sshpass" (in directory "/var/jenkins_home/workspace/test-ansible-on-remote"): error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at hudson.Proc$LocalProc.<init>(Proc.java:250)
at hudson.Proc$LocalProc.<init>(Proc.java:219)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:937)
at hudson.Launcher$ProcStarter.start(Launcher.java:455)
at hudson.Launcher$ProcStarter.join(Launcher.java:466)
at org.jenkinsci.plugins.ansible.CLIRunner.execute(CLIRunner.java:49)
at org.jenkinsci.plugins.ansible.AbstractAnsibleInvocation.execute(AbstractAnsibleInvocation.java:290)
at org.jenkinsci.plugins.ansible.AnsiblePlaybookInvocation.execute(AnsiblePlaybookInvocation.java:31)
at org.jenkinsci.plugins.ansible.AnsiblePlaybookBuilder.perform(AnsiblePlaybookBuilder.java:261)
at org.jenkinsci.plugins.ansible.AnsiblePlaybookBuilder.perform(AnsiblePlaybookBuilder.java:232)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:79)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonito
Am I missing anything in the configuration?
sshpass needs to be installed as part of the host Jenkins docker image which Ansible uses for making the ssh connections to the hosts.

Docker Desktop, Windows 10 - waiting for SSH to be available / certificate signed by unknown authority

Sharing a solution to this issue in case it's helpful to somebody, as the exact issue / fix doesn't seem to be covered by other threads with similar titles.
The symptom was that on attempting to create a new vm with docker-machine create --driver hyperv testvm, the process hung at:
Running pre-create checks...
Creating machine...
(testvm) Copying F:\Virtual\Docker\cache\boot2docker.iso to
F:\Virtual\Docker\machines\testvm\boot2docker.iso...
(testvm) Creating SSH key...
(testvm) Creating VM...
(testvm) Using switch "Docker External Switch"
(testvm) Creating VHD
(testvm) Starting VM...
(testvm) Waiting for host to start...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
This was with Windows 10 Pro, Hyper-V, and a fresh install of Docker Desktop Community 2.0.0.3 (although I suspect that Hyper-V is irrelevant to this issue).
When I ctrl-c'd out of the create command I could docker-machine ls and see that the VM was up, but was showing an error:
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
testvm - hyperv Running tcp://192.168.5.61:2376 Unknown Unable to query docker version: Get https://192.168.5.60:2376/v1.15/version: x509: certificate signed by unknown authority
All attempts to docker-machine ssh to it failed similarly:
PS C:\> docker-machine ssh testvm
exit status 255
I tried using git bash as suggested in various threads elsewhere, but was seeing, eg:
$ docker-machine ssh testvm
Error: Cannot run SSH command: Host "testvm" is not running
(Likely some kind of configuration issue with my git bash install, but was unable to figure out what it was!)
The problem turned out to be some kind of compatibility issue with my installation of OpenSSH here:
PS C:\> get-command ssh
CommandType Name Version Source
----------- ---- ------- ------
Application ssh.exe 7.7.2.1 C:\Windows\System32\OpenSSH\ssh.exe
This was producing debug output (when docker-machine was run with the -debug switch) along these lines:
(testvm) Calling .GetSSHPort
(testvm) Calling .GetSSHKeyPath
(testvm) Calling .GetSSHKeyPath
(testvm) Calling .GetSSHUsername
Using SSH client type: external
&{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker#192.168.5.61 -o IdentitiesOnly=yes -i F:\Virtual\Docker\machines\testvm\id_rsa -p 22] C:\Windows\System32\OpenSSH\ssh.exe <nil>}
About to run SSH command:
exit 0
SSH cmd err, output: exit status 255:
Error getting ssh command 'exit 0' : ssh command error:
command : exit 0
err : exit status 255
output :
Everything started to work when I used the --native-ssh switch which is documented here. I was then able to:
docker-machine --native-ssh regenerate-certs testvm
..to resolve the certificate issue, and:
PS C:\> docker-machine --native-ssh ssh testvm ps
PID TTY TIME CMD
3301 pts/0 00:00:00 ps
..etc.
Probably better though to:
docker-machine rm -y testvm
docker-machine --native-ssh create --driver hyperv testvm
Everything was working for me without the switch at one point - my guess is that I didn't have OpenSSH installed at that time, and docker-machine was using its native version by default.

Docker-machine Google (GCE) driver not working, unable to create instance

Okay, this is very easy to reproduce and incredibly frustrating. Would be super grateful for any help or advice! I'm using Docker for Mac, running on OS X El Capitan (10.11.6). The gist is that Docker seems to not work with Google Compute Engine (GCE) via the Docker GCE driver (Docker official docs reference here).
1: Sign up for a new free GCP (Google Cloud) account at http://console.cloud.google.com/. Also download and install the Google Cloud SDK from here: https://cloud.google.com/sdk/.
2: Create a new Google Cloud project.
3: Go to "API Manager" in the Google Cloud console and click on "Credentials"
4: Click on "Create Credentials" and select "Service Account Key". Select "Compute Engine default service account", make sure JSON is selected as the output type, and click "Create". Move the outputted JSON file to your user root directory (/Users/MYUSERNAME).
5: Add the following line to your .bash_profile config:
export GOOGLE_APPLICATION_CREDENTIALS=/Users/MYUSERNAME/NAME_OF_CREDENTIALS_FILE.json. Save the file.
6: Exit the terminal and open up a new one so that the env variable is now set.
7: Run gcloud config set project PROJECT_ID (where PROJECT_ID is the name of the project just created in the Google Cloud Console).
8: Run gcloud auth login which will open a browser tab to log you into Google and grant permissions. Click 'Allow'.
9: Now the fun part, run the following command, per the Docker documentation (I've added a --debug flag):
docker-machine --debug create --driver google --google-project PROJECT_ID vm01
('vm01' is the name of the virtual machine here, this could be anything you want.)
At the end of the very lengthy output I get the following, concluded by the error message at the very bottom:
(LOTS OF OTHER OUTPUT BEFORE THIS, NOT ABLE TO COPY-PASTE EVERYTHING DUE TO STACKOVERFLOW 30000 CHAR LIMIT)
(vm01) Calling .GetURL
(vm01) Calling .DriverName
Setting Docker configuration on the remote daemon...
(vm01) Calling .GetSSHHostname
(vm01) Calling .GetSSHPort
(vm01) Calling .GetSSHKeyPath
(vm01) Calling .GetSSHKeyPath
(vm01) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/nathan/.docker/machine/machines/vm01/id_rsa (-rw-------)
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker-user#104.198.166.134 -o IdentitiesOnly=yes -i /Users/nathan/.docker/machine/machines/vm01/id_rsa -p 22] /usr/bin/ssh <nil>}
About to run SSH command:
printf %s "[Service]
ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver aufs --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=google
MountFlags=slave
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
Environment=
[Install]
WantedBy=multi-user.target
" | sudo tee /etc/systemd/system/docker.service
SSH cmd err, output: <nil>: [Service]
ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver aufs --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=google
MountFlags=slave
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
Environment=
[Install]
WantedBy=multi-user.target
(vm01) Calling .GetSSHHostname
(vm01) Calling .GetSSHPort
(vm01) Calling .GetSSHKeyPath
(vm01) Calling .GetSSHKeyPath
(vm01) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/nathan/.docker/machine/machines/vm01/id_rsa (-rw-------)
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker-user#104.198.166.134 -o IdentitiesOnly=yes -i /Users/nathan/.docker/machine/machines/vm01/id_rsa -p 22] /usr/bin/ssh <nil>}
About to run SSH command:
sudo systemctl daemon-reload
SSH cmd err, output: <nil>:
(vm01) Calling .GetSSHHostname
(vm01) Calling .GetSSHPort
(vm01) Calling .GetSSHKeyPath
(vm01) Calling .GetSSHKeyPath
(vm01) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/nathan/.docker/machine/machines/vm01/id_rsa (-rw-------)
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker-user#104.198.166.134 -o IdentitiesOnly=yes -i /Users/nathan/.docker/machine/machines/vm01/id_rsa -p 22] /usr/bin/ssh <nil>}
About to run SSH command:
sudo systemctl -f start docker
SSH cmd err, output: <nil>:
(vm01) Calling .GetSSHHostname
(vm01) Calling .GetSSHPort
(vm01) Calling .GetSSHKeyPath
(vm01) Calling .GetSSHKeyPath
(vm01) Calling .GetSSHUsername
Using SSH client type: external
Using SSH private key: /Users/nathan/.docker/machine/machines/vm01/id_rsa (-rw-------)
&{[-F /dev/null -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker-user#104.198.166.134 -o IdentitiesOnly=yes -i /Users/nathan/.docker/machine/machines/vm01/id_rsa -p 22] /usr/bin/ssh <nil>}
About to run SSH command:
netstat -tln
SSH cmd err, output: <nil>: Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 10.0.3.1:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp6 0 0 :::22 :::* LISTEN
Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded
notifying bugsnag: [Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded]
Solved this just now. I used an updated image from the Google registry (Ubuntu 16.04 LTS, versus the default Ubuntu 15 that gets used by the docker-machine --driver google command) and it seems to have worked properly. Not sure why. The full command was:
docker-machine --debug create --driver google --google-project PROJECT_ID --google-machine-image https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/ubuntu-1604-xenial-v20161205 vm02

Resources