I want to build a docker image from a Dockerfile. I can do this by using the bash like this:
[root#srv01 ~]# docker build -t appname/tomcat:someTag /root/Documents/myDockerfiles/tomcat
The good thing about having the image build using the bash is, that it prints to stdout what it executes step-by-step:
Step 1 : FROM tomcat:8.0.32-jre8
8.0.32-jre8: Pulling from library/tomcat
fdd5d7827f33: Already exists
...
When using Ansible in the following fashion from bash:
[root#localhost ansiblescripts]# ansible-playbook -vvvvv build-docker-image.yml:
Where the file build-docker-image.yml contains this content:
- name: "my build-docker-image.yml playbook"
hosts: myHost
tasks:
- name: "simple ping"
ping:
- name: "build the docker image"
become: yes
become_method: root
become_method: su
command: /bin/docker build -t something/tomcat:ver1 /home/docker/tomcat
#async: 1
#poll: 0
It waits for the whole build command to finish and then prints all the stdout as verbose output together in one piece.
Commenting in async:1 and poll:0 doesn't solve my problem, since it doesn't print the stdout at all.
Related
I am having issues when trying to connect to a docker-compose network from inside of a container. These are the files I am working with. The whole thing runs when I ./run.sh.
Dockerfile:
FROM docker/compose:latest
WORKDIR .
# EXPOSE 8228
RUN apk update
RUN apk add py-pip
RUN apk add jq
RUN pip install anchorecli
COPY dockertest.sh ./dockertest.sh
COPY docker-compose.yaml docker-compose.yaml
CMD ["./dockertest.sh"]
docker-compose.yaml
services:
# The primary API endpoint service
engine-api:
image: anchore/anchore-engine:v0.6.0
depends_on:
- anchore-db
- engine-catalog
#volumes:
#- ./config-engine.yaml:/config/config.yaml:z
ports:
- "8228:8228"
..................
## A NUMBER OF OTHER CONTAINERS THAT ANCHORE-ENGINE USES ##
..................
networks:
default:
external:
name: anchore-net
dockertest.sh
echo "------------- INSTALL ANCHORE CLI ---------------------"
engineid=`docker ps | grep engine-api | cut -f 1 -d ' '`
engine_ip=`docker inspect $engineid | jq -r '.[0].NetworkSettings.Networks."cws-anchore-net".IPAddress'`
export ANCHORE_CLI_URL=http://$engine_ip:8228/v1
export ANCHORE_CLI_USER='user'
export ANCHORE_CLI_PASS='pass'
echo "System status"
anchore-cli --debug system status #This line throws error (see below)
run.sh:
#!/bin/bash
docker build . -t anchore-runner
docker network create anchore-net
docker-compose up -d
docker run --network="anchore-net" -v //var/run/docker.sock:/var/run/docker.sock anchore-runner
#docker network rm anchore-net
Error Message:
System status
INFO:anchorecli.clients.apiexternal:As Account = None
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 172.19.0.6:8228
Error: could not access anchore service (user=user url=http://172.19.0.6:8228/v1): HTTPConnectionPool(host='172.19.0.6', port=8228): Max retries exceeded with url: /v1
(Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))
Steps:
run.sh builds container image and creates network anchore-net
the container has an entrypoint script, which does multiple things
firstly, it brings up the docker-compose network as detached FROM inside the container
secondly, nstalls anchore-cli so I can run commands against container network
lastly, attempts to get a system status of the anchore-engine (d.c network) but thats where I am running into HTTP request connection issues.
I am dynamically getting the IP of the api endpoint container of anchore-engine and setting the URL of the request to do that. I have also tried passing those variables from command line such as:
anchore-cli --u user --p pass --url http://$engine_ip/8228/v1 system status but that throws the same error.
For those of you who took the time to read through this, I highly appreciate any input you can give me as to where the issue may be lying. Thank you very much.
I have a python script I want to run in a kubernetes job. I have used a configMap to upload it to the container located for example in dir/script.py.
The container is run normally with the args["load"].
I have tried using a postStart lifecycle in the Job manifest but it appears not to run.
lifecycle:
preStop:
exec:
command:
- /bin/sh
- -c
- /usr/bin/python /opt/config-init/db/tls_generator.py
Below is the snippet of the manifest
containers:
- name: {{ template "gluu.name" . }}-load
image: gluufederation/config-init:4.0.0_dev
lifecycle:
preStop:
exec:
command:
- /bin/sh
- -c
- /usr/bin/python /opt/config-init/db/tls_generator.py
volumeMounts:
- mountPath: /opt/config-init/db/
name: {{ template "gluu.name" . }}-config
- mountPath: /opt/config-init/db/generate.json
name: {{ template "gluu.fullname" . }}-mount-gen-file
subPath: generate.json
- mountPath: /opt/config-init/db/tls_generator.py
name: {{ template "gluu.fullname" . }}-tls-script
envFrom:
- configMapRef:
name: {{ template "gluu.fullname" . }}-config-cm
args: [ "load" ]
How can I run the tls_generator.py scipt after the args["load"].
The dockerFile part looks like
ENTRYPOINT ["tini", "-g", "--", "/app/scripts/entrypoint.sh"]
CMD ["--help"]
You are using Container Lifecycle Hooks, to be more specific PreStop.
This hook is called immediately before a container is terminated due to an API request or management event such as liveness probe failure, preemption, resource contention and others.
If you want to execute a command when pod is starting you should consider using PostStart.
This hook executes immediately after a container is created. However, there is no guarantee that the hook will execute before the container ENTRYPOINT. No parameters are passed to the handler.
Another option would be using Init Containers, and here are few ideas with examples:
Wait for a Service to be created, using a shell one-line command like:
for i in {1..100}; do sleep 1; if dig myservice; then exit 0; fi; done; exit 1
Register this Pod with a remote server from the downward API with a command like:
curl -X POST http://$MANAGEMENT_SERVICE_HOST:$MANAGEMENT_SERVICE_PORT/register -d >'instance=$(<POD_NAME>)&ip=$(<POD_IP>)'
Wait for some time before starting the app container with a command like
sleep 60
Please read the documentation on how to use Init containers for more details.
My end goal was to run tls_generator.py right after the load command has completed. This is what I came with and is working fine.
command: ["/bin/sh", "-c"]
args: ["tini -g -- /app/scripts/entrypoint.sh load && /usr/bin/python
/scripts/tls_generator.py"]
In this case the default command when running "tini -g -- /app/scripts/entrypoint.sh" will be the --help command. But adding load passes it as a command.
So Jenkins is installed inside the cluster with this official helm chart. And this is my installed plugins as per helm release values:
installPlugins:
- kubernetes:1.18.1
- workflow-job:2.33
- workflow-aggregator:2.6
- credentials-binding:1.19
- git:3.11.0
- blueocean:1.19.0
my Jenkinsfile relies on the following pod template to spin up slaves:
kind: Pod
spec:
# dnsConfig:
# options:
# - name: ndots
# value: "1"
containers:
- name: dind
image: docker:19-dind
command:
- cat
tty: true
volumeMounts:
- name: dockersock
readOnly: true
mountPath: /var/run/docker.sock
resources:
limits:
cpu: 500m
memory: 512Mi
volumes:
- name: dockersock
hostPath:
path: /var/run/docker.sock
Slaves (pod /dind container) starts nicely as expected whenever there is new Build.
However, it broke at the step of "docker build" in ( Jenkinsfile pipeline
docker build -t ... ) and it breaks there :
Step 16/24 : RUN ../gradlew clean bootJar
---> Running in f14b6418b3dd
Downloading https://services.gradle.org/distributions/gradle-5.5-all.zip
Exception in thread "main" java.net.UnknownHostException: services.gradle.org
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:220)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
at java.base/java.net.Socket.connect(Socket.java:591)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:285)
at java.base/sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:182)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:265)
at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:372)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1515)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250)
at org.gradle.wrapper.Download.downloadInternal(Download.java:67)
at org.gradle.wrapper.Download.download(Download.java:52)
at org.gradle.wrapper.Install$1.call(Install.java:62)
at org.gradle.wrapper.Install$1.call(Install.java:48)
at org.gradle.wrapper.ExclusiveFileAccessManager.access(ExclusiveFileAccessManager.java:69)
at org.gradle.wrapper.Install.createDist(Install.java:48)
at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:107)
at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:63)
The command '/bin/sh -c ../gradlew clean bootJar' returned a non-zero code:
At the first galance, I thought it's DNS resolution issue with the Slave container (docker:19-dind) since it is alpine.
That's why I debug its /etc/resolv.conf by adding sh "cat /etc/resolv.conf" in the Jenkinsfile.
I got :
nameserver 172.20.0.10
search cicd.svc.cluster.local svc.cluster.local cluster.local ap-southeast-1.compute.internal
options ndots:5
I removed the last line options ndots:5 as per recommendation of many thread on the internet.
But it does not fix the issue. 😔
I thought again and again and I realized that the container responsible for this error is not the Slave (docker:19-dind), instead, it is the intermediate containers that are opened to satisfy docker build.
As consequence, I added RUN cat /etc/resolv.conf as another layer in the Dockerfile (which starts by FROM gradle:5.5-jdk11).
Now, the resolv.conf is different :
Step 15/24 : RUN cat /etc/resolv.conf
---> Running in 91377c9dd519
; generated by /usr/sbin/dhclient-script
search ap-southeast-1.compute.internal
options timeout:2 attempts:5
nameserver 10.0.0.2
Removing intermediate container 91377c9dd519
---> abf33839df9a
Step 16/24 : RUN ../gradlew clean bootJar
---> Running in f14b6418b3dd
Downloading https://services.gradle.org/distributions/gradle-5.5-all.zip
Exception in thread "main" java.net.UnknownHostException: services.gradle.org
Basically, it is a different nameserver 10.0.0.2 than the nameserver of the slave container 172.20.0.10. There is NO ndots:5 in resolv.conf this intermediate container.
I was confused after all these debugging steps and a lot of attempts.
Architecture
Jenkins Server (Container )
||
(spin up slaves)
||__ SlaveA (Container, image: docker:19-dind)
||
( run "docker build" )
||
||_ intermediate (container, image: gradle:5.5-jdk11 )
Just add --network=host to docker build or docker run.
docker build --network=host foo/bar:latest .
Found the answer here
I'm using ansible to execute some tasks on a local docker container, as such:
hosts: name-of-docker-container
connection: docker
tasks:
- name: setting up ssh_config
template:
src: ssh_config.j2
dest: /home/user/.ssh/ssh_config
mode: "0600"
owner: user
group: user
Something as simple as this takes a 2-5 seconds to run. Shouldn't this take less than a second? How can I make ansible run faster? I've tried the pipelining, but it doesn't seem to help:
ansible-playbook -v -e 'pipelining=True' -i inventories/staging/hosts.yml staging-deploy.yml
I use an ansible script to load & start the https://hub.docker.com/r/rastasheep/ubuntu-sshd/ container.
so it starts well of course :
bash-4.4$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8bedbd3b7d88 rastasheep/ubuntu-sshd "/usr/sbin/sshd -D" 37 minutes ago Up 36 minutes 0.0.0.0:49154->22/tcp test
bash-4.4$
so after ansible failure on ssh access to it I tested manually from shell
this is also ok.
bash-4.4$ ssh root#172.17.0.2
The authenticity of host '172.17.0.2 (172.17.0.2)' can't be established.
ECDSA key fingerprint is SHA256:YtTfuoRRR5qStSVA5UuznGamA/dvf+djbIT6Y48IYD0.
ECDSA key fingerprint is MD5:43:3f:41:e9:89:45:06:6f:f6:42:c4:6a:70:37:f8:1d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.17.0.2' (ECDSA) to the list of known hosts.
root#172.17.0.2's password:
root#8bedbd3b7d88:~# logout
Connection to 172.17.0.2 closed.
bash-4.4$
so the step that failed is trying to get on it from ansible script & make access to ssh-copy-id
ansible error message is :
Fatal: [172.17.0.2]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '172.17.0.2' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey,password).\r\n", "unreachable": true}
---
- hosts: 127.0.0.1
tasks:
- name: start docker service
service:
name: docker
state: started
- name: load and start the container we wanna use
docker_container:
name: test
image: rastasheep/ubuntu-sshd
state: started
ports:
- "49154:22"
- name: Wait maximum of 300 seconds for ports to be available
wait_for:
host: 0.0.0.0
port: 49154
state: started
- hosts: 172.17.0.2
vars:
passwordadmin: $6$pbE6yznA$AeFIdI.....K0
passwordroot: $6$TMrxQUxT$I8.JIzR.....TV1
ansible_ssh_extra_args: "-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
tasks:
- name: Build test container root user rsa ssh-key
shell: docker exec test ssh-keygen -b 2048 -t rsa -f /root/.ssh/id_rsa -q -N ""
so I cannot even run the needed step to build ssh
how to do then ??
1st step (ansible task) : load docker container
2cd step (ansible task on only 172.17.0.2) : connect to it & setup it
there will be 3rd step to run application on it after that.
the problem occurs only when starting the 2cd step
Ok after many trys on a second container
conclusion is my procedure was bad
what I have done to solve that :
build a diroctory tree separating ./ ./inventory ./includes
build 1 yaml file by host (local, docker, labo)
build 1 main yaml file on ./
build 1 new host file in ./inventory
connect forced by sshpass to docker on default password
changed it
add the host key on authorized key to a login dedicated usage
installed pyhton (needed to answer ansible host else it makes
randomly module errors or refused connections depending on current
action)
setup a ssh login user in sudoers
then I can un the docker.yaml actions
then only at last I can run the labo.yaml actions.
Thanks for help
now I'm able to build the missing tools.