Using Ubuntu 16.04 and docker 18.03.1-ce
In a Dockerfile I have:
RUN curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar
But when I build that with:
sudo docker build -t myimage .
I get:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: mynexus
The command '/bin/sh -c curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar' returned a non-zero code: 6
If I run the exact same command line from a terminal it works fine:
$ curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3613k 100 3613k 0 0 1413k 0 0:00:02 0:00:02 --:--:-- 1413k
Why can I curl that file from command line but not from the DockerFile?
You can see the curl error message when running the last commited image from the build with curl command:
docker run -it --rm lastCommitedImageHash curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar
I had the same problem last week - curl worked directly from host but not when ran during build from Dockerfile.
Restart of Docker daemon helped to me.
I haven't found what was the reason of that state. It was on the machine which is also as a Kubernetes master and I did Kubernetes cluster upgrade last week and probably this was the source of that problem.
Another cause of that problem in your case can be if you have mynexus hostname defined only in the /etc/hosts file on the host machine and then it is unknown for the running container during the build.
Related
I am trying to make a health check during my circleci workflow to see if the app compiles successfully. Despite the app building on circleci, curl is never able to find to the localhost port.
I have tried this locally on my machine and it works great.
Below is my workflow job:
health-check:
docker:
- image: image_name
steps:
- setup_remote_docker
- restore_cache:
key: image-cache-ci-{{ .Environment.CIRCLE_KEY }}
- run:
name: start
command: |
docker load < image.tar
docker run -d graphql-ci:$CIRCLE_KEY "./node_modules/nodemon/bin/nodemon.js index.js"
docker run graphql-ci:$CIRCLE_KEY curl http://localhost:4000/.well-known/apollo/server-health
The app successfully compiles. However, the second line always returns the following:
12354314532412342: Loading layer 1.803MB/1.803MB
Loaded image: graphql-ci:234523451325424
34523453153145235345234523452345234
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 4000: Connection refused
Exited with code exit status 7
I have also tried to use wait-on - wait-on http://localhost:4000/.well-known/apollo/server-health && curl http://localhost:4000/.well-known/apollo/server-health
I have also placed the curl called in a separate run step after the build one
- run:
name: check that the server is up
command: |
docker load < image.tar
docker run graphql-ci:$CIRCLE_KEY curl http://localhost:4000/.well-known/apollo/server-health
I have also tried to use docker exec but then it just tells me that it cannot find my docker container (and then specifies the docker container image hash).
If I remove the detach flag -d then it will show that it successfully started.
[nodemon] 2.0.2
[nodemon] to restart at any time, enter `rs`
[nodemon] watching dir(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node index.js`
The default PORT set is 4000, I add a flag to the docker run command to include --env PORT=4000 to just confirm it. Didn't affect anything but I know that it is running.
I also gave pm2 a try for the server since then the same terminal will be available by default. This will run but makes no difference.
Really just want to curl a sever I know is loaded in one circleci job.
Entrypoint
CMD [ "node", "index.js" ]
EDIT:
I got closer to getting a successful result.
scripts
"curl": "if [[ $(curl -o /dev/null -w '%{http_code}' http://localhost:4000/.well-known/apollo/server-health) == 200 ]]; then echo 1; else echo 0; fi",
"ci": "./node_modules/nodemon/bin/nodemon.js index.js",
"health": "start-server-and-test ci http-get://localhost:4000/.well-known/apollo/server-health 'npm run curl'",
curl returns 1 if successful (returns 200) and 0 otherwise.
I also installed a package called start-server-and-test which starts the server, waits for it to finish then run the final script. It is normally used for cypress but works here too.
This was the result
> #bespokemetrics/graphql# curl /usr/src/app
> if [[ $(curl -o /dev/null -w '%{http_code}' http://localhost:3000/.well-known/apollo/server-health) == 200 ]]; then echo 1; else echo 0; fi
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 17 100 17 0 0 8500 0 --:--:-- --:--:-- --:--:-- 8500
1 // this was the returned
npm ERR! code ELIFECYCLE
npm ERR! errno 143
npm ERR! package_name: `./node_modules/nodemon/bin/nodemon.js index.js`
npm ERR! Exit status 143
Why does it return an ERR despite succeeding?
I also created an npm script to container all of the code needed to trigger the test on 1 line.
- run:
name: Start and Curl Server
command: |
docker load < image.tar
docker run graphql-ci:$CIRCLE_SHA1 npm run health
"curl": "if [[ $(curl -o /dev/null -w '%{http_code}' http://localhost:4000/.well-known/apollo/server-health) == 200 ]]; then echo 'Success'; pkill node; else echo 'Failure'; fi"
"ci": "./node_modules/nodemon/bin/nodemon.js index.js"
"health": "start-server-and-test ci http-get://localhost:4000/.well-known/apollo/server-health 'npm run curl'"
I installed a package called start-server-and-test which allows you to start your server, wait for it to build, then run whatever command you want all on 1 line.
My curl command only returns the status code, if 200 then return a success message and kill the node process. That was the err that was returned after the success earlier because it wasn't closed properly.
I am currently setting up a Cloudbees Jenkins cluster with the help of the docker swarm plugin. I got it to work once, but haven't gotten it to work ever since. I get several different errors related to the creation of the agent that executes the pipeline, but they all seem to be related to the environment variables that the plugin creates at runtime. Does someone know a solution to this?
[1:40:45 PM] Creating Service with Name : agt-vice_testing_59-72
[1:40:45 PM] ServiceSpec created with ID : kk0jxbuqvh4zyngruamxphl4b
[1:40:45 PM] ServiceSpec request JSON : {"TaskTemplate":{"ContainerSpec":{"Image":"docker:17.12","Command":["sh","-cx","docker run --rm --privileged java:8 sh -xc 'curl --connect-timeout 20 --max-time 60 -o agent.jar http://199.18.14.456:8080/jnlpJars/agent.jar && java -jar agent.jar -jnlpUrl $DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL' "],"Env":["DOCKER_SWARM_PLUGIN_JENKINS_AGENT_SECRET=5ee89222c397f86c7ca8ecc6af73268539fa3ea9f64e46f815c4cfc06322aa0f","DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JAR_URL=http://199.18.14.456:8080/jnlpJars/agent.jar","DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL=http://199.18.14.456:8080/computer/agt-vice_testing_59-72/slave-agent.jnlp","DOCKER_SWARM_PLUGIN_JENKINS_AGENT_NAME=agt-vice_testing_59-72"],"Dir":"/tmp","User":"root","DNSConfig":{"Nameservers":[],"Search":[],"Options":[]},"Mounts":[{"Target":"/var/run/docker.sock","Source":"/var/run/docker.sock","Type":"bind","VolumeOptions":null}],"Hosts":[],"Secrets":[],"Configs":[]},"RestartPolicy":{"Condition":"none","MaxAttempts":0},"Resources":{"Limits":{"NanoCPUs":0,"MemoryBytes":0},"Reservations":{"NanoCPUs":0,"MemoryBytes":0}},"Placement":{"Constraints":[]}},"EndpointSpec":{"Ports":[]},"Name":"agt-vice_testing_59-72","Labels":{"ROLE":"jenkins-agent"},"Networks":[]}
�+ docker run --rm --privileged java:8 sh -xc curl --connect-timeout 20 --max-time 60 -o agent.jar http://199.18.14.456:8080/jnlpJars/agent.jar && java -jar agent.jar -jnlpUrl $DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL
d+ curl --connect-timeout 20 --max-time 60 -o agent.jar http://199.18.14.456:8080/jnlpJars/agent.jar
P % Total % Received % Xferd Average Speed Time Time Time Current
N Dload Upload Total Spent Left Speed
�
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 770k 100 770k 0 0 63.9M 0 --:--:-- --:--:-- --:--:-- 68.4M
+ java -jar agent.jar -jnlpUrl
#Option "-jnlpUrl" takes an operand
!java -jar agent.jar [options...]
I am trying to execute a ./login.sh as a part of docker run now when i run the ./login.sh in ubuntu machine i see success response but when i create a docker image i see output as sh: abc: unknown operand “invalid url”
Here is contents of ./login.sh
#!/bin/sh
# Black Box Tester!
content=$(curl --location --request POST "https://api.platform.abc.com.
/auth/oauth/token" --header 'Content-Type: application/x-www-form-
urlencoded' \
--header 'Authorization: Basic V0zSA==' --data-raw 'grant_type=password&
username=test#example.com&password=123456'| jq -r '.domain_id' )
if [ $content = abc ]
then
echo “Valid Login Token”
else
echo “invalid url”
fi
and here is contents of docker file
FROM ubuntu:16.04
RUN apt-get update && apt-get install -y curl \
&& apt-get -y install jq
FROM openjdk:8-jre-alpine
RUN apk --no-cache add curl jq
WORKDIR /opt
ADD login.sh /opt
RUN pwd \
&& find /opt
CMD ["./login.sh"]
When same ./login.sh i execute normally in Ubuntu machine i get successful output as
ubuntu#ip-172-31-29-248:~$ ./login.sh
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left Speed
100 2034 100 1958 100 76 9458 367 --:--:-- --:--:-- --:--:--
9873
“Valid Login Token”
Here is output of sudo docker run lots/loginimage
ubuntu#ip-172-31-29-248:~$ sudo docker run lots/loginimage
./login.sh: line 6: jq: not found
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left Speed
92 2034 92 1807 100 76 7199 302 --:--:-- --:--:-- --:--:--
7501
curl: (23) Failed writing body (0 != 1024)
sh: abc: unknown operand
“invalid url”
Can some one let me know why the login script which is simply a curl command runs sucessfully in ubuntu machine but same login script shows else condition in docker run?
Also i have included a code in docker file to download and install Json path jq but why image still says ./login.sh: line 6: jq: not found??
your help is highly appreciated
While creating docker you need to install jq sudo apt-get install jq
I am writing a Dockerfile based on wildfly image. I have isolated these lines where I am having some headache. The curl command doesn't work during build process. I have already uninstalled and installed Docker again but the error persists. My system is a Linux Mint.
In addition I tried to build that same Dockerfile in a RHEL and it worked like a charm.
Here's the Dockerfile:
FROM jboss/wildfly
RUN cd $HOME \
&& curl -O "http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.44/mysql-connector-java-5.1.44.jar"
Here's the error output:
Sending build context to Docker daemon 1.03MB
Step 1/6 : FROM jboss/wildfly
---> b695bdcce374
Step 2/6 : RUN cd $HOME && curl -O "http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.44/mysql-connector-java-5.1.44.jar"
---> Running in 4fdcef7dbda1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:39 --:--:-- 0
curl: (6) Could not resolve host: central.maven.org; Unknown error
The command '/bin/sh -c cd $HOME && curl -O "http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.44/mysql-connector-java-5.1.44.jar"' returned a non-zero code: 6
I could workaround the problem doing something like this:
docker build --add-host central.maven.org:151.101.56.209 .
but I'm not happy with that. I would like to say Docker to use my DNS instead of set fixed IP. It would be more elegant.
The error is a DNS problem resolving the hostname. You could try running docker with the --dns option (https://docs.docker.com/engine/reference/run/#network-settings)
docker run --dns=8.8.8.8
which will set the container /etc/resolv.conf for you.
I need to get some containers to dead state, as I want to check if a script of mine is working. Any advice is welcome. Thank you.
You've asked for a dead container.
TL;DR: This is how to create a dead container
Don't do this at home:
ID=$(docker run --name dead-experiment -d -t alpine sh)
docker kill dead-experiment
test "$ID" != "" && chattr +i -R /var/lib/docker/containers/$ID
docker rm -f dead-experiment
And voila, docker could not delete the container root directory, so it falls to a status=dead:
docker ps -a -f status=dead
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
616c2e79b75a alpine "sh" 6 minutes ago Dead dead-experiment
Explanation
I've inspected the source code of docker and saw this state transition:
container.SetDead()
// (...)
if err := system.EnsureRemoveAll(container.Root); err != nil {
return errors.Wrapf(err, "unable to remove filesystem for %s", container.ID)
}
// (...)
container.SetRemoved()
So, if docker cannot remove the container root directory, it remain as dead and does not continue to the Removed state. So I've forced the file permissions to not permit root remove files (chattr -i).
PS: to revert the directory permissions do this: chattr -i -R /var/lib/docker/containers/$ID
For docker-1.12+, instruction HEALTHCHECK can help you.
The HEALTHCHECK instruction tells Docker how to test a container to check that it is still working. This can detect cases such as a web server that is stuck in an infinite loop and unable to handle new connections, even though the server process is still running.
For example, we have a Dockerfile to define my own webapp:
FROM nginx:1.13.1
RUN apt-get update \
&& apt-get install -y curl \
&& rm -rf /var/lib/apt/lists/*
HEALTHCHECK --interval=15s --timeout=3s \
CMD curl -fs http://localhost:80/ || exit 1
check every five minutes or so that a web-server is able to serve the site’s main page within three seconds.
The command’s exit status indicates the health status of the container. The possible values are:
0: success - the container is healthy and ready for use
1: unhealthy - the container is not working correctly
2: reserved - do not use this exit code
Then use docker build command to build an image:
$ docker build -t myapp:v1 .
And run a container using this image:
$ docker run -d --name healthcheck-demo -p 80:80 myapp:v1
check the status of container:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b812c8d6f43a myapp:v1 "nginx -g 'daemon ..." 3 seconds ago Up 2 seconds (health: starting) 0.0.0.0:80->80/tcp healthcheck-demo
At the beginning, the status of container is (health: starting); after a while, it changes to be healthy:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d2bb640a6036 myapp:v1 "nginx -g 'daemon ..." 2 minutes ago Up 5 minutes (healthy) 0.0.0.0:80->80/tcp healthcheck-demo
It takes retries consecutive failures of the health check for the container to be considered unhealthy.
You can use your own script to replace the command curl -fs http://localhost:80/ || exit 1. What's more, stdout and stderr of your script can be fetched from docker inspect command:
$ docker inspect --format '{{json .State.Health}}' healthcheck-demo |python -m json.tool
{
"FailingStreak": 0,
"Log": [
{
"End": "2017-06-09T19:39:58.379906565+08:00",
"ExitCode": 0,
"Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 612 100 612 0 0 97297 0 --:--:-- --:--:-- --:--:-- 99k\n<!DOCTYPE html>\n<html>\n<head>\n<title>Welcome to nginx!</title>\n<style>\n body {\n width: 35em;\n margin: 0 auto;\n font-family: Tahoma, Verdana, Arial, sans-serif;\n }\n</style>\n</head>\n<body>\n<h1>Welcome to nginx!</h1>\n<p>If you see this page, the nginx web server is successfully installed and\nworking. Further configuration is required.</p>\n\n<p>For online documentation and support please refer to\nnginx.org.<br/>\nCommercial support is available at\nnginx.com.</p>\n\n<p><em>Thank you for using nginx.</em></p>\n</body>\n</html>\n",
"Start": "2017-06-09T19:39:58.229550952+08:00"
}
],
"Status": "healthy"
}
Hope this helps!