I am currently setting up a Cloudbees Jenkins cluster with the help of the docker swarm plugin. I got it to work once, but haven't gotten it to work ever since. I get several different errors related to the creation of the agent that executes the pipeline, but they all seem to be related to the environment variables that the plugin creates at runtime. Does someone know a solution to this?
[1:40:45 PM] Creating Service with Name : agt-vice_testing_59-72
[1:40:45 PM] ServiceSpec created with ID : kk0jxbuqvh4zyngruamxphl4b
[1:40:45 PM] ServiceSpec request JSON : {"TaskTemplate":{"ContainerSpec":{"Image":"docker:17.12","Command":["sh","-cx","docker run --rm --privileged java:8 sh -xc 'curl --connect-timeout 20 --max-time 60 -o agent.jar http://199.18.14.456:8080/jnlpJars/agent.jar && java -jar agent.jar -jnlpUrl $DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL' "],"Env":["DOCKER_SWARM_PLUGIN_JENKINS_AGENT_SECRET=5ee89222c397f86c7ca8ecc6af73268539fa3ea9f64e46f815c4cfc06322aa0f","DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JAR_URL=http://199.18.14.456:8080/jnlpJars/agent.jar","DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL=http://199.18.14.456:8080/computer/agt-vice_testing_59-72/slave-agent.jnlp","DOCKER_SWARM_PLUGIN_JENKINS_AGENT_NAME=agt-vice_testing_59-72"],"Dir":"/tmp","User":"root","DNSConfig":{"Nameservers":[],"Search":[],"Options":[]},"Mounts":[{"Target":"/var/run/docker.sock","Source":"/var/run/docker.sock","Type":"bind","VolumeOptions":null}],"Hosts":[],"Secrets":[],"Configs":[]},"RestartPolicy":{"Condition":"none","MaxAttempts":0},"Resources":{"Limits":{"NanoCPUs":0,"MemoryBytes":0},"Reservations":{"NanoCPUs":0,"MemoryBytes":0}},"Placement":{"Constraints":[]}},"EndpointSpec":{"Ports":[]},"Name":"agt-vice_testing_59-72","Labels":{"ROLE":"jenkins-agent"},"Networks":[]}
�+ docker run --rm --privileged java:8 sh -xc curl --connect-timeout 20 --max-time 60 -o agent.jar http://199.18.14.456:8080/jnlpJars/agent.jar && java -jar agent.jar -jnlpUrl $DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL
d+ curl --connect-timeout 20 --max-time 60 -o agent.jar http://199.18.14.456:8080/jnlpJars/agent.jar
P % Total % Received % Xferd Average Speed Time Time Time Current
N Dload Upload Total Spent Left Speed
�
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 770k 100 770k 0 0 63.9M 0 --:--:-- --:--:-- --:--:-- 68.4M
+ java -jar agent.jar -jnlpUrl
#Option "-jnlpUrl" takes an operand
!java -jar agent.jar [options...]
Related
I am running docker build with following Dockerfile, with main idea to use feature repo as described https://github.com/WASdev/ci.docker#installing-liberty-features-from-local-repository-19008:
FROM websphere-liberty-kernel-ubi-min:19.0.0.9
COPY usr/ /opt/ibm/wlp/usr/
USER root
ARG FEATURE_REPO_URL=http://xyz.openshift.local/19.0.0.9/repo.zip
ARG VERBOSE=true
RUN configure.sh
RUN chown -R 1001:0 /tmp \
&& chmod -R g+rw /tmp \
&& chown -R 1001:0 /opt/ibm/wlp/output \
&& chmod -R g+rw /opt/ibm/wlp/output \
&& chown -R 1001:0 /opt/ibm/wlp/usr/servers/defaultServer \
&& chmod -R g+rw /opt/ibm/wlp/usr/servers/defaultServer \
&& chown -R 1001:0 /opt/ibm/wlp/usr/shared/resources \
&& chmod -R g+rw /opt/ibm/wlp/usr/shared/resources
USER 1001
Docker build output shows that repo.zip is downloaded and missing features are detected:
+ '[' http://xyz.openshift.local/19.0.0.9/repo.zip ']'
+ curl -k --fail http://xyz.openshift.local/19.0.0.9/repo.zip % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 14 311M 14 44.8M 0 0 55.0M 0 0:00:05 --:--:-- 0:00:05 54.9M 31 311M 31 99.4M 0 0 54.8M 0 0:00:05 0:00:01 0:00:04 54.7M 49 311M 49 152M 0 0 54.2M 0 0:00:05 0:00:02 0:00:03 54.2M 66 311M 66 207M 0 0 54.4M 0 0:00:05 0:00:03 0:00:02 54.4M 83 311M 83 260M 0 0 54.0M 0 0:00:05 0:00:04 0:00:01 54.0M 99 311M 99 310M 0 0 53.4M 0 0:00:05 0:00:05 --:--:-- 53.1M100 311M 100 311M 0 0 53.4M 0 0:00:05 0:00:05 --:--:-- 52.7M
+ installUtility install --acceptLicense defaultServer --from=/tmp/repo.zip
Checking for missing features required by the server ...
The server requires the following additional features: mpconfig-1.3 transportsecurity-1.0 cdi-2.0 mpopenapi-1.0 jaxws-2.2 jsonp-1.1 jpa-2.2 mprestclient-1.3 mphealth-2.1 wssecurity-1.1 jaxrs-2.1. Installing features from the repository ...
Successfully connected to the configured repository.
but then installation of all features fails:
Preparing assets for installation. This process might take several minutes to complete.
CWWKF1259E: Unable to obtain the following assets: mpconfig-1.3 transportsecurity-1.0 cdi-2.0 mpopenapi-1.0 jaxws-2.2 jsonp-1.1 jpa-2.2 mprestclient-1.3 mphealth-2.1 wssecurity-1.1 jaxrs-2.1. Ensure that the specified assets are valid. To find the IDs of applicable assets, run the installUtility find command.
I have taken a look in downloaded repo.zip and I can find following file (that should match one of the missing features):
So, what is the reason for the error?
Using different image (ibmcom/websphere-liberty:some21.0.0.3version) and pointing FEATURE_REPO_URL to place where 21.0.0.3 repo is hosted works.
So whoever prepared 19.0.0.3 image for me and said which feature repo to use, pointed me to some incompatible artefacts.
What is interesting, when I combined different versions of image vs repo, configure.sh was nicely verbose (it explained there is incompatibility), but this error CWWKF1259E: Unable to obtain the following assets is really unhelpful.
I am trying to make a health check during my circleci workflow to see if the app compiles successfully. Despite the app building on circleci, curl is never able to find to the localhost port.
I have tried this locally on my machine and it works great.
Below is my workflow job:
health-check:
docker:
- image: image_name
steps:
- setup_remote_docker
- restore_cache:
key: image-cache-ci-{{ .Environment.CIRCLE_KEY }}
- run:
name: start
command: |
docker load < image.tar
docker run -d graphql-ci:$CIRCLE_KEY "./node_modules/nodemon/bin/nodemon.js index.js"
docker run graphql-ci:$CIRCLE_KEY curl http://localhost:4000/.well-known/apollo/server-health
The app successfully compiles. However, the second line always returns the following:
12354314532412342: Loading layer 1.803MB/1.803MB
Loaded image: graphql-ci:234523451325424
34523453153145235345234523452345234
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 4000: Connection refused
Exited with code exit status 7
I have also tried to use wait-on - wait-on http://localhost:4000/.well-known/apollo/server-health && curl http://localhost:4000/.well-known/apollo/server-health
I have also placed the curl called in a separate run step after the build one
- run:
name: check that the server is up
command: |
docker load < image.tar
docker run graphql-ci:$CIRCLE_KEY curl http://localhost:4000/.well-known/apollo/server-health
I have also tried to use docker exec but then it just tells me that it cannot find my docker container (and then specifies the docker container image hash).
If I remove the detach flag -d then it will show that it successfully started.
[nodemon] 2.0.2
[nodemon] to restart at any time, enter `rs`
[nodemon] watching dir(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node index.js`
The default PORT set is 4000, I add a flag to the docker run command to include --env PORT=4000 to just confirm it. Didn't affect anything but I know that it is running.
I also gave pm2 a try for the server since then the same terminal will be available by default. This will run but makes no difference.
Really just want to curl a sever I know is loaded in one circleci job.
Entrypoint
CMD [ "node", "index.js" ]
EDIT:
I got closer to getting a successful result.
scripts
"curl": "if [[ $(curl -o /dev/null -w '%{http_code}' http://localhost:4000/.well-known/apollo/server-health) == 200 ]]; then echo 1; else echo 0; fi",
"ci": "./node_modules/nodemon/bin/nodemon.js index.js",
"health": "start-server-and-test ci http-get://localhost:4000/.well-known/apollo/server-health 'npm run curl'",
curl returns 1 if successful (returns 200) and 0 otherwise.
I also installed a package called start-server-and-test which starts the server, waits for it to finish then run the final script. It is normally used for cypress but works here too.
This was the result
> #bespokemetrics/graphql# curl /usr/src/app
> if [[ $(curl -o /dev/null -w '%{http_code}' http://localhost:3000/.well-known/apollo/server-health) == 200 ]]; then echo 1; else echo 0; fi
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 17 100 17 0 0 8500 0 --:--:-- --:--:-- --:--:-- 8500
1 // this was the returned
npm ERR! code ELIFECYCLE
npm ERR! errno 143
npm ERR! package_name: `./node_modules/nodemon/bin/nodemon.js index.js`
npm ERR! Exit status 143
Why does it return an ERR despite succeeding?
I also created an npm script to container all of the code needed to trigger the test on 1 line.
- run:
name: Start and Curl Server
command: |
docker load < image.tar
docker run graphql-ci:$CIRCLE_SHA1 npm run health
"curl": "if [[ $(curl -o /dev/null -w '%{http_code}' http://localhost:4000/.well-known/apollo/server-health) == 200 ]]; then echo 'Success'; pkill node; else echo 'Failure'; fi"
"ci": "./node_modules/nodemon/bin/nodemon.js index.js"
"health": "start-server-and-test ci http-get://localhost:4000/.well-known/apollo/server-health 'npm run curl'"
I installed a package called start-server-and-test which allows you to start your server, wait for it to build, then run whatever command you want all on 1 line.
My curl command only returns the status code, if 200 then return a success message and kill the node process. That was the err that was returned after the success earlier because it wasn't closed properly.
Using Ubuntu 16.04 and docker 18.03.1-ce
In a Dockerfile I have:
RUN curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar
But when I build that with:
sudo docker build -t myimage .
I get:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: mynexus
The command '/bin/sh -c curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar' returned a non-zero code: 6
If I run the exact same command line from a terminal it works fine:
$ curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3613k 100 3613k 0 0 1413k 0 0:00:02 0:00:02 --:--:-- 1413k
Why can I curl that file from command line but not from the DockerFile?
You can see the curl error message when running the last commited image from the build with curl command:
docker run -it --rm lastCommitedImageHash curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar
I had the same problem last week - curl worked directly from host but not when ran during build from Dockerfile.
Restart of Docker daemon helped to me.
I haven't found what was the reason of that state. It was on the machine which is also as a Kubernetes master and I did Kubernetes cluster upgrade last week and probably this was the source of that problem.
Another cause of that problem in your case can be if you have mynexus hostname defined only in the /etc/hosts file on the host machine and then it is unknown for the running container during the build.
Hi total Docker noob here, is there a dependency i can run so this error goes away?
Step 19/40 : RUN curl -OL https://www.haskell.org/ghc/dist/${ghc%[a-z]}/ghc-${ghc}-x86_64-deb7-linux.tar.xz | tar -xJf- && cd ghc-${ghc} && ./configure && make install
---> Running in 0141996d759f
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 161 100 161 0 0 335 0 --:--:-- --:--:-- --:--:-- 336
100 85.5M 100 85.5M 0 0 308k 0 0:04:44 0:04:44 --:--:-- 358k
xz: (stdin): File format not recognized
tar: Child returned status 1
tar: Error is not recoverable: exiting now
The command '/bin/sh -c curl -OL https://www.haskell.org/ghc/dist/${ghc%[a-z]}/ghc-${ghc}-x86_64-deb7-linux.tar.xz | tar -xJf- && cd ghc-${ghc} && ./configure && make install' returned a non-zero code: 2
The problem is the -O option to curl that writes to disk instead of stdout.
Try with: curl -L https://...
I'm trying to deploy rails application in Cent-OS. When I run curl -L get.rvm.io | bash -s stable to setup ruby environment.
I got the following output
[root#186-aven-vps marketeurepo]# curl -L get.rvm.io | bash -s stable
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 184 100 184 0 0 581 0 --:--:-- --:--:-- --:--:-- 580
100 22721 100 22721 0 0 35092 0 --:--:-- --:--:-- --:--:-- 35092
ERROR: Could not find 'which' command, make sure it's available first before continuing installation.
I'm using Mac OS X on local machine and I'm new to this.Any suggestion will be appreciated.Thank you.
I solved it by running this command:
$ yum install which