How to make a Health-Check curl call in Circle-Ci - docker

I am trying to make a health check during my circleci workflow to see if the app compiles successfully. Despite the app building on circleci, curl is never able to find to the localhost port.
I have tried this locally on my machine and it works great.
Below is my workflow job:
health-check:
docker:
- image: image_name
steps:
- setup_remote_docker
- restore_cache:
key: image-cache-ci-{{ .Environment.CIRCLE_KEY }}
- run:
name: start
command: |
docker load < image.tar
docker run -d graphql-ci:$CIRCLE_KEY "./node_modules/nodemon/bin/nodemon.js index.js"
docker run graphql-ci:$CIRCLE_KEY curl http://localhost:4000/.well-known/apollo/server-health
The app successfully compiles. However, the second line always returns the following:
12354314532412342: Loading layer 1.803MB/1.803MB
Loaded image: graphql-ci:234523451325424
34523453153145235345234523452345234
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 4000: Connection refused
Exited with code exit status 7
I have also tried to use wait-on - wait-on http://localhost:4000/.well-known/apollo/server-health && curl http://localhost:4000/.well-known/apollo/server-health
I have also placed the curl called in a separate run step after the build one
- run:
name: check that the server is up
command: |
docker load < image.tar
docker run graphql-ci:$CIRCLE_KEY curl http://localhost:4000/.well-known/apollo/server-health
I have also tried to use docker exec but then it just tells me that it cannot find my docker container (and then specifies the docker container image hash).
If I remove the detach flag -d then it will show that it successfully started.
[nodemon] 2.0.2
[nodemon] to restart at any time, enter `rs`
[nodemon] watching dir(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node index.js`
The default PORT set is 4000, I add a flag to the docker run command to include --env PORT=4000 to just confirm it. Didn't affect anything but I know that it is running.
I also gave pm2 a try for the server since then the same terminal will be available by default. This will run but makes no difference.
Really just want to curl a sever I know is loaded in one circleci job.
Entrypoint
CMD [ "node", "index.js" ]
EDIT:
I got closer to getting a successful result.
scripts
"curl": "if [[ $(curl -o /dev/null -w '%{http_code}' http://localhost:4000/.well-known/apollo/server-health) == 200 ]]; then echo 1; else echo 0; fi",
"ci": "./node_modules/nodemon/bin/nodemon.js index.js",
"health": "start-server-and-test ci http-get://localhost:4000/.well-known/apollo/server-health 'npm run curl'",
curl returns 1 if successful (returns 200) and 0 otherwise.
I also installed a package called start-server-and-test which starts the server, waits for it to finish then run the final script. It is normally used for cypress but works here too.
This was the result
> #bespokemetrics/graphql# curl /usr/src/app
> if [[ $(curl -o /dev/null -w '%{http_code}' http://localhost:3000/.well-known/apollo/server-health) == 200 ]]; then echo 1; else echo 0; fi
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 17 100 17 0 0 8500 0 --:--:-- --:--:-- --:--:-- 8500
1 // this was the returned
npm ERR! code ELIFECYCLE
npm ERR! errno 143
npm ERR! package_name: `./node_modules/nodemon/bin/nodemon.js index.js`
npm ERR! Exit status 143
Why does it return an ERR despite succeeding?

I also created an npm script to container all of the code needed to trigger the test on 1 line.
- run:
name: Start and Curl Server
command: |
docker load < image.tar
docker run graphql-ci:$CIRCLE_SHA1 npm run health
"curl": "if [[ $(curl -o /dev/null -w '%{http_code}' http://localhost:4000/.well-known/apollo/server-health) == 200 ]]; then echo 'Success'; pkill node; else echo 'Failure'; fi"
"ci": "./node_modules/nodemon/bin/nodemon.js index.js"
"health": "start-server-and-test ci http-get://localhost:4000/.well-known/apollo/server-health 'npm run curl'"
I installed a package called start-server-and-test which allows you to start your server, wait for it to build, then run whatever command you want all on 1 line.
My curl command only returns the status code, if 200 then return a success message and kill the node process. That was the err that was returned after the success earlier because it wasn't closed properly.

Related

Why docker run saying Unknown operand whilst same shell script runs perfectly in ubuntu machine

I am trying to execute a ./login.sh as a part of docker run now when i run the ./login.sh in ubuntu machine i see success response but when i create a docker image i see output as sh: abc: unknown operand “invalid url”
Here is contents of ./login.sh
#!/bin/sh
# Black Box Tester!
content=$(curl --location --request POST "https://api.platform.abc.com.
/auth/oauth/token" --header 'Content-Type: application/x-www-form-
urlencoded' \
--header 'Authorization: Basic V0zSA==' --data-raw 'grant_type=password&
username=test#example.com&password=123456'| jq -r '.domain_id' )
if [ $content = abc ]
then
echo “Valid Login Token”
else
echo “invalid url”
fi
and here is contents of docker file
FROM ubuntu:16.04
RUN apt-get update && apt-get install -y curl \
&& apt-get -y install jq
FROM openjdk:8-jre-alpine
RUN apk --no-cache add curl jq
WORKDIR /opt
ADD login.sh /opt
RUN pwd \
&& find /opt
CMD ["./login.sh"]
When same ./login.sh i execute normally in Ubuntu machine i get successful output as
ubuntu#ip-172-31-29-248:~$ ./login.sh
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left Speed
100 2034 100 1958 100 76 9458 367 --:--:-- --:--:-- --:--:--
9873
“Valid Login Token”
Here is output of sudo docker run lots/loginimage
ubuntu#ip-172-31-29-248:~$ sudo docker run lots/loginimage
./login.sh: line 6: jq: not found
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left Speed
92 2034 92 1807 100 76 7199 302 --:--:-- --:--:-- --:--:--
7501
curl: (23) Failed writing body (0 != 1024)
sh: abc: unknown operand
“invalid url”
Can some one let me know why the login script which is simply a curl command runs sucessfully in ubuntu machine but same login script shows else condition in docker run?
Also i have included a code in docker file to download and install Json path jq but why image still says ./login.sh: line 6: jq: not found??
your help is highly appreciated
While creating docker you need to install jq sudo apt-get install jq

Gitlab CI emulator not starting

I have a problem with the CI on GitLab. It was working before and then, few days ago without changing anything it stopped to work. The emulator doesn't start anymore and the job always finish in timeout.
Here are the logs of the job when it fails (full log):
[ ... ]
$ wget --quiet --output-document=android-wait-for-emulator https://raw.githubusercontent.com/travis-ci/travis-cookbooks/0f497eb71291b52a703143c5cd63a217c8766dc9/community-cookbooks/android-sdk/files/default/android-wait-for-emulator
$ chmod +x android-wait-for-emulator
$ android-sdk-linux/tools/bin/sdkmanager --update > update.log
Warning: File /root/.android/repositories.cfg could not be loaded.
$ android-sdk-linux/tools/bin/sdkmanager "platform-tools" "emulator" "system-images;android-${EMULATOR_VERSION};google_apis;x86_64" > installEmulator.log
Warning: File /root/.android/repositories.cfg could not be loaded.
$ echo no | android-sdk-linux/tools/bin/avdmanager create avd --force --name test --abi google_apis/x86_64 --package "system-images;android-${EMULATOR_VERSION};google_apis;x86_64"
Loading local repository...
[========= ] 25% Loading local repository...
[========= ] 25% Fetch remote repository...
[========= ] 25% Fetch remote repository...
[========= ] 25% Fetch remote repository...
[=======================================] 100% Fetch remote repository...
Do you wish to create a custom hardware profile? [no] $ android-sdk-linux/emulator/emulator -avd test -no-window -no-audio &
$ ./android-wait-for-emulator
statvfs('/root/.android/avd/test.avd/snapshots/default_boot/ram.img') failed: No such file or directory
Waiting for emulator to start
Pulling docker image gitlab/gitlab-runner-helper:x86_64-577f813d ...
ERROR: Job failed: execution took longer than 1h0m0s seconds
Here are the logs of the EXACT SAME job when it worked (full log):
[ ... ]
$ wget --quiet --output-document=android-wait-for-emulator https://raw.githubusercontent.com/travis-ci/travis-cookbooks/0f497eb71291b52a703143c5cd63a217c8766dc9/community-cookbooks/android-sdk/files/default/android-wait-for-emulator
$ chmod +x android-wait-for-emulator
$ android-sdk-linux/tools/bin/sdkmanager --update > update.log
Warning: File /root/.android/repositories.cfg could not be loaded.
$ android-sdk-linux/tools/bin/sdkmanager "platform-tools" "emulator" "system-images;android-${EMULATOR_VERSION};google_apis;x86_64" > installEmulator.log
Warning: File /root/.android/repositories.cfg could not be loaded.
$ echo no | android-sdk-linux/tools/bin/avdmanager create avd --force --name test --abi google_apis/x86_64 --package "system-images;android-${EMULATOR_VERSION};google_apis;x86_64"
Loading local repository...
[========= ] 25% Loading local repository...
[========= ] 25% Fetch remote repository...
[========= ] 25% Fetch remote repository...
[========= ] 25% Fetch remote repository...
[=======================================] 100% Fetch remote repository...
Do you wish to create a custom hardware profile? [no] $ android-sdk-linux/emulator/emulator -avd test -no-window -no-audio &
$ ./android-wait-for-emulator
statvfs('/root/.android/avd/test.avd/snapshots/default_boot/ram.img') failed: No such file or directory
Waiting for emulator to start
Your emulator is out of date, please update by launching Android Studio:
- Start Android Studio
- Select menu "Tools > Android > SDK Manager"
- Click "SDK Tools" tab
- Check "Android Emulator" checkbox
- Click "OK"
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
Waiting for emulator to start
emulator: INFO: boot completed
emulator: INFO: boot time 23881 ms
emulator: Increasing screen off timeout, logcat buffer size to 2M.
emulator: Revoking microphone permissions for Google App.
Emulator is ready
$ adb shell settings put global window_animation_scale 0 &
$ adb shell settings put global transition_animation_scale 0 &
$ adb shell settings put global animator_duration_scale 0 &
$ adb shell input keyevent 82
$ cd ./DenisAppProject
$ ./gradlew connectedCheck
[ ... ]
$ adb emu kill
OK: killing emulator, bye bye
OK
emulator: Saving state on exit with session uptime 146931 ms
Job succeeded
And here is my gitlab-ci.yml
image: openjdk:8-jdk
variables:
ANDROID_COMPILE_SDK: "28"
ANDROID_BUILD_TOOLS: "28.0.3"
ANDROID_SDK_TOOLS: "4333796"
EMULATOR_VERSION: "26"
before_script:
- apt-get --quiet update --yes
- apt-get --quiet install --yes wget tar unzip lib32stdc++6 lib32z1
- wget --quiet --output-document=android-sdk.zip https://dl.google.com/android/repository/sdk-tools-linux-${ANDROID_SDK_TOOLS}.zip
- unzip -d android-sdk-linux android-sdk.zip
- echo y | android-sdk-linux/tools/bin/sdkmanager "platforms;android-${ANDROID_COMPILE_SDK}" >/dev/null
- echo y | android-sdk-linux/tools/bin/sdkmanager "platform-tools" >/dev/null
- echo y | android-sdk-linux/tools/bin/sdkmanager "build-tools;${ANDROID_BUILD_TOOLS}" >/dev/null
- export ANDROID_HOME=$PWD/android-sdk-linux
- export PATH=$PATH:$PWD/android-sdk-linux/platform-tools/
- chmod +x ./DenisAppProject/gradlew
# temporarily disable checking for EPIPE error and use yes to accept all licenses
- set +o pipefail
- yes | android-sdk-linux/tools/bin/sdkmanager --licenses
- set -o pipefail
stages:
- build
- unit-test
- instrumental-test
lintDebug:
tags: ["android"]
stage: build
script:
- cd DenisAppProject/
- ./gradlew -Pci --console=plain :DenisApp:lintDebug -PbuildDir=lint
assembleDebug:
tags: ["android"]
stage: build
script:
- cd DenisAppProject/
- ./gradlew assembleDebug
artifacts:
paths:
- DenisApp/build/outputs/
debugTests:
tags: ["android"]
stage: unit-test
script:
- cd DenisAppProject/
- ./gradlew -Pci --console=plain :DenisApp:testDebug
instrumentation_tests:
tags: ["android"]
stage: instrumental-test
script:
- apt-get --quiet update --yes
- apt-get --quiet install --yes libx11-dev libpulse0 libgl1 libnss3 libxcomposite-dev libxcursor1 libasound2
- wget --quiet --output-document=android-wait-for-emulator https://raw.githubusercontent.com/travis-ci/travis-cookbooks/0f497eb71291b52a703143c5cd63a217c8766dc9/community-cookbooks/android-sdk/files/default/android-wait-for-emulator
- chmod +x android-wait-for-emulator
- android-sdk-linux/tools/bin/sdkmanager --update > update.log
- android-sdk-linux/tools/bin/sdkmanager "platform-tools" "emulator" "system-images;android-${EMULATOR_VERSION};google_apis;x86_64" > installEmulator.log
- echo no | android-sdk-linux/tools/bin/avdmanager create avd --force --name test --abi google_apis/x86_64 --package "system-images;android-${EMULATOR_VERSION};google_apis;x86_64"
- android-sdk-linux/emulator/emulator -avd test -no-window -no-audio &
- ./android-wait-for-emulator
# Turn off animations
- adb shell settings put global window_animation_scale 0 &
- adb shell settings put global transition_animation_scale 0 &
- adb shell settings put global animator_duration_scale 0 &
- adb shell input keyevent 82
- cd ./DenisAppProject
- ./gradlew connectedCheck
- adb emu kill
Screen of the job where you can see that it worked the first time and then it stopped (all the jobs before were working).
#Denis-Pinna
Hi, I tried to set gitlab CI first time. I am getting following error when ii start the emulator after creation.
Waiting for emulator to start
emulator: ERROR: x86 emulation currently requires hardware acceleration!
Please ensure KVM is properly installed and usable.
CPU acceleration status: KVM requires a CPU that supports vmx or svm
More info on configuring VM acceleration on Linux:
https://developer.android.com/studio/run/emulator-acceleration#vm-linux
General information on acceleration: https://developer.android.com/studio/run/emulator-acceleration.
At the end, I reproduced the same environment locally with docker. Then I could try to start the emulator with logs and I found out that I had a segmentation fault error, after some other tests I found that taking the normal emulator version and not the 64 one solved my problem, I also added some parameters as disabling the gpu, the starting animation etc. It doesn't explain why it suddenly stopped to work but at least this command work :
- android-sdk-linux/emulator/emulator -avd test -no-boot-anim -no-snapshot-save -no-audio -no-window -gpu off -debug -all
and use the emulator image
system-images;android-${ANDROID_COMPILE_SDK};google_apis_playstore;x86

RUN curl fails in dockerfile but works from command line

Using Ubuntu 16.04 and docker 18.03.1-ce
In a Dockerfile I have:
RUN curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar
But when I build that with:
sudo docker build -t myimage .
I get:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: mynexus
The command '/bin/sh -c curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar' returned a non-zero code: 6
If I run the exact same command line from a terminal it works fine:
$ curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3613k 100 3613k 0 0 1413k 0 0:00:02 0:00:02 --:--:-- 1413k
Why can I curl that file from command line but not from the DockerFile?
You can see the curl error message when running the last commited image from the build with curl command:
docker run -it --rm lastCommitedImageHash curl -k -o app.jar -kfSL https://mynexus:9006/repository/legacy/1.0.2/app.jar
I had the same problem last week - curl worked directly from host but not when ran during build from Dockerfile.
Restart of Docker daemon helped to me.
I haven't found what was the reason of that state. It was on the machine which is also as a Kubernetes master and I did Kubernetes cluster upgrade last week and probably this was the source of that problem.
Another cause of that problem in your case can be if you have mynexus hostname defined only in the /etc/hosts file on the host machine and then it is unknown for the running container during the build.

curl doesn't work during docker build

I am writing a Dockerfile based on wildfly image. I have isolated these lines where I am having some headache. The curl command doesn't work during build process. I have already uninstalled and installed Docker again but the error persists. My system is a Linux Mint.
In addition I tried to build that same Dockerfile in a RHEL and it worked like a charm.
Here's the Dockerfile:
FROM jboss/wildfly
RUN cd $HOME \
&& curl -O "http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.44/mysql-connector-java-5.1.44.jar"
Here's the error output:
Sending build context to Docker daemon 1.03MB
Step 1/6 : FROM jboss/wildfly
---> b695bdcce374
Step 2/6 : RUN cd $HOME && curl -O "http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.44/mysql-connector-java-5.1.44.jar"
---> Running in 4fdcef7dbda1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:39 --:--:-- 0
curl: (6) Could not resolve host: central.maven.org; Unknown error
The command '/bin/sh -c cd $HOME && curl -O "http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.44/mysql-connector-java-5.1.44.jar"' returned a non-zero code: 6
I could workaround the problem doing something like this:
docker build --add-host central.maven.org:151.101.56.209 .
but I'm not happy with that. I would like to say Docker to use my DNS instead of set fixed IP. It would be more elegant.
The error is a DNS problem resolving the hostname. You could try running docker with the --dns option (https://docs.docker.com/engine/reference/run/#network-settings)
docker run --dns=8.8.8.8
which will set the container /etc/resolv.conf for you.

How to get a docker container to the state: dead for debugging?

I need to get some containers to dead state, as I want to check if a script of mine is working. Any advice is welcome. Thank you.
You've asked for a dead container.
TL;DR: This is how to create a dead container
Don't do this at home:
ID=$(docker run --name dead-experiment -d -t alpine sh)
docker kill dead-experiment
test "$ID" != "" && chattr +i -R /var/lib/docker/containers/$ID
docker rm -f dead-experiment
And voila, docker could not delete the container root directory, so it falls to a status=dead:
docker ps -a -f status=dead
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
616c2e79b75a alpine "sh" 6 minutes ago Dead dead-experiment
Explanation
I've inspected the source code of docker and saw this state transition:
container.SetDead()
// (...)
if err := system.EnsureRemoveAll(container.Root); err != nil {
return errors.Wrapf(err, "unable to remove filesystem for %s", container.ID)
}
// (...)
container.SetRemoved()
So, if docker cannot remove the container root directory, it remain as dead and does not continue to the Removed state. So I've forced the file permissions to not permit root remove files (chattr -i).
PS: to revert the directory permissions do this: chattr -i -R /var/lib/docker/containers/$ID
For docker-1.12+, instruction HEALTHCHECK can help you.
The HEALTHCHECK instruction tells Docker how to test a container to check that it is still working. This can detect cases such as a web server that is stuck in an infinite loop and unable to handle new connections, even though the server process is still running.
For example, we have a Dockerfile to define my own webapp:
FROM nginx:1.13.1
RUN apt-get update \
&& apt-get install -y curl \
&& rm -rf /var/lib/apt/lists/*
HEALTHCHECK --interval=15s --timeout=3s \
CMD curl -fs http://localhost:80/ || exit 1
check every five minutes or so that a web-server is able to serve the site’s main page within three seconds.
The command’s exit status indicates the health status of the container. The possible values are:
0: success - the container is healthy and ready for use
1: unhealthy - the container is not working correctly
2: reserved - do not use this exit code
Then use docker build command to build an image:
$ docker build -t myapp:v1 .
And run a container using this image:
$ docker run -d --name healthcheck-demo -p 80:80 myapp:v1
check the status of container:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b812c8d6f43a myapp:v1 "nginx -g 'daemon ..." 3 seconds ago Up 2 seconds (health: starting) 0.0.0.0:80->80/tcp healthcheck-demo
At the beginning, the status of container is (health: starting); after a while, it changes to be healthy:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d2bb640a6036 myapp:v1 "nginx -g 'daemon ..." 2 minutes ago Up 5 minutes (healthy) 0.0.0.0:80->80/tcp healthcheck-demo
It takes retries consecutive failures of the health check for the container to be considered unhealthy.
You can use your own script to replace the command curl -fs http://localhost:80/ || exit 1. What's more, stdout and stderr of your script can be fetched from docker inspect command:
$ docker inspect --format '{{json .State.Health}}' healthcheck-demo |python -m json.tool
{
"FailingStreak": 0,
"Log": [
{
"End": "2017-06-09T19:39:58.379906565+08:00",
"ExitCode": 0,
"Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 612 100 612 0 0 97297 0 --:--:-- --:--:-- --:--:-- 99k\n<!DOCTYPE html>\n<html>\n<head>\n<title>Welcome to nginx!</title>\n<style>\n body {\n width: 35em;\n margin: 0 auto;\n font-family: Tahoma, Verdana, Arial, sans-serif;\n }\n</style>\n</head>\n<body>\n<h1>Welcome to nginx!</h1>\n<p>If you see this page, the nginx web server is successfully installed and\nworking. Further configuration is required.</p>\n\n<p>For online documentation and support please refer to\nnginx.org.<br/>\nCommercial support is available at\nnginx.com.</p>\n\n<p><em>Thank you for using nginx.</em></p>\n</body>\n</html>\n",
"Start": "2017-06-09T19:39:58.229550952+08:00"
}
],
"Status": "healthy"
}
Hope this helps!

Resources