I have a Jenkinsfile in which I am trying to execute npm run test inside a container.
When I run with inside it fails but when I run with withRun it runs as I want to.
Code for reference
with inside
stage('Test') {
docker.image('justinribeiro/chrome-headless').inside ("-p 9222:9222 --security-opt seccomp=$WORKSPACE/chrome.json") {
sh label:
'Running npm test',
script: '''
npm run test
'''
}
}
With withRun
stage('Test') {
docker.image('justinribeiro/chrome-headless').withRun ("-p 9222:9222 --security-opt seccomp=$WORKSPACE/chrome.json") {
sh label:
'Running npm test',
script: '''
npm run test
'''
}
}
Now I want to understand what is the difference between them.
I have observed that inside adds volumes and runs cat on container whereas withRun does not.
I also read the documentation https://jenkins.io/doc/book/pipeline/docker/ but did not understand well enough.
A much more detailed explanation will be much appreciated.
Thanks.
Image.run([args, command])
Uses docker run to run the image, and returns a Container which you could stop later. Additional args may be added, such as '-p 8080:8080 --memory-swap=-1'. Optional command is equivalent to Docker command specified after the image. Records a run fingerprint in the build.
Image.withRun[(args[, command])] {…}
Like run but stops the container as soon as its body exits, so you do not need a try-finally block.
Image.inside[(args)] {…}
Like withRun this starts a container for the duration of the body, but all external commands (sh) launched by the body run inside the container rather than on the host. These commands run in the same working directory (normally a Jenkins agent workspace), which means that the Docker server must be on localhost.
So as you can tell from the above, your sh method commands (specifically the npm commands) will execute on the host for withRun, but within the container for inside.
Link to the docs: https://opensource.triology.de/jenkins/pipeline-syntax/globals
Related
In a Jenkinsfile, how can I run a Docker container and capture the output, without reverting to sh instructions?
This is how I build my container image during the Build stage:
dockerImage = docker.build(env.DOCKER_COORDS, "--build-arg BUILD_VERSION=${env.BUILD_VERSION} .")
And this is how I push the image in a later Publish stage:
withDockerRegistry(credentialsId: env.DOCKER_REG_CREDS_ID) {
dockerImage.push('latest')
}
In between the Build and Publish stages, in a Test stage, I would like to validate that the output from the container, when passing a --version argument, is equal to ${env.BUILD_VERSION}.
How can I run the container and capture the output, without having to use sh "docker ..." instructions again?
I am new to this but according to the documentation of Docker Pipeline plugin, you can use Image.run([args]), for your case it will be dockerImage.run()
To pass environment variables:
dockerImage.run(['-e your_variable=X'])
Normally you can run and capture the output using the sh method (on a Linux slave and bat for windows). For example, to retrieve the python version of the official python docker, it will look like this:
def version = sh label: '', returnStdout: true, script: 'docker run python python --version'
And then you could just compare the output of the commande with ${env.BUILD_VERSION}. This shouldn't be to different with your docker image.
My Jenkins pipeline uses the docker-workflow plugin. It builds a Docker image and tags it app. The build step fetches some dependencies and bakes them into the image along with my app.
I want to run two commands inside a container based on that image. The command should be executed in the built environment, with access to the dependencies. I tried using Image.inside, but it seems to fail because inside mounts the project directory over the working directory (?) and so the dependencies aren't available.
docker.image("app").inside {
sh './run prepare-tests'
sh './run tests'
}
I tried using docker.script.withDockerContainer too, but the commands don't seem to run inside the container. The same seems to be true for Image.withRun. At least with that I could specify a command, but it seems that I'd have to run specify both commands in one statement. Also it seems that withRun doesn't fail the build if the command doesn't exit cleanly.
docker
.image("app")
.withRun('', 'bash -c "./app prepare-tests && ./app tests"') { container ->
sh "exit \$(docker wait ${container.id})"
}
Is there a way to use Image.inside without mounting the project directory? Or is there are more elegant way of doing this?
docker DSL, like docker.image().inside() {} etc will mount jenkins job workspace dir to container and make it as the WORKDIR which will overwrite the WORKDIR in Dockerfile.
You can verify that from jenkins console output .
1) CD workdir fristly
docker.image("app").inside {
sh '''
cd <WORKDIR of image specifyed in Dockerfile>
./run prepare-tests
./run tests
'''
}
2) Run container in sh , rather than via docker DSL
sh '''
docker run -i app bash -c "./run prepare-tests && ./run tests"
'''
I'm doing a Jenkins freestyle build, ssh'ing into a VM and running some existing scripts. During a build step "Execute shell script on remote host using ssh", everything is working fine until I get to a command: "chroot . sh". This is mounting a rootfs which we build in, if I do this step manually it is bringing me to a sh prompt where I can run another script to do the actual build, but Jenkins just hangs forever at this point.
From looking around it looks like this is because the command does not return a completion signal?, so Jenkins is waiting indefinitely. I've also just tried doing the same steps in Putty, using a text file containing the commands I need. The Putty "script" also fails at this point, stopping any input because of the new sh prompt.
Is there any way around this? I've tried various solutions, like:
nohup chroot . sh 1>&2 - of course this doesnt work
and running the command in the background doesnt put me into the chroot environment I need.
Kind of confused at this point.
Edit:
Code snippet:
cd /home/dev/root_env
chroot . sh
cd /home/dev
./build.sh
Thats literally all I'm doing, however I freeze forever at line 2 of that.
I'm trying to run a Python image in Jenkins to perform a series of unit tests with pytest, but I'm getting some strange behavior with Docker.
My Jenkinsfile pipeline is
agent {
docker { image 'python:3.6-jessie' }
}
stages {
stage('Run tests') {
steps {
withCredentials([
string(credentialsId: 'a-secret', variable: 'A_SECRET')
{
sh label: "Install dependencies", script: 'pip install -r requirements.txt'
sh label: 'Execute tests', script: "pytest mytests.py"
}
}
}
}
However, when I run the pipeline, Docker appears to be executing a very long instruction (with significantly more -e environment variables than I defined as credentials?), followed by cat.
The build then simply hangs and never finishes:
Jenkins does not seem to be running inside a container
$ docker run -t -d -u 996:994
-w /var/lib/jenkins/workspace/myproject
-v /var/lib/jenkins/workspace/myproject:/var/lib/jenkins/workspace/myproject:rw,z
-v /var/lib/jenkins/workspace/myproject#tmp:/var/lib/jenkins/workspace/myproject#tmp:rw,z
-e ******** -e ******** python:3.6-jessie cat
When I SSH into my instance and run docker ps, I see
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
240d00459d92 python:3.6-jessie "cat" About a minute ago Up About a minute kind_wright
Why is Jenkins running cat? Why does Jenkins say I am not running inside a container, when it has clearly created a container for me? And most importantly, why are my pip install -r requirements and other steps not executing?
I finally figured this out. If you have empty global environment variables in your Jenkins configuration, it appears that you'll get a malformed docker run command since Jenkins will write the command, with your empty string environment variable, as docker run -e some_env_var=some_value -e = ...
This will cause the container to simply hang.
A telltale sign that this is happening is you'll get the error message:
invalid argument "=" for "-e, --env" flag: invalid environment variable: =
This is initially difficult to diagnose since Jenkins (rightfully) hides your actual credentials with ***, so the empty environment strings do not show up as empty.
You need to check your Jenkins global configuration and make sure you don't have any empty environment variables accidentally defined:
If these exist, you need to delete them and rerun.
I have an Jenkins Server running in an docker container and have access to docker an the host system, so far it is working well. Now I want to set up a pipeline testing an script inside an docker container.
Jenkinsfile:
pipeline {
agent { docker 'nginx:1.11' }
stages {
stage('build') {
steps {
sh 'nginx -t'
}
}
}
}
Error Message:
> + docker pull nginx:1.11
>
> Warning: failed to get default registry endpoint from daemon (Got
> permission denied while trying to connect to the Docker daemon socket
> at unix:///var/run/docker.sock: Get
> http://%2Fvar%2Frun%2Fdocker.sock/v1.29/info: dial unix
> /var/run/docker.sock: connect: permission denied). Using system
> default: https://index.docker.io/v1/
>
> Got permission denied while trying to connect to the Docker daemon
> socket at unix:///var/run/docker.sock: Post
> http://%2Fvar%2Frun%2Fdocker.sock/v1.29/images/create?fromImage=nginx&tag=1.11:
> dial unix /var/run/docker.sock: connect: permission denied
>
> script returned exit code 1
My problem is that jenkins needs to run the docker command with sudo, but how to say the agent running the command with sudo?
I have faced the same issue. After analysing the console log, I have found that the reason is that the Docker Jenkins Plugin starts a new container with a specific option -u 107:112:
...
docker run -t -d -u 107:112 ...
...
After trying many options such as: add jenkins to sudo group (it did not work because jenkins user does not exist in container), add USER root into Dockerfile, ... but none of them do the trick.
Finally I have found a solution that is using args in docker agent to overwrite the -u option. This is my Jenkinsfile:
pipeline {
agent {
docker {
image 'ubuntu'
args '-u root:sudo -v $HOME/workspace/myproject:/myproject'
}
}
stages {
stage("setup_env") {
steps {
sh 'apt-get update -y'
sh 'apt-get install -y git build-essential gcc cmake make'
}
}
stage("install_dependencies") {
steps {
sh 'apt-get install -y libxml2-dev'
}
}
stage("compile_dpi") {
steps {
sh 'cd /myproject && make clean && make -j4'
}
}
stage("install_dpi") {
steps {
sh 'cd /myproject && make install'
}
}
stage("test") {
steps {
sh 'do some test here'
}
}
}
post {
success {
echo 'Do something when it is successful'
bitbucketStatusNotify(buildState: 'SUCCESSFUL')
}
failure {
echo 'Do something when it is failed'
bitbucketStatusNotify(buildState: 'FAILED')
}
}
}
There's maybe a security issue here but it is not the problem in my case.
I'd solve the problem differently, matching the jenkins group id inside the container to that of the docker socket you've mounted a volume. I do this with an entrypoint that runs as root, looks up the gid of the socket, and if that doesn't match that of the gid inside the current container, it does a groupmod to correct it inside the container. Then I drop privileges to the jenkins user to launch Jenkins. This entrypoint run on every startup, but fairly transparently to the Jenkins app that is launched.
All the steps to perform this are included in this github repo: https://github.com/sudo-bmitch/jenkins-docker/
You can work around that by:
1- In your Dockerfile add jenkins to the sudoers file:
RUN echo "jenkins ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
2- Add an extra step in your Jenkinsfile to give jenkins the right permissions to use docker:
pipeline {
agent none
stages {
stage("Fix the permission issue") {
agent any
steps {
sh "sudo chown root:jenkins /run/docker.sock"
}
}
stage('Step 1') {
agent {
docker {
image 'nezarfadle/tools'
reuseNode true
}
}
steps {
sh "ls /"
}
}
}
}
As others have suggested, the issue is that jenkins does not have permission to run docker containers. Let's go over the ways you could launch jenkins first, and then see what could be done in each of these ways.
1. running jenkins manually
Surely you could download & run jenkins with java as suggested in here. In this method, you could do several things to allow your jenkins user to use docker:
a. give jenkins user root access:
I do not suggest this way, after all you are giving your pipelines access to everything! So you probably do not want this to happen.
b. add jenkins user to docker group
Like explained here you could manage docker as non-root user. just add your user to docker group and thats all. I recommend it if you know who is going to use docker (cause well, you are giving him root access in docker in a way).
c. make docker rootless
This is a new feature docker added to its arsenal recently. You could read in detail what it implies here. To tell you the truth I am not a fan of this feature! The reason is that you can not (at least I could not find a way) to make it work for a user in a container (as you need to stop docker service to make it happen), also I had some difficulties configing dns when using rootless mode. But it should be fine if you are not in a container.
2. running jenkins in docker
This method is more troublesome actually! I struggled with the ways I could use docker in jenkins container, but in the end got the results needed, so was worth the effort.
To run docker in jenkins (which is also a docker container itself) you have three ways:
1. use dind (docker in docker)
It is pretty straight forward, you run dind image & connect docker in jenkins container to the dind, without any special permission handling you can use docker at will.
2. use dood (docker outside of docker)
mount docker path as a volume in the docker run script for your jenkins, note that you need to use one of the two ways I explained above (in running jenkins manually) to be able to use docker, it could be a bit tricky but possible.
3. run agent as a docker in a different environment & connect remote agent in jenkins
At last it is possible to run the agent separately & connecting the remote agent in jenkins. Although this does not exactly answer your question, but is a way you could use.
These ways for just running a docker in jenkins, you will probably have some issues after you ran a docker as agent, like having permission issues in the agent container itself, which is most likely because of the agent's user (if you like, you could access the user with command
docker exec -it [agent container id] whoami
e.g. in this sample the user in agent is node
agent {
docker { image 'node:14-alpine' }
}
steps{
sh 'npm i -g random'
}
so it would throw an error because the node user does not have permission to install npm module globally (I know, it is weird!)
so as luongnv89 mentioned, you could change the user running the docker like this
agent {
docker { image 'node:14-alpine' args '-u root' }
}
Hope this was helpful understanding the whole picture. 😊
What worked for me was
node() {
String jenkinsUserId = sh(returnStdout: true, script: 'id -u jenkins').trim()
String dockerGroupId = sh(returnStdout: true, script: 'getent group docker | cut -d: -f3').trim()
String containerUserMapping = "-u $jenkinsUserId:$dockerGroupId "
docker.image('image')
.inside(containerUserMapping + ' -v /var/run/docker.sock:/var/run/docker.sock:ro') {
sh "..."
}
}
This way the user in the container still uses the jenkins user id + group id to avoid permissions conflicts with shared data but is also member of the docker group inside container which is required to access the docker socket (/var/run/docker.sock)
I prefer this solution as it doesn't require any additional scripts or dockerfiles
I just had the same exact issue. You need to add jenkins user to docker group:
DOCKER_SOCKET=/var/run/docker.sock
DOCKER_GROUP=docker
JENKINS_USER=jenkins
if [ -S ${DOCKER_SOCKET} ]; then
DOCKER_GID=$(stat -c '%g' ${DOCKER_SOCKET})
sudo groupadd -for -g ${DOCKER_GID} ${DOCKER_GROUP}
sudo usermod -aG ${DOCKER_GROUP} ${JENKINS_USER}
fi
# Start Jenkins service
sudo service jenkins restart
After you run the above, pipelines successfully start docker
I might have found a reasonably good solution for this.
Setup
I run Jenkins as a container and use it to build containers on the dockerhost it's running on. To do this, I pass /var/run/docker.sock as a volume to the container.
Just to reiterate the disclaimer some other people already stated: Giving access to the docker socket is essentially like giving root access to the machine - be careful!
I assume that you've already installed docker into your Jenkins Image.
Solution
This is based on the fact, that the docker binary is not in the first directory of $PATH. We basically place a shell script that runs sudo docker instead of just the plain docker command (and passes the parameters along).
Add a file like this to your jenkins repository and call it docker_sudo_overwrite.sh:
#! /bin/sh
# This basically is a workaround to add sudo to the docker command, because aliases don't seem to work
# To be honest, this is a horrible workaround that depends on the order in $PATH
# This file needs to be place in /usr/local/bin with execute permissions
sudo /usr/bin/docker $#
Then extend your Jenkins Dockerfile like this:
# Now we need to allow jenkins to run docker commands! (This is not elegant, but at least it's semi-portable...)
USER root
## allowing jenkins user to run docker without specifying a password
RUN echo "jenkins ALL=(ALL) NOPASSWD: /usr/bin/docker" >> /etc/sudoers
# Create our alias file that allows us to use docker as sudo without writing sudo
COPY docker_sudo_overwrite.sh /usr/local/bin/docker
RUN chmod +x /usr/local/bin/docker
# switch back to the jenkins-user
USER jenkins
This gives the jenkins service user the ability to run the docker binary as root with sudo (without providing a password). Then we copy our script to /usr/local/bin/docker which "overlays" the actual binary and runs it with sudo. If it helps, you can look at my example on Github.
Same issue here where.
[...]
agent { docker 'whatever_I_try_doesnt_work'} # sudo, jenkins user in dockerroot group etc
[...]
So my workaround is to add it as one of the steps in the the build stage of the pipeline as follow:
pipeline {
agent any
stages {
stage('Build') {
steps {
sh 'sudo docker pull python:3.5.1'
}
}
}
}