Do docker pull using jenkins - jenkins

I would like to do next steps using jenkins:
1- docker pull <image_name>
2- docker run -i -t <command>
I´ve installed docker plugin on jenkins but is it this prossible? The documentations in docker plugin page is very poor .

These steps are executed programmatically by the plugin.
Alternatively you can execute an script into a jenkins slave with docker installed in build->execute shell:
#!/bin/bash
export image=`docker images httpd|wc -l`
echo image $image
if [ "$image" -lt "1" ];
then
docker pull httpd
fi
export container=`docker ps -all -f="name=webcontainer"|wc -l`
echo container $container
if [ "$container" -gt "1" ];
then
echo "Deleting webcontainer"
docker rm -f webcontainer
fi
BUILD_ID=dontKillMe docker run -d -t -p8888:80 --name webcontainer httpd
You can interact with created docker with below command:
`docker exec -it webcontainer /bin/bash`

These days (mid 2017, more than a year after the OP's question), you would use an inside directive of a Jenkins pipeline to pull and run within a docker image some commands.
For instance (Using Jenkins Pipelines with Docker), using the Docker Pipeline plugin:
docker.image('ruby:2.3.1').inside {
stage("Install Bundler") {
sh "gem install bundler --no-rdoc --no-ri"
}
stage("Use Bundler to install dependencies") {
sh "bundle install"
}
}

Related

jenkins pipeline : docker not found [duplicate]

This question already has answers here:
Docker not found when building docker image using Docker Jenkins container pipeline
(7 answers)
Closed 9 months ago.
i wanted to create a docker image with jenkins but docker not found
how can i add jenkins to docker groupe on windows ?
i tried to add docker plugin and didn't work
this is my pipeline
pipeline {
agent any
options { buildDiscarder(logRotator(numToKeepStr:'5'))}
environment {DOCKERHUB_CREDENTIALS = credentials('tfkben-dockerhub')}
stages {
stage('build'){ steps { sh 'docker build -t tfkben/ben:latest .' } }
stage('Login'){ steps { sh 'echo $DOCKERHUB_CREDENTIALS_PSW | docker login -u $DOCKERHUB_CREDENTIALS_USR --password-stdin ' }}
stage('Push'){ steps { sh 'docker push tfkben/ben:latest'} }
}
post { always { sh 'docker logout' }}
}
my Dockerfile :
FROM python:3.11-rc-bullseye
WORKDIR /app
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
COPY . .
CMD [ "python3", "manage.py", "runserver", "0.0.0.0:8000"]
and this is the error message :
docker build -t tfkben/ben:latest .
/var/jenkins_home/workspace/dockerhub-auth_master#tmp/durable-d7adec4b/script.sh: 1: docker: not found
If you try to run Jenkins inside a container instead :
docker run -u 0 --privileged --name jenkins -d -p 8080:8080 -p 50000:50000 -v /var/run/docker.sock:/var/run/docker.sock -v $(which docker):/usr/bin/docker -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts
Of course you could replace $(which docker) directly by your docker path if your host machine doesn't recognize the command.
You should be able to run docker command inside your pipeline.

Docker mount volume in Jenkins Docker container

I am following the Jenkins tutorial with some modification.
I run the Jenkins docker container by:
docker run --rm --privileged -u root -p 8080:8080 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$PWD"/vol:/var/jenkins_home \
jenkinsci/blueocean
With my Jenkinsfiles:
stage('Test') {
agent {
docker {
image 'qnib/pytest'
}
}
steps {
sh 'ls' ##### 1
sh 'py.test --junit-xml test-reports/results.xml sources/test_calc.py' ##### 2
}
}
stage('Deliver') {
agent any
environment {
VOLUME = '$(pwd)/sources:/src'
ABS_WS = '/home/myname/vol/workspace'
JOB_WS = "\${PWD##*/}"
IMAGE = 'cdrx/pyinstaller-linux:python2'
}
steps {
dir(path: env.BUILD_ID) {
unstash(name: 'compiled-results')
sh "pwd" ##### 3
sh "ls" ##### 4
sh "docker run -v '${ABS_WS}/${JOB_WS}/sources:/src' ${IMAGE} 'ls'" ##### 5
sh "docker run -v ${ABS_WS}/${JOB_WS}/sources:/src ${IMAGE} 'ls'" ##### 6
sh "docker run -v ${VOLUME} ${IMAGE} 'ls'" ##### 7
}
}
}
The output and my questions for ####1~6:
####1: ls here including the /sources/*.py that docker container(qnib/pytest) can process.
####3: output: /var/jenkins_home/workspace/simple-python-pyinstaller-app/32
####4: ls here also including the /soucres/*.py we need
####5: ls here didn't include /sources/*.py, due to docker volume mounted failed.
I already tried with different solution from here, still not working.
docker run -v '/home/myname/vol/workspace/${PWD##*/}/sources:/src' cdrx/pyinstaller-linux:python2 ls
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
ls
add2vals.spec
build
dist
BUT ####6, similar to ####5 just without Single quotation, nothing output from ls (WHY?):
docker run -v /home/myname/vol/workspace/32/sources:/src cdrx/pyinstaller-linux:python2 ls
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
ls
####7. the output is identical to ####5
docker run
-v /var/jenkins_home/workspace/simple-python-pyinstaller-app/32/sources:/src cdrx/pyinstaller-linux:python2 ls
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
ls
add2vals.spec
build
dist
My questions are:
In Deliver stage, how can I map docker container volume to the host or Jenkins container?
In ####3,4 the path in Jenkins container is /var/jenkins_home/workspace/simple-python-pyinstaller-app/32 , this path including the /sources/*.py; and #####7 we can see /var/jenkins_home/workspace/simple-python-pyinstaller-app/32/sources:/src, I thought it was mounted on the correct path to /src in pyinstaller-linux container.
I am not very clear why in Test stage we don't need to mount any volume when running pytest docker?
And why not Deliver stage going the same way as Test stage? (like ####2)
What is difference between ####6 and ####5 ?

Run host Docker from within Jenkins Docker

Is it possible to create and run Docker containers for CI/CD from within a running Jenkins Docker Container? So basically access Docker on the host server from within a running container.
On my host server (Ubuntu 19.04) Docker (Docker version 19.03.3) is installed. By running the following command I create a Jenkins Container that I give permissions to Docker (so I thought):
mkdir /home/myuser/Desktop/jenkins_home
docker run -dit --name jenkins -v /home/myuser/Desktop/jenkins_home:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock -p 8080:8080 jenkins/jenkins:lts
Within Jenkins I create a Pipeline that loads a Jenkinsfile from Git that looks like this:
pipeline {
agent {
docker {
image 'ubuntu:19.04'
args '-u root:sudo -p 3000:3000'
}
}
stages {
stage('Install') {
steps {
sh 'apt-get update'
sh 'apt-get install -y curl'
sh 'curl -sL https://deb.nodesource.com/setup_13.x | sh -'
sh 'curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -'
sh 'echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list'
sh 'apt-get update'
sh 'apt-get install -y nodejs yarn'
}
}
stage('Build') {
steps {
sh './build.sh'
}
}
}
}
When I run the Pipeline it crashes when trying to instruct Docker to pull the ubuntu:19.04 Docker image. The error is docker: not found.
Somewhere a connection between my Jenkins Container and the host Docker access files is misconfigured. What configuration is necessary to run Docker commands on the host server from within the Docker Container?
If you want to create and run Docker containers for CI/CD from Jenkins container,
This can be achieved creating a shell command on Jenkins job that runs an ssh command on Docker host.
This needs as requirements that Jenkins container ssh public key is authorized on Docker host, so authorized_keys file should exist on Docker host.
To use the same ssh keys inside Jenkins container can be used a bind mount with ssh keys on Jenkins containers.
Example using docker-compose:
volumes:
- /home/user/.ssh/id_rsa:/var/jenkins_home/.ssh/id_rsa
- /home/user/.ssh/id_rsa.pub:/var/jenkins_home/.ssh/id_rsa.pub
This is an example content of a shell command used to launch and update containers on Docker host from a Jenkins job:
cat ./target/stack/installer-*.tar | ssh root#${DOCKER_HOST} \
/home/user/Build-Server/remote-installer.sh
In the command above an installer is launched on Docker host. As result new containers are deployed/updated on Docker host.
The remote-installer.sh script receive the file from standard input and unpack it using tar command.
TEMPDIR=`mktemp -d`
echo "unarchiving to $TEMPDIR"
tar xv -C "$TEMPDIR"
...
This works for both cases having Docker containers on same server as Jenkins container or having Docker containers and Jenkins container on different servers.

Unable to run systemd inside docker which is being run inside jenkins

I'm trying to get Jenkins to run Docker that runs SystemD.
So far I've been able to run systemd inside docker locally without Jenkins. Here are the steps to run it locally without jenkins:
# pull unop/fedora-systemd and create and run the container for it
sudo docker run --cap-add=SYS_ADMIN -e container=docker --tmpfs /run --tmpfs /tmp -v /sys/fs/cgroup:/sys/fs/cgroup:ro -t -i unop/fedora-systemd
# on a different terminal window, I can:
# get the container id of the "unop/fedora-systemd" image
sudo docker ps
# then exec bash on it
sudo docker container exec -t -i a98aa2bcd19e bash # where a98aa2bcd19e is the container id found above
# once inside the container, I can run systemd without any problems. examples:
systemctl status
systemctl start dbus.service
systemctl status dbus.service
The above works locally and I am able to run systemd inside the docker container.
The problem I get is when I try the same thing, but inside Jenkins.
I've tried to tweak Jenkinsfile several times, but not of my previous tries seemed to work. I always get an error when running under Jenkins similar to this:
+ systemctl status
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
This is my latest Jenkinsfile that I've tried
pipeline {
agent {
docker {
image 'unop/fedora-systemd'
args '--cap-add=SYS_ADMIN -e container=docker --tmpfs /run --tmpfs /tmp -v /sys/fs/cgroup:/sys/fs/cgroup:ro -t -i'
}
}
stages {
stage('test') {
steps {
sh "echo hello world"
sh "systemctl status"
sh "systemctl start dbus.service"
sh "systemctl dbus.service"
}
}
}
}
On previous iterations of the Jenkinsfile, I've tried to replace -cap-add=SYS_ADMIN -e container=docker for --privileged, but that didn't help, I still got the same errors
Anyone have an idea of how can I get this to work? Why does the above work locally, but not on Jenkins? what am I missing here?
Note: Jenkins version: 2.150.2 and this is the Dockerfile used by unop/fedora-systemd
FROM fedora:rawhide
MAINTAINER http://fedoraproject.org/wiki/Cloud
ENV container docker
RUN dnf -y update && dnf clean all
RUN dnf -y install systemd && dnf clean all && \
(cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup", "/tmp", "/run" ]
CMD ["/usr/sbin/init"]
PS: I've seen a related question, but what they were asking is different
I did not know about the related question. Let me point out again that you do not need to run a systemd daemon in a systemd controlled container if it is just about running multiple services in it. Simply overwrite /usr/bin/systemctl with the docker-systemctl-replacement script. Then go to register it with CMD ["/usr/bin/systemctl"] as the init process of the container.
That's it. Now you can run any systemctl-start process from the operating system. It works to the extent that even provisioning with ansible/puppet scripts have no problem at all. An specficially, I am using that to provision Jenkins images with the operating system that the developers like to have as a basis. No priviledged mode required.
You may try an image that has Fedora with System D already active with this command:
docker run -d --name systemd-fedora --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:ro jrei/systemd-fedora
Then you just need to run:
docker exec -it systemd-fedora /bin/bash
and there you can just install, start and restart any service you need.

Jenkins docker - running a container, executing shell script etc

I'm trying to run docker containers in the Jenkins Pipeline.
I have the following in my Jenkinsfile:
stage('test') {
steps {
script {
parallel (
"gatling" : {
sh 'bash ./test-gatling.sh'
},
"python" : {
sh 'bash ./test-python.sh'
})
}
}
}
In the test-gatling.sh I have this:
#!/bin/bash
docker cp RecordedSimulation.scala denvazh/gatling:/RecordedSimulation.scala
docker run -it -m denvazh/gatling /bin/bash
ls
./gatling.sh
The ls command is there just for test, but when it's executed it lists files and folders of my github repository, rather than the files inside the denvazh/gatling container. Why is that? I thought the docker run -it [...] command would open the container so that commands could be run inside it?
================
Also, how do I run a container and just have it running, without executing any commands inside it? (In the Jenkins Pipeline ofc)
I'd like to run: docker run -d -p 8080:8080 -t [my_container] and access it on port 8080. How do I do that...?
If anyone has the same or similar problem, here are the answers:
Use docker exec [name of container] command, and to run any terminal commands inside a container, add /bin/bash -c "[command]"
To be able to access a container/app that is running on any port from a second container, when starting the second container run it with --net=host parameter

Resources