I am currently experimenting with Docker in combination with Jenkins to streamline the CI/CD workflow for a new project. I do so on a Mac with Docker 1.12 installed.
This is what I do:
Use docker machine to create a new Docker server
Use the official Jenkins Docker image to spin up a Jenkins instance on that server
Install the "Yet Another Docker Plugin" and "CloudBees Docker Pipeline" plugins.
Add a "Docker Cloud" using the IP of the Docker server above and the third party Docker DinD image tehranian/dind-jenkins-slave
With this setup, I run a very simple pipeline job like this:
node('docker') {
docker.image('hseeberger/scala-sbt').inside {
stage 'Checkout'
echo 'We got here!'
}
}
Jenkins spins up a Docker instance as expected and executes the job. So the basic Docker setup is working as expected.
But the Docker command within the job fails. Log output looks something like this:
[Pipeline] node
Still waiting to schedule task
Docker-23ebf3d8dd4f is offline
Running on Docker-23ebf3d8dd4f in /home/jenkins/workspace/docker-test
[Pipeline] {
[Pipeline] sh
[docker-test] Running shell script
+ docker inspect -f . hseeberger/scala-sbt
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
[Pipeline] sh
[docker-test] Running shell script
+ docker pull hseeberger/scala-sbt
Using default tag: latest
Warning: failed to get default registry endpoint from daemon (Cannot connect to the Docker daemon. Is the docker daemon running on this host?). Using system default: https://index.docker.io/v1/
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Now when I browse around for solutions, it is usually mentioned that the Docker socket needs to be provided to the container as a volume, but that doesn't seem to work either.
Since the general setup seems to be working, wouldn't the slave simply have to do the same thing as the Jenkins plugin does to spin up the Docker slave in the first place? That is, use the URL of the Docker server to control it? Since I assume this is an extremely common use-case, there must be a Docker image for Jenkins Docker slaves that can do this out of the box, right? What am I missing?
You might need to set the docker env and use the content of docker-machine env node in your running shellscript.
Related
This question already has answers here:
Docker not found when building docker image using Docker Jenkins container pipeline
(7 answers)
Closed 1 year ago.
I've added the BitBucket server integration plugin (https://plugins.jenkins.io/atlassian-bitbucket-server-integration/) and can connect to the BitBucket cloud repo from Jenkins:
But I receive an error when I try to build:
/var/jenkins_home/workspace/bb_add-jenkins-file#tmp/durable-c49dbeca/script.sh: 1: /var/jenkins_home/workspace/bb_add-jenkins-file#tmp/durable-c49dbeca/script.sh: docker: not found
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
[Bitbucket] Notifying commit build result
[Bitbucket] Build result notified
ERROR: script returned exit code 127
Finished: FAILURE
So it seems I need to install Docker on the Jenkins instance ?
https://plugins.jenkins.io/docker-build-publish/
I'm following this tutorial to configure Docker with Jenkins: https://medium.com/#karthi.net/docker-tutorial-build-docker-images-using-jenkins-d2880e65b74
and have reached this step:
On my own Jenkins Docker setup page I have :
I'm unsure what Docker URL should be used? Do I need to provision a new container instance within the Kubernetes cluster and run docker within this new instance? This new Docker instance is then the Docker Host URI field?
I think that plugin requires the docker cli to be present.
If you run jenkins as a docker image itself, use an image that provides the docker cli, for example https://hub.docker.com/r/trion/jenkins-docker-client
If you want to use the host docker daemon for building, you need to bind-mount the docker socket.
If you want to use a sidecar container to provide the docker daemon, for example using a docker-in-docker setup you can usually use the container name as docker host or kubernetes service name. This depends on how you provide the sidecar container and there is no general answer to that.
I have a jenkins pipeline script which creates the Docker image and deploys it to the docker hub. I have installed the docker plugin but it complains about "docker command not found". I am not sure if I need to install docker in the same machine or something else need to happen?
Yes you have to install docker on the slave machine which is running that pipeline script of docker plugin. I would suggest adding a label docker to the slave that has docker installed and then use the pipeline script as:
node('docker') {
...
}
I'm using Docker Pipeline Plugin version 1.10.
I have my Jenkins installed in a container. I have a remote server that runs a Docker daemon. The daemon is reachable from the Jenkins machine via TCP (tested). I disabled TLS security on the Docker daemon.
I'm not able to make the docker.withServer(...) step work.
As a basic test I simply put following content in a Jenkinsfile (if I'm correct this is a valid pipeline content):
docker.withServer('tcp://my.docker.host:2345') {
def myImage = docker.build('myImage')
}
When the pipeline executes I get this error: script.sh: line 2: docker: command not found like the docker command was still trying to execute locally (there is no docker command installed locally) rather than on my remote Docker daemon.
Am I missing anything ? Is it required to have the docker command installed locally when trying to execute Docker commands on a remote server..?
have you tried
withDockerServer('tcp://my.docker.host:2345') {
.....
}
Documentation here
docker needs to be installed on jenkins master in order for jenkins to be able to launch the docker on my.docker.host.
the first docker command runs on jenkins master, but with a parameter to pass the command to my.docker.host
the container itself will then run on my.docker.host
Note that you only need to install docker on the jenkins master; the daemon does not need to be running on jenkins master.
Check if you have set up port correctly. Default port for daemon is 2375. It has to be checked on both docker daemon (option -H 0.0.0.0:2375) and on the jenkins client
I've got a jenkins declarative pipeline build that runs gradle and uses a gradle plugin to create a docker image. I'm also using a dockerfile agent directive, so the entire thing runs inside a docker container. This was working great with jenkins itself installed in docker (I know, that's a lot of docker). I had jenkins installed in a docker container on docker for mac, with -v /var/run/docker.sock:/var/run/docker.sock (DooD) per https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/. With this setup, the pipeline docker agent ran fine, and the docker build command within the pipeline docker agent ran fine as well. I assumed jenkins also mounted the docker socket on its inner docker container.
Now I'm trying to run this on jenkins installed on an ec2 instance with docker installed properly. The jenkins user has the docker group as its primary group. The jenkins user is able to run "docker run hello-world" successfully. My pipeline build starts the docker agent container (based on the gradle image with various things added) but when gradle attempts to run the docker build command, I get the following:
* What went wrong:
Execution failed for task ':docker'.
> Docker execution failed
Command line [docker build -t config-server:latest /var/lib/****/workspace/nfig-server_feature_****-HRUNPR3ZFDVG23XNVY6SFE4P36MRY2PZAHVTIOZE2CO5EVMTGCGA/build/docker] returned:
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
Is it possible to build docker images inside a docker agent using declarative pipeline?
Yes, it is.
The problem is not with Jenkins' declarative pipeline, but how you're setting up and running things.
From the error above, looks like there's a missing permission which needs to be granted.
Maybe if you share what your configuration looks like and how your're running things, more people can help.
In my Manage Jenkins > Global Tool Configuration, i have already configured a tool called "docker" as follows:
name: docker
install automatically: CHECKED
docker version: latest
Then all I have in my jenkinsfile is the following and nothing else:
node {
DOCKER_HOME = tool "docker"
sh """
echo $DOCKER_HOME
ls $DOCKER_HOME/bin/
$DOCKER_HOME/bin/docker images
$DOCKER_HOME/bin/docker ps -a
"""
}
I get an error like this "Cannot connect to the Docker daemon. Is the docker daemon running on this host?".
Following is the full console log:
Started by user Syed Rakib Al Hasan
[Pipeline] node
Running on master in /var/jenkins_home/workspace/helloDocker
[Pipeline] {
[Pipeline] tool
[Pipeline] sh
[helloDocker] Running shell script
+ echo /var/jenkins_home/tools/org.jenkinsci.plugins.docker.commons.tools.DockerTool/docker
/var/jenkins_home/tools/org.jenkinsci.plugins.docker.commons.tools.DockerTool/docker
+ ls /var/jenkins_home/tools/org.jenkinsci.plugins.docker.commons.tools.DockerTool/docker/bin/
docker
+ /var/jenkins_home/tools/org.jenkinsci.plugins.docker.commons.tools.DockerTool/docker/bin/docker images
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
How do i ensure that the docker daemon/service is running/started before my pipeline reaches the line to run docker commands.
Is there any other native docker-build-step plugin way to achieve what I am doing here? Like docker ps -a or docker images or docker build -t?
Some assumptions:
Let's say my chosen node do not already have docker/docker-engine installed/running in my host machine. That's the purpose of the tool command to automatically install docker in the node if it is not already there.
This Jenkins plugin is for the docker client; I'd solve (work around) by:
setting up jenkins slaves where docker daemon is reachable, add a label
setting up a housekeeping job which will fail if docker daemon was not reachable (so we can notify the infra team without having the QA to figure out and escalate the problem)
assign jobs which assumes the docker daemon to be reachable to this label
I hope it helps, and I'm curious if any of you have a better solution!