I know my issue is already discussed in How to run shell script on host from docker container? but i think my issue is a littel bit more complicated.
At first I try to explain my situation. I'm using jenkins 2.x from a docker container in CentOS VM (Host). In jenkins i created a Job which checks out 3 files from SVN (2 Shell scripts and 1 .jar file). these files will be downloaded in jenkins workspace in jenkins docker container and also on host in a mounted directory like that:
volumes:
- ${DATA_HOME}/jenkins/data:/var/jenkins_home
One of these scripts will be executed from jenkins job and that executes the other script. The second script checks out a SVN directory and does much more stuffs.
So I want a new mounted volume in that directory all results of executed second script will be placed on Host. I think to connect to the host over 'SSH' and execute the script seems to be fine but how can i do that.
I hope I could explain my issue understandable
I will answer regarding "I think to connect to the host over 'SSH' and execute the script seems to be fine but how can i do that"
Pass Host machine Ip to your run command.
docker run --name redis --env pass=pass_my --add-host="hostmachine:192.168.1.23" -dit redis
Now,
docker exec -it redis ash
and run this command. This will do SSH from the container to host
ssh user_name#hostmachine 'ls; bash /home/user_name/Desktop/test.sh; docker run --name db -dit db; docker ps'
If you want something without password then set ssh-key in a container or you can also try
sshpass -p $pass ssh user_name#hostmachine 'ls;/home/user_name/Desktop/test.sh; docker run --name db -d
it db; docker ps'
or if you want to run the script that is inside container you can also do that just pass the script to ssh.
sshpass -p $pass ssh user_name#hostmachine < ./ab.sh
Note: $pass is password of host from ENV and hostmachine is host the we set during run command.
Based on comments in ans:
We can simply install any SSH plugin (SSH) or (Publish over SSH) and
it will work after providing username/password.
Only thing to watch out is that host name resolution does not work and we will need to provide an IP address.
As pointed out this is not the best approach, but sometimes in migration from older systems, we need to move one step at a time and this is the easiest step to take.
Related
The Problem:
Let's say you need to be able to create containers in your host from inside a container, Why?!!! Imagine you have your "continuous everything" process automated in a Jenkins Pipeline and this process includes creation of container or services for testing.
Even Though container and virtual machines enforces isolation from the host, this is a valid scenario.
The solution:
Sorry WinTel guys, did you expect this answer includes Windows?... Well just a clue, you can enable tcp://localhost:2375
Coming back to production grade answer, follow the next steps:
Spin up your instance binding "/var/run/docker.sock" from your host to your container:
docker container run --name container -v /var/run/docker.sock:/var/run/docker.sock image
docker.sock as any file exposes its user id and group id, any user having as group "docker" is allowed to "talk" with docker using the client, so run the following script:
#!/usr/bin/env bash
DOCKER_SOCKET=/var/run/docker.sock
DOCKER_GROUP=docker
if [ -S ${DOCKER_SOCKET} ]; then
DOCKER_GID=$(stat -c '%g' ${DOCKER_SOCKET})
groupadd -for -g ${DOCKER_GID} ${DOCKER_GROUP}
usermod -aG ${DOCKER_GROUP} youruser
fi
Don't freak out, this won't harm your system, basically, if the file (socket)
docker.sock exists (as it should), the script will get it group id, will create a group call
docker and will set the same group id as the docker's group one in the host
(confused?!?!, remember that we are inside the container we want to have access
to host docker, we executed "docker container exec -it -u root container bash"
in order to access the container), then, the user called "youruser" will be
modified by being added to "docker" group.
(Almost there!!!) Install docker client inside your container, use your
favorite package manager and install the docker client, I have the same version
of client and server and works like a charm but I suppose I could work with
other versions but come on!! mixing versions??? seriously???
After following these steps, you will be able to run docker commands using the common process, just remember that it is possible to do anything!!! even shooting you in the foot!!!
I'm new to Jenkins and Docker both. I am currently working on project where I allow users to submit jobs to jenkins. And I was wondering if there is a way to use docker to dynamically spun-up Jenkins server and redirect all the user jobs coming from my application to this server and then destroy this jenkins once the work is done. Is it possible? if yes, how ? if no, Why? Also, I need to setup maven for this jenkins server, do I need another container for that?
You can try the following but I can't guarantee if it's that easy to move your jenkins content from your dedicated server to your docker container. I did not try it before .
But the main thing is the following.
Create a backup of the content of your dedicated jenkins server using
tar
Create a named docker volume:
$ docker volume create --name jenkins-volume
Extract your jenkins-backup.tar inside your docker volume. This
volume is in /var/lib/docker/volumes/jenkins-volume/_data
$ sudo tar -xvpzf jenkins-backup.tar -C /var/lib/docker/volumes/jenkins-volume/_data/
Now you can start your jenkins container and telling it to use the jenkins-volume. Use the jenkins-image which is the same version as the dedicated jenkins.
$ docker run -d -u jenkins --name jenkins -p 50000:50000 -p 443:8443 -v jenkins-volume:/var/jenkins_home --restart=always jenkins:your-version
For me this worked to move the content from our jenkins container on AWS to a jenkins container on another cloud provider. I did not try it for a dedicated server. But you can't break anything of your existing jenkins when you try it.
I'd like to package Selenium grid exrtas into a docker image.
This service being run without using docker container can reboot the OS it's running in. I wonder if I can setup the container to restart by Selemiun grid extras service running inside the container.
I am not familiar with Selenium Grid, but as a general idea: you could mount a folder from the host as data volume, then let Selenium write information to there, like a flag file.
On the host, you have a scheduled task / cronjob running on the host that would check for this flag in the shared folder and if it has a certain status, you would invoke a docker restart from there.
Not sure if there are other more elegant solutions for this, but this is what came to my mind adhoc.
Update:
I just found this on the Docker forum:
https://forums.docker.com/t/how-can-i-run-docker-command-inside-a-docker-container/337
I'm not sure about CoreOS but normally you can manage your host
containers from within a container by mounting the Docker socket.
Such as
docker run -it -v /var/run/docker.sock:/var/run/docker.sock ubuntu:latest sh -c "apt-get update ; apt-get install docker.io -y ;
bash"
or
https://registry.hub.docker.com/u/abh1nav/dockerui/
In Docker i have installed Jenkins successfully. When i create a new job and i would like to execute a sh file from my workspace, what is the best way to add a file to my workspace with Docker? I started my container with this: docker run --name myjenkins -p 8080:8080 -p 50000:50000 -v /var/jenkins_home jenkins
You could copy a file from your file system to the container with a simple command from your terminal.
docker cp [OPTIONS] LOCALPATH|- CONTAINER:PATH
https://docs.docker.com/engine/reference/commandline/cp/
example:
docker cp /yourpaht/yourfile <containerId>:/var/jenkins_home
It depends a bit on how the planned lifecycle of your Jenkins container is. If it is just used temporarily and does no harm if the data is gone, docker cp as NickGnd suggested will do the trick.
But since the working data of Jenkins like jobconfigs, system configs and workspaces will only live inside the container, all of it will be gone once the container is removed, so if you plan to have a longer running Jenkins environment, you might want to persist the data outside of the container so it will survive recreating the container, launching new container versions and so on. This can be done with the option --volume /path/on/host:/path/in/container or its short form -v on docker run.
There is also the option of --volumes-from which you can use to mount to keep the data in one "data container" and mount it into your Jenkins container.
For further information on this, please have a look at The docker volumes documentation
I'm trying to run Docker inside a Jenkins container that is also running in Docker (i.e. Docker in Docker). What I want to know is how to properly start the Docker service when booting Jenkins. The only solution I've found today is to build my own Jenkins image based on the official Jenkins image but change the jenkins script loaded by the entry point to also start up Docker:
# I've added this line just before Jenkins is started from the script:
sudo service docker start
# I've also removed "exec" from the original file which used "exec java $JAVA_TOPS ..." but that didn't work
java $JAVA_OPTS -jar /usr/share/jenkins/jenkins.war $JENKINS_OPTS "$#"
This works when I run (using docker run) a new container but the problem is that if I do (docker start) on stopped container the Docker service is not started.
I strongly suspect that this is not the right way to start my Docker service. My plan is to perhaps use supervisord to start Jenkins and Docker separately (I suppose container linking is out of the question since Docker should be executed as a service on the same container that Jenkins is running on?). My concern with this approach is that I'm going to lose the EntryPoint specified in the Jenkins Dockerfile which allows me to pass arguments to the Jenkins container when starting the container, for example:
docker run -p 8080:8080 -v /your/home:/var/jenkins_home jenkins -- <jenkins_arguments>
Does anyone have any recommendations on a good way to solve this preferably by not forking the official Jenkins image?
I'm pretty you cannot do that.
Docker in Docker doesn't mean you have to run docker inside docker with 3 level : host > First level container > Second Level Container
In fact, you just need to share docker with host, and this is your host who will run others containers.
To do that, you have to mount volume with -v parameter
-v /var/run/docker.sock:/var/run/docker.sock
with this command, when you will docker run inside you jenkins container, the docker client will communicate with docker deamon from your host in order to run new container.
To do that, you should run your jenkins container with privileged
--privileged
To resume, here is the full command line
docker run -d -v /var/run/docker.sock:/var/run/docker.sock --privileged myimage
And you you don't need to create a new jenkins image for that.
Hoping to have helped you
http://container-solutions.com/running-docker-in-jenkins-in-docker/