Here's the situation:
I have a docker container (jenkins). I've mounted the sockets to my container so that I can perform docker commands inside my jenkins container.
Manually, everything works in the container. However, when Jenkins executes the job, it doesn't "wait" for the docker exec command to run to completion.
Below, is an extract from the Jenkinsfile. The short-lived printenv command runs correctly, and prints the environment variables. The next command (python) just gets run and then Jenkins moves on immediately, not waiting for completion. The Jenkins agent (slave) is running on an Ubuntu image. Running all these commands outside Jenkins work as expected.
echo "Running the app docker container in detached tty mode to keep it up"
docker run --detach --tty --name "${CONTAINER_NAME}" "${IMAGE_NAME}"
echo "Listing environment variables"
docker exec --interactive "${CONTAINER_NAME}" bash -c "printenv"
echo "Running test coverage"
docker exec --interactive "${CONTAINER_NAME}" bash -c "python -m coverage run --source . --branch -m pytest -vs"
It seems maybe related to this question.
Please can anyone explain how to get Jenkins to wait for the docker exec command to complete before proceeding to the next step.
Have considered alternatives, like the Docker Pipeline Plugin, but would much prefer to use something close to what I have above where possible.
Ok, another approach, I've tried using Docker Pipeline plugin here.
You can use docker.sock as volume mount to orchestrate containers on your host machine like this in your docker-compose.yml
volumes:
- /var/run/docker.sock:/var/run/docker.sock
Depending on your setup you might need to run
chmod 666 /var/run/docker.sock
to get going in the first place.
This works on macOS as well as Linux.
Ugh. This was down to the way that I'd set up docker support on the slave container.
I'd used socat to provide a TCP server proxy. Instead, switched that out for a plain old docker.sock volume between host & container.
volumes:
- /var/run/docker.sock:/var/run/docker.sock
The very first time, I had to also sort out a permissions issue by doing (inside the container):
rm -Rf ~/.docker
chmod 666 /var/run/docker.sock
After that, everything "just worked". Very painful experience.
Related
Can I execute a local shell script within a docker container using docker run -it ?
Here is what I can do:
$ docker run -it 5ee0b7440be5
bash-4.2# echo "Hello"
Hello
bash-4.2# exit
exit
I have a shell script on my local machine
hello.sh:
echo "Hello"
I would like to execute the local shell script within the container and read the value returned:
$ docker run -it 5e3337440be5 #Some way of passing a reference to hello.sh to the container.
Hello
A specific design goal of Docker is that you can't. A container can't access the host filesystem at all, except to the extent that an administrator explicitly mounts parts of the filesystem into the container. (See #tentative's answer for a way to do this for your use case.)
In most cases this means you need to COPY all of the scripts and support tools into your image. You can create a container running any command you want, and one typical approach is to set the image's CMD to do "the normal thing the container will normally do" (like run a Web server) but to allow running the container with a different command (an admin task, a background worker, ...).
# Dockerfile
FROM alpine
...
COPY hello.sh /usr/local/bin
...
EXPOSE 80
CMD httpd -f -h /var/www
docker build -t my/image .
docker run -d -p 8000:80 --name web my/image
docker run --rm --name hello my/image \
hello.sh
In normal operation you should not need docker exec, though it's really useful for debugging. If you are in a situation where you're really stuck, you need more diagnostic tools to be understand how to reproduce a situation, and you don't have a choice but to look inside the running container, you can also docker cp the script or tool into the container before you docker exec there. If you do this, remember that the image also needs to contain any dependencies for the tool (interpreters like Python or GNU Bash, C shared libraries), and that any docker cpd files will be lost when the container exits.
You can use a bind-mount to mount a local file to the container and execute it. When you do that, however, be aware that you'll need to be providing the container process with write/execute access to the folder or specific script you want to run. Depending on your objective, using Docker for this purpose may not be the best idea.
See #David Maze's answer for reasons why. However, here's how you can do it:
Assuming you're on a Unix based system and the hello.sh script is in your current directory, you can mount that single script to the container with -v $(pwd)/hello.sh:/home/hello.sh.
This command will mount the file to your container, start your shell in the folder where you mounted it, and run a shell:
docker run -it -v $(pwd)/hello.sh:/home/hello.sh --workdir /home ubuntu:20.04 /bin/sh
root#987eb876b:/home ./hello.sh
Hello World!
This command will run that script directly and save the output into the variable output:
output=$(docker run -it -v $(pwd)/hello.sh:/home/test.sh ubuntu:20.04 /home/hello.sh)
echo $output
Hello World!
References for more information:
https://docs.docker.com/storage/bind-mounts/#start-a-container-with-a-bind-mount
https://docs.docker.com/storage/bind-mounts/#use-a-read-only-bind-mount
I am new to docker container. Can someone please tell me what is difference between these two commands. In my knowledge, have the same out put than why we use the bash command.
docker run -it ubuntu
docker run -it ubuntu bash
In docker, we run a linux container. As you know, a linux system is alive when it's init 0 service is alive. 'init 0' is kind of the heart of a linux system. when 'init 0' is killed, the linux system also dies.
In a containerized architecture, you run a container for simply one purpose i.e. to simply run one service. we want if the service fails, the container also dies. so we define the servcie as init 0 job for the container.
when you run docker run -it ubuntu bash, here, bash is the init 0 job for the container. As soon as you exit from bash, the container stops working.
Instead of using bash you can also try another commands like #Shmuel suggested.
Well, when we create custom images, often we want to pre-define default 'init 0' job for our custom image. If the init 0' is predefined, you don't need to mention it in docker run command.
In ubuntu image, the pre-defined 'init 0' job is bash. So, if you don't mention bash in the run command, it works the same.
docker run -it ubuntu let's you run command inside the container.
The bash is the command to run.
For example instead you can run
docker run -it ubuntu ls /home
This will list the /home dir inside the container.
I started to experiment with the docker but have some questions regarding how to develop on it and regarding its use cases. If anyone could guide me through these questions, it will be much appreciated.
First,
As far as I understood, docker is used mainly for developing applications on custom environments, thus avoiding the tidious installation processes. This is initially my intention, why I'd like to use docker for.
I've created a docker file which builds successfuly, and which has basic C++ development tools based upon library/gcc. I want to be able to develop in this docker container as you would do on your terminal.
What I did is I created a docker image from a Dockerfile. (I can observe that it is successfully created)
docker build -t mydockerimage .
Then run the docker in detached mode.
docker run -d mydockerimage
At this point, I am notified with the ID of the docker container. However docker container does not seem to be running when I check the output of:
docker container ls
Here comes the first question, why is my docker container not running?
To my understanding, simplest way to interact with the docker container is as follows:
docker exec -it <container_id_or_name> echo "Hello from container!"
Is this true? Is this a use case of docker in which I simply can start the container and exec some Linux command on it?
Moreover, I get a permission denied on /var/lib/docker.sock when I try to execute docker commands without sudo. What am I missing here?
Thank you in advance.
Do you provide an entrypoint or CMD in your dockerfile? This will be executed inside your container and keeps the container running. You can find some details here.
In short. Docker has a default entrypoint: /bin/sh -c, but no default CMD.
Check the dockerfile of ubuntu. This has bash as CMD so it's executing /bin/sh -c bash.
$ docker run -it ubuntu bash
root#9855e779cab2:/#
This will result in an interactive shell in which you can execute commands like on an ubuntu. If you exit the container the container will stop running.
To keep a container running you can use the -d option. It will run the container in the background as a daemon:
$ docker run -d -it ubuntu bash
2606ad8e095baa0237cc30e599a26a4d727d99d47392d779fb83cd50f1a39614
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2606ad8e095b ubuntu "bash" 18 seconds ago Up 17 seconds cranky_johnson
Now you can exec inside the container to "go inside" the container and execute ubuntu commands.
$ docker exec -it 2606ad8e095b bash
root#2606ad8e095b:/#
When you exit the container it remains running in the background.
Now we can execute your command too:
$ docker exec -it 2606ad8e095b echo "Hello from container!"
Hello from container!
This will open a bash session in your container and echo the string.
I think it's important in your case you define some entrypoint (which can also be a script) or a CMD. Probably you need something very similar to Ubuntu when you just want to use bash inside your container.
Moreover, I get a permission denied on /var/lib/docker.sock when I try to execute docker commands without sudo. What am I missing here?
This is normal. The Docker daemon currently requires root privileges. So you have to use docker with your root user or users which have root priviledges and you have to add sudo every time. You can add your user to a docker group. Every time the daemon starts, it makes the ownership of the Unix socket read/writable by the docker group. This means you can use docker without using sudo everytime when that user is inside your docker group.
To add your user to the docker group:
$ sudo groupadd docker
$ sudo usermod -aG docker $USER
$ exit
ssh back or open new shell
I need to set environment variable in a running docker container. I am already aware of the way of setting environment variable while creating a container. As far I found there is no available straight forward way to do this with docker and docker is planning to add something with new version 1.13.
But I found that some people able to manage it which is not working for me now. I tried following ways but did not work for me-
docker exec -it -u=root test /bin/bash -c "export port=8090"
echo "export port=8090" to /etc/bash.bashrc using a script and then source it
docker exec -it test /bin/bash -c "source /etc/bash.bashrc"
configuring the whole thing in a script and run it from host also did not work. While running script from host all the other command successfully executes except "export port=8090" or "source /etc/bash.bashrc" or "source /root/.bashrc".
Can anyone explain why sourcing file from host does not work in docker container even when I set user("-u=root")? Can anyone help me to solve this? When I source the file from inside the container it works perfectly. But in my case I have to do it from host machine
NOTE:, I am using docker 1.12 and tried the above in ubuntu:16.04 and ubuntu:14.04
If you have a running process in the docker and you are attempting to change the environment variable in the docker so the running process will dynamically change - this will not work. The environment variables of a process are set when it starts. You can see here ways to overcome that, but I don't think that is the right way to go.
I would instead, have a configuration file that the file reads (or listens to) periodically. And when you want to change the configuration change the file.
If this isn't your scenario, please describe your scenario so we can better assist you.
I find a way to provide environment variable to a running container. Fist upgrade your docker-engine. I am using V1.12.5.
create a script with environment variables-
#!/bin/bash
echo "export VAR1=VAL1
export VAR2=VAL2" >> /etc/bash.bashrc
source /etc/bash.bashrc
Now start a container. Here, 'test' is the container name:
docker run -idt --name=test ubuntu
Copy your script to container:
docker cp script.sh test:/
Run the script :
docker exec -it test /bin/bash -c "/script.sh"
Restart your container:
docker restart test
Go to container shell
docker exec -it test /bin/bash
Check the variable
echo $VAR1
A jar need to deploy in docker.I know how to write Dockerfile for a running jar.
this jar is a commandline option application.it has serveral arguments.and will be needed to run serveral times with different arguments.
for example. It has arg1,arg2.
User can run with arg1=A,arg2=B then run with arg1=A2. No arg2.
Docker cannot run this, i have specified these arguments when they run and the container stop once the jar main task finished. I need to start another container to run jar.
Don't think this is friendly. My question is in this case, is it not suitable to deploy with docker?
You can configure the container to run a script that will never end just to keep the container running.
As an example you can include the following in the Dockerfile:
RUN echo 'sleep infinity' >> /bootstrap.sh && chmod +x /bootstrap.sh
You can start the container in the following way:
docker run -d --name <container-name> <image> ./bootstrap.sh
To run the jar you can use:
docker exec <container-name> java [arguments]
Having in mind it is a java program and it is OS agnostic you don't have a huge benefit in running inside a container but is possible.
You can use a simple "hack" for this purpose... But I do not think this is the best solution.
Start a container with a process that is not supposed to be ended soon, e.g. bash. Also, lets say you want to use the latest ubuntu image. Then you can start the container with:
$ docker run -d -it ubuntu bash
This starts a ubuntu container and keeps it running as a daemon edit: detached (-d) in the background.
Lets lookup the container's name:
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS NAMES
59104211e795 ubuntu "bash" 2 seconds ago Up 1 seconds jolly_hawking
It is jolly_hawking. Your commands (here: ls /) can then be sent to the container with this command:
$ docker exec jolly_hawking ls /
But that is definitely not the best solution. Maybe just keep this as an example how this might work for you and how Docker containers are working.