I am setting up a test infrastructure using docker-compose. I want to use the docker-compose option --exit-code-from to return the exit code from the container that is running tests. However, I also have a container that runs migrations on my database container using the sequelize cli. This migrations container exits with code 0 when migrations are complete and then my tests run. This causes an issue with both the --exit-code-from and --abort-on-container-exit options. Is there a way to ignore when the migration container exits?
Don't use docker-compose up for running one-off tasks. Use docker-compose run instead, as the documentation suggests:
The docker-compose run command is for running “one-off” or “adhoc” tasks. It requires the service name you want to run and only starts containers for services that the running service depends on. Use run to run tests or perform an administrative task such as removing or adding data to a data volume container. The run command acts like docker run -ti in that it opens an interactive terminal to the container and returns an exit status matching the exit status of the process in the container.
Source: https://docs.docker.com/compose/faq/
For example:
docker-compose build my_app
docker-compose run db_migrations # this starts the services it depends on, such as the db
docker-compose run my_app_tests
--exit-code-from implies --abort-on-container-exit, which according to documentation
--abort-on-container-exit Stops all containers if any container was stopped.
But you could try:
docker inspect <container ID> --format='{{.State.ExitCode}}'
You can get a list of all (including stopped) containers with
docker container ls -a
Here's a nice example: Checking the Exit Code of Stopped Containers
Related
I run the one of the open source microservices from here. When i run docker ps then all the containers status are UP, means they keep running. My issue is when I separately run a container then it did not keep running and exits. Below is one of the service defined in docker-compose file.
social-graph-service:
image: yg397/social-network-microservices
hostname: social-graph-service
restart: always
entrypoint: SocialGraphService
when i run it using command
sudo docker run -d --restart always --entrypoint SocialGraphService --hostname social-graph-service yg397/social-network-microservices
then its status does not UP, it exits after running. Why all the containers run continuously when i run them using sudo docker-compose up? and exit when i run them individually?
It looks like the graph service depends on MongoDB in order to run. My guess is it crashes when you run it individually because the mongo instance doesn't exist and it fails to connect.
The author of the repo wrote the docker-compose file to hide away some of the complexity from you, but that's a substantial tree of relationships between microservices, and most of them seem to depend on others existing in order to boot up.
-- Update --
The real issue is in the comments below. OP was already running the docker-compose stack while attempting to start another container, but forgot to connect the container to the docker network generated by docker-compose.
I'm new in docker.
What is the difference between these?
docker run 'an image'
docker-compose run 'something'
docker-compose start 'docker-compose.yml'
docker-compose up 'docker-compose.yml'
Thanks in advance.
https://docs.docker.com/compose/faq/#whats-the-difference-between-up-run-and-start
What’s the difference between up, run, and start?
Typically, you want docker-compose up. Use up to start or restart all the services defined in a docker-compose.yml. In the default “attached” mode, you see all the logs from all the containers. In “detached” mode (-d), Compose exits after starting the containers, but the containers continue to run in the background.
The docker-compose run command is for running “one-off” or “adhoc” tasks. It requires the service name you want to run and only starts containers for services that the running service depends on. Use run to run tests or perform an administrative task such as removing or adding data to a data volume container. The run command acts like docker run -ti in that it opens an interactive terminal to the container and returns an exit status matching the exit status of the process in the container.
The docker-compose start command is useful only to restart containers that were previously created, but were stopped. It never creates new containers.
Also: https://docs.docker.com/compose/reference/
In practice to start a container I do:
docker run a8asd8f9asdf0
If thats the case, what does:
docker start
do?
In the manual it says
Start one or more stopped containers
This is a very important question and the answer is very simple, but fundamental:
Run: create a new container of an image, and execute the container. You can create N clones of the same image. The command is:
docker run IMAGE_ID and not docker run CONTAINER_ID
Start: Launch a container previously stopped. For example, if you had stopped a database with the command docker stop CONTAINER_ID, you can relaunch the same container with the command docker start CONTAINER_ID, and the data and settings will be the same.
run runs an image
start starts a container.
The docker run doc does mention:
The docker run command first creates a writeable container layer over the specified image, and then starts it using the specified command.
That is, docker run is equivalent to the API /containers/create then /containers/(id)/start.
You do not run an existing container, you docker exec to it (since docker 1.3).
You can restart an exited container.
Explanation with an example:
Consider you have a game (iso) image in your computer.
When you run (mount your image as a virtual drive), a virtual drive is created with all the game contents in the virtual drive and the game installation file is automatically launched. [Running your docker image - creating a container and then starting it.]
But when you stop (similar to docker stop) it, the virtual drive still exists but stopping all the processes. [As the container exists till it is not deleted]
And when you do start (similar to docker start), from the virtual drive the games files start its execution. [starting the existing container]
In this example - The game image is your Docker image and virtual drive is your container.
run command creates a container from the image and then starts the root process on this container. Running it with run --rm flag would save you the trouble of removing the useless dead container afterward and would allow you to ignore the existence of docker start and docker remove altogether.
run command does a few different things:
docker run --name dname image_name bash -c "whoami"
Creates a Container from the image. At this point container would have an id, might have a name if one is given, will show up in docker ps
Starts/executes the root process of the container. In the code above that would execute bash -c "whoami". If one runs docker run --name dname image_name without a command to execute container would go into stopped state immediately.
Once the root process is finished, the container is stopped. At this point, it is pretty much useless. One can not execute anything anymore or resurrect the container. There are basically 2 ways out of stopped state: remove the container or create a checkpoint (i.e. an image) out of stopped container to run something else. One has to run docker remove before launching container under the same name.
How to remove container once it is stopped automatically? Add an --rm flag to run command:
docker run --rm --name dname image_name bash -c "whoami"
How to execute multiple commands in a single container? By preventing that root process from dying. This can be done by running some useless command at start with --detached flag and then using "execute" to run actual commands:
docker run --rm -d --name dname image_name tail -f /dev/null
docker exec dname bash -c "whoami"
docker exec dname bash -c "echo 'Nnice'"
Why do we need docker stop then? To stop this lingering container that we launched in the previous snippet with the endless command tail -f /dev/null.
daniele3004's answer is already pretty good.
Just a quick and dirty formula for people like me who mixes up run and start from time to time:
docker run [...] = docker pull [...] + docker start [...]
It would have been wiser to name the command "new" instead of "run".
Run creates a container instance of an existing (or downloadable) image and starts it.
Up until recently, when one was doing docker-compose up for a bunch of containers and one of the started containers stopped, all of the containers were stopped. This is not the case anymore since https://github.com/docker/compose/issues/741 and this is a really annoying for us: We use docker-compose to run selenium tests which means starting application server, starting selenium hub + nodes, starting tests driver, then exiting when tests driver stops.
Is there a way to get back old behaviour?
You can use:
docker-compose up --abort-on-container-exit
Which will stop all containers if one of your containers stops
In your docker compose file, setup your test driver container to depend on other containers (with depends_on parameter). Your docker compose file should look like this:
services:
application_server:
...
selenium:
...
test_driver:
entry_point: YOUR_TEST_COMMAND
depends_on:
- application_server
- selenium
With dependencies expressed this way, run:
docker-compose run test_driver
and all the other containers will shut down when the test_driver container is finished.
This solution is an alternative to the docker-compose up --abort-on-container-exit answer. The latter will also shut down all other containers if any of them exits (not only the test driver). It depends on your use case which one is more adequate.
Did you try the work around suggested on the link you provided?
Assuming your test script looked similar to this:
$ docker-compose rm -f
$ docker-compose build
$ docker-compose up --timeout 1 --no-build
When the application tests end, compose would exit and the tests finish.
In this case, with the new docker-compose version, change your test container to have a default no-op command (something like echo, or true), and change your test script as follows:
$ docker-compose rm -f
$ docker-compose build
$ docker-compose up --timeout 1 --no-build -d
$ docker-compose run tests test_command...
$ docker-compose stop
Using run allows you to get the exit status from the test run, and you only see the output of the tests (not all the dependencies).
Reference
If this is not acceptable, you could refer to Docker Remote API and watch for the stop event for the containers and act on it.
An example usage is this docker-gen tool written in golang which watches for container start events, to automatically regenerate configuration files.
I'm not sure this is the perfect answer to your problem, but maestro for Docker, lets you manage mulitple Docker containers as single unit.
It should feel familiar as you group them using a YAML file.
I'm trying to use docker-compose to orchestrate several containers. To troubleshoot, I frequently end up running bash from within a container by doing:
$ docker-compose run --rm run web bash
I always try pass the --rm switch so that these containers are removed when I exit the bash session. Sometimes though, they remain, and I see them at the output of docker-compose ps.
Name Command State Ports
----------------------------------------------------------------------------------
project_nginx_1 /usr/sbin/nginx Exit 0
project_nginx_run_1 bash Up 80/tcp
project_web_1 python manage.py runserver ... Exit 128
project_web_run_1 bash Up 8000/tcp
At this point, I am trying to stop and remove these components manually, but I can not manage to do this. I tried:
$ docker-compose stop project_nginx_run_1
No such service: project_nginx_run_1
I also tried the other commands rm, kill, etc..
What should I do to get rid of these containers?
Edit:
Fixed the output of docker-compose ps.
just stop those test containers with the docker stop command instead of using docker-compose.
docker-compose shines when it comes to start together many containers, but using docker-compose to start containers does not prevent you from using the docker command to do whatever you need to do with individual containers.
docker stop project_nginx_run_1 project_web_run_1
Also, since you are debugging containers, I suggest to use docker-compose exec <service id> bash to get a shell in a running container. This has the advantage of not starting a new container.
With docker-compose, services can be stopped in two ways, but I would like add some detailed info about both options.
In short
docker-compose down
Stop and remove containers, networks, images, and volumes
docker-compose stop
Stop services
In detail
If docker-compose run starts services project_nginx_run_1 and project_web_run_1, then
docker-compose down log will be
$ docker-compose down
Stopping project_nginx_run_1 ...
Stopping project_web_run_1 ...
.
. some service logs goes here
Stopping project_web_run_1 ... done
Stopping project_nginx_run_1 ... done
Removing project_web_run_1 ... done
Removing project_nginx_run_1 ... done
Removing network project_default
docker-compose stop log will be
$ docker-compose stop
Stopping project_nginx_run_1 ...
Stopping project_web_run_1 ...
.
. some service logs goes here
Stopping project_web_run_1 ... done
Stopping project_nginx_run_1 ... done
The docker-compose, unlike docker, use the names for it's containers defined in the yml file. Therefore, to stop just one container the command will be:
docker-compose stop nginx_run
docker-compose down
from within the directory where it was launched, is the only way I managed to confirm it was stopped, as in docker-compose ps no longer yields it!