I have a docker-compose project where each service has its own command to run upon startup.
I want to add an extra command before the one given in the docker-compose.yml that will execute only when the container is first created.
Since I'm not sure this is possible, it is also acceptable for me to manually run commands before calling docker-compose up.
Given the following docker-compose.yml
version: '2.3'
version: '2.3'
services:
serv1:
image: my_image
command: some startup command
serv2:
image: my_image
command: some startup command 2
I have tried to run the following
$ docker-compose run serv1 my_creation_command
$ docker-compose run serv2 my_creation_command
$ docker-compose up
But this results in the first two commands creating containers projectname_serv1_run_1 and projectname_serv2_run_1 and the third command creating projectname_serv1_1 and projectname_serv2_1
This is obviously a problem since the my_creation_command never ran on the containers created by the up command
Any solution that will manage to get a custom command executed only on the creation part will be acceptable
A possible and easy solution is to wrap init command and run command into a unique script (let's call it init_or_run) :
version: '2.3'
services:
serv1:
image: my_image
command: init_and_run
serv2:
image: my_image
command: init_and_run_2
These scripts have to handle init and run cases. I've already done that for databases (see example below).
Example (with a database)
in init_and_run's init case, creates database/tables
in init_and_run's run case, just starts the database daemon
Checking if you have to run init part of the script here could be just verifying if database exist :
if database_does_not_exist
then
create_database_and_tables # init case
fi
run_database_daemon # in both cases, run database daemon
With this solution, you just have to run docker-compose up to start your services, without caring if service have already been initialized, because init_and_run script will handle both cases (init and run or just run).
Related
I'm writing an automated test that involves running several containers at once. The test submits some workload to the tested service, and expects a callback from it after a time.
To run the whole system, I use docker compose run with the following docker-compose file:
version: "3.9"
services:
service:
build: ...
ports: ...
tester:
image: alpine
depends_on:
- service
profiles:
- testing
The problem is, I can see "service" from "tester", but not the other way around, so the callback from the service could not land to "tester":
$ docker compose -f .docker/docker-compose.yaml run --rm tester \
nslookup service
Name: service
Address 1: ...
$ docker compose -f .docker/docker-compose.yaml run --rm service \
nslookup tester
** server can't find tester: NXDOMAIN
I tried specifying the same network for them, and giving them "links", but the result is the same.
It seems like a very basic issue, so perhaps I'm missing something?
When you docker-compose run some-container, it starts a temporary container based on that description plus the things it depends_on:. So, when you docker-compose run service ..., it doesn't depends_on: anything, and Compose only starts the temporary container, which is why the tester container doesn't exist at that point.
If you need the whole stack up to make connections both ways between containers, you need to run docker-compose up -d. You can still docker-compose run temporary containers on top of these.
Given the following Docker Compose file....
version: '3.8'
services:
producer:
image: producer
container_name: producer
depends_on: [db]
build:
context: ./producer
dockerfile: ./Dockerfile
db:
image: some-db-image
container_name: db
When I do docker-compose up producer obviously the db service gets started too. When I CTRL+C both services are stopped. This is expected and fine.
But sometimes, the db service is started before, on a different shell and so doing docker-compose up producer understands that db is running and only starts producer. But when I hit CTRL+C, both producer and db are stopped even though db was not started as part of this docker compose up command.
Is there a way to avoid getting the dependencies services stopped when stopping its "parent" ?
When running just docker-compose up, the CTRL+C command always stops all running services in the current compose scope. It doesn't care about depends_on.
You would need to spin it up with detach option -d, like
docker-compose up -d producer
Then you can do
docker stop producer
And db service should still be running.
As I understand your question: You want to stop a container A which depends on another container B. But when stopping A, you don't want docker-compose to stop B.
Docker-compose stops the dependent containers ('B' in this case) when 'A' is stopped.
How I would approach this:
Split up the docker-compose files into A and B
In docker-compose for A create a health check testing (and waiting) for container B to be alive.
Since this is a database, you could do this with a dummy query.
Then you still have dependency, but not the docker-compose connection of stopping dependant containers.
You can't simply do that with CTRL+C.
Your docker-compose file and the services defined in it are treated as a project. You may notice that all containers, networks and volumes are prefixed with the name of the directory where the docker-compose file is located by default. This is the project name. It can be changed via an environment variable or the -p flag of the docker-compose command.
What docker-compose does is it keeps track of all the resources for a given project.
In your case there are two services: db and producer. Whenever you run docker-compose up, both of them start up. They both end up being part of the same project. The same applies when you only start one of the services (e.g. with docker-compose up db). You can later start the other service and it will still be part of the same project.
One more thing to note here: Whenever you run docker-compose without the -d (detached) flag, you get attached to the whole project, meaning whenever you hit CTRL+C, you'll stop all services. It does not matter if the last compose command started only one of the services or if they depend on each other. Attaching to the project and hitting CTRL+C will stop them.
A possible solution to your problem would be the following:
Start up your services via docker-compose up -d (both db and producer will get created). They are now in detached mode. If you still want to check the logs in real time (kinda like attaching), use docker-compose logs -f. Now, however, if you want to stop only one of the services you can simply do docker-compose stop $SVC_NAME (where $SVC_NAME is either db or producer) and this will keep the other one running. This way, whatever happens to your terminal session, your services won't stop, unless you explicitly tell them to.
Is there a way to avoid getting the dependencies services stopped when stopping its "parent" ?
Yes.
Using the new version docker compose instead of docker-compose might solve your problem Reference.
Simple example
Assuming now you are using the new version, your process could be something like this.
docker-compose.yml
version: "3.8"
services:
db:
build: .
producer:
build: .
depends_on: [db]
extra:
build: .
Dockerfile
FROM node:alpine
WORKDIR /app
COPY . .
ENTRYPOINT [ "/bin/sh", "script.sh" ]
script.sh
while :; do sleep 1; done
Suppose db has started before with
$ docker compose up -d db.
Then later,
$ docker compose up -d producer.
Now you can stop only producer with
$ docker compose stop producer.
You can check if db is still running with
$ docker compose ps.
Notice the use of -d flag for detached mode, as pointed out in another answer, so you don't need to kill the process with CTRL+C. Also, using detached flag allows you to check the services that are running with docker compose ps.
A similar issue as yours was reported and fixed a while ago, as you can see here.
I was not able to reproduce the behavior you observe with a complete minimal example. Namely, when running docker compose stop producer, the underlying db is not stopped AFAICT.
Anyway, you may be interested in an alternative command that is a bit more flexible than docker compose up, regarding how to run "one-off commands": docker compose run.
The typical use cases are as follows:
docker compose run db bash → run the db service, replacing the default CMD with bash
docker compose run -d db → run the db service in the background (detach mode)
docker compose run --service-ports producer → run the service producer and its dependencies (unless they were run with docker compose up), enabling the ports mapping.
So for your specific use case, you could run:
docker compose up -d db
docker compose run --service-ports producer
docker compose v3
I'm trying to run some app-specific commands like composer update whenever I run docker-compose up, having my docker-compose.yml file look something along the lines of this
version: '3'
services:
app1:
image: laraedit/laraedit
ports:
- 3000:80
volumes:
- ./appfolder:/var/www/appfolder
If I run my first-run commands in the entrypoint, it will override all the commands that default laraedit/laraedit is running. (At least I think so, because the container always stops when my entrypoint commands finish)
I don't want to bother the process of laraedit/laraedit starting up, I just want to execute a couple of commands on the side.
If I weren't using docker-compose, then I would have laraedit/laraedit Dockerfile locally, and I could then edit it and add a RUN statement somewhere in there.
But since I don't have the Dockerfile, and I can't make an entrypoint without throwing off the container's normal startup, I don't know how to go about automating the process of running these boring commands every single time I run docker-compose up.
Things I've tried:
adding my own Dockerfile (that replaces laraedit's)
running an entrypoint script (that blocks laraedit's startup)
running them as a command (the commands did not execute)
You need to extend the laraedit/laraedit image with a custom one.
You can use a Dockerfile as simple as this:
FROM laraedit/laraedit
COPY my_entrypoint.sh /my_entrypoint.sh
RUN chmod +x /my_entrypoint.sh
ENTRYPOINT /my_entrypoint.sh
my_entrypoint.sh is a script that contains your initialization commands and calls the original entrypoint at its end, for example:
#!/bin/sh
my_init_cmd1
my_init_cmd2
...
/original/entrypoint/script/path
You can get /original/entrypoint/script/path value by reading the original laraedit Dockerfile
Let's say you put the 2 files above in a directory called docker alongside your docker-compose.yml, than you need to adjust your docker-compose.yml like this:
version: '3'
services:
app1:
build: ./docker/
ports:
- 3000:80
volumes:
- ./appfolder:/var/www/appfolder
I want to start a service with docker-compose and keep the container running so I can get its IP-address via 'docker inspect'. However, the container always exits right after starting up.
I tried to add "command: ["sleep", "60"]" and other things to the docker-compose.yml but whenever I add the line with "command:..." I cant call "docker-compose up" as I will get the message "Cannot start container ..... System error: invalid character 'k' looking for beginning of value"
I also tried adding "CMD sleep 60" and whatnot to the Dockerfile itself but these commands do not seem to be executed.
Is there an easy way to keep the container alive or to fix one of my problems?
EDIT:
Here is the Compose file I want to run:
version: '2'
services:
my-test:
image: ubuntu
command: bash -c "while true; do echo hello; sleep 2; done"
It's working fine If I start this with docker-compose under OS X, but if I try the same under Ubuntu 16.04 it gives me above error message.
If I try the approach with the Dockerfile, the Dockerfile looks like this:
FROM ubuntu:latest
CMD ["sleep", "60"]
Which does not seem to do anything
EDIT 2:
I have to correct myself, turned out it was the same problem with the Dockerfile and the docker-compose.yml:
Each time I add either "CMD ..." to the Dockerfile OR add "command ..." to the compose file, I get above error with the invalid character. If I remove both commands, it works flawlessly.
To keep a container running when you start it with docker-compose, use the following command
command: tail -F anything
In the above command the last part anything should be included literally, and the assumption is that such a file is not present in the container, but with the -F option (capital -F not to be confused with -f which in contrast will terminate immediateley if the file is not found) the tail command will wait forever for the file anything to appear. A forever waiting process is basically what we need.
So your docker-compose.yml becomes
version: '2'
services:
my-test:
image: ubuntu
command: tail -F anything
and you can run a shell to get into the container using the following command
docker exec -i -t composename_my-test_1 bash
where composename is the name that docker-compose prepends to your containers.
You can use tty configuration option.
version: '3'
services:
app:
image: node:8
tty: true # <-- This option
Note: If you use Dockerfile for image and CMD in Dockerfile, this option won't work; however, you can use the entrypoint option in the compose file which clears the CMD from the Dockerfile.
Based on the comment of #aanand on GitHub Aug 26, 2015, one could use tail -f /dev/null in docker-compose to keep the container running.
docker-compose.yml example
version: '3'
services:
some-app:
command: tail -f /dev/null
Why this command?
The only reason for choosing this option was that it received a lot of thumbs up on GitHub, but the highest voted answer does not mean that it is the best answer. The second reason was a pragmatic one as issues had to be solved as soon as possible due to deadlines.
Create a file called docker-compose.yml
Add the following to the file
version: "3"
services:
ubuntu:
image: ubuntu:latest
tty: true
Staying in the same directory, run docker-compose up -d from the terminal
Run docker ps to get the container id or name
You can run docker inspect $container_id
You can enter the container and get a bash shell running docker-compose exec ubuntu /bin/bash or docker-compose exec ubuntu /bin/sh
When done, make sure you are outside the container and run docker-compose down
Here's a small bash script (my-docker-shell.sh) to create the docker compose file, run the container, login to the container and then finally cleanup the docker container and the docker compose file when you log out.
#!/bin/bash
cat << 'EOF' > ./docker-compose.yml
---
version: "3"
services:
ubuntu:
image: ubuntu:latest
command: /bin/bash
# tty: true
...
EOF
printf "Now entering the container...\n"
docker-compose run ubuntu bash
docker-compose down
rm -v ./docker-compose.yml
In the Dockerfile you can use the command:
{CMD sleep infinity}
Some people here write about overwriting the entrypoint so that the command can also have its effect. But no one gives an example. I then:
docker-compose.yml:
version: '3'
services:
etfwebapp:
# For messed up volumes and `sudo docker cp`:
command: "-f /dev/null"
entrypoint: /usr/bin/tail
tty: true
# ...
I am not sure if tty is needed at this point. Is it better to do it twice? In my case it did no harm and worked perfectly. Without entrypoint it didn't work for me because then command had no effect. So I guess for this solution tty is optional.
To understand which command is executed at start-up, simply read the entrypoint before the command (concat with space): /usr/bin/tail -f /dev/null.
I'm late to the party, but you can simply use: stdin_open: true
version: '2'
services:
my-test:
image: ubuntu
stdin_open: true
Blocking command is all you need.
I have been struggling with this problem for half a day.
. There are many answers below, but not clear enough. And nobody said why.
In short, there are two methods, but it can also be said that there is only one, running a Blocking processes in background.
This first one is using COMMAND:
version: '3'
services:
some-app:
command: ["some block command"]
put some block command like sleep infinity, tail -f /dev/null, watch anything, while true ...
Here I recommend sleep infinity.
The second is enable tty=true, then open a shell in command like /bin/bash.
services:
ubuntu:
image: ubuntu:latest
tty: true
command: "/bin/bash"
Since the tty is enabled, bash will keep running background, you can put some other block commands before it if you want.
Be careful, you must excute shell command at the end, like
command: /bin/bash -c "/root/.init-service && /bin/bash"
As you can see, all you need is blocking command.
Just a quick note
I have tested single image based on golang, so when I call docker-compose down here what I get:
version: "3.1"
...
command: tail -f /dev/null # stopping container takes about 10 sec.
tty: true # stopping container takes about 2 sec.
My system info:
Ubuntu 18.04.4 LTS (64-bit)
Docker version 19.03.6, build 369ce74a3c
docker-compose version 1.26.0, build d4451659
As the commenter stated, we'd have to see the Dockerfile in question to give you a complete answer, but this is a very common mistake. I can pretty much guarantee that the command you're trying to run is starting a background process. This might be the command you'd run in non-Docker situations, but it's the wrong thing to do in a Dockerfile. For instance, if what you're running is typically defined as a system service, you might use something like "systemctl start". That would start the process in the background, which will not work. You have to run the process in the foreground, so the entire process will block.
Okay I found my mistake. In the Dockerfile for the image used for compose I specified that the base image should be ubuntu:latest, but I previously created an image called ubuntu by myself and that image did not work. So I did not use the original ubuntu image but rather a corrupt version of my own image also called ubuntu.
What I'm trying to do
I want to run a yesod web application in one docker container, linked to a postgres database in another docker container.
What I've tried
I have the following file hierarchy:
/
api/
Dockerfile
database/
Dockerfile
docker-compose.yml
The docker-compose.yml looks like this:
database:
build: database
api:
build: api
command: .cabal/bin/yesod devel # dev setting
environment:
- HOST=0.0.0.0
- PGHOST=database
- PGPORT=5432
- PGUSER=postgres
- PGPASS
- PGDATABASE=postgres
links:
- database
volumes:
- api:/home/haskell/
ports:
- "3000:3000"
Running sudo docker-compose up fails either to start the api container at all or, just as often, with the following error:
api_1 | Yesod devel server. Press ENTER to quit
api_1 | yesod: <stdin>: hGetLine: end of file
personal_api_1 exited with code 1
If, however, I run sudo docker-compose database up & then start up the api container without using compose but instead using
sudo docker run -p 3000:3000 -itv /home/me/projects/personal/api/:/home/haskell --link personal_database_1:database personal_api /bin/bash
I can export the environment variables being set up in the docker-compose.yml file then manually run yesod devel and visit my site successfully on localhost.
Finally, I obtain a third different behaviour if I run sudo docker-compose run api on its own. This seems to start successfully but I can't access the page in my browser. By running sudo docker-compose run api /bin/bash I've been able to explore this container and I can confirm the environment variables being set in docker-compose.yml are all set correctly.
Desired behaviour
I would like to get the result I achieve from running the database in the background then manually setting the environment in the api container's shell simply by running sudo docker-compose up.
Question
Clearly the three different approaches I'm trying do slightly different things. But from my understanding of docker and docker-compose I would expect them to be essentially equivalent. Please could someone explain how and why they differ and, if possible, how I might achieve my desired result?
The error-message suggests the API container is expecting input from the command-line, which expects a TTY to be present in your container.
In your "manual" start, you tell docker to create a TTY in the container via the -t flag (-itv is shorthand for -i -t -v), so the API container runs successfully.
To achieve the same in docker-compose, you'll have to add a tty key to the API service in your docker-compose.yml and set it to true;
database:
build: database
api:
build: api
tty: true # <--- enable TTY for this service
command: .cabal/bin/yesod devel # dev setting