Why does docker compose exit right after starting? - docker

I'm trying to configure docker-compose to use GreenPlum db in Ubuntu 16.04. Here is my docker-compose.yml:
version: '2'
services:
greenplum:
image: "pivotaldata/gpdb-base"
ports:
- "5432:5432"
volumes:
- gp_data:/tmp/gp
volumes:
gp_data:
The issue is when I run it with sudo docker-compose up the GrrenPlum db is shutdowm immedately after starting. It looks as this:
greenplum_1 | 20170602:09:01:01:000050 gpstart:e1ae49da386c:gpadmin-[INFO]:-Starting Master instance 72ba20be3774 directory /gpdata/master/gpseg-1
greenplum_1 | 20170602:09:01:02:000050 gpstart:e1ae49da386c:gpadmin-[INFO]:-Command pg_ctl reports Master 72ba20be3774 instance active
greenplum_1 | 20170602:09:01:02:000050 gpstart:e1ae49da386c:gpadmin-[INFO]:-No standby master configured. skipping...
greenplum_1 | 20170602:09:01:02:000050 gpstart:e1ae49da386c:gpadmin-[INFO]:-Database successfully started
greenplum_1 | ALTER ROLE
dockergreenplumn_greenplum_1 exited with code 0 <<----- Here
Actually, when I start it with just sudo docker run pivotaldata/gpdb-base it's ok.
What's wrong with the docker compose?

First of all, be cautious running this image: the image looks to be badly maintained, and the information on Docker Hub indicates it's neither "official", nor "supported" in any way;
2017-01-09: Toolsmiths reviewed this image; it is not one we create. We make no promises about whether this is up to date or if it works. Feel free to email pa-toolsmiths#pivotal.io if you are the owner and are interested in collaborating with us on this image.
When using images from Docker Hub, it's recommended to either use official images, or when not available, prefer automated builds (in which case the source code of the image can be verified to see what's used to build theimage).
I think the image is built from this GitHub repository, which means it has not been updated for over a year, and uses an outdated (CentOS 6.7) base image that has a huge amount of critical vulnerabilities
Back to your question;
I tried starting the image, both with docker-compose and docker run, and both resulted in the same for me.
Looking at that image, it is designed to be run interactively, or to be used as a base image (and overriding the command).
I inspected the image to find out what the container's command is;
docker inspect --format='{{json .Config.Cmd}}' pivotaldata/gpdb-base
["/bin/sh","-c","echo \"127.0.0.1 $(cat ~/orig_hostname)\" >> /etc/hosts && service sshd start && su gpadmin -l -c \"/usr/local/bin/run.sh\" && /bin/bash"]
So, this is what's executed when the container is started;
echo "127.0.0.1 $(cat ~/orig_hostname)" >> /etc/hosts \
&& service sshd start \
&& su gpadmin -l -c "/usr/local/bin/run.sh" \
&& /bin/bash"
Based on the above, there is no "foreground" process in the container, so the moment /usr/local/bin/run.sh finishes, a bash shell is started. A bash shell wothout a tty attached, exits immediately, at which point the container exits.
To run this image
(Again; be cautious running this image)
Either run the image interactively, by passing it stdin and a tty (-i -t, or -it as a shorthand);
docker run -it pivotaldata/gpdb-base
Or can run it "detached", as long as a tty is passed as well (add the -d and -t flags, or -dt as a shorthand); doing so, keeps the container running in the background;
docker run -dit pivotaldata/gpdb-base
To do the same in docker-compose, add a tty to your service;
tty: true
Your compose file will then look like this;
version: '2'
services:
greenplum:
image: "pivotaldata/gpdb-base"
ports:
- "5432:5432"
tty: true
volumes:
- gp_data:/tmp/gp
volumes:
gp_data:

Related

Docker keeps restarting without error on logs

I'm trying to create a simple instance using docker-compose with this simple yml(It's a Laravel project, but right now i'm not initializing anything):
version: '3'
services:
api:
image: amazonlinux
container_name: test-backend
restart: unless-stopped
working_dir: /var/www
ports:
- 8000:80
volumes:
- .:/var/www
- ~/.ssh:/root/.ssh
Here i'm just trying to create an AmazonLinux instance to test some libraries i'm installing to let the backend run, but for some reason, when i make docker-compose up, the instance keeps restarting. I tried checking the logs, but they are empty, there is no error message, warning, or anything that tells me what is happening.
I tried running the instance manually with docker run -dit amazonlinux:latest and that works, create an instance with AmazonLinux that doesn't restart, but the compose one keeps doint it. I have also tried wiping everything with
- docker rm $(docker ps -aq) -f
- docker volume rm $(docker volume ls -q) -f
- docker rmi $(docker images --filter "dangling=true" -q --no-trunc)
Yet keeps happening, i restarted Docker, still happens. Other instances of other projects don't have any problem, they can be launched with docker-compose up, it's just this one the one causing a problem, Does someone know what i might be doing wrong? As an aditional detail, i believe this started happening after i made an accidental ctrl+c in the middle of a docker-compose up, but i'm not sure if that might be the cause.
You're not giving your container anything to do.
If you docker image inspect amazonlinux, you can see that the default behavior of the image is to start an interactive bash shell:
[...]
"Cmd": [
"/bin/sh",
"-c",
"#(nop) ",
"CMD [\"/bin/bash\"]"
],
[...]
When you start a container via docker-compose up, this is similar to running docker run <containername>: by default, the container is not attached to stdin and does not allocate a tty. When bash is run in this environment, it's default behavior is to exit immediately. You can simulate this yourself by running:
bash < /dev/null
This is why you container keeps restarting: it starts up, bash attempts to run an interactive shell but it can't, so it exits.
The solution here is to run something besides bash. What are you trying to do with this container? You would typically run some sort of service (like a web server, or a database server, or a message broker, or...etc). Set an appropriate command key in the service in your docker-compose.yml.
If you're really just playing around, a good option is sleep inf, which means "do nothing, forever". This will allow the container to start and keep running, and you can then use docker-compose exec to run commands inside wthe container:
version: '3'
services:
api:
image: amazonlinux
container_name: test-backend
restart: unless-stopped
working_dir: /var/www
ports:
- 8000:80
volumes:
- .:/var/www
- ~/.ssh:/root/.ssh
command:
- sleep
- inf

First attempt at docker compose, status is "restarting" what have I done wrong?

This is my first attempt at docker composer (and docker since yesterday), however the docker is in a restarting state.
The application is Grafana which I normally run with:
docker volume create grafana-storage
docker run -d -p 3000:3000 --name=grafana -v grafana-storage:/var/lib/grafana grafana/grafana
Today I thought I'd try using docker composer, here is what I have done:Create a folder for the Docker App(s)
sudo mkdir Docker_Applications
cd Docker_Applications
sudo mkdir Grafana
Go into the directory
cd Grafana
sudo nano docker-compose.yml
add
version: '3'
services:
grafana:
image: "grafana/grafana:7.3.7"
volumes:
# Data persistency
# sudo mkdir -p /Docker_Applications/Grafana
- "./database:/var/lib/grafana"
- "./config:/etc/grafana"
ports:
- 3000:3000
restart: always
Then ran it
root#grafana-dev:/Docker_Applications/Grafana$ sudo docker-compose up -d
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Starting grafana_grafana_1 ... done
status
root#grafana-dev:/Docker_Applications/Grafana$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a347b12ae9a3 grafana/grafana:7.3.7 "/run.sh" 18 minutes ago Restarting (1) 4 seconds ago grafana_grafana_1
Hopefully you can see I've tried my best. I wonder if it's to do with the volumes.
Any help would be really appreciated.
I removed the volumes section of the yaml file and it runs now. It looks like permissions or it can't locate my folders, i'm not sure what permissions/commands to try.
grafana-dev:/Docker_Applications/Grafana$ sudo docker-compose up
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Creating grafana_grafana_1 ... done
Attaching to grafana_grafana_1
grafana_1 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana_1 | GF_PATHS_CONFIG='/etc/grafana/grafana.ini' is not readable.
grafana_1 | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana_1 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migration-from-a-previous-version-of-the-docker-container-to-5-1-or-later
Most likely the container exits during startup due to an error and since you've set restart: always in your docker-compose file, the container automatically restarts.
Check the logs or just run docker-compose up non-detached by removing the -d flag to find out what the problem is, fix that and your container will stop restarting itself continuously.

Setting up Rails console on Docker container not taking any input

I was trying to setup Rails console in my dockerized container. The entire application has multiple components and I have set up the orchestration using docker-compose. Here is the relevant service from my docker-compose.yml file.
app:
image: <image_name>
# mount the current directory (on the host) to /usr/src/app on the container, any changes in either would be reflected in both the host and the container
tty: true
volumes:
- .:/usr/src/app
# expose application on localhost:36081
ports:
- "36081:36081"
# application restarts if stops for any reason - required for the container to restart when the application fails to start due to the database containers not being ready
restart: always
depends_on:
- other-db
# the environment variables are used in docker/config/env_config.rb to connect to different database containers
container_name: application
environment:
- CONSOLE=$CONSOLE
My Dockerfile has the following command ENTRYPOINT /usr/src/app/docker-entrypoint.sh
And in the docker-entrypoint.sh
#!/bin/bash
echo "waiting for all db connections to be healthy... Sleeping..."
sleep 1m
mkdir -p /usr/src/app/tmp/pids/
mkdir -p /usr/src/app/tmp/sockets/
if [ "$CONSOLE" = "Y" ];
then
echo "Starting Padrino console"
bundle exec padrino console
fi
When I run
export CONSOLE=Y
docker-compose -f docker-compose.yml up -d && docker attach application
The console starts up and I see >> but I cannot type in it. Where am I going wrong?
Try starting your container with -i mode.
-i, --interactive Attach container's STDIN
something like
docker-compose -f docker-compose.yml up -i && docker attach application
you can also mix -d and -i as per need.
With help from this post, I figured that I was missing stdin_open: true in the docker-compose.yml. Adding it worked like a breeze.

Adding files to standard images using docker-compose

I'm unsure if something obvious escapes me or if it's just not possible but I'm trying to compose an entire application stack with images from docker hub.
One of them is mysql and it supports adding custom configuration files through volumes and to run .sql-files from a mounted directory.
But, I have these files on the machine where I'm running docker-compose, not on the host. Is there no way to specify files from the local machine to copy into the container before it runs it entrypoint/cmd? Do I really have to create local images of everything just for this case?
Option A: Include the files inside your image. This is less than ideal since you are mixing configuration files with your image (that should really only contain your binaries, not your config), but satisfies the requirement to use only docker-compose to send the files.
This option is achieved by using docker-compose to build your image, and that build will send over any files from the build directory to the remote docker engine. Your docker-compose.yml would look like:
version: '2'
services:
my-db-app:
build: db/.
image: custom-db
And db/Dockerfile would look like:
FROM mysql:latest
COPY ./sql /sql
The entrypoint/cmd would remain unchanged. You would need to run docker-compose up --build if the image already exists and you need to change the sql files.
Option B: Use a volume to store your data. This cannot be done directly inside of docker-compose. However it's the preferred way to include files from outside of the image into the container. You can populate the volume across the network by using the docker CLI and input redirection along with a command like tar to pack and unpack those files being sent over stdin:
tar -cC sql . | docker run --rm -it -v sql-files:/sql \
busybox /bin/sh -c "tar -xC /sql"
Run that via a script and then have that same script bounce the db container to reload that config.
Option C: Use some kind of network attached filesystem. If you can configure NFS on the host where you are running your docker CLI, you can connect to those NFS shares from the remote docker node using one of the below options:
# create a reusable volume
$ docker volume create --driver local \
--opt type=nfs \
--opt o=addr=192.168.1.1,rw \
--opt device=:/path/to/dir \
foo
# or from the docker run command
$ docker run -it --rm \
--mount type=volume,dst=/container/path,volume-driver=local,volume-opt=type=nfs,volume-opt=o=addr=192.168.1.1,volume-opt=device=:/host/path \
foo
# or to create a service
$ docker service create \
--mount type=volume,dst=/container/path,volume-driver=local,volume-opt=type=nfs,volume-opt=o=addr=192.168.1.1,volume-opt=device=:/host/path \
foo
Option D: With swarm mode, you can include files as configs in your image. This allows configuration files, that would normally need to be pushed to any node in the swarm, to be sent on demand to the node where your service is running. This uses a docker-compose.yml file to define it, but swarm mode isn't using docker-compose itself, so this may not fit your specific requirements. You can run a single node swarm mode cluster, so this option is available even if you only have a single node. This option does require that each of your sql files are added as a separate config. The docker-compose.yml would look like:
version: '3.4'
configs:
sql_file_1:
file: ./file_1.sql
services
my-db-app:
image: my-db-app:latest
configs:
- source: sql_file_1
target: /sql/file_1.sql
mode: 444
Then instead of a docker-compose up, you'd run a docker stack deploy -c docker-compose.yml my-db-stack.
If you can not use volumes (wants stateless docker-compose.yml and using remote machine), you can have config file written by command.
Example for nginx config in official image:
version: "3.7"
services:
nginx:
image: nginx:alpine
ports:
- 80:80
environment:
NGINX_CONFIG: |
server {
server_name "~^www\.(.*)$$" ;
return 301 $$scheme://$$1$$request_uri ;
}
server {
server_name example.com
...
}
command:
/bin/sh -c "echo \"$$NGINX_CONFIG\" > /etc/nginx/conf.d/redir.conf; nginx -g \"daemon off;\""
Environment variable could also be saved in .env file, you can use Compose's extend feature or load it from shell environment (where you fetched it from enywhere else):
https://docs.docker.com/compose/compose-file/#env_file
https://docs.docker.com/compose/compose-file/#variable-substitution
To get the original entrypoint command of a container:
docker container inspect [container] | jq --raw-output .[0].Config.Cmd
To investigate which file to modify this usually will work:
docker exec --interactive --tty [container] sh
This is how I'm doing it with volumes:
services:
my-db-app:
command: /shell_scripts/go.sh
volumes:
- ./shell_scripts:/shell_scripts
i think you had to do in a compose file:
volumes:
- src/file:dest/path
As a more recent update to this question: with a docker swarm hosted on Amazon, for example, you can define a volume that can be shared by services and is available across all nodes of the swarm (using the cloudstor driver, which in turn has AWS EFS underlying for persistence).
version: '3.3'
services:
my-db-app:
command: /shell_scripts/go.sh
volumes:
shell_scripts:/shell_scripts
volumes:
shell_scripts:
driver: "cloudstor:aws"
With Compose V2 you can simply do (as in the documentation) :
docker compose cp src [service:]dest
Before v2 you can use the workaround using docker cp explained in the associated issue
docker cp /path/to/my-local-file.sql "$(docker-compose ps -q mycontainer)":/file-on-container.sql

Docker Compose keep container running

I want to start a service with docker-compose and keep the container running so I can get its IP-address via 'docker inspect'. However, the container always exits right after starting up.
I tried to add "command: ["sleep", "60"]" and other things to the docker-compose.yml but whenever I add the line with "command:..." I cant call "docker-compose up" as I will get the message "Cannot start container ..... System error: invalid character 'k' looking for beginning of value"
I also tried adding "CMD sleep 60" and whatnot to the Dockerfile itself but these commands do not seem to be executed.
Is there an easy way to keep the container alive or to fix one of my problems?
EDIT:
Here is the Compose file I want to run:
version: '2'
services:
my-test:
image: ubuntu
command: bash -c "while true; do echo hello; sleep 2; done"
It's working fine If I start this with docker-compose under OS X, but if I try the same under Ubuntu 16.04 it gives me above error message.
If I try the approach with the Dockerfile, the Dockerfile looks like this:
FROM ubuntu:latest
CMD ["sleep", "60"]
Which does not seem to do anything
EDIT 2:
I have to correct myself, turned out it was the same problem with the Dockerfile and the docker-compose.yml:
Each time I add either "CMD ..." to the Dockerfile OR add "command ..." to the compose file, I get above error with the invalid character. If I remove both commands, it works flawlessly.
To keep a container running when you start it with docker-compose, use the following command
command: tail -F anything
In the above command the last part anything should be included literally, and the assumption is that such a file is not present in the container, but with the -F option (capital -F not to be confused with -f which in contrast will terminate immediateley if the file is not found) the tail command will wait forever for the file anything to appear. A forever waiting process is basically what we need.
So your docker-compose.yml becomes
version: '2'
services:
my-test:
image: ubuntu
command: tail -F anything
and you can run a shell to get into the container using the following command
docker exec -i -t composename_my-test_1 bash
where composename is the name that docker-compose prepends to your containers.
You can use tty configuration option.
version: '3'
services:
app:
image: node:8
tty: true # <-- This option
Note: If you use Dockerfile for image and CMD in Dockerfile, this option won't work; however, you can use the entrypoint option in the compose file which clears the CMD from the Dockerfile.
Based on the comment of #aanand on GitHub Aug 26, 2015, one could use tail -f /dev/null in docker-compose to keep the container running.
docker-compose.yml example
version: '3'
services:
some-app:
command: tail -f /dev/null
Why this command?
The only reason for choosing this option was that it received a lot of thumbs up on GitHub, but the highest voted answer does not mean that it is the best answer. The second reason was a pragmatic one as issues had to be solved as soon as possible due to deadlines.
Create a file called docker-compose.yml
Add the following to the file
version: "3"
services:
ubuntu:
image: ubuntu:latest
tty: true
Staying in the same directory, run docker-compose up -d from the terminal
Run docker ps to get the container id or name
You can run docker inspect $container_id
You can enter the container and get a bash shell running docker-compose exec ubuntu /bin/bash or docker-compose exec ubuntu /bin/sh
When done, make sure you are outside the container and run docker-compose down
Here's a small bash script (my-docker-shell.sh) to create the docker compose file, run the container, login to the container and then finally cleanup the docker container and the docker compose file when you log out.
#!/bin/bash
cat << 'EOF' > ./docker-compose.yml
---
version: "3"
services:
ubuntu:
image: ubuntu:latest
command: /bin/bash
# tty: true
...
EOF
printf "Now entering the container...\n"
docker-compose run ubuntu bash
docker-compose down
rm -v ./docker-compose.yml
In the Dockerfile you can use the command:
{CMD sleep infinity}
Some people here write about overwriting the entrypoint so that the command can also have its effect. But no one gives an example. I then:
docker-compose.yml:
version: '3'
services:
etfwebapp:
# For messed up volumes and `sudo docker cp`:
command: "-f /dev/null"
entrypoint: /usr/bin/tail
tty: true
# ...
I am not sure if tty is needed at this point. Is it better to do it twice? In my case it did no harm and worked perfectly. Without entrypoint it didn't work for me because then command had no effect. So I guess for this solution tty is optional.
To understand which command is executed at start-up, simply read the entrypoint before the command (concat with space): /usr/bin/tail -f /dev/null.
I'm late to the party, but you can simply use: stdin_open: true
version: '2'
services:
my-test:
image: ubuntu
stdin_open: true
Blocking command is all you need.
I have been struggling with this problem for half a day.
. There are many answers below, but not clear enough. And nobody said why.
In short, there are two methods, but it can also be said that there is only one, running a Blocking processes in background.
This first one is using COMMAND:
version: '3'
services:
some-app:
command: ["some block command"]
put some block command like sleep infinity, tail -f /dev/null, watch anything, while true ...
Here I recommend sleep infinity.
The second is enable tty=true, then open a shell in command like /bin/bash.
services:
ubuntu:
image: ubuntu:latest
tty: true
command: "/bin/bash"
Since the tty is enabled, bash will keep running background, you can put some other block commands before it if you want.
Be careful, you must excute shell command at the end, like
command: /bin/bash -c "/root/.init-service && /bin/bash"
As you can see, all you need is blocking command.
Just a quick note
I have tested single image based on golang, so when I call docker-compose down here what I get:
version: "3.1"
...
command: tail -f /dev/null # stopping container takes about 10 sec.
tty: true # stopping container takes about 2 sec.
My system info:
Ubuntu 18.04.4 LTS (64-bit)
Docker version 19.03.6, build 369ce74a3c
docker-compose version 1.26.0, build d4451659
As the commenter stated, we'd have to see the Dockerfile in question to give you a complete answer, but this is a very common mistake. I can pretty much guarantee that the command you're trying to run is starting a background process. This might be the command you'd run in non-Docker situations, but it's the wrong thing to do in a Dockerfile. For instance, if what you're running is typically defined as a system service, you might use something like "systemctl start". That would start the process in the background, which will not work. You have to run the process in the foreground, so the entire process will block.
Okay I found my mistake. In the Dockerfile for the image used for compose I specified that the base image should be ubuntu:latest, but I previously created an image called ubuntu by myself and that image did not work. So I did not use the original ubuntu image but rather a corrupt version of my own image also called ubuntu.

Resources