Problem
I want to run a webapp via Docker by running 2 containers as a unit.
1 container runs my web-server (Tomcat 7).
The other container runs my database (Postgres 9.4).
I can run docker-compose up and Docker is able to spin up my two containers as specified in my docker-compose.yml:
web:
build: .
ports:
- "5000"
links:
- db
db:
image: postgres
I'd like to be able to spin up another copy of my webapp by running docker-compose up again, but this results in Docker telling me that there are already containers running:
$ docker-compose up -d
Creating composetest_db_1
Creating composetest_web_1
$ docker-compose up -d
composetest_db_1 is up-to-date
composetest_web_1 is up-to-date
My work around
I've gotten around this issue by using the -p option to give new copies different project names:
$ docker-compose -p project1 up -d
...
Successfully built d3268e345f3d
Creating project1_web_1
$ docker-compose -p project2 up -d
...
Successfully built d3268e345f3d
Creating project2_web_1
Unfortunately, this creating new images for each copy:
$ docker images
project1_web latest d3268e345f3d 2 hours ago 682 MB
project2_web latest d3268e345f3d 2 hours ago 682 MB
Question
Is there a way to use docker-compose to spin up multiple instances of a multi-container app by using a single image?
You can re-use your docker compose template by specifying the project name (which defaults to the directory name):
$ docker-compose --project-name inst1 up -d
Creating inst1_web_1
$ docker-compose --project-name inst2 up -d
Creating inst2_web_1
You could also scale up the container instances within a project:
$ docker-compose --project-name inst2 scale web=5
Creating and starting 2 ... done
Creating and starting 3 ... done
Creating and starting 4 ... done
Creating and starting 5 ... done
There should now be 6 containers running:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5e4ab4cebacf tomcat:8.0 "catalina.sh run" 43 seconds ago Up 42 seconds 0.0.0.0:32772->8080/tcp inst2_web_2
ced61f9ac2db tomcat:8.0 "catalina.sh run" 43 seconds ago Up 42 seconds 0.0.0.0:32773->8080/tcp inst2_web_5
efb1ef13147c tomcat:8.0 "catalina.sh run" 43 seconds ago Up 42 seconds 0.0.0.0:32771->8080/tcp inst2_web_4
58e524da3473 tomcat:8.0 "catalina.sh run" 43 seconds ago Up 42 seconds 0.0.0.0:32770->8080/tcp inst2_web_3
0f58c3c3b0ed tomcat:8.0 "catalina.sh run" 2 minutes ago Up 2 minutes 0.0.0.0:32769->8080/tcp inst2_web_1
377e3e5b03e4 tomcat:8.0 "catalina.sh run" 2 minutes ago Up 2 minutes 0.0.0.0:32768->8080/tcp inst1_web_1
If you want to reuse the image, you should build the image independent of the compose script.
run docker build -t somewebapp/web:latest
Then change your build section of docker-compose.yml to reference an image.
Related
I have what is prob a Docker Compose 101 question. I can spin up a simple Docker container with docker run:
me#host1:~/dc$
me#host1:~/dc$ sudo docker run -dit --name myContainer 54c9d81cbb44
60d254728f0a763bdda3078bd1c708176ca21e3eced475cb7e2c3edc7859a12c
me#host1:~/dc$
me#host1:~/dc$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
60d254728f0a 54c9d81cbb44 "bash" 2 minutes ago Up 2 minutes myContainer
me#host1:~/dc$
Easy! So then, I translate the above into a docker-compose.yaml file:
version: "2"
services:
myService:
container_name: myContainer2
image: 54c9d81cbb44
When I run the above file, the container exits immediately:
me#host1:~/dockerCompose$
me#host1:~/dockerCompose$ sudo docker-compose up
Creating myContainer2 ...
Creating myContainer2 ... done
Attaching to myContainer2
myContainer2 exited with code 0
me#host1:~/dockerCompose$
me#host1:~/dockerCompose$
me#host1:~/dockerCompose$ sudo docker ps --all
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
110f648fadbe 54c9d81cbb44 "bash" 14 seconds ago Exited (0) 13 seconds ago myContainer2
60d254728f0a 54c9d81cbb44 "bash" 6 minutes ago Up 6 minutes myContainer
me#host1:~/dockerCompose$
(The above happens even when I try to spin up myContainer2 without myContainer running.)
So what gives? I'm tempted to say the docker run "-dit" options are what are making the difference here; that is the only difference I see between the docker run and docker-compose versions. I've been Googling for "How to set docker run options in docker-compose file" for an hour, but pulling up information that isn't applicable here. I'm clearly missing something fundamental. Anyone see what? Thank you.
If you set stdin_open and tty to true, it'll stay up. Those are equivalent to the i and t options.
version: "2"
services:
myService:
container_name: myContainer2
image: 54c9d81cbb44
stdin_open: true
tty: true
Start the container with docker-compose up -d to run it detached.
Apoligies if this question is dumb or naive... we are still learning docker. We are running Airflow in docker. Here are the docker images on our GCP compute engine:
ubuntu#our-airflow:~/airflow-dir$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
our-airflow_webserver latest aaaaaaaaaaaa 17 minutes ago 968MB
<none> <none> bbbbbbbbbbbb 22 minutes ago 2.13GB
apache/airflow 2.1.4 cccccccccccc 5 weeks ago 968MB
<none> <none> dddddddddddd 2 months ago 2.01GB
python 3.7-slim-buster eeeeeeeeeeee 17 months ago 155MB
postgres 9.6 ffffffffffff 17 months ago 200MB
ubuntu#our-airflow:~/airflow-dir$
dddddddddddd was the image that used to run when we ran docker-compose up from the command line. However, we were testing a new Dockerfile, and built the new image aaaaaaaaaaaa with the tag our-airflow_webserver. dddddddddddd used to have this tag, but it was changed to <none> when we built aaaaaaaaaaaa.
We'd like to run docker-compose up dddddddddddd, however this does not work. We get the error ERROR: No such service: dddddddddddd. How can we create a container using the image dddddddddddd with docker-compose up? Is this possible?
Edit: If I simply run docker run dddddddddddd, I do not get the desired output. I think this is because our docker-compose file is launching all of the requisite services we need for airflow (webserver, scheduler, metadata db).
Edit2: Here's the seemingly relevant webserver part of our docker-compose file:
webserver:
# image:
build:
dockerfile: Dockerfile.Self
context: .
can we simply uncomment image, and set it to image: dddddddddddd and then comment out the build part?
can we simply uncomment image, and set it to image: dddddddddddd
Yes, you can. If you want to start the service with another image you must change the docker-compose.yml file.
and then comment out the build part?
You don't need to comment the build part. The build just takes action when the image specified is not found or the --build option is passed as argument.
If you want to ensure that the image is not gonna be build, just pass the argument --no-build to docker-compose up command. This will avoid to build the image even if it's missing.
Check the docker-compose up docs for further information.
I am trying to share Compose configurations between different projects using the same postgres database and redis server. For that, I have three different Compose configurations.
./docker-compose.base.yml
./apps/app1/docker-compose.extended.yml
./apps/app2/docker-compose.extended.yml
I create and start the containers with the following command:
docker-compose -f docker-compose.base.yml -f apps/app1/docker-compose.extended.yml -f apps/app2/docker-compose.extended.yml up -d
All the services in three configuration files are in the same network: myapp-backend. All services (postgres, redis, elasticsearch, kafka, zookeeper, app1, app2) run without any problems, except one catch. app2 does not show up when I write docker-compose ps or it does not stop when I type docker-compose down.
Creating network "myapp_myapp-backend" with the default driver
Creating myapp_postgres_1 ... done
Creating myapp_zookeeper_1 ... done
Creating myapp_redis_1 ... done
Creating myapp_elasticsearch_1 ... done
Creating myapp_kafka_1 ... done
Creating myapp_app1_1 ... done
Creating myapp_app2_1 ... done
docker-compose ps
Name Command State Ports
-----------------------------------------------------------------------------------------------------------
myapp_elasticsearch_1 /usr/local/bin/docker-entr ... Up 127.0.0.1:9200->9200/tcp, 9300/tcp
myapp_kafka_1 start-kafka.sh Up 127.0.0.1:9092->9092/tcp
myapp_postgres_1 docker-entrypoint.sh postgres Up 127.0.0.1:5434->5432/tcp
myapp_redis_1 docker-entrypoint.sh redis ... Up 6379/tcp
myapp_app1_1 /bin/sh -c bundle install ... Up 127.0.0.1:3000->3000/tcp
myapp_zookeeper_1 /bin/sh -c /usr/sbin/sshd ... Up 2181/tcp, 22/tcp, 2888/tcp, 3888/tcp
As you can see, app2 service does not show up here. But it shows up on docker ps:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
75bc1704c559 myapp_app1 "/bin/sh -c '(bundle…" 33 minutes ago Up 22 minutes 127.0.0.1:3020->3000/tcp myapp_app1_1
4ab2294f7a2c myapp_app2 "/bin/sh -c 'bundle …" 33 minutes ago Up 32 minutes 127.0.0.1:3000->3000/tcp myapp_app2_1
...
I can share the compose files if necessary, but I cannot understand why this is happening.
docker-compose ps doesn't remember the full stack of Compose files you initially ran docker-compose up with. You need to repeat all of the -f options on every docker-compose command.
If you don't want to repeat this, Compose also supports a COMPOSE_FILE environment variable that is the same as the -f options, so you should also be able to:
export COMPOSE_FILE=docker-compose.base.yml:apps/app1/docker-compose.extended.yml:apps/app2/docker-compose.extended.yml
docker-compose up -d
docker-compose ps
docker-compose down
it shows up on docker ps:
That's part of a feature: Multiple Isolated environments on a single host.
Compose is using the project app_1 or _2 to isolate environments from each other. From the docs
on a shared host or dev host, to prevent different projects, which may use the same service names, from interfering with each other
The default project name is the basename of the project directory. And that's the reason you can see the output myapp_app2_1 meaning service_folder_servicenumber
You can set a custom project name by using the -p command line option or the COMPOSE_PROJECT_NAME environment variable.
Multiple isolated environments on a single host
I created ubuntu image using docker-compose. Here is the relevant code from docker-compose.yml:
ubuntu-os:
container_name: ubuntu
image: ubuntu
volumes:
- ubuntu-datavolume:/home/username/docker/.os/ubuntu/
volumes:
ubuntu-datavolume:
It gets stopped as soon as it is started. I can not interract with the container. Here is relevant docker ps -a:
03dae5416b67 ubuntu "/bin/bash" 12 minutes ago Exited (0) 3 minutes ago ubuntu
I have tried every possible combo of docker start -a ubuntu but with no luck. I want this image to persist data across restart so I created the volume. Any suggestions?
Creating a new container is not what I am looking for but to start the existing container. I don't want to run the container but start and interact.
You use a ubuntu image which have an entry point /bin/sh. If you launch this without interactive/terminal linked, it will just run and exit with code 0. Your container finish successfully.
You can add the option:
stdin_open: true
tty: true
Referenced https://docs.docker.com/compose/compose-file/#domainname-hostname-ipc-mac_address-privileged-read_only-shm_size-stdin_open-tty-user-working_dir
or add command line that do something. like: command: sleep 600000
I understand that you can user docker-compose with the scale command to spin up multiple containers. However, they will all have the same configuration used.
Is it possible to launch a container on the same host with different configurations (different .yml files) on the same host?
Using the following commands:
docker-compose -f dev.yml up -d
docker-compose -f qa.yml up -d
only the qa.yml container will be running, which is not what I want.
-- edit --
Here's what happens when I try running both commands.
$ docker-compose -f compose/dev.yml up -d
compose_mydocker_1 is up-to-date
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
905912df6e48 compose_mydocker "/sbin/my_init" 2 days ago Up 2 days 0.0.0.0:1234->80/tcp compose_mydocker_1
$ docker-compose -f compose/qa.yml up -d
Recreating compose_mydocker_1...
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3fc912201224 compose_mydocker "/sbin/my_init" 5 seconds ago Up 5 seconds 0.0.0.0:1235->80/tcp compose_mydocker_1
My qa.yml and dev.yml look like this:
mydocker:
build: ..
ports:
- "1234:80" #for dev.yml
#- "1235:80" for qa.yml
environment:
- ENVIRONMENT=dev #and vice-versa for qa
volumes:
- ../assets/images:/var/www/assets
What you need to do is change the project name. By default, compose uses a project named based on the current directory. In your case, you want separate environments, so you need different project names.
You can use either docker-compose -p <project_name> or set COMPOSE_PROJECT_NAME in the environment.
There is also some discussion about providing a way to persist the project name: https://github.com/docker/compose/issues/745