I want to fail docker compose if my first task fail and it should not move to other tasks mentioned in the compose file.
Docker-compose is not a script; it's the definition of an environment. The environment is described in the docker-compose file by defining each service that make up the environment.
Of course, it is sometimes necessary to have a service fail during initialization.
This can be done by, for example:
defining a Dockerfile for the service, which performs initialization during image creation, or
defining an entrypoint script, which does initialization when starting a container based on the image.
The point is, docker-compose shouldn't fail. The services should fail.
If you absolutely need to fail on startup, you can define a dependency between services by using depends_on to define the order that containers are started; like this:
version: "3.7"
services:
web:
build: .
depends_on:
- db
- redis
redis:
image: redis
db:
image: postgres
Please note, the dependent services will start as soon as the parent services are started... It will not wait for them to be in a ready state.
Related
I am fairly new to docker.
My problem is - I have a multi container application(zookeeper, kafka, producer, consumer) in which I want to run the producer and the consumer containers only after the zookeeper and kafka containers are up and running. How to edit the docker-compose file and achieve that? Thanks in advance
The depends_on instruction only allows you to manage the starting order.
However, it does not wait for your service to be up and running before triggering the next.
If you want to start a service after your dependance is up, then you'll have to look at tools like wait-for-it or dockerize.
You can find more info on the official Docker documentation
Check the depends_on directive in the docker-compose documentation.
__ from the docs __
depends_on Express dependency between services, Service dependencies
cause the following behaviors:
docker-compose up starts services in dependency order. In the
following example, db and redis are started before web.
docker-compose up SERVICE automatically includes SERVICE’s
dependencies. In the following example, docker-compose up web also
creates and starts db and redis.
docker-compose stop stops services in dependency order. In the
following example, web is stopped before db and redis.
Simple example:
version: "3.7"
services:
web:
build: .
depends_on:
- db
- redis
redis:
image: redis
db:
image: postgres
I need to start backend-container after start database-container. How can I do it with docker-compose?
Use a depends_on clause on your backend-container. Something like that :
version: "3.7"
services:
web:
build: .
depends_on:
- db
db:
image: postgres
Here is the documentation about it.
Have fun!
You should look into the depends_on configuration for docker compose.
In short, you should be able to do something like:
services:
database-container:
# configuration
backend-container:
depends_on:
- database-container
# configuration
The depends_on field will work with docker-compose, but you will find it is not supported if you upgrade to swarm mode. It also guarantees the database container is created, but not necessarily ready to receive connections.
For that, there are several options:
Let the backend container fail and configure a restart policy. This is ugly, leads to false errors being reported, but it's also the easiest to implement.
Perform a connection from your app with a retry loop, a sleep between retries, and a timeout in case the database doesn't start in a timely fashion. This is usually my preferred method, but it requires a change to your application.
Use an entrypoint script with a command like wait-for-it.sh that waits for a remote resource to become available, and once that command succeeds, launch your app. This doesn't cover all the scenarios as a complete client connection, but can be less intrusive to implement since it only requires changes to an entrypoint script rather than the app itself.
I have a docker-compose.yml file which works with docker-compose up --build. My app works and everything is fine.
version: '3'
services:
myapp:
container_name: myapp
restart: always
build: ./myapp
ports:
- "8000:8000"
command: /usr/local/bin/gunicorn -w 2 -b :8000 flaskplot:app
nginx:
container_name: nginx
restart: always
build: ./nginx
ports:
- "80:80"
depends_on:
- myapp
But when I use docker stack deploy -c docker-compose.yml myapp, I get the following error:
Ignoring unsupported options: build, restart
Ignoring deprecated options:
container_name: Setting the container name is not supported.
Creating network myapp_default
Creating service myapp_myapp
failed to create service myapp_myapp: Error response from daemon: rpc error: code = InvalidArgument desc = ContainerSpec: image reference must be provided
any hints how I should "translate" the docker-compose.yml file to make it compatible with docker stack deploy?
To run containers in swarm mode, you do not build them on each swarm node individually. Instead you build the image once, typically on a CI server, push to a registry server (often locally hosted, or you can use docker hub), and specify the image name inside your compose file with an "image" section for each service.
Doing that will get rid of the hard error. You'll likely remove the build section of the compose file since it no longer applies.
Specifying "container_name" is unsupported because it would break the ability to scale or perform updates (a container name must be unique within the docker engine). Let swarm name the containers and reference your app on the docker network by it's service name.
Specifying "depends_on" is not supported because containers may be started on different nodes, and rolling updates/failure recovery may remove some containers providing a service after the app started. Docker can retry the failing app until the other service starts up, or preferably you configure an entrypoint that waits for the dependencies to become available with some kind of ping for a minute or two.
Without seeing your Dockerfile, I'd also recommend setting up a healthcheck on each image. Swarm mode uses this to control rolling updates and recover from application failures.
Lastly, consider adding a "deploy" section to your compose file. This tells swarm mode how to deploy and update your service, including how many replicas, constraints on where to run, memory and CPU limits and requirements, and how fast to update the service. You can define a restart policy here as well but I recommend against it since I've seen docker engines restarting containers that conflict with swarm mode deploying containers on other nodes, or even a new container on the same node.
You can see the full compose file documentation with all of these options here: https://docs.docker.com/compose/compose-file/
Our team is new to running a micro-service ecosystem and I am curious how one would achieve conditionally loading docker containers from a compose, or another variable-cased script.
An example use-case would be.
Doing front-end development that depends on a few different services. We will label those DockerA/D
Dependency Matrix
Feature1 - DockerA
Feature2 - DockerA and DockerB
Feature3 - DockerA and DockerD
I would like to be able to run something like the following
docker-compose --feature1
or
magic-script -service DockerA -service DockerB
Basically, I would like to run the command to conditionally start the APIs that I need.
I am already aware of using various mock servers for UI development, but want to avoid them.
Any thoughts on how to configure this?
You can stop all services after creating them and then selectively starting them one by one. E.g.:
version: "3"
services:
web1:
image: nginx
ports:
- "80:80"
web2:
image: nginx
ports:
- "8080:80"
docker-compose up -d
Creating network "composenginx_default" with the default driver
Creating composenginx_web2_1 ... done
Creating composenginx_web1_1 ... done
docker-compose stop
Stopping composenginx_web1_1 ... done
Stopping composenginx_web2_1 ... done
Now any service can be started using, e.g.,
docker-compose start web2
Starting web2 ... done
Also, using linked services, there's the scale command that can change the number of running services (can add containers without restart).
We are using docker-compose in our dev environment to start all the services.
Does any one know if I can start just 1 service using docker-compose without checking if dependencies are running or not (since I know they are running)?
$ docker-compose -f docker-compose-service1.yml up
it gives an error: ERROR: Service ‘service1’ depends on service ‘service2’ which is undefined.
The yml file looks something like:
version: '2'
services:
service1:
build: ./service1
dns: 192.168.1.100
depends_on:
- "service2"
container_name: service1
I just want to start service1 since I know all the dependencies are already running.
If other services are running and you want to restart only one, you can use
docker-compose -f docker-compose.yml restart service1
Edit:
Regarding error:
ERROR: Service ‘service1’ depends on service ‘service2’ which is
undefined.
It is because the docker-compose.yml which is used to up the services is not acceptable to docker-compose. First the yml file is compiled (the point of failure in our case) to see if everything is as per proper syntax, and then it is executed.