I just get a strange behavior when I run ‘docker-compose up -d’, it spin-up the container but it also remove one of my existing container.
This behavior does not happen at my other environment when I try to run the same docker-compose file.
Is the anyone have same experience and able to solve it ?
Any suggestion how to trace the root cause of this behavior ?
Thanks before.
I do not know what the structure of your project is, It would be better if you attached the docker-compose.yml file.
However when running docker-compose, if the containers name is the same, the previous container will be stopped and a container with the same name will be created again.
try the following method, this method may help you.
Have two or more separate docker-compose.yml files and execute each one you want as needed as follows.
docker-compose -f docker-compose1.yml up -d
docker-compose -f docker-compose2.yml up -d
...
Related
I'm new to the world of containers and specially when it comes to Docker Compose. I'm confused about some concepts and I can't find information about them.
Basically I want to know if I can handle settings in different "docker-compose.yml" files in a isolated manner. I explain better... I would like to know if I can up or stop resources referring to a specific "docker-compose.yml" individually.
PLUS:
To better explain my doubt I'll show you some conjectures about what I'm trying to explain.
It seems to me that it is possible to have multiple configurations for Docker Compose using different ".yml" files like the example below...
EXAMPLE
docker-compose -f docker-compose.a.yml -f docker-compose.b.yml up -d
... and that I can also handle each of these settings individually, such as stopping all the resources referring to a specific docker-compose.yml...
EXAMPLE
docker-compose -f docker-compose.b.yml stop
[Ref(s).: https://gabrieltanner.org/blog/docker-compose#using-multiple-docker-compose-files , https://stackoverflow.com/q/29835905/3223785 , https://stackoverflow.com/questions/42287426/docker-multiple-environments , https://runnable.com/docker/advanced-docker-compose-configuration ]
Yes, it is possible. I'm not exactly sure what you are trying to do, but to be able to manage the services using -f option the way that you described, there shouldn't be a service with the same name on multiple files.
For example, if you have a service called db in docker-compose.a.yml and one other db service in docker-compose.b.yml. The following command will only built one container for db service:
docker-compose -f docker-compose.a.yml -f docker-compose.b.yml up -d
Take a look at -p option. It will make a project with the services isolated inside it. Then you can manage them using following commands with the same docker-compose.yml file:
docker-compose -p foo up -d
docker-compose -p foo stop [service_name]
yes you can.
It is just a matter of preference but i usually create a folder for every project i have, each of them have a unique docker-compose.yml file in it with all its dependencies (frontend / database /redis)
Then to start a specific project i just go inside its folder and run docker-compose up. it then only starts this project without touching others.
you can also type this if you only want to start redis.
docker-compose up redis
All docker-compose subcommands (up, stop, down...) must be executed consuming a docker-compose<.SOME_OPT_VAL>.yml file.
This docker-compose.yml file must be in the folder where the docker-compose command is executed or must be informed via the -f flag. This way, these subcommands will be executed on the "services" (resources) defined in the docker-compose.yml file.
There is also the possibility of defining the service where a certain subcommand will be executed...
MODELS
docker-compose <SUBCOMAMND> <OPT_SERVICE_NAME>
docker-compose -f <DOCKER_COMPOSE_YML> <SUBCOMAMND> <OPT_SERVICE_NAME>
EXAMPLES
docker-compose stop api
docker-compose -f docker-compose.yml stop api
I have ran into this problem when opening the project in container.
Setting up container for folder or workspace: c:\Work\playground\moodle\lms_administrace
Run: docker-compose -f c:\Work\playground\moodle\lms_administrace\docker\docker-compose-dev.yml config --services
app
redis
db
phpmyadmin
Run: docker-compose --project-name docker -f c:\Work\playground\moodle\lms_administrace\docker\docker-compose-dev.yml up -d --build
Creating volume "docker_mysql_data_volume" with default driver
Pulling app (nodejs:)...
ERROR: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.
Continue with the new image? [yN]
The problem is that I cannot press y or N. I know why I'm having this problem - because I have used that docker compose file before and containers and volumes were created with the directory prefix (docker).
There's a way how to change the compose project name through .env file, but it does not work (I put the file in the root directory, in the directory where compose file is, and in the .devcontainer folder). And also there is -p parameter, but the MS GitHub page does not provide any information.
I can probably fix it by renaming everything, but this may be a serious issue since you can't continue in the process ...
Did anybody experienced similar problem and fixed that?
Thanks,
Karel
You probably mistyped service docker image name in docker-compose.yml.
You are trying to pull nodejs image instead of node
Also, there is could be same error with case postgresql and postgres.
I had the same problem,My problem is using the wrong mirror name.
I’m starting up a few containers with a docker-compose.yml file. I would like all of them to start in detached mode except for one, which i would like to run attached.
I can pass the detach flag on the command line with
docker-compose up -d
But I wonder if it’s possible to specify this in the yml file?
You could try docker-compose up -d service_name for the dettached ones and just docker-compose up service_name for the other.
Not possible to declare in the compose file as far as I know.
I enjoy a lot using docker-compose.
Eg. on my server, when I want to update my app with minor changes, I only need to git pull origin master && docker-compose restart, works perfectly.
But sometimes, I need to rebuild (eg. I added an npm dependency, need to run npm install again).
In this case, I do docker-compose build --no-cache && docker-compose restart.
I would expect this to :
create a new instance of my container
stop the existing container (after the newer has finished building)
start the new one
optionally remove the old one, but this could be done manually
But in practice it seems to restart the former one again.
Is it the expected behavior?
How can I handle a rebuild and start the new one after it is built?
Maybe I missed a specific command? Or would it make sense to have it?
from the manual docker-compose restart
If you make changes to your docker-compose.yml configuration these
changes will not be reflected after running this command.
you should be able to do
$docker-compose up -d --no-deps --build <service_name>
The --no-deps will not start linked services.
The problem is that restart will restart your current containers, which is not what you want.
As an example, I just did this
change the docker file for one of the images
call docker-compose build to build the images
call docker-compose down1 and docker-compose up
docker-compose restart will NOT work here
using docker-compose start instead also does not work
To be honest, i'm not completly sure you need to do a down first, but that should be easy to check.1 The bottomline is that you need to call up. You will see the containers of unchanged images restarting, but for the changed image you'll see recreating.
The advantage of this over just calling up --build is that you can see the building-process first before you restart.
1: from the comments; down is not needed, you can just call up --build. Down has some "down"-sides, including possible being destructive to your (volume-)data.
Use the --build flag to the up command, along with the -d flag to run your containers in the background:
docker-compose up -d --build
This will rebuild all images defined in your compose file, then restart any containers whose images have changed.
-d assumes that you don't want to keep everything running in your shell foreground. This makes it act more like restart, but it's not required.
Don't manage your application environment directly. Use deployment tool like Rancher / Kubernetes. Using one you will be able to upgrade your dockerized application without any downtime and even downgrade it should you need to.
Running Rancher is as easy as running another docker container as this tool is available in the Docker Hub.
You can use Swarm. Init swarm first by docker swarm init command and use healthcheck in docker-compose.yml.
Then run below command:
docker stack deploy -c docker-compose.yml project_name
instead of
docker-compose up -d.
When docker-compose.yml file is updated only run this command again:
docker stack deploy -c docker-compose.yml project_name
Docker Swarm will create new version of services and stop old version after that.
Though the accepted answer shall work to rebuild the container before starting the new one as a replacement, it is ok for simple use case, but the container will still be down during new container initialization process. If this is quite long, it can be an issue.
I managed to achieve rolling updates with docker-compose (along with a nginx reverse proxy), and detailed how I built that in this github issue: https://github.com/docker/compose/issues/1786#issuecomment-579794865
Hope it can help!
Run the following commands:
docker-compose pull
docker-compose up -d --no-deps --build <service_name>
As the top rated answer mentioned
docker-compose up -d --no-deps --build <service_name>
will restart a single service without taking down the whole compose.
I just wanted to add to the top answer in case anyone is unsure how to update an image without restarting the container.
Another way:
docker-compose restart in your case could be replaced with docker-compose up -d --force-recreate, see https://docs.docker.com/compose/reference/up/
Running docker-compose up while docker-compose is in the running state, will recreate container that got their configuration changed.
Thats the easiest way, and it will only affect containers that got their configuration changed.
root#docker:~# docker-compose up
traefik is up-to-date
nginx is up-to-date
Recreating php ... done
My docker images are built on a Jenkins CI server and are pushed to our private Docker Registry. My goal is to provision environments with docker-compose which always start the originally built state of the images.
I am currently using docker-compose 1.3.2 as well as 1.4.0 on different machines but we also used older versions previously.
I always used the docker-compose pull && docker-compose up -d commands to fetch the fresh images from the registry and start them up. I believe my preferred behaviour was working as expected up to a certain point in time, but since then docker-compose up started to re-run previously stopped containers instead of starting the originally built images every time.
Is there a way to get rid of this behaviour? Could that way be one which is wired in the docker-compose.yml configuration file to not depend "not forgetting" something on the command line upon every invocation?
ps. Besides finding a way to achieve my goal, I would also love to know a bit more about the background of this behaviour. I think the basic idea of Docker is to build an immutable infrastructure. The current behaviour of docker-compose just seem to plain clash with this approach.. or do I miss some points here?
docker-compose up --force-recreate is one option, but if you're using it for CI, I would start the build with docker-compose rm -f to stop and remove the containers and volumes (then follow it with pull and up).
This is what I use:
docker-compose rm -f
docker-compose pull
docker-compose up --build -d
# Run some tests
./tests
docker-compose stop -t 1
The reason containers are recreated is to preserve any data volumes that might be used (and it also happens to make up a lot faster).
If you're doing CI you don't want that, so just removing everything should get you want you want.
Update: use up --build which was added in docker-compose 1.7
The only solution that worked for me was the --no-cache flag:
docker-compose build --no-cache
This will automatically pull a fresh image from the repo. It also won't use the cached version that is prebuilt with any parameters you've been using before.
By current official documentation there is a shortcut that stops and removes containers, networks, volumes, and images created by up, if they are already stopped or partially removed and so on, then it will do the trick too:
docker-compose down
Then if you have new changes on your images or Dockerfiles use:
docker-compose build --no-cache
Finally:docker-compose up
In one command: docker-compose down && docker-compose build --no-cache && docker-compose up
docker-compose up --build # still use image cache
OR
docker-compose build --no-cache # never use cache
You can pass --force-recreate to docker compose up, which should use fresh containers.
I think the reasoning behind reusing containers is to preserve any changes during development. Note that Compose does something similar with volumes, which will also persist between container recreation (a recreated container will attach to its predecessor's volumes). This can be helpful, for example, if you have a Redis container used as a cache and you don't want to lose the cache each time you make a small change. At other times it's just confusing.
I don't believe there is any way you can force this from the Compose file.
Arguably it does clash with immutable infrastructure principles. The counter-argument is probably that you don't use Compose in production (yet). Also, I'm not sure I agree that immutable infra is the basic idea of Docker, although it's certainly a good use case/selling point.
docker-compose up --build --force-recreate
I claimed 3.5gb space in ubuntu AWS through this.
clean docker
docker stop $(docker ps -qa) && docker system prune -af --volumes
build again
docker build .
docker-compose build
docker-compose up
Also if the compose has several services and we only want to force build one of those:
docker-compose build --no-cache <service>
together with --force-recreate,
you might want to consider using this flag too:
-V, --renew-anon-volumes Recreate anonymous volumes instead of retrieving
data from the previous containers.
I'm not sure from which version this flag is available, so check your docker-compose up --help if you have it or not
$docker-compose build
If there is something new it will be rebuilt.