I edited a file in a running docker container and restarted it, unfortunately my last edit was not correct. So every time I start the container with:
docker start <containerId>
It always exits immediately.
Now I can not even modify my edit back, since
docker exec -it <containerId> bash
can only run on a running docker.
The question is how can I change it and restart the container now? Or I had to abandon it and start a new container from an existing image?
You didn't supply any details regarding your container's purpose, or what you modified. Conceptually, you could create the file that needs to be modified in a place on your filesystem and mount that file into the container as a volume when you start it, like:
docker run -it -v /Users/<path_to_file>:<container_path_to_file> <container>
Hovever, this is bad form, as your container loses portability at that point without committing a new image.
Ideally, changes that need to be made inside of a Docker container are made in the Dockerfile, and the container image re-built. This way, your initial, working container state is represented in your Dockerfile code, making your configuration repeatable, portable, and immutable.
The file system of exited containers can still be changed. The preferable way is probably:
docker cp <fixedFile> <containerId>:<brokenFile>
But you can also circumvent docker completely; see here.
Related
I am trying Docker for the first time and do not yet have a "mental model". Total beginner.
All the examples that I am looking at have included the --rm flag to run, such as
docker run -it --rm ...
docker container run -it --rm ...
Question:
Why do these commands include the --rm flag? I would think that if I were to go through the trouble of setting up or downloading a container with the good stuff in it, why remove it? I want to keep it to use again.
So, I know I have the wrong idea of Docker.
Containers are merely an instance of the image you use to run them.
The state of mind when creating a containerized app is not by taking a fresh, clean ubuntu container for instance, and downloading the apps and configurations you wish to have in it, and then let it run.
You should treat the container as an instance of your application, but your application is embedded into an image.
The proper usage would be creating a custom image, where you embed all your files, configurations, environment variables etc, into the image. Read more about Dockerfile and how it is done here
Once you did that, you have an image that contains everything, and in order to use your application, you just run the image with proper port settings or other dynamic variables, using docker run <your-image>
Running containers with --rm flag is good for those containers that you use for very short while just to accomplish something, e.g., compile your application inside a container, or just testing something that it works, and then you are know it's a short lived container and you tell your Docker daemon that once it's done running, erase everything related to it and save the disk space.
The flag --rm is used when you need the container to be deleted after the task for it is complete.
This is suitable for small testing or POC purposes and saves the headache for house keeping.
From https://docs.docker.com/engine/reference/run/#clean-up---rm
By default a container’s file system persists even after the container exits. This makes debugging a lot easier (since you can inspect the final state) and you retain all your data by default. But if you are running short-term foreground processes, these container file systems can really pile up. If instead you’d like Docker to automatically clean up the container and remove the file system when the container exits, you can add the --rm flag
In short, it's useful to keep the host clean from stopped and unused containers.
When you run a container from an image using a simple command like (docker run -it ubuntu), it spins up a container. You attach to your container using docker attach container-name (or using exec for different session).
So, when you're within your container and working on it and you type exit or ctrl+z or any other way to come out of the container, other than ctrl+p+q, your container exits. That means that your container has stopped, but it is still available on your disk and you can start it again with : docker start container-name/ID.
But when you run the container with —rm tag, on exit, the container is deleted permanently.
I use --rm when connecting to running containers to perform some actions such as database backup or file copy. Here is an example:
docker run -v $(pwd):/mnt --link app_postgres_1:pg --rm postgres:9.5 pg_dump -U postgres -h pg -f /mnt/docker_pg.dump1 app_db
The above will connect a running container named 'app_postgres_1' and create a backup. Once the backup command completes, the container is fully deleted.
The "docker run rm " command makes us run a new container and later when our work is completed then it is deleted by saving the disk space.
The important thing to note is, the container is just like a class instance and not for data storage. We better delete them once the work is complete. When we start again, it starts fresh.
The question comes then If the container is deleted then what about the data in a container? The data is actually saved in the local system and get linked to it when the container is started. The concept is named as "Volume or shared volume".
I am trying Docker for the first time and do not yet have a "mental model". Total beginner.
All the examples that I am looking at have included the --rm flag to run, such as
docker run -it --rm ...
docker container run -it --rm ...
Question:
Why do these commands include the --rm flag? I would think that if I were to go through the trouble of setting up or downloading a container with the good stuff in it, why remove it? I want to keep it to use again.
So, I know I have the wrong idea of Docker.
Containers are merely an instance of the image you use to run them.
The state of mind when creating a containerized app is not by taking a fresh, clean ubuntu container for instance, and downloading the apps and configurations you wish to have in it, and then let it run.
You should treat the container as an instance of your application, but your application is embedded into an image.
The proper usage would be creating a custom image, where you embed all your files, configurations, environment variables etc, into the image. Read more about Dockerfile and how it is done here
Once you did that, you have an image that contains everything, and in order to use your application, you just run the image with proper port settings or other dynamic variables, using docker run <your-image>
Running containers with --rm flag is good for those containers that you use for very short while just to accomplish something, e.g., compile your application inside a container, or just testing something that it works, and then you are know it's a short lived container and you tell your Docker daemon that once it's done running, erase everything related to it and save the disk space.
The flag --rm is used when you need the container to be deleted after the task for it is complete.
This is suitable for small testing or POC purposes and saves the headache for house keeping.
From https://docs.docker.com/engine/reference/run/#clean-up---rm
By default a container’s file system persists even after the container exits. This makes debugging a lot easier (since you can inspect the final state) and you retain all your data by default. But if you are running short-term foreground processes, these container file systems can really pile up. If instead you’d like Docker to automatically clean up the container and remove the file system when the container exits, you can add the --rm flag
In short, it's useful to keep the host clean from stopped and unused containers.
When you run a container from an image using a simple command like (docker run -it ubuntu), it spins up a container. You attach to your container using docker attach container-name (or using exec for different session).
So, when you're within your container and working on it and you type exit or ctrl+z or any other way to come out of the container, other than ctrl+p+q, your container exits. That means that your container has stopped, but it is still available on your disk and you can start it again with : docker start container-name/ID.
But when you run the container with —rm tag, on exit, the container is deleted permanently.
I use --rm when connecting to running containers to perform some actions such as database backup or file copy. Here is an example:
docker run -v $(pwd):/mnt --link app_postgres_1:pg --rm postgres:9.5 pg_dump -U postgres -h pg -f /mnt/docker_pg.dump1 app_db
The above will connect a running container named 'app_postgres_1' and create a backup. Once the backup command completes, the container is fully deleted.
The "docker run rm " command makes us run a new container and later when our work is completed then it is deleted by saving the disk space.
The important thing to note is, the container is just like a class instance and not for data storage. We better delete them once the work is complete. When we start again, it starts fresh.
The question comes then If the container is deleted then what about the data in a container? The data is actually saved in the local system and get linked to it when the container is started. The concept is named as "Volume or shared volume".
I'm running the official solr 6.6 container used in a docker-compose environment without any relevant volumes.
If i modify a running solr container the data survives a restart.
I dont see any volumes mounted and it works for a plain solr container:
docker run --name solr_test -d -p 8983:8983 -t library/solr:6.6
docker exec -it solr_test /bin/bash -c 'echo woot > /opt/solr/server/solr/testfile'
docker stop solr_test
docker start solr_test
docker exec -it solr_test cat /opt/solr/server/solr/testfile
Above example prints 'woot'. I thought that a container doesnt persist any data? Also the documentation mentions that the solr cores are persisted in the container.
All i found, regarding container persistence is that i need to add volumes on my own like mentioned here.
So i'm confused: do containers store the data changed within the container or not? And how does the solr container achive this behaviour? The only option i see is that i misunderstood peristence in case of docker or the build of the container can set some kind of option to achieve this which i dont know about and didnt see in the solr Dockerfile.
This is expected behaviour.
The data you create inside a container persist as long as you don't delete the container.
But think containers in some way of throw away mentality. Normally you would want to be able to remove the container with docker rm and spawn a new instance including your modified config files. That's why you would need an e.g. named volume here, which survives a container life cycle on your host.
The Dockerfile, because you mention it in your question, actually only defines the image. When you call docker run you create a container from it. Exactly as defined in the image. A fresh instance without any modifications.
When you call docker commit on your container you snapshot it (including the changes you made to the files) and create a new image out of it. They achieve the data persistence this way.
The documentation you referring to explains this in detail.
In some places when I read about Docker containers, I found some people talking that they lose their data (saved inside the container and not a part of volume data) when they restart the container.
I tried to create a simple Ubuntu container like this: docker run -it ubuntu /bin/bash, and created some files inside the container and then restarted it, but my data still there. Why does that actually happen? why do my data still there? Is this something new in the newer versions of Docker or do I have misunderstanding for something?
The data is lost when the container is removed, not when it's stopped or restarted.
Basically, if you do docker ps, if the containers keeps the same id (the big ugly hexadecimal id), the data is not lost.
It gets complicated when somehow your docker containers are not managed by you, but by some kind of automated-managing method. Tools like these usually start new containers if there is failure. In that case you should mount a volume to store your data on the host.
You might want to look at the Container Lifecycle: https://github.com/wsargent/docker-cheat-sheet#lifecycle
docker create creates a container but does not start it.
docker rename allows the container to be renamed.
docker run creates and starts a container in one operation.
docker rm deletes a container.
docker update updates a container's resource limits.
If you do docker rm and docker run again your state will not be there anymore.
If you want a transient container, docker run --rm will remove the container after it stops.
When you initially run a Docker container from an image you can specify the option:
--restart="always"
This ensures that the container is always restarted by the Docker daemon if for some reason it stops. So you could run a container like so:
docker run --restart="always" <IMAGE>
Also you can restart an existing Docker container by specifying its container ID, i.e.:
docker start <CONTAINER ID>
However I can't determine if it's possible to change an existing container, that originally was not run with the --restart="always" option, to convert it to always restart in future.
Currently the only way I can think to do this is to save the container as a new image and then run that image as a new container with the --restart="always" option. Would this in fact be the correct way to do this?
EDIT: What I perhaps didn't make clear enough originally is that I am thinking about the situation where there have been changes in the container since it was originally run, which need to be persisted. So just running a new container from the original image would not be sufficient.
We now have docker update, which allows changing the restart policy of a running container.
docker update --restart=always <CONTAINER ID>
There are three other options:
no (default)
on-failure
unless-stopped
Please refer to the link for details.
Ok, so to answer my own question, it seems that it's not possible just to restart the same container with --restart=always, because that's something you have to do when you run a container for the first time and not a parameter that you can use when you start an existing container.
There are three possible work-arounds to this:
As #user2915097 stated, you can abandon the original container (stopping it and then deleting it with docker rm <CONTAINER ID>to tidy up). Then just run a new container from the original image specifying the -restart=always option this time.
If no volumes were used, so the changes are internal to the container, you need to commit the container to a new image and then run a new container from that image.
docker commit <CONTAINER ID> <NEW IMAGE NAME>
docker run -d --restart=always ... <NEW IMAGE NAME>
If volumes were used and all changes are restricted to the volumes, then you can run a second container with the --volumes-from parameter without having to commit a new version of the image. i.e.
docker stop <CONTAINER 1 NAME>
docker run -d --restart=always --volumes-from <CONTAINER 1 NAME> ... <ORIGINAL IMAGE NAME>
It would then be safe to delete Container 1, as the volumes will not be deleted whilst another container continues to use them.
I guess there is a fourth possibility too; if you used a volume(s) and you know that there have been changes to the container that aren't on the volume, then you'll have to use a combination of (2) and (3).
Update: This worked to enable restart. But setting it back to no and it gets reset back to always and the container starts again! :( I'm going to leave this answer here in case someone figures out how this really works. I must be close!
Folks, I've found the most hacky solution that gets around copying containers etc.
vi /var/lib/docker/containers/$(docker inspect -f '{{ .Id }}' $ContainerID)/hostconfig.json
Search for "RestartPolicy". Set it to "no", "always" etc
Maybe someone could wrap that up in a script!?
Anyway, that piece of json along with the config.json would allow you to modify all sorts of things that you missed when creating your container.
extract from http://www.brandpending.com/blog/2014/11/21/setting-and-re-setting-the-restart-behaviour-of-a-docker-container
So let say you want to change the restart policy of this container
from always to on-failure. To do this, you need to stop the container,
remove it and re-run it with the new restart policy.