I'm running the official solr 6.6 container used in a docker-compose environment without any relevant volumes.
If i modify a running solr container the data survives a restart.
I dont see any volumes mounted and it works for a plain solr container:
docker run --name solr_test -d -p 8983:8983 -t library/solr:6.6
docker exec -it solr_test /bin/bash -c 'echo woot > /opt/solr/server/solr/testfile'
docker stop solr_test
docker start solr_test
docker exec -it solr_test cat /opt/solr/server/solr/testfile
Above example prints 'woot'. I thought that a container doesnt persist any data? Also the documentation mentions that the solr cores are persisted in the container.
All i found, regarding container persistence is that i need to add volumes on my own like mentioned here.
So i'm confused: do containers store the data changed within the container or not? And how does the solr container achive this behaviour? The only option i see is that i misunderstood peristence in case of docker or the build of the container can set some kind of option to achieve this which i dont know about and didnt see in the solr Dockerfile.
This is expected behaviour.
The data you create inside a container persist as long as you don't delete the container.
But think containers in some way of throw away mentality. Normally you would want to be able to remove the container with docker rm and spawn a new instance including your modified config files. That's why you would need an e.g. named volume here, which survives a container life cycle on your host.
The Dockerfile, because you mention it in your question, actually only defines the image. When you call docker run you create a container from it. Exactly as defined in the image. A fresh instance without any modifications.
When you call docker commit on your container you snapshot it (including the changes you made to the files) and create a new image out of it. They achieve the data persistence this way.
The documentation you referring to explains this in detail.
Related
I am trying Docker for the first time and do not yet have a "mental model". Total beginner.
All the examples that I am looking at have included the --rm flag to run, such as
docker run -it --rm ...
docker container run -it --rm ...
Question:
Why do these commands include the --rm flag? I would think that if I were to go through the trouble of setting up or downloading a container with the good stuff in it, why remove it? I want to keep it to use again.
So, I know I have the wrong idea of Docker.
Containers are merely an instance of the image you use to run them.
The state of mind when creating a containerized app is not by taking a fresh, clean ubuntu container for instance, and downloading the apps and configurations you wish to have in it, and then let it run.
You should treat the container as an instance of your application, but your application is embedded into an image.
The proper usage would be creating a custom image, where you embed all your files, configurations, environment variables etc, into the image. Read more about Dockerfile and how it is done here
Once you did that, you have an image that contains everything, and in order to use your application, you just run the image with proper port settings or other dynamic variables, using docker run <your-image>
Running containers with --rm flag is good for those containers that you use for very short while just to accomplish something, e.g., compile your application inside a container, or just testing something that it works, and then you are know it's a short lived container and you tell your Docker daemon that once it's done running, erase everything related to it and save the disk space.
The flag --rm is used when you need the container to be deleted after the task for it is complete.
This is suitable for small testing or POC purposes and saves the headache for house keeping.
From https://docs.docker.com/engine/reference/run/#clean-up---rm
By default a container’s file system persists even after the container exits. This makes debugging a lot easier (since you can inspect the final state) and you retain all your data by default. But if you are running short-term foreground processes, these container file systems can really pile up. If instead you’d like Docker to automatically clean up the container and remove the file system when the container exits, you can add the --rm flag
In short, it's useful to keep the host clean from stopped and unused containers.
When you run a container from an image using a simple command like (docker run -it ubuntu), it spins up a container. You attach to your container using docker attach container-name (or using exec for different session).
So, when you're within your container and working on it and you type exit or ctrl+z or any other way to come out of the container, other than ctrl+p+q, your container exits. That means that your container has stopped, but it is still available on your disk and you can start it again with : docker start container-name/ID.
But when you run the container with —rm tag, on exit, the container is deleted permanently.
I use --rm when connecting to running containers to perform some actions such as database backup or file copy. Here is an example:
docker run -v $(pwd):/mnt --link app_postgres_1:pg --rm postgres:9.5 pg_dump -U postgres -h pg -f /mnt/docker_pg.dump1 app_db
The above will connect a running container named 'app_postgres_1' and create a backup. Once the backup command completes, the container is fully deleted.
The "docker run rm " command makes us run a new container and later when our work is completed then it is deleted by saving the disk space.
The important thing to note is, the container is just like a class instance and not for data storage. We better delete them once the work is complete. When we start again, it starts fresh.
The question comes then If the container is deleted then what about the data in a container? The data is actually saved in the local system and get linked to it when the container is started. The concept is named as "Volume or shared volume".
I have an docker container running prometheus and sometimes I have to update an config file inside the container, the problem is that I don't know who I can update this file without deleting and creating the container again.
docker run --network="host" -d --name=prometheus -p 9090:9090 -v ~/prometheus.yaml:/etc/prometheus/prometheus.yml prom/prometheus --config.file=/etc/prometheus/prometheus.yml
I want to know how I can update the prometheus.yaml without deleting and creating again the docker container.
You should VOLUME the data path of Prometheus outside of your container.
So if the container creates again, you can have your previous data.
The default data path of Prometheus is ./data but in docker it depends on your base-image.
In theory you can't since by principle containers are ephemeral. Meaning they're supposed to be disposable upon exiting. However, there are a few ways out of your predicament:
#1. Create a new Image from your running container: https://www.scalyr.com/blog/create-docker-image/ to persist the state.
#2. Copy your data from within the container to the "outside world" as a backup, if option 1 is not the right option for you (here's an explanation how to do so: https://linuxhandbook.com/docker-cp-example/). You could also log in to the container (docker exec -it <container-name> bash) and then use yum or apt install (depending on your base image) to install the necessary tools to make your backup (rsync, ...), if the sometimes very barebones baseimage does not provide these.
#3. As #Amir already mentioned, you should always create a Volume inside your Container and map it to the outside world to have a persistent data storage. You create a Volume by the VOLUME-Keyword in the Dockerfile: https://docs.docker.com/storage/volumes/ ..by doing so you can restart the container everytime if the config changes without worrying about data loss.
HTH
Use the reload url
Prometheus can reload its configuration at runtime. If the new configuration is not well-formed, the changes will not be applied. A configuration reload is triggered by sending a SIGHUP to the Prometheus process or sending a HTTP POST request to the /-/reload endpoint (when the --web.enable-lifecycle flag is enabled). This will also reload any configured rule files.
Use the following the change the config inside the container using:
docker exec -it <container_name> sh
Map the config to outside the docker container for persistence using
-v <host-path>:<container_path>
I have a container (i'm using this container https://hub.docker.com/_/mysql/) which had started before, with ID 5f96e9570d1b1475a888d7a615acdd9a7715c1ed6f0c40900f2e9c1ab485c7cf, but now how can i restart it ? I tried this command but not work
$ docker run --name mysql -e MYSQL_ROOT_PASSWORD=*Abcd1234 -d mysql:5.7
D:\CWindow10\Docker Toolbox\docker.exe: Error response from daemon: Conflict. The container name "/mysql" is already in use by container "5f96e9570d1b1475a888d7a615acdd9a7715c1ed6f0c40900f2e9c1ab485c7cf". You have to remove (or rename) that container to be able to reuse that name.
See 'D:\CWindow10\Docker Toolbox\docker.exe run --help'.
If i delete the container and retype the command, will the old data still exist in new container?
To restart an existing container, simply run docker start <container_name_or_id>.
Regarding the data: docker uses the concept of volumes to put data. For the mysql image, there's a section "Where to Store Data" on the docker hub site. If you don't manually declare where the image should go, docker will create one for you. If you want your data to be kept, the easiest way is to create a folder and tell the docker run command to map that volume. That way, you can still use it if you throw away your container.
use this command to restart container docker restart <CONTAINER>
starting new container will not preserve your data unless you have mounted external volume and stored data on it. Have a look at this blog http://blog.arungupta.me/docker-mysql-persistence/
In some places when I read about Docker containers, I found some people talking that they lose their data (saved inside the container and not a part of volume data) when they restart the container.
I tried to create a simple Ubuntu container like this: docker run -it ubuntu /bin/bash, and created some files inside the container and then restarted it, but my data still there. Why does that actually happen? why do my data still there? Is this something new in the newer versions of Docker or do I have misunderstanding for something?
The data is lost when the container is removed, not when it's stopped or restarted.
Basically, if you do docker ps, if the containers keeps the same id (the big ugly hexadecimal id), the data is not lost.
It gets complicated when somehow your docker containers are not managed by you, but by some kind of automated-managing method. Tools like these usually start new containers if there is failure. In that case you should mount a volume to store your data on the host.
You might want to look at the Container Lifecycle: https://github.com/wsargent/docker-cheat-sheet#lifecycle
docker create creates a container but does not start it.
docker rename allows the container to be renamed.
docker run creates and starts a container in one operation.
docker rm deletes a container.
docker update updates a container's resource limits.
If you do docker rm and docker run again your state will not be there anymore.
If you want a transient container, docker run --rm will remove the container after it stops.
I edited a file in a running docker container and restarted it, unfortunately my last edit was not correct. So every time I start the container with:
docker start <containerId>
It always exits immediately.
Now I can not even modify my edit back, since
docker exec -it <containerId> bash
can only run on a running docker.
The question is how can I change it and restart the container now? Or I had to abandon it and start a new container from an existing image?
You didn't supply any details regarding your container's purpose, or what you modified. Conceptually, you could create the file that needs to be modified in a place on your filesystem and mount that file into the container as a volume when you start it, like:
docker run -it -v /Users/<path_to_file>:<container_path_to_file> <container>
Hovever, this is bad form, as your container loses portability at that point without committing a new image.
Ideally, changes that need to be made inside of a Docker container are made in the Dockerfile, and the container image re-built. This way, your initial, working container state is represented in your Dockerfile code, making your configuration repeatable, portable, and immutable.
The file system of exited containers can still be changed. The preferable way is probably:
docker cp <fixedFile> <containerId>:<brokenFile>
But you can also circumvent docker completely; see here.