I have a running docker container using an ancestor my_base_image. Now when the container is running, can I set an environment variable using export command with docker exec? if yes, how?
I tried using the following, but doesn't work
docker exec -i -t $(docker ps -q --filter ancestor=`my_base_image`) bash -c "export my_env_var=hey"
Basically I want to set my_env_var=hey as env variable inside docker container. I know this can be done in may ways using .env_file or env key docker-compose & ENV in Dockerfile. But I just want to know if it is possible using docker exec command
This is impossible. A process can never change the environment of any other process beyond itself, except that it can specify the initial environment of processes it starts itself. In this case, your docker exec shell isn’t launching the main container process, so it can’t change that process’s environment variables.
This is one of a number of changes that you will need to stop, delete, and recreate the container for. You should treat this as extremely routine container maintenance and plan to delete the container eventually. That means, for example, keeping any data that needs to be persisted outside the container, ideally in an external database but possibly in a mounted volume.
Related
Is it possible to change the settings of docker container like entrypoint, ports or memory-limits without having to delete the container and run using docker run command? Example: docker stop <container_id>, change settings and then docker start <container_id>?
When you use docker run -d image_name, some images tries to initialize from start and as a result I can't use the same volume.
Is it possible to change the settings by stopping the container instead of re-run?
You need to stop, delete, and recreate the container.
# this is absolutely totally 100% normal and routine
docker stop my_container
docker rm my_container
# docker build -t image_name .
docker run -d -p 12345:8000 --name my_container image_name
This isn't specific to Docker. If you run any command in any Unix-like environment and you want to change its command-line parameters or environment variables, you need to stop the process and create a new one. A Docker container is a wrapper around a process with some additional isolation features, and for a great many routine things you're required to delete the container. In cluster container environments like Kubernetes, this is routine enough that changing any property of a Deployment object will cause all of the associated containers (Kubernetes Pods) to get recreated automatically.
There are a handful of Docker commands that exist but are almost never used in normal operation. docker start is among these; just skip over it in the documentation.
When you use docker run -d image_name, some images tries to initialize from start and as a result I can't use the same volume.
In fact, the normal behavior of docker run is that you're always beginning the program from a known "clean" initial state; this is easier to set up as an application developer than trying to recover from whatever state the previous run of the application might have been left in.
If you need to debug the image startup, an easy thing to do is to tell the container to run an interactive shell instead of its default command
docker run --rm -it image_name /bin/sh
(Some images may have bash available which will be more comfortable to work in; some images may require an awkward docker run --entrypoint option.) From this shell you can try to manually run the container startup commands and see what happens. You don't need to worry about damaging the container code in any particular way, since anything you change in this shell will get lost as soon as the container exits.
I am trying Docker for the first time and do not yet have a "mental model". Total beginner.
All the examples that I am looking at have included the --rm flag to run, such as
docker run -it --rm ...
docker container run -it --rm ...
Question:
Why do these commands include the --rm flag? I would think that if I were to go through the trouble of setting up or downloading a container with the good stuff in it, why remove it? I want to keep it to use again.
So, I know I have the wrong idea of Docker.
Containers are merely an instance of the image you use to run them.
The state of mind when creating a containerized app is not by taking a fresh, clean ubuntu container for instance, and downloading the apps and configurations you wish to have in it, and then let it run.
You should treat the container as an instance of your application, but your application is embedded into an image.
The proper usage would be creating a custom image, where you embed all your files, configurations, environment variables etc, into the image. Read more about Dockerfile and how it is done here
Once you did that, you have an image that contains everything, and in order to use your application, you just run the image with proper port settings or other dynamic variables, using docker run <your-image>
Running containers with --rm flag is good for those containers that you use for very short while just to accomplish something, e.g., compile your application inside a container, or just testing something that it works, and then you are know it's a short lived container and you tell your Docker daemon that once it's done running, erase everything related to it and save the disk space.
The flag --rm is used when you need the container to be deleted after the task for it is complete.
This is suitable for small testing or POC purposes and saves the headache for house keeping.
From https://docs.docker.com/engine/reference/run/#clean-up---rm
By default a container’s file system persists even after the container exits. This makes debugging a lot easier (since you can inspect the final state) and you retain all your data by default. But if you are running short-term foreground processes, these container file systems can really pile up. If instead you’d like Docker to automatically clean up the container and remove the file system when the container exits, you can add the --rm flag
In short, it's useful to keep the host clean from stopped and unused containers.
When you run a container from an image using a simple command like (docker run -it ubuntu), it spins up a container. You attach to your container using docker attach container-name (or using exec for different session).
So, when you're within your container and working on it and you type exit or ctrl+z or any other way to come out of the container, other than ctrl+p+q, your container exits. That means that your container has stopped, but it is still available on your disk and you can start it again with : docker start container-name/ID.
But when you run the container with —rm tag, on exit, the container is deleted permanently.
I use --rm when connecting to running containers to perform some actions such as database backup or file copy. Here is an example:
docker run -v $(pwd):/mnt --link app_postgres_1:pg --rm postgres:9.5 pg_dump -U postgres -h pg -f /mnt/docker_pg.dump1 app_db
The above will connect a running container named 'app_postgres_1' and create a backup. Once the backup command completes, the container is fully deleted.
The "docker run rm " command makes us run a new container and later when our work is completed then it is deleted by saving the disk space.
The important thing to note is, the container is just like a class instance and not for data storage. We better delete them once the work is complete. When we start again, it starts fresh.
The question comes then If the container is deleted then what about the data in a container? The data is actually saved in the local system and get linked to it when the container is started. The concept is named as "Volume or shared volume".
I am trying Docker for the first time and do not yet have a "mental model". Total beginner.
All the examples that I am looking at have included the --rm flag to run, such as
docker run -it --rm ...
docker container run -it --rm ...
Question:
Why do these commands include the --rm flag? I would think that if I were to go through the trouble of setting up or downloading a container with the good stuff in it, why remove it? I want to keep it to use again.
So, I know I have the wrong idea of Docker.
Containers are merely an instance of the image you use to run them.
The state of mind when creating a containerized app is not by taking a fresh, clean ubuntu container for instance, and downloading the apps and configurations you wish to have in it, and then let it run.
You should treat the container as an instance of your application, but your application is embedded into an image.
The proper usage would be creating a custom image, where you embed all your files, configurations, environment variables etc, into the image. Read more about Dockerfile and how it is done here
Once you did that, you have an image that contains everything, and in order to use your application, you just run the image with proper port settings or other dynamic variables, using docker run <your-image>
Running containers with --rm flag is good for those containers that you use for very short while just to accomplish something, e.g., compile your application inside a container, or just testing something that it works, and then you are know it's a short lived container and you tell your Docker daemon that once it's done running, erase everything related to it and save the disk space.
The flag --rm is used when you need the container to be deleted after the task for it is complete.
This is suitable for small testing or POC purposes and saves the headache for house keeping.
From https://docs.docker.com/engine/reference/run/#clean-up---rm
By default a container’s file system persists even after the container exits. This makes debugging a lot easier (since you can inspect the final state) and you retain all your data by default. But if you are running short-term foreground processes, these container file systems can really pile up. If instead you’d like Docker to automatically clean up the container and remove the file system when the container exits, you can add the --rm flag
In short, it's useful to keep the host clean from stopped and unused containers.
When you run a container from an image using a simple command like (docker run -it ubuntu), it spins up a container. You attach to your container using docker attach container-name (or using exec for different session).
So, when you're within your container and working on it and you type exit or ctrl+z or any other way to come out of the container, other than ctrl+p+q, your container exits. That means that your container has stopped, but it is still available on your disk and you can start it again with : docker start container-name/ID.
But when you run the container with —rm tag, on exit, the container is deleted permanently.
I use --rm when connecting to running containers to perform some actions such as database backup or file copy. Here is an example:
docker run -v $(pwd):/mnt --link app_postgres_1:pg --rm postgres:9.5 pg_dump -U postgres -h pg -f /mnt/docker_pg.dump1 app_db
The above will connect a running container named 'app_postgres_1' and create a backup. Once the backup command completes, the container is fully deleted.
The "docker run rm " command makes us run a new container and later when our work is completed then it is deleted by saving the disk space.
The important thing to note is, the container is just like a class instance and not for data storage. We better delete them once the work is complete. When we start again, it starts fresh.
The question comes then If the container is deleted then what about the data in a container? The data is actually saved in the local system and get linked to it when the container is started. The concept is named as "Volume or shared volume".
I'm running the official solr 6.6 container used in a docker-compose environment without any relevant volumes.
If i modify a running solr container the data survives a restart.
I dont see any volumes mounted and it works for a plain solr container:
docker run --name solr_test -d -p 8983:8983 -t library/solr:6.6
docker exec -it solr_test /bin/bash -c 'echo woot > /opt/solr/server/solr/testfile'
docker stop solr_test
docker start solr_test
docker exec -it solr_test cat /opt/solr/server/solr/testfile
Above example prints 'woot'. I thought that a container doesnt persist any data? Also the documentation mentions that the solr cores are persisted in the container.
All i found, regarding container persistence is that i need to add volumes on my own like mentioned here.
So i'm confused: do containers store the data changed within the container or not? And how does the solr container achive this behaviour? The only option i see is that i misunderstood peristence in case of docker or the build of the container can set some kind of option to achieve this which i dont know about and didnt see in the solr Dockerfile.
This is expected behaviour.
The data you create inside a container persist as long as you don't delete the container.
But think containers in some way of throw away mentality. Normally you would want to be able to remove the container with docker rm and spawn a new instance including your modified config files. That's why you would need an e.g. named volume here, which survives a container life cycle on your host.
The Dockerfile, because you mention it in your question, actually only defines the image. When you call docker run you create a container from it. Exactly as defined in the image. A fresh instance without any modifications.
When you call docker commit on your container you snapshot it (including the changes you made to the files) and create a new image out of it. They achieve the data persistence this way.
The documentation you referring to explains this in detail.
I am running a docker container using --env VAR="foo" to set a few variables. When I run commands on this running container from the same shell / environment that I started the container, everything is fine.
The problem is that now I want to run commands against this container from cron. When cron runs the same command, the ENV variables in the container no longer exist.
How can I persist these ENV variables in the container regardless of where it is accessed from?
Edit:
To clarify, I docker run from a standard shell. In the cron job, I use docker exec and that is when the ENV vars disappear.
I have also noticed that some host machines I can't do any exec's on docker containers from a cron job.
I presume you use docker run inside your cron task.
If that's the case that's normal. You are starting a new container from the same image.
If you want to use the same container (with all your env variables set), you can use docker exec.
https://docs.docker.com/reference/commandline/exec/