I have a Meteor App deployed with Meteor UP to Ubuntu.
From this App I need to read a file which is located outside App container on the host server.
How can I do that?
I've tried to set up volumes in the mup.js but no luck. It seems that I'm missing how to correctly provide /host/path and /container/path
volumes: {
// passed as '-v /host/path:/container/path' to the docker run command
'/host/path': '/container/path',
'/second/host/path': '/second/container/path'
},
Read the docs for Docker mounting volumes but obviously can't understand it.
Let's say file is in /home/dirname/filename.csv.
How to correctly mount it into App to be able to access it from the Application?
Or maybe there are other possibilities to access it?
Welcome to Stack Overflow. Let me suggest another way of thinking about this...
In a scalable cluster, docker instances can be spun up and down as the load on the app changes. These may or may not be on the same host computer, so building a dependency on the file system of the host isn't a great idea.
You might be better to think of using a file storage mechanism such as S3, which will scale on its own, and disk storage limits won't apply.
Another option is to determine if the files could be stored in the database.
I hope that helps
Let's try to narrow the problem down.
Meteor UP is passing the configuration parameter volumes directly on to docker, as they also mention in the comment you included. It therefore might be easier to test it against docker directly - narrowing the components involved down as much as possible:
sudo docker run \
-it \
--rm \
-v "/host/path:/container/path" \
-v "/second/host/path:/second/container/path" \
busybox \
/bin/sh
Let me explain this:
sudo because Meteor UP uses sudo to start the container. See: https://github.com/zodern/meteor-up/blob/3c7120a75c12ea12fdd5688e33574c12e158fd07/src/plugins/meteor/assets/templates/start.sh#L63
docker run we want to start a container.
-it to access the container (think of it like SSH'ing into the container).
--rm to automatically clean up - remove the container - after we're done.
-v - here we give the volumes as you define it (I here took the two directories example you provided).
busybox - an image with some useful tools.
/bin/sh - the application to start the container with
I'd expect that you also cannot access the files here. In this case, dig deeper on why you can't make a folder accessible in Docker.
If you can, which would sound weird to me, you can start the container and try to access into the container by running the following command:
docker exec -it my-mup-container /bin/sh
You can think of this command like SSH'ing into a running container. Now you can check around if it really isn't there, if the credentials inside the container are correct, etc.
At last, I have to agree it #mikkel, that it's not a good option to mount a local directoy, but you can now start looking into how to use docker volume to mount a remote directory. He mentioned S3 by AWS, I've worked with AzureFiles on Azure, there are plenty of possibilities.
Related
I've been trying the whole day to accomplish a simplistic example of sharing a Windows directory to Linux container running on Windows Docker host.
Have read all the guidelines and run the following:
docker run -it --rm -p 5002:80 --name mount-test --mount type=bind,source=D:\DockerArea\PortScanner,target=/app/PortScannerWorkingDirectory barebonewebapi:latest
The origin PortScanner directory on host machine has got some text file in it. The container is created successfully.
The issue is that when I'm trying to
docker exec -it mount-test /bin/bash
and then list the mounted directory in the container PortScannerWorkingDirectory - it just shows that it's empty. Nor the C# code can read the contents of the host file in the mapped directory.
Am I missing something simple here? I feel like I got stuck and can't share files on the host Windows machine to Linux container.
After several days of dealing with the issue I've found quite apparent answer. Although I had had C and D drives already shared to Docker in Docker settings I did an experiment and re-shared both drives (there's a special button Reset Credentials for that purpose in Docker agent settings for Windows). After that the issue is resolved. So saving it here with the hope that it may help someone else since this seems to be a glitch with permissions or similar.
The issue is quite hard to diagnose - when there's an issue the Docker container just silently writes into its writable layer and no error pops up.
Go to the docker settings -> shared drives -> reset credentials.
and then click the drive and click apply button.
then execute following command as suggested by docker
docker run --rm -v c:/Users:/data alpine ls /data
I use portainer to manage containers and it works great.
https://portainer.io/
But when I connect to console, I get the command prompt of container. Is there any way to run simple commands like ls /home/ that will list the files on host?
In other words is there any image that will mount the file system of host server "as-is"?
Here's an example using docker command line:
$ docker run --rm -it -v ~/Desktop:/Desktop alpine:latest /bin/sh
/ # ls /Desktop/
You can extend the approach to as far as you need to. Experiment with it. Learn about the different mount options.
I know the Docker app on MacOS provides a way for default volume mounts. Portainer also claims to provide a volume management screen, am yet to use it.
Hope this helps.
If you're dealing with services, or an existing, running container, you can in most cases access the shell directly. Let's say you have a container called "meow". You can run:
docker exec -it meow bash
and it will drop you into the bash shell. You'll actually need to know if bash is installed, or try calling sh instead.
The "i" option indicates it should be interactive, and the "t" option indicates it should emulate a TTY terminal. When you're done, you can hit Ctrl+D to exit out of the container.
First of all: You never ever want to do so.
Volumes mounted to containers are used to persist the container's data as containers are designed to be volatile -(the container itself shouldn't persist it s state so restarting the container n number of times should result in the same container state each time it starts)- so think of the volume as a the database where all the data (state of the container) should be stored.
Seeing volumes this way makes it easier to decide against sharing the host's entire file system, as this container would have read write permissions over the host OS files itself which is a huge security threat .
Sharing volumes across containers is considered a bad container architecture let alone sharing the entirety of the host file system.
I would propose simple ssh (or remote desktop) to your host if you require access to it to run commands or tasks on your host.
OR if your container requires access to a specific folder for some reason then you should consider mounting or binding that folder to the container
docker run -d --name devtest --mount source=myvol2,target=/app nginx:latest
I would recommend copying the content of that folder into a docker managed volume (a folder under the docker/volumes tree) and binding the container to this volume instead of the original folder to minimize the impact of your container on your host's OS.
I'm trying to use this Vanilla Minecraft docker image to run a Minecraft server for friends, but can't seem to get its data to live beyond the lifetime of the container itself.
I am running on an Ubuntu host, and on that host I have created /opt/minecraft where I'd like any outside-of-container persisted data to live.
To test my approaches I would:
start the container (commands provided below with each approach I tried)
join the server
dig a circle around myself as an indicator that I've been there
disconnect from the server
stop the container docker stop generated_name
start the container again using the same command as above
join the server again
is the circle I dug still there / has the world changed?
I have yet to find an approach that doesn't lose my data. Here's what I've tried:
Approach 1: all in one
I tried just mounting /opt/minecraft as /minecraft in the container:
docker run -d -v /opt/minecraft:/minecraft -p 25565:25565 webhippie/minecraft-vanilla
Approach 2: individual volumes
I noted in the Dockerfile that there are 3 volumes listed:
VOLUME ["/minecraft/merge", "/minecraft/world", "/minecraft/logs"]
So after destroying and recreating an empty /opt/minecraft and creating the three folders to mount, I tried this instead:
docker run -d -v /opt/minecraft/world:/minecraft/world -v /opt/minecraft/merge:/minecraft/merge -v /opt/minecraft/logs:/minecraft/logs -p 25565:25565 webhippie/minecraft-vanilla
In both approaches 1 and 2 I can see that some files have been created in the folder(s) mounted as volumes (so I don't think it's a permissions issue), but it doesn't seem to be enough to keep my world from being created from scratch again after a container restart. What am I missing?
I'm also new to Minecraft. Maybe there's some nuance about the game that I'm missing, instead?
Your second approach should be totally fine to keep the world, at least that's how I'm doing that.
So, I'm trying to package my WordPress image in a way that all files except the uploads are persisted. In order to do so, I have created my Dockerfile which uses the official WordPress image as its base, and adds the files from an archive (containing all the WordPress files, themes, plugins, etc.), like so:
FROM wordpress
ADD archive.tar.gz /var/www/html/
Since I want the uploads to be persisted, I have created a separate data volume container, e.g. test2.com-wp-data:
docker create -v /var/www/html/wp-content/uploads —name test2.com-wp-data wordpress
Then I simply mount it via —-volumes-from flag:
docker run —name test2.com --volumes-from test2.com-wp-data -d --link test2.com-mysql:mysql myimage
However, when I inspect my newly created container, I cannot find /var/www/html/wp-content/uploads:
# docker inspect -f '{{.HostConfig.VolumesFrom}}' test2.com
[test2.com-wp-data]
# docker inspect -f '{{.Volumes}}' test2.com
map[/var/www/html:/var/lib/docker/vfs/dir/4fff1d36d5aacd0b2c73977acf8fe680bda6fd891f2c4410a90f6c2dca4aaedf]
I can see that both /var/www/html and /var/www/html/wp-content/uploads are set up as volumes in my test2.com-wp-data data container:
# docker inspect -f '{{.Config.Volumes}}' test2.com-wp-data
map[/var/www/html:map[] /var/www/html/wp-content/uploads:map[]]
I know that the wordpress image by default creates a /var/www/html volume, for which I don't really mind, but does that mean that anything that is below that folder is ignored if mounted separately? Will I need to build my own WordPress image in order to have /var/www/html/wp-content/uploads set as a volume in my WordPress container?
Thank you very much for your time!
EDIT: I've tested a different setup with a folder that has nothing to do with /var/www/html, and the result is the same: —-volumes-from is ignored.
Version 1.4 + of docker should be what you need to get this working. Older versions of docker don't seem to play nicely with data-only containers instantiated with "create" rather than "run".
Well, after some further testing I've realised that despite what the documentation indicates, docker create alone is not enough to get a working data volume working. I've only managed to get working data volumes by instantiating them with the docker run command, as follows:
docker run —-name data -v /var/www/html/wp-content/uploads mysql true
This way the container exits immediately, but if I use it to attach the data volume to another container it works as expected.
If anyone knows any specific reason behind this behaviour, I'd be glad to learn more, especially since the documentation seems to be misleading.
Thanks!
EDIT: It turns out I was using Docker 1.3.x, which hadn't implemented this feature yet, hence why the documentation was misleading for me!
What is the best way to persist containers data with docker? I would like to be able to retain some data and be able to get them back when restarting my container. I have read this interesting post but it does not exactly answer my question.
As far as I understand, I only have one option:
docker run -v /home/host/app:/home/container/app
This will mount the countainer folder onto the host.
Is there any other option? FYI, I don't use linking containers (--link )
Using volumes is the best way of handling data which you want to keep from a container. Using the -v flag works well and you shouldn't run into issues with this.
You can also use the VOLUME instruction in the Dockerfile which means you will not have to add any more options at run time, however they're quite tightly coupled with the specific container, you'd need to use docker start, rather than docker run to get the data back (or of course -v to the volume which was created in the past, likely in /var/ somewhere).
A common way of handling volumes is to create a data volume container with volumes defined by -v Then when you create your app container, use the --volumes-from flag. This will make your new container use the same volumes as the container you used the -v on (your data volume container). Of course this may seem like you're shifting the issue somewhere else.
This makes it quite simple to share volumes over multiple containers. Perhaps you have a container for your application, and another for logstash.
create a volume-container: this format of -v creates a volume, directory e.g. /var/lib/docker/volume/d3b0d5b781b7f92771b7342824c9f136c883af321a6e9fbe9740e18b93f29b69
which is still a bind mounted /container/path/vol
docker run -v /foo/bar/vol --name volbox ubuntu
I can now use this container, as my volume.
docker run --volumes-from volbox --name foobox ubuntu /bin/bash
root#foobox# ls /container/path/vol
Now, if I distribute these two containers, they will just work. The volume will always be available to foobox, regardless which host it is deployed to.
The snag of course comes if you don't want your storage to be in /var/lib/docker/volumes...
I suggest you take a look at some of the excellent post by Michael Crosby
https://docs.docker.com/userguide/dockervolumes/
and the docker docs
https://docs.docker.com/userguide/dockervolumes/