I am trying to use docker to build my project in a specific environment, but I am wondering if there's a way to avoid having to docker run -v [mypath]:[workdir] -it [name] to simply get some files out.
Can I tell a dockerfile to run some commands and then copy the resulting files to the host?
No, images and containers are intentionally isolated from the host's filesystem, and there's intentionally no way for an image to read or write files on the host without being explicitly granted the privilege in a docker run -v option.
If your real goal is to run something to interact with files on the host then you'll often be better off installing and running it directly on the host than writing the very long docker run command line (and needing root privileges to do anything).
You can't write file directly to you host system. But you can still achieve that by printing your result into console, and the write it directly to your file.
Take an example of this docker image: docker/whalesay. when you run docker run docker/whalesay cowsay Hello there!, It will print into console the text Hello there! and the whale together. So you can write these thing back to your file by doing:
docker run docker/whalesay cowsay Hello there! | tee ./somefiletowrite.txt
Related
I have a Meteor App deployed with Meteor UP to Ubuntu.
From this App I need to read a file which is located outside App container on the host server.
How can I do that?
I've tried to set up volumes in the mup.js but no luck. It seems that I'm missing how to correctly provide /host/path and /container/path
volumes: {
// passed as '-v /host/path:/container/path' to the docker run command
'/host/path': '/container/path',
'/second/host/path': '/second/container/path'
},
Read the docs for Docker mounting volumes but obviously can't understand it.
Let's say file is in /home/dirname/filename.csv.
How to correctly mount it into App to be able to access it from the Application?
Or maybe there are other possibilities to access it?
Welcome to Stack Overflow. Let me suggest another way of thinking about this...
In a scalable cluster, docker instances can be spun up and down as the load on the app changes. These may or may not be on the same host computer, so building a dependency on the file system of the host isn't a great idea.
You might be better to think of using a file storage mechanism such as S3, which will scale on its own, and disk storage limits won't apply.
Another option is to determine if the files could be stored in the database.
I hope that helps
Let's try to narrow the problem down.
Meteor UP is passing the configuration parameter volumes directly on to docker, as they also mention in the comment you included. It therefore might be easier to test it against docker directly - narrowing the components involved down as much as possible:
sudo docker run \
-it \
--rm \
-v "/host/path:/container/path" \
-v "/second/host/path:/second/container/path" \
busybox \
/bin/sh
Let me explain this:
sudo because Meteor UP uses sudo to start the container. See: https://github.com/zodern/meteor-up/blob/3c7120a75c12ea12fdd5688e33574c12e158fd07/src/plugins/meteor/assets/templates/start.sh#L63
docker run we want to start a container.
-it to access the container (think of it like SSH'ing into the container).
--rm to automatically clean up - remove the container - after we're done.
-v - here we give the volumes as you define it (I here took the two directories example you provided).
busybox - an image with some useful tools.
/bin/sh - the application to start the container with
I'd expect that you also cannot access the files here. In this case, dig deeper on why you can't make a folder accessible in Docker.
If you can, which would sound weird to me, you can start the container and try to access into the container by running the following command:
docker exec -it my-mup-container /bin/sh
You can think of this command like SSH'ing into a running container. Now you can check around if it really isn't there, if the credentials inside the container are correct, etc.
At last, I have to agree it #mikkel, that it's not a good option to mount a local directoy, but you can now start looking into how to use docker volume to mount a remote directory. He mentioned S3 by AWS, I've worked with AzureFiles on Azure, there are plenty of possibilities.
I am new to docker and containers. I have a container consisting of an MRI analysis software. Within this container are many other software the main software draws its commands from. I would like to run a single command from one of the softwares in this container using research data that is located on an external hard drive which is plugged into my local machine that is running docker.
I know there is a cp command for copying files (such as scripts) into containers and most other questions along these lines seem to recommend copying the files from your local machine into the container and then running the script (or whatever) from the container. In my case I need the container to access data from separate folders in a directory structure and copying over the entire directory is not feasible since it is quite large.
I honestly just want to know how I can run a single command inside the docker using inputs present on my local machine. I have run docker ps to get the CONTAINER_ID which is d8dbcf705ee7. Having looked into executing commands inside containers I tried the following command:
docker exec d8dbcf705ee7 /bin/bash -c "mcflirt -in /Volumes/DISS/FMRIPREP/sub-S06V1A/func/sub-S06V1A_task-compound_run-01_bold.nii -out sub-S06V1A_task-compound_run-01_bold_mcf_COMMAND_TEST.nii.gz -reffile /Volumes/DISS/FMRIPREP_TMP/sub-S06V1A_dof6_ver1.2.5/fmriprep_wf/single_subject_S06V1A_wf/func_preproc_task_compound_run_01_wf/bold_reference_wf/gen_ref/ref_image.nii.gz -mats -plots"
mcflirt is the command I want to run inside the container. I believe the exec command would do what I hope since if I run docker exec d8dbcf705ee7 /bin/bash -c "mcflirt" I will get help output for the mcflirt command which is the expected outcome in that case. The files inside of the /Volume/... paths are the files on my local machine I would like to access. I understand that the location of the files is the problem since I cannot tab complete the paths within this command; when I run this I get the following output:
Image Exception : #22 :: ERROR: Could not open image /Volumes/DISS/FMRIPREP/sub-S06V1A/func/sub-S06V1A_task-compound_run-01_bold
terminate called after throwing an instance of 'RBD_COMMON::BaseException'
Can anyone point me in the right direction?
So if I got you right, you need to execute some shell script and provide the context (like local files).
The way is straightforward.
Lets say your script and all needed files are located in /hello folder of your host PC (no matter really if they are stored together or not, just showing the technique).
/hello
- runme.sh
- datafile1
- datafile1
You mount this folder into your container to make the files accessible inside. If you dont need container to modify them, better mount in readonly mode.
You launch docker like this:
docker run -it -v /hello:/hello2:ro ubuntu /hello2/runme.sh
And that's it! Your script runme.sh gets executed inside container and it has access to nearby files. Thanks to -v /hello:/hello2:ro directive. It maps host's folder /hello into container's folder /hello2 in readonly ro mode.
Note you can have same names, I've just differed them to show the difference.
I would like to know if there is a specific way to upload data to Docker, I've been stuck on this during a week and I am sure the answer will be something simple.
Does anyone know? I am working with a windows 10 machine.
You can mount directories on the host system inside the container and access their contents that way, if that's what you mean by 'data'.
You should check out Manage data in containers for more info.
You can use the docker cp command to copy the file.
For eg: If you want to copy abc.txt to location /usr/local/folder inside some docker container(you can get docker container name from NAMES column by executing command docker ps.) then you just execute,
docker cp abc.txt ContainerName:/usr/local/folder
(abc.txt is a local to the foler from where you are executing the command. You can provide the full path of the file.)
After this just get into the container by,
docker exec -it ContainerName bash
then cd /usr/local/folder. you will see your file copied their.
I am currently exploring on how we can remove the file/folder resides inside docker container programmatically . I know we can copy files from container to host using docker cp. However, I am looking for something like docker mv or docker rm which allows me to move or remove files/folders inside docker.
The scenario is, We are writing the automated test cases in Java and for one case we need to copy the log file from the server's log folder to assert the log written by the test case. We are using the docker cp to copy the log files. However it contains the log of old test cases as well. So I was thinking if I can remove the log files before executing my test case. It make sure the log I have copied is written by my test case only. Is there any other way around?
You can use below command to remove the files from program running on host
docker exec <container> rm -rf <YourFile>
However if old files exist because the container were never removed, then general practice is to remove the container once the all test suite execution is complete,
New job should create the new container.
In the docker file definition you can use
RUN rm [folder-path]
However anything added using ADD or COPY will still increase the size of your image.
EDIT: For accessing a running container from an external program running on your host try this.
Mount a host folder as a volume when starting the instance.
Run a script or program on the host to delete desired folders on the host which will affect the container file system as well.
You can access the container bash console
#console control over container
docker exec -it a5866aee4e90 bash
Then when you are inside the container you can do anything with console.
Im using this command to find and rename files in my jboss deployments directory. Modify it how you need to serve you. You can delete files also insted of mv use rm
find /home/jboss/jbossas/standalone/deployments -depth -name "*.failed" -exec sh -c 'mv "$1" "${1%.failed}.dodeploy"' _ {} \;
I created a data only container containing static HTML files that are intended to be consumed by a nginx container. Goal is that my webapp is providing a volume that nginx can use.
For this reason I created a simple Dockerfile:
FROM scratch
MAINTAINER me <me#me.com>
ADD dist/ /webappp/
When I run the created container from command line run -d -v /webappp --name webapp myOrg/webapp echo yo
I get the error Error response from daemon: Cannot start container db7fd5cd40d76311f8776b1710b4fe6d66284fe75253a806e281cd8ae5169637: exec: "echo": executable file not found in $PATH which if of course correct because the image has no commands at all the can be executed. Running a container without a command is not possible.
While this error on command line is not a big problem for me because I know the data container is still created and can now be accessed by nginx it turns out to be a no go if I want to automate it with Vagrant. The automated processes always fail because of this error.
My only solution so far is to extend my little handy image from from a distro which IMHO doesn't make sense for a data only container in order just to call echo or true!
Is there a NOP exec command in docker or does docker need always to execute something, is it possible to run a scratch container that does nothing or does not produce an error.
As mentioned in the Docker manual: The container don't need to be running. It also doesn't say that the container "should" be able to run at all.
So instead of echoing something stupid by running a data only container e.g. docker run -v /webappp --name webapp myOrg/webapp echo yo
It is already enough to just create the container and never run/start it.
docker create -v /webappp --name webapp myOrg/webapp
Note to self: Vagrant does not support docker create when provisioning!
Why are you using scratch?
Just use the nginx image as a base. You already have the image cached so it won't take up any more space and you'll be able to call echo.
Some references for data containers:
Data-only container madness
Understanding Volumes in Docker
Offiical docs on data containers