I am new to docker and containers. I have a container consisting of an MRI analysis software. Within this container are many other software the main software draws its commands from. I would like to run a single command from one of the softwares in this container using research data that is located on an external hard drive which is plugged into my local machine that is running docker.
I know there is a cp command for copying files (such as scripts) into containers and most other questions along these lines seem to recommend copying the files from your local machine into the container and then running the script (or whatever) from the container. In my case I need the container to access data from separate folders in a directory structure and copying over the entire directory is not feasible since it is quite large.
I honestly just want to know how I can run a single command inside the docker using inputs present on my local machine. I have run docker ps to get the CONTAINER_ID which is d8dbcf705ee7. Having looked into executing commands inside containers I tried the following command:
docker exec d8dbcf705ee7 /bin/bash -c "mcflirt -in /Volumes/DISS/FMRIPREP/sub-S06V1A/func/sub-S06V1A_task-compound_run-01_bold.nii -out sub-S06V1A_task-compound_run-01_bold_mcf_COMMAND_TEST.nii.gz -reffile /Volumes/DISS/FMRIPREP_TMP/sub-S06V1A_dof6_ver1.2.5/fmriprep_wf/single_subject_S06V1A_wf/func_preproc_task_compound_run_01_wf/bold_reference_wf/gen_ref/ref_image.nii.gz -mats -plots"
mcflirt is the command I want to run inside the container. I believe the exec command would do what I hope since if I run docker exec d8dbcf705ee7 /bin/bash -c "mcflirt" I will get help output for the mcflirt command which is the expected outcome in that case. The files inside of the /Volume/... paths are the files on my local machine I would like to access. I understand that the location of the files is the problem since I cannot tab complete the paths within this command; when I run this I get the following output:
Image Exception : #22 :: ERROR: Could not open image /Volumes/DISS/FMRIPREP/sub-S06V1A/func/sub-S06V1A_task-compound_run-01_bold
terminate called after throwing an instance of 'RBD_COMMON::BaseException'
Can anyone point me in the right direction?
So if I got you right, you need to execute some shell script and provide the context (like local files).
The way is straightforward.
Lets say your script and all needed files are located in /hello folder of your host PC (no matter really if they are stored together or not, just showing the technique).
/hello
- runme.sh
- datafile1
- datafile1
You mount this folder into your container to make the files accessible inside. If you dont need container to modify them, better mount in readonly mode.
You launch docker like this:
docker run -it -v /hello:/hello2:ro ubuntu /hello2/runme.sh
And that's it! Your script runme.sh gets executed inside container and it has access to nearby files. Thanks to -v /hello:/hello2:ro directive. It maps host's folder /hello into container's folder /hello2 in readonly ro mode.
Note you can have same names, I've just differed them to show the difference.
Related
what I want to accomplish
I want to use the functions written in the .vimrc placed on the host side within Docker.
what I did
Put the .vimrc file in the /home/akihiro directory on the host side.
When using the docker run command, mount the /home/akihiro directory on the host side and run the python file in Docker with Vim.
akihiro#akihiro-thinkpad-x1-carbon-5th:~$ docker run --rm -it -v /home/akihiro:/home --name test cnn_study:latest
As a result, the settings written in the .vimrc file did not work.
Next, I started a new container without mounting.
Created /home/akihiro directory in the container.
I left the container.
I copied only the /home/akihiro/.vimrc file on the host side into the container, and re-entered the container.
docker cp ./.vimrc 52b28f1ffea8:/home/akihiro
Started up a Python file using Vim.
As a result, the settings written in the .vimrc file did not work.
What you are doing is mapping the complete /home or /home/akihiro directory on the Container.
You can't do that for 2 reasons:
There are more files in that directory. What do you expect happens with them?
It's not like a OR-function is done on files in both folder.
Mapping the volume complete replaces the internal folder
My gut feeling says that mapping the direcory comes too late in the process.
The directory is already there in the Container and therefore cannot be overwritten.
(At least that's how I understand the strange effects I get with mapping sometimes)
What you should do is only map the file:
$ docker run --rm -it -v /home/akihiro/.vimrc:/home/akihiro/.vimrc --name test cnn_study:latest
I do the same with .bashrc (to set the prompt to the name of the Container)
I am trying to pass a directory inside the container, eventually where this can be automated. However I don't see any alternative other than physically editing the Dockerfile and manually typing the specific directory to be added.
Note: I have tried mounted volumes, however that solution doesn't help my issue, as I want to eventually call the container on a directory which will eventually have a script run on the directory in the container--not simply copying the local directory inside the container.
Method 1:
$ --build-arg project_directory=/path/to/dir
ARG project_directory
ADD $project_directory .
My unsuccessful solution assumes that I can use the argument's value as a basic string that the ADD command can interpret just as if I was just manually entering the path.
not simply copying the local directory inside the container
That's exactly what you're doing now, by using ADD $project_directory. If you need to make changes from the container and have them reflected onto the host, use:
docker run -v $host_dir:$container_dir image:tag
The command above launches a new container, and it's quite possible for you to launch it with different directory names. You can do so in a loop, from a jenkins pipeline, a shell script, or whatever suits your development environment.
#!/bin/bash
container_dir=/workspace
for directory in /src /realsrc /kickasssrc
do
docker run -v $directory:$container_dir image:tag
done
I built myself an image for ROS. I run it while mounting my original home on the host and some tricks to get graphics as well. After starting the shell inside docker I always need to execute two source commands. One of the files to be sourced are actually inside the container, but the other resides in my home, which only gets mounted on starting the container. I would have these two files sourced automatically.
I tried adding
RUN bash -c "source /opt/ros/indigo/setup.bash"
to the image file, but this did not actually source it. Using CMD instead of run didn't drop me into the container's shell (I assume it finished executing source and then exited?). I don't even have an idea how to source the file that is only available after startup. What would I need to do?
TL;DR: you need to perform this step as part of your CMD or ENTRYPOINT, and for something like a source command, you need a step after that in the shell to run your app, or whatever shell you'd like. If you just want a bash shell as your command, then put your source command inside something like your .bashrc file. Or you can run something like:
bash -c "source /opt/ros/indigo/setup.bash && bash"
as your command.
One of the files to be sourced are actually inside the container, but the other resides in my home, which only gets mounted on starting the container.
...
I tried adding ... to the image file
Images are built using temporary containers that only see your Dockerfile instructions and the context sent with that to run the build. Containers use that built image and all of your configuration, like volumes, to run your application. There's a hard divider between those two steps, image build and container run, and your volumes are not available during that image build step.
Each of those RUN steps being performed for the image build are done in a temporary container that only stores the output of the filesystem when it's finished. Changes to your environment, a cd into another directory, spawned processes or services in the background, or anything else not written to the filesystem when the command spawned by RUN exits, will be lost. This is one reason you will see commands chained together in a single long RUN command, and it's why you have ENV and WORKDIR commands in the Dockerfile.
I would like to know if there is a specific way to upload data to Docker, I've been stuck on this during a week and I am sure the answer will be something simple.
Does anyone know? I am working with a windows 10 machine.
You can mount directories on the host system inside the container and access their contents that way, if that's what you mean by 'data'.
You should check out Manage data in containers for more info.
You can use the docker cp command to copy the file.
For eg: If you want to copy abc.txt to location /usr/local/folder inside some docker container(you can get docker container name from NAMES column by executing command docker ps.) then you just execute,
docker cp abc.txt ContainerName:/usr/local/folder
(abc.txt is a local to the foler from where you are executing the command. You can provide the full path of the file.)
After this just get into the container by,
docker exec -it ContainerName bash
then cd /usr/local/folder. you will see your file copied their.
I am currently exploring on how we can remove the file/folder resides inside docker container programmatically . I know we can copy files from container to host using docker cp. However, I am looking for something like docker mv or docker rm which allows me to move or remove files/folders inside docker.
The scenario is, We are writing the automated test cases in Java and for one case we need to copy the log file from the server's log folder to assert the log written by the test case. We are using the docker cp to copy the log files. However it contains the log of old test cases as well. So I was thinking if I can remove the log files before executing my test case. It make sure the log I have copied is written by my test case only. Is there any other way around?
You can use below command to remove the files from program running on host
docker exec <container> rm -rf <YourFile>
However if old files exist because the container were never removed, then general practice is to remove the container once the all test suite execution is complete,
New job should create the new container.
In the docker file definition you can use
RUN rm [folder-path]
However anything added using ADD or COPY will still increase the size of your image.
EDIT: For accessing a running container from an external program running on your host try this.
Mount a host folder as a volume when starting the instance.
Run a script or program on the host to delete desired folders on the host which will affect the container file system as well.
You can access the container bash console
#console control over container
docker exec -it a5866aee4e90 bash
Then when you are inside the container you can do anything with console.
Im using this command to find and rename files in my jboss deployments directory. Modify it how you need to serve you. You can delete files also insted of mv use rm
find /home/jboss/jbossas/standalone/deployments -depth -name "*.failed" -exec sh -c 'mv "$1" "${1%.failed}.dodeploy"' _ {} \;