I am running a docker image from deepai/densecap on my windows machine using docker toolbox. When i run image using docker CLI and pass the arguments for cp command as stated in below picture
It says that "docker cp" requires exactly 2 arguments". The various command i try to pass my image from local file system to container are:
docker cp C:\Users\piyush\Desktop\img1.jpg in1
docker cp densecap:C:\Users\piyush\Desktop\image1.jpg in1
docker cp C:\Users\piyush\Desktop\img1.jpg densecap:/shared/in1
I have just started using docker. Any help will be highly appreciated. I am also posting the container log:
It would seem on some versions of Docker, docker cp does not support parameter expansion...
For example
WORKS Docker version 19.03.4-ce, build 9013bf583a
CTR_ID=$(docker ps -q -f name=containername)
docker cp patches $CTR_ID:/home/build
FAILS Docker version 19.03.4-ce, build 9013bf583a
BUILDHOME=/home/build
docker cp patches containeridliteral:$BUILDHOME
In your case, maybe the pwd is not expanding properly.
Related
I'm using the node docker images as a container for my build pipelines.
An issue I frequently run into is that a binary that I expect to exist, doesn't and I have to wait for it fail in the build pipeline. The zip command is one such example.
I can run the docker image on my local machine and ssh in to test commands.
Is there a way to summarise what commands are available for a given image?
Is there a way to summarise what commands are available for a given image?
You could look at the contents of /bin:
$ docker run --rm -it --entrypoint=ls node /bin
or /usr/local/bin:
$ docker run --rm -it --entrypoint=ls node /usr/local/bin
etc...
I did a docker pull and can list the image that's downloaded. I want to see the contents of this image. Did a search on the net but no straight answer.
If the image contains a shell, you can run an interactive shell container using that image and explore whatever content that image has. If sh is not available, the busybox ash shell might be.
For instance:
docker run -it image_name sh
Or following for images with an entrypoint
docker run -it --entrypoint sh image_name
Or if you want to see how the image was built, meaning the steps in its Dockerfile, you can:
docker image history --no-trunc image_name > image_history
The steps will be logged into the image_history file.
You should not start a container just to see the image contents. For instance, you might want to look for malicious content, not run it. Use "create" instead of "run";
docker create --name="tmp_$$" image:tag
docker export tmp_$$ | tar t
docker rm tmp_$$
The accepted answer here is problematic, because there is no guarantee that an image will have any sort of interactive shell. For example, the drone/drone image contains on a single command /drone, and it has an ENTRYPOINT as well, so this will fail:
$ docker run -it drone/drone sh
FATA[0000] DRONE_HOST is not properly configured
And this will fail:
$ docker run --rm -it --entrypoint sh drone/drone
docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "exec: \"sh\": executable file not found in $PATH".
This is not an uncommon configuration; many minimal images contain only the binaries necessary to support the target service. Fortunately, there are mechanisms for exploring an image filesystem that do not depend on the contents of the image. The easiest is probably the docker export command, which will export a container filesystem as a tar archive. So, start a container (it does not matter if it fails or not):
$ docker run -it drone/drone sh
FATA[0000] DRONE_HOST is not properly configured
Then use docker export to export the filesystem to tar:
$ docker export $(docker ps -lq) | tar tf -
The docker ps -lq there means "give me the id of the most recent docker container". You could replace that with an explicit container name or id.
docker save nginx > nginx.tar
tar -xvf nginx.tar
Following files are present:
manifest.json – Describes filesystem layers and name of json file that has the Container properties.
.json – Container properties
– Each “layerid” directory contains json file describing layer property and filesystem associated with that layer. Docker stores Container images as layers to optimize storage space by reusing layers across images.
https://sreeninet.wordpress.com/2016/06/11/looking-inside-container-images/
OR
you can use dive to view the image content interactively with TUI
https://github.com/wagoodman/dive
EXPLORING DOCKER IMAGE!
Figure out what kind of shell is in there bash or sh...
Inspect the image first: docker inspect name-of-container-or-image
Look for entrypoint or cmd in the JSON return.
Then do: docker run --rm -it --entrypoint=/bin/bash name-of-image
once inside do: ls -lsa or any other shell command like: cd ..
The -it stands for interactive... and TTY. The --rm stands for remove container after run.
If there are no common tools like ls or bash present and you have access to the Dockerfile simple add the common tool as a layer.
example (alpine Linux):
RUN apk add --no-cache bash
And when you don't have access to the Dockerfile then just copy/extract the files from a newly created container and look through them:
docker create <image> # returns container ID the container is never started.
docker cp <container ID>:<source_path> <destination_path>
docker rm <container ID>
cd <destination_path> && ls -lsah
To list the detailed content of an image you have to run docker run --rm image/name ls -alR where --rm means remove as soon as exits form a container.
If you want to list the files in an image without starting a container :
docker create --name listfiles <image name>
docker export listfiles | tar -t
docker rm listfiles
We can try a simpler one as follows:
docker image inspect image_id
This worked in Docker version:
DockerVersion": "18.05.0-ce"
if you want to check the image contents without running it you can do this:
$ sudo bash
...
$ cd /var/lib/docker # default path in most installations
$ find . -iname a_file_inside_the_image.ext
... (will find the base path here)
This works fine with the current default BTRFS storage driver.
Oneliner, no docker run (based on responses above)
IMAGE=your_image docker create --name filelist $IMAGE command && docker export filelist | tar tf - | tree --fromfile . && docker rm filelist
Same, but report tree structure to result.txt
IMAGE=your_image docker create --name filelist $IMAGE command && docker export filelist | tar tf - | tree --noreport --fromfile . | tee result.txt && docker rm filelist
I tried this tool - https://github.com/wagoodman/dive
I found it quite helpful to explore the content of the docker image.
Perhaps this is nota very straight forward approach but this one worked for me.
I had an ECR Repo (Amazon Container Service Repository) whose code i wanted to see.
First we need to save the repo you want to access as a tar file. In my case the command went like - docker save .dkr.ecr.us-east-1.amazonaws.com/<name_of_repo>:image-tag > saved-repo.tar
UNTAR the file using the command - tar -xvf saved-repo.tar. You could see many folders and files
Now try to find the file which contain the code you are looking for (if you know some part of the code)
Command for searching the file - grep -iRl "string you want to search" ./
This will make you reach the file. It can happen that even that file is tarred, so untar it using the command mentioned in step 2.
If you dont know the code you are searching for, you will need to go through all the files that you got after step 2 and this can be bit tiring.
All the Best !
There is a free open source tool called Anchore-CLI that you can use to scan container images. This command will allow you to list all files in a container image
anchore-cli image content myrepo/app:latest files
https://anchore.com/opensource/
EDIT: not available from anchore.com anymore, It's a python program you can install from https://github.com/anchore/anchore-cli
With Docker EE for Windows (17.06.2-ee-6 on Hyper-V Server 2016) all contents of Windows Containers can be examined at C:\ProgramData\docker\windowsfilter\ path of the host OS.
No special mounting needed.
Folder prefix can be found by container id from docker ps -a output.
I am trying to build a new docker image using docker provided base Ubuntu image. I'll be using docker file to run few scripts and install applications on the base image. However my script requirement is that the hostname should remain same. I couldn't find any information on OS names for docker images. Does anybody has an idea that once we add layers to a docker image does the OS name remains same.
You can set the hostname with the -h argument to Docker run, otherwise it gets the short form of the container ID as the hostname:
$ docker run --rm -it debian bash
root#0d36e1b1ac93:/# exit
exit
$ docker run --rm -h myhost -it debian bash
root#myhost:/# exit
exit
As far as I know, you can't tell docker build to use a given hostname, but see Dockerfile HOSTNAME Instruction for docker build like docker run -h.
I am getting into Docker and am trying to better understand how it works out there in the "real world".
It occurs to me that, in practice:
You need a way to version Docker images
You need a way to tell the Docker engine (running on a VM) to stop/start/restart a particular container
You need a way to tell the Docker engine which version of a image to run
Does Docker ship with built-in commands for handling each of these? If not what tools/strategies are used for accomplishing them? Also, when I build a Docker image (via, say, docker build -t myapp .), what file type is produced and where is it located on the machine?
docker has all you need to build images and run containers. You can create your own image by writing a Dockerfile or by pulling it from the docker hub.
In the Dockerfile you specify another image as the basis for your image, run command install things. Images can have tags, for example the ubuntu image can have the latest or 12.04 tag, that can be specified with ubuntu:latest notation.
Once you have built the image with docker build -t image-name . you can create containers from that image with `docker run --name container-name image-name.
docker ps to see running containers
docker rm <container name/id> to remove containers
Suppose we have a docker file like bellow:
->Build from git without versioning:
sudo docker build https://github.com/lordash/mswpw.git#fecomments:comments
in here:
fecomments is branch name and comments is the folder name.
->building from git with tag and version:
sudo docker build https://github.com/lordash/mswpw.git#fecomments:comments -t lordash/comments:v1.0
->Now if you want to build from a directory: first go to comments directory the run command sudo docker build .
->if you want to add tag you can use -t or -tag flag to do that:
sudo docker build -t lordash . or sudo docker build -t lordash/comments .
-> Now you can version your image with the help of tag:
sudo docker build -t lordash/comments:v1.0 .
->you can also apply multiple tag to an image:
sudo docker build -t lordash/comments:latest -t lordash/comments:v1.0 .
My question is related to this question on copying files from containers to hosts; I have a Dockerfile that fetches dependencies, compiles a build artifact from source, and runs an executable. I also want to copy the build artifact (in my case it's a .zip produced by sbt dist in '../target/`, but I think this question also applies to jars, binaries, etc.
docker cp works on containers, not images; do I need to start a container just to get a file out of it? In a script, I tried running /bin/bash in interactive mode in the background, copying the file out, and then killing the container, but this seems kludgey. Is there a better way?
On the other hand, I would like to avoid unpacking a .tar file after running docker save $IMAGENAME just to get one file out (but that seems like the simplest, if slowest, option right now).
I would use docker volumes, e.g.:
docker run -v hostdir:out $IMAGENAME /bin/cp/../blah.zip /out
but I'm running boot2docker in OSX and I don't know how to directly write to my mac host filesystem (read-write volumes are mounting inside my boot2docker VM, which means I can't easily share a script to extract blah.zip from an image with others. Thoughts?
To copy a file from an image, create a temporary container, copy the file from it and then delete it:
id=$(docker create image-name)
docker cp $id:path - > local-tar-file
docker rm -v $id
Unfortunately there doesn't seem to be a way to copy files directly from Docker images. You need to create a container first and then copy the file from the container.
However, if your image contains a cat command (and it will do in many cases), you can do it with a single command:
docker run --rm --entrypoint cat yourimage /path/to/file > path/to/destination
If your image doesn't contain cat, simply create a container and use the docker cp command as suggested in Igor's answer.
docker cp $(docker create --name tc registry.example.com/ansible-base:latest):/home/ansible/.ssh/id_rsa ./hacked_ssh_key && docker rm tc
wanted to supply a one line solution based on pure docker functionality (no bash needed)
edit: container does not even has to be run in this solution
edit2: thanks to #Jonathan Dumaine for --rm so the container will be removed after, i just never tried, because it sounded illogical to copy something from somewhere which has been already removed by the previous command, but i tried it and it works
edit3: due the comments we found out --rm is not working as expected, it does not remove the container because it never runs, so I added functionality to delete the created container afterwards(--name tc=temporary-container)
edit 4: this error appeared, seems like a bug in docker, because t is in a-z and this did not happen a few months before.
Error response from daemon: Invalid container name (t), only [a-zA-Z0-9][a-zA-Z0-9_.-] are allowed
A much faster option is to copy the file from running container to a mounted volume:
docker run -v $PWD:/opt/mount --rm --entrypoint cp image:version /data/libraries.tgz /opt/mount/libraries.tgz
real 0m0.446s
** VS **
docker run --rm --entrypoint cat image:version /data/libraries.tgz > libraries.tgz
real 0m9.014s
Parent comment already showed how to use cat. You could also use tar in a similar fashion:
docker run yourimage tar -c -C /my/directory subfolder | tar x
Another (short) answer to this problem:
docker run -v $PWD:/opt/mount --rm -ti image:version bash -c "cp /source/file /opt/mount/"
Update - as noted by #Elytscha Smith this only works if your image has bash built in
Not a direct answer to the question details, but in general, once you pulled an image, the image is stored on your system and so are all its files. Depending on the storage driver of the local Docker installation, these files can usually be found in /var/lib/docker/overlay2 (requires root access). overlay2 should be the most common storage driver nowadays, but the path may differ.
The layers associated with an image can be found using $ docker inspect image IMAGE_NAME:TAG, look for a GraphDriver attribute.
At least in my local environment, the following also works to quickly see all layers associated with an image:
docker inspect image IMAGE_NAME:TAG | jq ".[0].GraphDriver.Data"
In one of these diff directories, the wanted file can be found.
So in theory, there's no need to create a temporary container. Ofc this solution is pretty inconvenient.
First pull docker image using docker pull
docker pull <IMG>:<TAG>
Then, create a container using docker create command and store the container id is a variable
img_id=$(docker create <IMG>:<TAG>)
Now, run the docker cp command to copy folders and files from docker container to host
docker cp $img_id:/path/in/container /path/in/host
Once the files/folders are moved, delete the container using docker rm
docker rm -v $img_id
You essentially had the best solution already. Have the container copy out the files for you, and then remove itself when it's complete.
This will copy the files from /inside/container/ to your machine at /path/to/hostdir/.
docker run --rm -v /path/to/hostdir:/mnt/out "$IMAGENAME" /bin/cp -r /inside/container/ /mnt/out/
Update - here's a better version without the tar file:
$id = & docker create image-name
docker cp ${id}:path .
docker rm -v $id
Old answer
PowerShell variant of Igor Bukanov's answer:
$id = & docker create image-name
docker cp ${id}:path - > local-file.tar
docker rm -v $id
I am using boot2docker on MacOS. I can assure you that scripts based on "docker cp" are portable. Because any command is relayed inside boot2docker but then the binary stream is relayed back to the docker command line client running on your mac. So write operations from the docker client are executed inside the server and written back to the executing client instance!
I am sharing a backup script for docker volumes with any docker container I provide and my backup scripts are tested both on linux and MacOS with boot2docker. The backups can be easily exchanged between platforms. Basically I am executing the following command inside my script:
docker run --name=bckp_for_volume --rm --volumes-from jenkins_jenkins_1 -v /Users/github/jenkins/backups:/backup busybox tar cf /backup/JenkinsBackup-2015-07-09-14-26-15.tar /jenkins
Runs a new busybox container and mounts the volume of my jenkins container with the name jenkins_jenkins_1. The whole volume is written to the file backups/JenkinsBackup-2015-07-09-14-26-15.tar
I have already moved archives between the linux container and my mac container without any adjustments to the backup or restore script. If this is what you want you find the whole script an tutorial here: blacklabelops/jenkins
You could bind a local path on the host to a path on the container, and then cp the desired file(s) to that path at the end of your script.
$ docker run -d \
-it \
--name devtest \
--mount type=bind,source="$(pwd)"/target,target=/app \
nginx:latest
Then there is no need to copy afterwards.