Recently the centos images in the global docker registry were updated (~4 days ago it seems). Before the update, I could successfully build off of the Centos 6 image but now I have some installation errors for some packages. When I look at the images from before and after, and then attempt to follow version history it looks something like
f1b10cd84249 --> b9aeeaeb5e17 (originally worked)
\-> fb9cc58bde0c --> a005304e4e74 (current version where my code breaks)
where the arrows show how the image was updated (left is oldest while right is newest). I'm curious as to how the images are different. Is there anyway to do a diff of the a005304e4e74 and b9aeeaeb5e17 images?
I've never tried, but I guess you could do:
$ docker export -o f1.tar b9aeeaeb5e17
$ docker export -o f2.tar a005304e4e74
$ diff <(tar -tvf f1.tar | sort) <(tar -tvf f2.tar | sort)
Export will create a tar of the filesystem, which we then use to get a diff of the file differences. (I got the diff syntax from Diff between two .tar.gz file lists in liunx )
You might find just running docker history on the images gives you enough information though.
Related
Is there a singularity equivalent to docker image list?
The Singularity 3.8 documentation states that
All cache entries are named using a content hash
which makes sense when checking for identical layers/images, but has no semantic meaning whatsoever. Compare this to docker's docker image list which lists the images that you already have with semantically meaningful names (e.g. ubuntu, ros).
Example:
$ singularity cache list -v
NAME DATE CREATED SIZE TYPE
496a66ed93a0244167905e 2021-07-23 18:20:26 0.57 KiB blob
6b05187eae388023ea3f9d 2021-07-23 18:20:26 0.34 KiB blob
a31c7b29f4ad2bd9467389 2021-07-23 18:20:26 27.24 MiB blob
sha256.9b0dafaadb1cd1d 2021-07-24 14:37:56 90.46 MiB library
sha256.cb37e547a142499 2021-07-23 18:10:22 55.39 MiB library
b3e2e47d016c08b3396b5e 2021-07-23 18:20:29 26.45 MiB oci-tmp
versus
$ docker image list
REPOSITORY TAG IMAGE ID CREATED SIZE
foxy-moveit2-tutorials latest ca302410c537 3 weeks ago 5.09GB
foxy-moveit2 latest ba2e29d36a81 3 weeks ago 4.85GB
ros galactic 717c275e4609 5 weeks ago 675MB
ros foxy 5459f09fa2f1 5 weeks ago 723MB
devrt/xserver latest 31e1d3e1887d 13 months ago 293MB
An alternative would be to define a specific directory where all the pulled images are placed, and lits its contents:
$ singularity pull --dir ~/singularity_images/ docker://ros:foxy
$ ls singularity_images/
ros_foxy.sif
In short, singularity leaves you wondering about the images that you have already downloaded, because the content hashes lack semantic meaning.
The Singularity equivalent of docker image ls is ls *.sif.
This is one of the key differences between Docker and Singularity: a service with a centralized collection of local images available to all group members vs. images as files that can be run by anyone on any computer with singularity installed. The cache listing you show is also specific to your user (generally ~/.singularity/cache), though build actions do end up with a shared cache at /root/.singularity/cache as they require sudo privileges.
Another important distinction between docker image ls and singularity cache list is after clearing your singularity cache, the sif you generated still works as expected. Removing a docker image is the equivalent of deleting that sif file.
If an image is likely to be used by others, it's worthwhile to set up a shared location on the filesystem. This way you can also separate users who have permission to create images from those who just need to use them.
docker pull jaegertracing/jaeger-agent:latest
Jaeger is just for illustration. But my question is more generic. The above command pulls the latest version of the jaeger-agent from docker-hub.
The docker-hub page for this is : https://hub.docker.com/r/jaegertracing/jaeger-agent
My question is how do I find the actual version of latest ?
I looked in to the tags here, but there is not much info :
https://hub.docker.com/r/jaegertracing/jaeger-agent/tags
Also I tried doing an inspect after pulling the image, but could not get necessary details.
docker image inspect jaegertracing/jaeger-agent:latest
Where can we get this information from ?
As #max-gasner mentioned, it's common for latest to be tracking the master branch of a git repository. This allows the engineers to quickly build and test images before they are released and version tagged. This is one of the reasons why it's not recommended to ever use latest tags for anything critical where you need reproducibility.
jaegertracing/jaeger-agent:latest doesn't have any other tags so the only way to determine which "version" of latest you are using is to look at the digest. This uniquely identifies the image build. Tags actually resolve to digests. So when a new image is built with the latest tag, that tag will then resolve to the digest of the new image.
DockerHub only shows the short version. You can inspect the full digest like this:
docker image inspect --format '{{.RepoDigests}}' jaegertracing/jaeger-agent:latest
> [jaegertracing/jaeger-agent#sha256:bacc749faba325fbe39dc13294c2222fb0babc9ecd344c25d9d577720a80b5ca]
latest is just a tag like any other -- you will want docker image inspect, which will give you information about the other tags on your image.
In the case of jaegertracing/jaeger-agent:latest, it doesn't look this image has any other tags, so it's probable that this image is tracking something like the master branch of a source control repository, i.e., it doesn't correspond to a published version at all.
There is an issue Digests on Dockerhub and those fetched by docker pull do not match not solved yet.
#peterevans's answer and this answer can help. https://stackoverflow.com/a/64309017/1543768
But if the machine can't install some tool easily, Created is an easy tool to use.
$docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
your-image latest 4b10e**** 22 months ago 15.1MB
$IMAGE_ID=4b10e
$docker image inspect --format '{{.Created }}' $IMAGE_ID
2020-11-15T18:39:27.727222621Z
Check the date with Dockerhub.
get the image id and then replace IMAGE_ID with it.
docker image inspect --format '{{json .}}' "$IMAGE_ID" | jq -r '. | {Id: .Id, Digest: .Digest, RepoDigests: .RepoDigests, Labels: .Config.Labels}'
I get following error on executing the jq -r
jq : The term 'jq' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:61
... ocker image inspect --format '{{json .}}' "3fda1307ec65" | jq -r '. | ...
~~
CategoryInfo : ObjectNotFound: (jq:String) [], CommandNotFoundException
FullyQualifiedErrorId : CommandNotFoundException
I am pretty new to docker and I started playing around with it.
I downloaded the latest mongo docker image: docker pull mongo
Then I view the image I downloaded (docker images) but the only information i have regarding the version of mongo is the tag which is just 'latest'
I found that I can use the docker inspect command to determine the version, which did prove helpful, but it is also very inconvenient.
since I read that each Image can have multiple tags, I am assuming the mongo image I downloaded has another tag with the version number.
How can I view all tags of an image I downloaded?
You can use registry api to do it, reference to this & this.
For your case, you can just use next command:
wget -q https://registry.hub.docker.com/v1/repositories/mongo/tags -O - | sed -e 's/[][]//g' -e 's/"//g' -e 's/ //g' | tr '}' '\n' | awk -F: '{print $3}'
Change the mongo to others if you need tags of other images.
You just downloaded one tag. in your case latest which is an alias to one other tag, most of the times but not always the highest version number. In this Case it's 4, 4.0 and 4.0.10 which are all the same images. But there is an even newer image which is released unstable and which is 4.1.
The mongo tags you can find here. It's always a good idea to check the dockerhub description of the images, there you will find a lot of information.
In order to see all the tags available for one image, you need to check on the registry where your image is stored. In your case, you're using the default public registry which you can find here.
You can find all the mongo tags, directly here.
In the description tab, you'll also see that, currently, the latest tag is also associated with the versions 4.0.10, 4.0 and 4.
Furthermore, in order to get a specific tag, you need to use this command:
docker image pull image:tag
For example, if you want to pull the version 4.0.10 of mongo then you need to type :
docker image pull mongo:4.0.10
I'd like to pull the images of CentOS, Tomcat, ... using their sha256 code, like in
docker pull myimage#sha256:0ecb2ad60
But I can't find the sha256-code to use anywhere.
I checked the DockerHub repository for any hint of the sha256-code, but couldn't find any. I downloaded the images by their tag
docker pull tomcat:7-jre8
and checked the image with docker inspect to see if there's a sha256 code in the metadata, but there is none (adding the sha256 code of the image would probably change the sha256 code).
Do I have to compute the sha256 code of an image myself and use that?
Latest answer
Edit suggested by OhJeez in the comments.
docker inspect --format='{{index .RepoDigests 0}}' $IMAGE
Original answer
I believe you can also get this using
docker inspect --format='{{.RepoDigests}}' $IMAGE
Works only in Docker 1.9 and if the image was originally pulled by the digest. Details are on the docker issue tracker.
You can get it by docker images --digests
REPOSITORY TAG DIGEST IMAGE ID CREATED SIZE
docker/ucp-agent 2.1.0 sha256:a428de44a9059f31a59237a5881c2d2cffa93757d99026156e4ea544577ab7f3 583407a61900 3 weeks ago 22.3 MB
Simplest and most concise way is:
docker images --no-trunc --quiet $IMAGE
This returns only the sha256:... string and nothing else.
e.g.:
$ docker images --no-trunc --quiet debian:stretch-slim
sha256:220611111e8c9bbe242e9dc1367c0fa89eef83f26203ee3f7c3764046e02b248
Edit:
NOTE: this only works for images that are local. You can docker pull $IMAGE first, if required.
Just saw it:
When I pull an image, the sha256 code is diplayed at the bottom of the output (Digest: sha....):
docker pull tomcat:7-jre8
7-jre8: Pulling from library/tomcat
902b87aaaec9: Already exists
9a61b6b1315e: Already exists
...
4dcef5c50d60: Already exists
Digest: sha256:c34ce3c1fcc0c7431e1392cc3abd0dfe2192ffea1898d5250f199d3ac8d8720f
Status: Image is up to date for tomcat:7-jre8
This sha code
sha256:c34ce3c1fcc0c7431e1392cc3abd0dfe2192ffea1898d5250f199d3ac8d8720f
can be used to pull the image afterwards with
docker pull tomcat#sha256:c34ce3c1fcc0c7431e1392cc3abd0dfe2192ffea1898d5250f199d3ac8d8720f
This way you can be sure that the image is not changed and can be safely used for production.
I found the above methods to not work in some cases. They either:
don't deal well with multiple images with the same hash (in the case of .RepoDigests suggestion - when you want to use a specific registry path)
don't work well when pushing the image to registries
(in the case of .Id where it's a local hash, not the hash in the
registry).
The below method is delicate, but works for extracting the specific full 'name' and hash for a specific pushed container.
Here's the scenario - An image is uploaded separately to 2 different projects in the same repo, so querying RepoDigests returns 2 results.
$ docker inspect --format='{{.RepoDigests}}' gcr.io/alpha/homeapp:latest
[gcr.io/alpha/homeapp#sha256:ce7395d681afeb6afd68e73a8044e4a965ede52cd0799de7f97198cca6ece7ed gcr.io/beta/homeapp#sha256:ce7395d681afeb6afd68e73a8044e4a965ede52cd0799de7f97198cca6ece7ed]
I want to use the alpha result, but I can't predict which index it will be. So I need to manipulate the text output to remove the brackets and get each entry on a separate line. From there I can easily grep the result.
$ docker inspect --format='{{.RepoDigests}}' gcr.io/alpha/homeapp:latest | sed 's:^.\(.*\).$:\1:' | tr " " "\n" | grep alpha
gcr.io/alpha/homeapp#sha256:ce7395d681afeb6afd68e73a8044e4a965ede52cd0799de7f97198cca6ece7ed
In addition to the existing answers, you can use the --digests option while doing docker images to get a list of digests for all the images you have.
docker images --digests
You can add a grep to drill down further
docker images --digests | grep tomcat
You can find it at the time of pulling the image from the respective repository. Below command mentions Digest: sha256 at the time of pulling the docker image.
09:33 AM##~::>docker --version
Docker version 19.03.4, build 9013bf5
Digest: sha256:6e9f67fa63b0323e9a1e587fd71c561ba48a034504fb804fd26fd8800039835d
09:28 AM##~::>docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
7ddbc47eeb70: Pull complete
c1bbdc448b72: Pull complete
8c3b70e39044: Pull complete
45d437916d57: Pull complete
**Digest: sha256:6e9f67fa63b0323e9a1e587fd71c561ba48a034504fb804fd26fd8800039835d**
Status: Downloaded newer image for ubuntu:latest
docker.io/library/ubuntu:latest
Once, the image is downloaded, we can do the following
"ubuntu#sha256:6e9f67fa63b0323e9a1e587fd71c561ba48a034504fb804fd26fd8800039835d"
09:36 AM##~::>docker inspect ubuntu | grep -i sha256
"Id": "sha256:775349758637aff77bf85e2ff0597e86e3e859183ef0baba8b3e8fc8d3cba51c",
**"ubuntu#sha256:6e9f67fa63b0323e9a1e587fd71c561ba48a034504fb804fd26fd8800039835d"**
"Image": "sha256:f0caea6f785de71fe8c8b1b276a7094151df6058aa3f22d2902fe6b51f1a7a8f",
"Image": "sha256:f0caea6f785de71fe8c8b1b276a7094151df6058aa3f22d2902fe6b51f1a7a8f",
"sha256:cc967c529ced563b7746b663d98248bc571afdb3c012019d7f54d6c092793b8b",
"sha256:2c6ac8e5063e35e91ab79dfb7330c6154b82f3a7e4724fb1b4475c0a95dfdd33",
"sha256:6c01b5a53aac53c66f02ea711295c7586061cbe083b110d54dafbeb6cf7636bf",
"sha256:e0b3afb09dc386786d49d6443bdfb20bc74d77dcf68e152db7e5bb36b1cca638"
This should have been the Id field, that you could see in the old deprecated Docker Hub API
GET /v1/repositories/foo/bar/images HTTP/1.1
Host: index.docker.io
Accept: application/json
Parameters:
namespace – the namespace for the repo
repo_name – the name for the repo
Example Response:
HTTP/1.1 200
Vary: Accept
Content-Type: application/json
[{"id": "9e89cc6f0bc3c38722009fe6857087b486531f9a779a0c17e3ed29dae8f12c4f",
"checksum": "b486531f9a779a0c17e3ed29dae8f12c4f9e89cc6f0bc3c38722009fe6857087"},
{"id": "ertwetewtwe38722009fe6857087b486531f9a779a0c1dfddgfgsdgdsgds",
"checksum": "34t23f23fc17e3ed29dae8f12c4f9e89cc6f0bsdfgfsdgdsgdsgerwgew"}]
BUT: this is not how it is working now with the new docker distribution.
See issue 628: "Get image ID with tag name"
The /v1/ registry response /repositories/<repo>/tags used to list the image ID along with the tag handle.
/v2/ only seems to give the handle.
It would be useful to get the ID to compare to the ID found locally. The only place I can find the ID is in the v1Compat section of the manifest (which is overkill for the info I want)
The current (mid 2015) answer is:
This property of the V1 API was very computationally expensive for the way images are stored on the backend. Only the tag names are enumerated to avoid a secondary lookup.
In addition, the V2 API does not deal in Image IDs. Rather, it uses digests to identify layers, which can be calculated as property of the layer and are independently verifiable.
As mentioned by #zelphir, using digests is not a good way since it doesn't exist for a local-only image. I assume the image ID sha is the most accurate and consistent across tags/pull/push etc.
docker inspect --format='{{index .Id}}' $IMAGE
Does the trick.
Just issue docker pull tomcat:7-jre8 again and you will get what you want.
According to the Docker documentation, to build your own image, you must always specify a base image using the FROM instruction.
Obviously, there are lots of images to choose from in the Docker index, but what if I wanted to build my own? Is that possible?
The image base is built off Ubuntu if I understand correctly, and I want to experiment with a Debian image. Plus, I want to really understand how Docker works, and the base image is still a blackbox for me.
Edit: official documentation on creating a base image
You can take a look at how the base images are created and go from there.
You can find them here: https://github.com/dotcloud/docker/tree/master/contrib.
There is mkimage-busybox.sh, mkimage-unittest.sh, mkimage-debian.sh
Quoting Solomon Hykes:
You can easily create a new container from any tarball with "docker import". For example:
debootstrap raring ./rootfs
tar -C ./rootfs -c . | docker import - flimm/mybase
(credit to fatherlinux) Get information from https://developers.redhat.com/blog/2014/05/15/practical-introduction-to-docker-containers/ , which explains better
Create the tar files for your file system, simply could be
tar --numeric-owner --exclude=/proc --exclude=/sys -cvf centos6-base.tar /
Transfer the tar file to other docker system if not installed locally and import it
cat centos6-base.tar | docker import - centos6-base
Now you can verify by running it.
docker run -i -t centos6-base cat /etc/redhat-release
The scripts from dotcloud combine first two steps together which make me confused and looks complicated in the beginning.
The docker official guideline using debootstrap also tries to make clean file system.
You can judge by yourself how to do step 1.
To start building your own image from scratch, you can use the scratch image.
Using the scratch “image” signals to the build process that you want the next command in the Dockerfile to be the first filesystem layer in your image.
FROM scratch
ADD hello /
CMD ["/hello"]
http://docs.docker.com/engine/articles/baseimages/#creating-a-simple-base-image-using-scratch
If you want to make your own base image I would first take a look at
Official Images, specifically stackbrew inside that repo.
Otherwise there are some great references for minimal OS images in the docker repo itself.
For example here is a script for making a minimal arch image and there are more here.