Each docker image layer correlates to overlay data on disk (in case of using "overlay" engine).
I.e.
We do inspect of docker image and get it's layers:
docker inspect <image> | grep 'RootFS' -A5
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:9c2f1836d49346677f8280bf0eb89c20853f6af4aa6e2fad87b0000bb181fad2",
"sha256:97a77835754fbd5f0883e663cd168ae6263551318d56476920ea05e500e371e6",
"sha256:7ea85bb6d4ef6de5e5ccaf47325b770b954142666b89ab034a0dff3cb98a2808",
...
Layer "sha256:9c2f1836d49346677f8280bf0eb89c20853f6af4aa6e2fad87b0000bb181fad2" correlates to data on disk, for example to "/var/lib/docker/overlay/0608b16c5e9a58f48cfe30ce9559b5c8676e23655719c7141fe75ae86076c3a9/root".
Is it possible to get this docker overlay root path /var/lib/docker/overlay/{hash}/root by image layer sha256?
Related
I am running WSL and I want to find where the docker images are stored in my filesystem.
I've tried docker info and it shows Docker Root Dir: /var/lib/docker
but when I do ls /var/lib/docker it shows ls: cannot access '/var/lib/docker/': No such file or directory
Tried finding some info and came across this fcc blog regarding Docker on Windows:
but mine contains only txt files and a tmp folder.
I tried inspecting an image I've built and it shows: `
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/tm64pahi18wlse3ibs2uex6dm/diff:/var/lib/docker/overlay2/s4jgik32xo1e9qoqie33bt6d7/diff:/var/lib/docker/overlay2/m9ea6j2mrehihllvnm1s51uyi/diff:/var/lib/docker/overlay2/1o1586gytwxyzhuap98m5a9v4/diff:/var/lib/docker/overlay2/437dfxzj5gmfrbs6f5a9pj0xt/diff:/var/lib/docker/overlay2/4d53adbcad7cc93f261de7f36303a7e1c54ae1cf1accb2768c881be550dd4e95/diff:/var/lib/docker/overlay2/62106246a0f1977a89f193792c0f066a10bc8179e1406d1931cb8c8f15dc47f4/diff:/var/lib/docker/overlay2/28cf7a8fe3d1eab10f09a1056a7a36f88425c331cd0681b6cf1159156238cf4c/diff:/var/lib/docker/overlay2/6fd3c82e02de85abceb47d0a58a06405c1cab0301c73a70939f507a3810fa540/diff:/var/lib/docker/overlay2/4d5eb280e3c8445439d9ee40ba3c066e6c11caf0a76f104e2795195c50ac8389/diff",
"MergedDir": "/var/lib/docker/overlay2/9az4itvoy9m4jv669niircfar/merged",
"UpperDir": "/var/lib/docker/overlay2/9az4itvoy9m4jv669niircfar/diff",
"WorkDir": "/var/lib/docker/overlay2/9az4itvoy9m4jv669niircfar/work"
},
"Name": "overlay2"
}
but doing ls gives the same ls: cannot access '/var/lib/docker/': No such file or directory
The only place I can see anything that I can understand is using docker images and in Docker Desktop where I can see the size an image takes up.
My OS info is below
NAME="Ubuntu"
VERSION="20.04 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
Problem
I am unable to determine the original image tag (and so the image version) on a running container in Kubernetes.
Description
Pure Docker:
When running a container in pure docker and the inspecting the container, I can always view the image tag. For example, when starting a container running ubuntu:18.04 and then inspecting it (using docker inspect) I see the following output (hugely shortened for brevity):
[
{
"Id": "e4109d8d4a3835f92629732d9dcb0967c16f9a716c6bbda8edf3a4423d714d01",
"Image": "sha256:2eb2d388e1a255c98029f40d6d7f8029fb13f1030abc8f11ccacbca686a8dc12",
"Config": {
"Hostname": "e4109d8d4a38",
"Env": ["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],
"Image": "ubuntu:18.04",
"Labels": {}
}
}
]
Here I can see that .Config.Image shows my original image + tag (ubuntu:18.04).
Kubernetes:
When pulling the same image into a Kubernetes cluster, it seems that Kubernetes always uses the digest when it creates the running container. Here is an example from a test pod also using ubuntu:18.04, showing output from kubectl desribe pod...
Normal Scheduled 109s default-scheduler Successfully assigned default/test-697b599788-t9xp9 to docker-desktop
Normal Pulling 108s kubelet, docker-desktop Pulling image "ubuntu:18.04"
Normal Pulled 104s kubelet, docker-desktop Successfully pulled image "ubuntu:18.04"
Normal Created 104s kubelet, docker-desktop Created container ubuntu
Normal Started 103s kubelet, docker-desktop Started container ubuntu
Above, the image that has been pulled is the image that is described in my pod specification (namely, ubuntu:18.04). So far, so good.
When I inspect the container from the docker API (via the mounted socket), .Config.Image is showing the digest, not the original tag. I have again amended the output for brevity:
curl --unix-socket /var/run/docker.sock https://localhost/container/99a29951d0b385875c54e586497dbbf1d3c6266bfc10351d0c75e6774394c682/json
{
"Id": "99a29951d0b385875c54e586497dbbf1d3c6266bfc10351d0c75e6774394c682",
"Image": "sha256:1e4467b07108685c38297025797890f0492c4ec509212e2e4b4822d367fe6bc8",
"Config": {
"Hostname": "test-7787dcf6d-h58rn",
"Env": [
"KUBERNETES_SERVICE_PORT=443",
"KUBERNETES_SERVICE_PORT_HTTPS=443",
"KUBERNETES_PORT=tcp://10.96.0.1:443",
"KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443",
"KUBERNETES_PORT_443_TCP_PROTO=tcp",
"KUBERNETES_PORT_443_TCP_PORT=443",
"KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1",
"KUBERNETES_SERVICE_HOST=10.96.0.1",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Image": "ubuntu#sha256:5d1d5407f353843ecf8b16524bc5565aa332e9e6a1297c73a92d3e754b8a636d",
"Labels": {
"annotation.io.kubernetes.container.hash": "3ee6fba4",
"annotation.io.kubernetes.container.restartCount": "0",
"annotation.io.kubernetes.container.terminationMessagePath": "/dev/termination-log",
"annotation.io.kubernetes.container.terminationMessagePolicy": "File",
"annotation.io.kubernetes.pod.terminationGracePeriod": "30",
"io.kubernetes.container.logpath": "/var/log/pods/default_test-7787dcf6d-h58rn_cf601af0-84fe-4df7-8126-736189b6f7a6/ubuntu/0.log",
"io.kubernetes.container.name": "ubuntu",
"io.kubernetes.docker.type": "container",
"io.kubernetes.pod.name": "test-7787dcf6d-h58rn",
"io.kubernetes.pod.namespace": "default",
"io.kubernetes.pod.uid": "cf601af0-84fe-4df7-8126-736189b6f7a6",
"io.kubernetes.sandbox.id": "cf212829401e97a4ed65fb7707d912be676f775a895977ef0c9c62bb28fec74f"
}
}
}
If I grep the complete output, at no point do I find the label "18.04". I understand the mutability aspect, and that using the digest is in line with best practice, but I am curious to know if there is a way to get my original tag.
If Kubernetes is pulling the digest of the image with the tag 18.04, is there an option to annotate the container with the original image tag, so that it becomes available in the docker API (in .Labels or .Env), or in short is there any way that the docker API can determine the original image tag, and thus the actual version of ubuntu (or any other application type)?
I have a requirement of converting an OCI image manifest to Docker v2.2 image format and vice versa. But I am not able to find any difference between the two , is there any actual difference or they are same ?
Docker Image Manifest V 2, Schema 2
Registry image manifests define the components that make up an image on a container registry (see section on container registries). The more common manifest format we’ll be working with is the Docker Image Manifest V2, Schema 2 (more simply, V2.2). There is also a V2, Schema 1 format that is commonly used but more complicated than V2.2 due to backwards-compatibility reasons against V1.
The V2.2 manifest format is a JSON blob with the following top-level fields:
schemaVersion - 2 in this case
mediaType - application/vnd.docker.distribution.manifest.v2+json
config - descriptor of container configuration blob
layers - list of descriptors of layer blobs, in the same order as the rootfs of the container configuration
Blob descriptors are JSON objects containing 3 fields:
mediaType - application/vnd.docker.container.image.v1+json for a container configuration or application/vnd.docker.image.rootfs.diff.tar.gzip for a layer
size - the size of the blob, in bytes
digest - the digest of the content
Here is an example of a V2.2 manifest format (for the Docker Hub busybox image):
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 1497,
"digest": "sha256:3a093384ac306cbac30b67f1585e12b30ab1a899374dabc3170b9bca246f1444"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 755724,
"digest": "sha256:57c14dd66db0390dbf6da578421c077f6de8e88edd0815b4caa94607ba5f4c09"
}
]
}
OCI Image Manifest
The OCI image format is essentially the same as the Docker V2.2 format, with a few differences.
mediaType - must be set to application/vnd.oci.image.manifest.v1+json
config.mediaType - must be set to application/vnd.oci.image.config.v1+json
Each object in layers must have mediaType be either application/vnd.oci.image.layer.v1.tar+gzip or application/vnd.oci.image.layer.v1.tar.
Source: https://containers.gitbook.io/build-containers-the-hard-way/#registry-format-oci-image-manifest
Below is the manifest file entry snippet(docker inspect image redis) of redis image
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/bd512eb256c8aa56cbe9243d440a311820712d1a245fe6f523d39d19cd6c862d/diff:/var/lib/docker/overlay2/7fa1e90f35c78fc83c3a
4b86e36e45d742383b394adf9ce4cf9b339d919c9cbe/diff:/var/lib/docker/overlay2/2c1869386b5b8542959da4f0173a5272b9703326d619f27258b4edff7a1dbbf9/diff:/var/lib/docker/overlay2
/23ba3955c5b72ec17b9c409bd5233a3d92cbd75543c7d144b364f8188765788e/diff:/var/lib/docker/overlay2/87d8a92919103e8ff723221200acb36e17c611fa499571ab183d0f51458e6f24/diff",
"MergedDir": "/var/lib/docker/overlay2/e503ed41978e99fe9b71a4225763a40b7988e9a4f31d4c06ef1ec1af46b0b6ab/merged",
"UpperDir": "/var/lib/docker/overlay2/e503ed41978e99fe9b71a4225763a40b7988e9a4f31d4c06ef1ec1af46b0b6ab/diff",
"WorkDir": "/var/lib/docker/overlay2/e503ed41978e99fe9b71a4225763a40b7988e9a4f31d4c06ef1ec1af46b0b6ab/work"
},
"Name": "overlay2"
},
where overlay2 filesystem is used by docker image and container.
Within GraphDriver entry of manifest,
what does LowerDir / MergedDir / UpperDir / WorkDir indicate?
LowerDir: these are the read-only layers of an overlay filesystem. For docker, these are the image layers assembled in order.
UpperDir: this is the read-write layer of an overlay filesystem. For docker, that is equivalent to the container specific layer which contains changes made by that container.
WorkDir: this is a required directory for overlay, it needs an empty directory for internal use.
MergedDir: this is the result of the overlay filesystem. Docker effectively chroot's into this directory when running the container.
For more on overlay filesystems (overlay2 is a newer release, but I don't believe there are any user visible changes), see the kernel docs: https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt
Background:
For Development purposes I do a lot of docker-compose up -d and docker-compose stop.
To view logs of a container I do either
- docker logs --details --since=1m -t -f container_name
or
- docker inspect --format='{{.LogPath}}' container_name
cat path-from-previous
The problem is when I want to view 10 days older logs, there are none, the logs just have todays logs.
when I do a docker inspect container_name I get the following
"Created": "todays-timestamp"
my logging is the default config.
"LogConfig": {
"Type": "json-file",
"Config": {}
},
the reason behind this is because there is no rotation in your docker-logs.
in case you are using a linux system go to:
/etc/logrotate.d/
and create the file docker-container like this => /etc/logrotate.d/docker-container
write this into the file:
/var/lib/docker/containers/*/*.log {
rotate 7
daily
compress
missingok
delaycompress
copytruncate
}
it takes all builded images and their daily log and rotates + compress them.
you can test this with:
logrotate -fv /etc/logrotate.d/docker-container
enter your docker folder /var/lib/docker/containers/[CONTAINER ID]/ and you can see the rotation.
reference: https://sandro-keil.de/blog/logrotate-for-docker-container/