Cached Docker images on Hosted Linux Preview agent - docker

We are building docker image on VSTS by using VSTS Hosted Linux Preview agent. microsoft/aspnetcore-build image is used to build asp.net core application. Each time build is triggered, an agent is pulling microsoft/aspnetcore-build image from registry and it takes some time. We would like to avoid this, by specifying specific image pre-cached on agents.
Is there a list of container images that have been cached on Hosted Linux Preview agent? Such information is available for Hosted VS2017 agent, but not for Linux one.

No official article mentions it, I submit a user voice here: Docker image cache on Hosted linux agent that you can vote and follow.

According to this documentation the complete list can be found in the following documents:
Details of Ubuntu 18.04 agent image *
Details of Ubuntu 20.04 agent image *
(*) The above are permalinks to a specific snapshot of code (dated 2022-03-26). For the most current data, switch to the "main" branch in GitHub. Note that files may disappear from the 'main' branch when image gets deprecated.
As of 2022-03-26, both images include the same set of pre-cached docker images which are:
alpine:3.12
alpine:3.13
alpine:3.14
buildpack-deps:bullseye
buildpack-deps:buster
buildpack-deps:stretch
debian:10
debian:11
debian:9
moby/buildkit:latest
node:12
node:12-alpine
node:14
node:14-alpine
node:16
node:16-alpine
ubuntu:16.04
ubuntu:18.04
ubuntu:20.04
This can also be confirmed by adding a simple task to the pipeline that runs docker image list. Here's an example of the command's output ran on Ubuntu-16.04 agent (before it was deprecated):
/usr/bin/docker image list
REPOSITORY TAG IMAGE ID CREATED SIZE
node 10-alpine 955e0e1f1a41 11 days ago 80.5MB
node 10 bb78c02ca3bf 11 days ago 908MB
node 12-alpine afd897e3184b 11 days ago 85.2MB
node 12 b6f455933a97 11 days ago 913MB
buildpack-deps stretch c82dad0a8baf 2 weeks ago 835MB
debian 9 92416e205014 2 weeks ago 101MB
debian 8 955c8f8160c2 2 weeks ago 129MB
alpine 3.8 c8bccc0af957 3 weeks ago 4.41MB
alpine 3.9 82f67be598eb 3 weeks ago 5.53MB
alpine 3.10 af341ccd2df8 3 weeks ago 5.56MB
jekyll/builder latest 4605d94a6b36 4 months ago 564MB
alpine 3.7 6d1ef012b567 11 months ago 4.21MB
mcr.microsoft.com/azure-pipelines/node8-typescript latest 9a948d360778 16 months ago 595MB

Related

How to run a CLion project in a docker container

I have the following CLion project:
/home/me/CLionProjects/myProject
This is the output of my docker images:
:sudo docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
tensorflow/tensorflow devel 61e124a9cc92 17 hours ago 1.96GB
ubuntu 16.04 b6f507652425 9 months ago 135MB
This project requires tensorflow-lite so I want to be able to compile the project inside the tensorflow/tensorflow image, which comes from this page in the Docker Hub.

Unable to remove python and ubuntu docker images

I am new to using dockers,so i found these python and ubuntu images when executing docker images and i am unable to remove it using docker rmi {image_id}.
Error response from daemon: conflict: unable to delete ba6acccedd29 (cannot be forced) - image has dependent child images
It was using up significant space,hence i was a bit concerned.
REPOSITORY TAG IMAGE ID CREATED SIZE
docker_jupyter latest 90708b9040df 28 hours ago 1.94GB
ubuntu 20.04 ba6acccedd29 7 days ago 72.8MB
testapi latest 82e970c819ff 6 weeks ago 1.33GB
python 3.9 a5210955ee89 6 weeks ago 911MB
python latest a5210955ee89 6 weeks ago 911MB
you have to delete the other images that depend on them first
I think "test_api" and "jupyter" images depends on python so u need to delete them first

How can we see cached images in kubernetes?

I am using kops as kubernetes deployment.
I noticed that whenever an image with same tag number is entered in deployment file the system takes the previous image if imagepullpolicy is not set to always
Is there any way in which I can see all the cached images of a container in kubernetes environment ?
Like suppose I have an image test:56 currently running in a deployment and test:1 to test:55 were used previously, so does kubernetes cache those images ? and if yes where can those be found ?
Comments on your environment:
I noticed that whenever an image with same tag number is entered in deployment file the system takes the previous image if imagepullpolicy is not set to always
A pre-pulled image can be used to preload certain images for speed or as an alternative to authenticating to a private registry, optimizing performance.
The docker will always cache all images that were used locally.
Since you are using EKS, keep in mind that if you have node health management (meaning a node will be replaced if it fails) the new node won't have the images cached from the old one so it's always a good idea to store your images on a Registry like your Cloud Provider Registry or a local registry.
Let's address your first question:
Is there any way in which I can see all the cached images of a container in kubernetes environment ?
Yes, you must use docker images to list the images stored in your environment.
Second question:
Like suppose I have an image test:56 currently running in a deployment and test:1 to test:55 were used previously, so does Kubernetes cache those images ? and if yes where can those be found ?
I prepared an example for you:
I deployed several pods based on the official busybox image:
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.28.4
pod/busy284 created
$ kubectl run busy293 --generator=run-pod/v1 --image=busybox:1.29.3
pod/busy284 created
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.28
pod/busy28 created
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.29
pod/busy29 created
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.30
pod/busy284 created
$ kubectl run busybox --generator=run-pod/v1 --image=busybox
pod/busybox created
Now let's check the images stored in docker images
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
k8s.gcr.io/kube-proxy v1.17.3 ae853e93800d 5 weeks ago 116MB
k8s.gcr.io/kube-controller-manager v1.17.3 b0f1517c1f4b 5 weeks ago 161MB
k8s.gcr.io/kube-apiserver v1.17.3 90d27391b780 5 weeks ago 171MB
k8s.gcr.io/kube-scheduler v1.17.3 d109c0821a2b 5 weeks ago 94.4MB
kubernetesui/dashboard v2.0.0-beta8 eb51a3597525 3 months ago 90.8MB
k8s.gcr.io/coredns 1.6.5 70f311871ae1 4 months ago 41.6MB
k8s.gcr.io/etcd 3.4.3-0 303ce5db0e90 4 months ago 288MB
kubernetesui/metrics-scraper v1.0.2 3b08661dc379 4 months ago 40.1MB
busybox latest 83aa35aa1c79 10 days ago 1.22MB
busybox 1.30 64f5d945efcc 10 months ago 1.2MB
busybox 1.29 758ec7f3a1ee 15 months ago 1.15MB
busybox 1.29.3 758ec7f3a1ee 15 months ago 1.15MB
busybox 1.28 8c811b4aec35 22 months ago 1.15MB
busybox 1.28.4 8c811b4aec35 22 months ago 1.15MB
You can see all the pushed images listed.
It's good to clean old resources from your system using the command docker system prune to free space on your server from time to time.
If you have any doubt, let me know in the comments.

Kitematic not showing any Images/Containers

I have pulled three images from docker hub using docker.I have listed those three images below
suju#suju-VirtualBox:~$ sudo docker images -a
REPOSITORY TAG IMAGE ID CREATED SIZE
mysql latest 71b5c7e10f9b 11 days ago 477MB
hello-world latest fce289e99eb9 4 weeks ago 1.84kB
java latest d23bdf5b1b1b 2 years ago 643MB
And i installed kitematic 0.17.6 for purpose of Managing Docker container
but when i open kitematic, it doent show any container, its just shows a white window without any options except exit

How to link container with a new image?

I am new to Docker and tried to check how to make changes in a docker container and commit them to reflect in the results. I have AWX running on Docker-CE and had created made changes in awx_web container to update the UI logo as mentioned in the Ansible Tower documentation:
docker cp custom_console_logo.png aea2551951d5:/var/lib/awx/public/static/assets
and created the local_settings.json in /var/lib/awx/public/static/ one of the running awx_web container using the following and then using the bash command to create and save file:
docker exec -it aea2551951d5 bash
Post this, i have commited the changes so that the changes are reflected permanently in the image file and in UI using:
docker commit aea2551951d5 ansible/awx_web
I have verified the changes in the container files and it appears to be fine however, the changes are not getting reflected when i verified them on browser. It appears like the existing container is pointing to the old image file:
CONTAINER ID IMAGE
aea2551951d5 b7c261b76010
I am not sure what was missed. Could anyone suggest if i have missed anything and how do i fix this?
Following are the containers:
[root#localhost~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
80ab6bf562a9 ansible/awx_task:latest "/tini -- /bin/sh -c…" 13 days ago Up 3 days 8052/tcp awx_task
aea2551951d5 b7c261b76010 "/tini -- /bin/sh -c…" 13 days ago Up 3 hours 0.0.0.0:80->8052/tcp awx_web
e789a4a82a9e memcached:alpine "docker-entrypoint.s…" 13 days ago Up 3 days 11211/tcp memcached
a8c74584255c ansible/awx_rabbitmq:3.7.4 "docker-entrypoint.s…" 13 days ago Up 3 days 4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 25672/tcp rabbitmq
25f6f6ca7766 postgres:9.6 "docker-entrypoint.s…" 13 days ago Up 3 days 5432/tcp postgres
Following are the Images:
[root#localhost~]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
ansible/awx_web latest 1accc59cbdee 2 hours ago 1.06GB
b7c261b76010 latest 998ec3e77fd8 3 hours ago 1.06GB
postgres 9.6 106bdfb062df 6 weeks ago 235MB
ansible/awx_task latest 535bb2b8e1f3 6 weeks ago 1.07GB
ansible/awx_web <none> b7c261b76010 6 weeks ago 1.04GB
hello-world latest 2cb0d9787c4d 2 months ago 1.85kB
memcached alpine b40e8fa7e3e5 2 months ago 8.69MB
ansible/awx_rabbitmq 3.7.4 e08fe791079e 5 months ago 85.6MB
[root#clab2726vw0001 ~]#
Environment Information:
Docker Version: 18.06.0-ce, build 0ffa825
AWX Version: 1.0.6.41
Ansible Version: 2.6.1
RHEL Version: Red Hat Enterprise Linux Server release 7.5 (Maipo)
Containers are the running instances of images and the container id is unique for the particular image.
We cannot use the same 12 digit container id as a reference for the other images. When we use "commit" it create a new fresh container for the new image.
This new container will contain all the changes needed.
You can try with naming the container.
Give a name to container (using docker rename command) .
Make your changes in container.
Commit it.
Give same name to new container generated for the new image.
Rename old container with some other name.
Now you have the same container name (as you were using before) but with new Image.

Resources