I am using kops as kubernetes deployment.
I noticed that whenever an image with same tag number is entered in deployment file the system takes the previous image if imagepullpolicy is not set to always
Is there any way in which I can see all the cached images of a container in kubernetes environment ?
Like suppose I have an image test:56 currently running in a deployment and test:1 to test:55 were used previously, so does kubernetes cache those images ? and if yes where can those be found ?
Comments on your environment:
I noticed that whenever an image with same tag number is entered in deployment file the system takes the previous image if imagepullpolicy is not set to always
A pre-pulled image can be used to preload certain images for speed or as an alternative to authenticating to a private registry, optimizing performance.
The docker will always cache all images that were used locally.
Since you are using EKS, keep in mind that if you have node health management (meaning a node will be replaced if it fails) the new node won't have the images cached from the old one so it's always a good idea to store your images on a Registry like your Cloud Provider Registry or a local registry.
Let's address your first question:
Is there any way in which I can see all the cached images of a container in kubernetes environment ?
Yes, you must use docker images to list the images stored in your environment.
Second question:
Like suppose I have an image test:56 currently running in a deployment and test:1 to test:55 were used previously, so does Kubernetes cache those images ? and if yes where can those be found ?
I prepared an example for you:
I deployed several pods based on the official busybox image:
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.28.4
pod/busy284 created
$ kubectl run busy293 --generator=run-pod/v1 --image=busybox:1.29.3
pod/busy284 created
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.28
pod/busy28 created
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.29
pod/busy29 created
$ kubectl run busy284 --generator=run-pod/v1 --image=busybox:1.30
pod/busy284 created
$ kubectl run busybox --generator=run-pod/v1 --image=busybox
pod/busybox created
Now let's check the images stored in docker images
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
k8s.gcr.io/kube-proxy v1.17.3 ae853e93800d 5 weeks ago 116MB
k8s.gcr.io/kube-controller-manager v1.17.3 b0f1517c1f4b 5 weeks ago 161MB
k8s.gcr.io/kube-apiserver v1.17.3 90d27391b780 5 weeks ago 171MB
k8s.gcr.io/kube-scheduler v1.17.3 d109c0821a2b 5 weeks ago 94.4MB
kubernetesui/dashboard v2.0.0-beta8 eb51a3597525 3 months ago 90.8MB
k8s.gcr.io/coredns 1.6.5 70f311871ae1 4 months ago 41.6MB
k8s.gcr.io/etcd 3.4.3-0 303ce5db0e90 4 months ago 288MB
kubernetesui/metrics-scraper v1.0.2 3b08661dc379 4 months ago 40.1MB
busybox latest 83aa35aa1c79 10 days ago 1.22MB
busybox 1.30 64f5d945efcc 10 months ago 1.2MB
busybox 1.29 758ec7f3a1ee 15 months ago 1.15MB
busybox 1.29.3 758ec7f3a1ee 15 months ago 1.15MB
busybox 1.28 8c811b4aec35 22 months ago 1.15MB
busybox 1.28.4 8c811b4aec35 22 months ago 1.15MB
You can see all the pushed images listed.
It's good to clean old resources from your system using the command docker system prune to free space on your server from time to time.
If you have any doubt, let me know in the comments.
Related
I made a goof while trying to rename an image by following the steps on this page that say to create a tag then delete the original
Docker how to change repository name or rename image?
Now when I list the images it doesn't show up anymore. However, when I list the containers the image still shows up.
PS C:\Users\Grail> docker images -a
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu latest 157be28c0fe3 7 days ago 668MB
fedora latest a368cbcfa678 2 months ago 183MB
PS C:\Users\Grail> docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS
1ea5ffd50852 157be28c0fe3 "/bin/bash" 7 hours ago Exited (0) 7 hours ago
fb81990e756c 0d120b6ccaa8 "/bin/bash" 10 hours ago Exited (0) 24 minutes ago
081641b3e600 a368cbcfa678 "/bin/bash" 11 hours ago Exited (0) 31 minutes ago
Not only that, the image (0d120b6ccaa8) still shows up in my Docker Dashboard (running on Windows) and I can start/stop it without any problems.
Clearly the image still exists. Can I restore it such that I can see it when I list the images?
Can it be restored from the container?
If it's in a weird state/unrecoverable, how do I actually delete it so it's not taking up space?
Update:
Thanks to #prashanna I went down a path where I exported the container and imported to get the image:
docker export -o mycontainer.tar fb81990e756c
docker import mycontainer.tar
**docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]**
is for creating a new image from a container, meaning when you update or add new config or install new software, thus creating a new template images.
ref:docker commit
I was following this URL: How to use local docker images with Minikube?
I couldn't add a comment, so thought of putting my question here:
On my laptop, I have Linux Mint OS. Details as below:
Mint version 19,
Code name : Tara,
PackageBase : Ubuntu Bionic
Cinnamon (64-bit)
As per one the answer on the above-referenced link:
I started minikube and checked pods and deployments
xxxxxxxxx:~$ pwd
/home/sj
xxxxxxxxxx:~$ minikube start
xxxxxxxxxx:~$ kubectl get pods
xxxxxxxxxx:~$ kubectl get deployments
I ran command docker images
xxxxxxxxx:~$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
<username>/spring-docker-01 latest e10f88e1308d 6 days ago 640MB
openjdk 8 81f83aac57d6 4 weeks ago 624MB
mysql 5.7 563a026a1511 4 weeks ago 372MB
I ran below command:
eval $(minikube docker-env)
Now when I check docker images, looks like as the README describes, it reuses the Docker daemon from Minikube with eval $(minikube docker-env).
xxxxxxxxxxxxx:~$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nginx alpine 33c5c6e11024 9 days ago 17.7MB
k8s.gcr.io/coredns 1.2.2 367cdc8433a4 5 weeks ago 39.2MB
k8s.gcr.io/kubernetes-dashboard-amd64 v1.10.0 0dab2435c100 5 weeks ago 122MB
k8s.gcr.io/kube-proxy-amd64 v1.10.0 bfc21aadc7d3 6 months ago 97MB
k8s.gcr.io/kube-controller-manager-amd64 v1.10.0 ad86dbed1555 6 months ago 148MB
k8s.gcr.io/kube-apiserver-amd64 v1.10.0 af20925d51a3 6 months ago 225MB
k8s.gcr.io/kube-scheduler-amd64 v1.10.0 704ba848e69a 6 months ago 50.4MB
k8s.gcr.io/etcd-amd64 3.1.12 52920ad46f5b 6 months ago 193MB
k8s.gcr.io/kube-addon-manager v8.6 9c16409588eb 7 months ago 78.4MB
k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64 1.14.8 c2ce1ffb51ed 9 months ago 41MB
k8s.gcr.io/k8s-dns-sidecar-amd64 1.14.8 6f7f2dc7fab5 9 months ago 42.2MB
k8s.gcr.io/k8s-dns-kube-dns-amd64 1.14.8 80cc5ea4b547 9 months ago 50.5MB
k8s.gcr.io/pause-amd64 3.1 da86e6ba6ca1 9 months ago 742kB
gcr.io/k8s-minikube/storage-provisioner v1.8.1 4689081edb10 11 months ago 80.8MB
k8s.gcr.io/echoserver 1.4 a90209bb39e3 2 years ago 140MB
Note: if noticed docker images command pulled different images before and after step 2.
As I didn't see the image that I wanted to put on minikube, I pulled it from my docker hub.
xxxxxxxxxxxxx:~$ docker pull <username>/spring-docker-01
Using default tag: latest
latest: Pulling from <username>/spring-docker-01
05d1a5232b46: Pull complete
5cee356eda6b: Pull complete
89d3385f0fd3: Pull complete
80ae6b477848: Pull complete
40624ba8b77e: Pull complete
8081dc39373d: Pull complete
8a4b3841871b: Pull complete
b919b8fd1620: Pull complete
2760538fe600: Pull complete
48e4bd518143: Pull complete
Digest: sha256:277e8f7cfffdfe782df86eb0cd0663823efc3f17bb5d4c164a149e6a59865e11
Status: Downloaded newer image for <username>/spring-docker-01:latest
Verified if I can see that image using "docker images" command.
xxxxxxxxxxxxx:~$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
<username>/spring-docker-01 latest e10f88e1308d 6 days ago 640MB
nginx alpine 33c5c6e11024 10 days ago 17.7MB
Then I tried to build image as stated in referenced link step.
xxxxxxxxxx:~$ docker build -t <username>/spring-docker-01 .
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /home/sj/Dockerfile: no such file or directory
As the error states that dockerfile doesn't exist at the location, I am not sure where exactly I can see dockerfile for the image I had pulled from docker hub.
Looks like I have to go to the location where the image has been pulled and from that location, I need to run the above-mentioned command. Please correct me wrong.
Below are the steps, I will be doing after I fix the above-mentioned issue.
# Run in minikube
kubectl run hello-foo --image=myImage --image-pull-policy=Never
# Check that it's running
kubectl get pods
UPDATE-1
There is mistake in above steps.
Step 6 is not needed. Image has already been pulled from docker hub, so no need of docker build command.
With that, I went ahead and followed instructions as mentioned by #aurelius in response.
xxxxxxxxx:~$ kubectl run sdk-02 --image=<username>/spring-docker-01:latest --image-pull-policy=Never
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/sdk-02 created
Checked pods and deployments
xxxxxxxxx:~$ kubectl get pods
NAME READY STATUS RESTARTS AGE
sdk-02-b6db97984-2znlt 1/1 Running 0 27s
xxxxxxxxx:~$ kubectl get deployments
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
sdk-02 1 1 1 1 35s
Then exposed deployment on port 8084 as I was using other ports like 8080 thru 8083
xxxxxxxxx:~$ kubectl expose deployment sdk-02 --type=NodePort --port=8084
service/sdk-02 exposed
Then verified if service has been started, checked if no issue on kubernetes dashboard and then checked the url
xxxxxxxxx:~$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h
sdk-02 NodePort 10.100.125.120 <none> 8084:30362/TCP 13s
xxxxxxxxx:~$ minikube service sdk-02 --url
http://192.168.99.101:30362
When I tried to open URL: http://192.168.99.101:30362 in browser I got message:
This site can’t be reached
192.168.99.101 refused to connect.
Search Google for 192 168 101 30362
ERR_CONNECTION_REFUSED
So the question : Is there any issue with steps performed?
UPDATE-2
The issue was with below step:
xxxxxxxxx:~$ kubectl expose deployment sdk-02 --type=NodePort --port=8084
service/sdk-02 exposed
Upon checking Dockerfile of my image : <username>/spring-docker-01:latest I was exposing it to 8083 something like EXPOSE 8083
May be that was causing issue.
So I went ahead and changed expose command:
xxxxxxxxx:~$ kubectl expose deployment sdk-02 --type=NodePort --port=8083
service/sdk-02 exposed
And then it started working.
If anyone has something to add to this, please feel free.
However I am still not sure where exactly I can see dockerfile for the image I had pulled from docker hub.
The docker build does not know what you mean by your command, because flag -t requires specific format:
--tag , -t Name and optionally a tag in the ‘name:tag’ format
xxxxxxxxxx:~/Downloads$ docker build -t shivnilesh1109/spring-docker-01 .
So the proper command here should be:
docker build -t shivnilesh1109/spring-docker-01:v1(1) .(2)
(1) desired name of your container:tag
(2) directory in which your dockerfile is.
After you proceed to minikube deployment, it will be enough just to run:
kubectl run *desired name of deployment/pod* --image=*name of the container with tag* --image-pull-policy=Never
If this would not fix your issue, try adding the path to Dockerfile manually. I've tested this on my machine and error stopped after using proper tagging of the image and tested this also with full path to Dockerfile otherwise I had the same error as you.
For you UPDATE-2 question, also to help you to understand the port exposed in the Dockerfile and in the command kubectl expose.
Dockerfile:
The EXPOSE instruction does not actually publish the port. It
functions as a type of documentation between the person who builds the
image and the person who runs the container, about which ports are
intended to be published.
For more details, see EXPOSE.
Kubectl expose:
--port: The port that the service should serve on. Copied from the resource being exposed, if unspecified
--target-port: Name or number for the port on the container that the service should direct traffic to. Optional.
For more details, see kubectl expose.
So I think you should add the parameters --target-port with the port that you exposed in the Dockerfile. And then the port mapping will be correct.
You can just create a Dockerfile with this content:
FROM shivnilesh1109/spring-docker-01
Then run:
docker build -t my-spring-docker-01 .
try adding your local docker image to minikube's cache , like so :
minikube cache add docker-image-name:latesttag
then set imagePullPolicy:Never in the yaml file.
I am trying to install additional python packages in AWX container awx_tasks so that the changes could enable the ansible modules like snow, ec2_elb_facts run (which have pre-requisites as Python modules). I have made the changes in the container using:
# docker exec -it 80ab6bf562a9 bash
where 80ab6bf562a9 is the container id for awx_task container.
and then installed the required packages inside the custom virtual environment (as mentioned in the AWX documentation). Post this, i have made the changes permanent by creating a new image with the container changes using:
# docker commit 80ab6bf562a9 ansible/awx_task:latest
Post this, ran the following command to map the new container with the newly created image with container changes.
# docker run --name awx_task -d 5290f9b3268c
Following are the containers post the above changes. Here, the newly created container which was mapped with the new image with changes in existing container is 968fb2a7da2f.
# docker container ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
968fb2a7da2f 5290f9b3268c "/tini -- /bin/sh -c…" 2 days ago Exited (143) 2 days ago awx_task
80ab6bf562a9 535bb2b8e1f3 "/tini -- /bin/sh -c…" 3 weeks ago Up 2 days 8052/tcp awx_task_OLD
aea2551951d5 b7c261b76010 "/tini -- /bin/sh -c…" 3 weeks ago Up 2 days 0.0.0.0:80->8052/tcp awx_web
e789a4a82a9e memcached:alpine "docker-entrypoint.s…" 3 weeks ago Up 2 days 11211/tcp memcached
a8c74584255c ansible/awx_rabbitmq:3.7.4 "docker-entrypoint.s…" 3 weeks ago Up 2 days 4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 25672/tcp rabbitmq
25f6f6ca7766 postgres:9.6 "docker-entrypoint.s…" 3 weeks ago Up 2 days 5432/tcp postgres
Following are my images post above changes. Here, the newly created image (with changes) is 5290f9b3268c.
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ansible/awx_task latest 5290f9b3268c 2 days ago 1.48GB
postgres 9.6 106bdfb062df 8 weeks ago 235MB
ansible/awx_task <none> 535bb2b8e1f3 8 weeks ago 1.07GB
ansible/awx_web <none> b7c261b76010 8 weeks ago 1.04GB
hello-world latest 2cb0d9787c4d 2 months ago 1.85kB
memcached alpine b40e8fa7e3e5 2 months ago 8.69MB
ansible/awx_rabbitmq 3.7.4 e08fe791079e 6 months ago 85.6MB
The new container is properly mapped with the new image (which has got the changes i wanted). The issue now is that when i stop the old container and start the new container AWX doesn't work. I can just view the UI, if i run any tasks like executing templates, it just freezes. It appears like the new container/images are not talking with the other containers like awx_rabbitmq, postgres etc. I have been reading multiple posts regarding this however, i couldn't find any single post which highlights anything regarding this.
I basically want the changes in the awx_task container to work so that i could achieve the goal of making the custom modules work. Could anyone suggest what can be done so that the new awx_task container could take the role of the older awx_task and AWX could work normally?
Since i found the way to do this, i will share the steps to make the required changes.
The python package versions can be controlled from the requirements directory, AWX Task and AWX Web Images related changes can be applied in the Dockerfile.j2 in the roles directory. Once the required changes are applied, we can run the setup using ansible-playbook install.yml -i inventory.
You should use the install.yml to restart the awx_task container, since it ensures the right environment variables are set, the right volumes are mapped, etc. Same command as you've used to install AWX:
ansible-playbook install.yml -i inventory.
See here for a full list of arguments that are used.
I am new to Docker and tried to check how to make changes in a docker container and commit them to reflect in the results. I have AWX running on Docker-CE and had created made changes in awx_web container to update the UI logo as mentioned in the Ansible Tower documentation:
docker cp custom_console_logo.png aea2551951d5:/var/lib/awx/public/static/assets
and created the local_settings.json in /var/lib/awx/public/static/ one of the running awx_web container using the following and then using the bash command to create and save file:
docker exec -it aea2551951d5 bash
Post this, i have commited the changes so that the changes are reflected permanently in the image file and in UI using:
docker commit aea2551951d5 ansible/awx_web
I have verified the changes in the container files and it appears to be fine however, the changes are not getting reflected when i verified them on browser. It appears like the existing container is pointing to the old image file:
CONTAINER ID IMAGE
aea2551951d5 b7c261b76010
I am not sure what was missed. Could anyone suggest if i have missed anything and how do i fix this?
Following are the containers:
[root#localhost~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
80ab6bf562a9 ansible/awx_task:latest "/tini -- /bin/sh -c…" 13 days ago Up 3 days 8052/tcp awx_task
aea2551951d5 b7c261b76010 "/tini -- /bin/sh -c…" 13 days ago Up 3 hours 0.0.0.0:80->8052/tcp awx_web
e789a4a82a9e memcached:alpine "docker-entrypoint.s…" 13 days ago Up 3 days 11211/tcp memcached
a8c74584255c ansible/awx_rabbitmq:3.7.4 "docker-entrypoint.s…" 13 days ago Up 3 days 4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 25672/tcp rabbitmq
25f6f6ca7766 postgres:9.6 "docker-entrypoint.s…" 13 days ago Up 3 days 5432/tcp postgres
Following are the Images:
[root#localhost~]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
ansible/awx_web latest 1accc59cbdee 2 hours ago 1.06GB
b7c261b76010 latest 998ec3e77fd8 3 hours ago 1.06GB
postgres 9.6 106bdfb062df 6 weeks ago 235MB
ansible/awx_task latest 535bb2b8e1f3 6 weeks ago 1.07GB
ansible/awx_web <none> b7c261b76010 6 weeks ago 1.04GB
hello-world latest 2cb0d9787c4d 2 months ago 1.85kB
memcached alpine b40e8fa7e3e5 2 months ago 8.69MB
ansible/awx_rabbitmq 3.7.4 e08fe791079e 5 months ago 85.6MB
[root#clab2726vw0001 ~]#
Environment Information:
Docker Version: 18.06.0-ce, build 0ffa825
AWX Version: 1.0.6.41
Ansible Version: 2.6.1
RHEL Version: Red Hat Enterprise Linux Server release 7.5 (Maipo)
Containers are the running instances of images and the container id is unique for the particular image.
We cannot use the same 12 digit container id as a reference for the other images. When we use "commit" it create a new fresh container for the new image.
This new container will contain all the changes needed.
You can try with naming the container.
Give a name to container (using docker rename command) .
Make your changes in container.
Commit it.
Give same name to new container generated for the new image.
Rename old container with some other name.
Now you have the same container name (as you were using before) but with new Image.
I am trying to deploy an application to a server. I am using Meteorup for deployment. This deployment config on this post is what I use Meteor-up terminates after running deploy.
This is part of the questions I have raised and yet none have solved it https://github.com/zodern/meteor-up/issues/734[1]. After much troubleshooting, I ran docker images, it actually added names to the images of mongodb and abernix/meteord but for the one I want to deploy it did not. Why? Since containers are created from images definitely it will not see the container as there is no image to reference for creation.
See this:
root#hostname:~# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
<none> <none> 06f352515b0a 41 minutes ago 659MB
<none> <none> e29de6b55976 About an hour ago 659MB
abernix/meteord base 7941fc48936e 2 months ago 520MB
mongo 3.4.1 0dffc7177b06 8 months ago 402MB
When I ran docker ps it showed that only mongo container was created:
root#christdoes:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2e7f21a50dfb mongo:3.4.1 "/entrypoint.sh mo..." 2 hours ago Up 2 hours 127.0.0.1:27017->27017/tcp mongodb
How do I resolve this issue?