I've running several containers on my host (Ubuntu Server)
I ran docker container via a command, like below
sudo docker run -d -p 5050:80 gitname/reponame
And after I call sudo docker ps and it shows
CONTAINER ID: e404ffa2bc6b
IMAGE: gitname/reponame
COMMAND: "dotnet run --server…"
CREATED: 14 seconds ago
STATUS: Up 12 seconds
PORTS: 5050/tcp, 0.0.0.0:5050->80/tcp
NAMES: reverent_mcnulty
And in a week I run sudo docker ps again and it shows another info where IMAGE was changed and it looks like ba2486f19dc0
I don't understand why.
It's problem for me because for stopping containers I use the command:
sudo docker stop $(sudo docker ps | awk '{ print $1,$2 }' | grep gitname/reponame | awk '{print $1 }')
And it doesn't work because image name already was changed
Every Docker image has a unique hex ID. These IDs will be different on different systems (even docker pull of the same image), but every image has exactly one ID.
Each image has some number of tags associated with it. This could be none, or multiple. docker tag will add a tag to an existing image; docker rmi will remove a tag, and also (if there are no other tags on the image, no other images using the image as a base, and no extant containers using the image) remove the image.
It's possible to "steal" the tag from an existing image. The most obvious way to do this is with docker build:
cat >Dockerfile <<EOF
FROM busybox
COPY file.txt /
EOF
echo foo > file.txt
docker build -t foo .
docker images
# note the ID for foo:latest
echo bar > file.txt
docker build -t foo .
docker images
# note the old ID will show as foo:<none>
# note a different ID for foo:latest
An explicit docker tag can do this too. Images on Docker Hub and other repositories can change too (ubuntu:16.04 is routinely re-published with security updates), and so if you docker pull an image you already have it can cause the old image to "lose its name" in favor of a newer version of that image.
How does this interact with docker run and docker ps? docker run can remember the image tag an image was started with, but if the image no longer has that tag, it forgets that datum. That's why your docker ps output reverts to showing the hex ID of the image.
There are a couple of ways around this for your application:
If the image is one you're building yourself, always use an explicit version tag (could just be the current datestamp) both when building and running the image. Then you'll never overwrite an existing image's tag. ("Don't use :latest tags.")
Use docker run --name to keep track of which container is which, and filter based on that, not the image tag. (#quentino's suggestion from comments.)
Don't explicitly docker pull in your workflow. If you don't have an image, docker run will automatically pull it for you. This will avoid existing images getting minor updates and also will avoid existing images losing their names.
Related
In my use case I always fetch the image tagged with "latest" tag. This "latest" tag gets updated regularly. So even if the latest tag image is updated on registry, the "docker run" command does not update it on local host. This is expected behavior as the "latest" image exists on local host.
But I want to make sure that if the "latest" image on local host and registry are different then it should pull the latest image from registry.
Is there any way to achieve this?
You can manually docker pull the image before you run it. This is fairly inexpensive, especially if the image hasn't changed. You can do it while the old container is still running to minimize downtime.
docker pull the-image
docker stop the-container
docker rm the-container
docker run -d ... --name the-container the-image
In an automated environment you might consider avoiding the latest tag and other similar fixed strings due to exactly this ambiguity. In Kubernetes, for example, the default behavior is to reuse a local image that has some name, which can result in different nodes running different latest images. If you label your images with a date stamp or source-control ID or something else such that every image has a unique tag, you can just use that tag.
Finding the tag value can be problematic outside the context of a continuous-deployment system; Docker doesn't have any built-in way to find the most recent tag for an image.
# docker pull the-image:20220704 # optional
docker stop the-container
docker rm the-container
docker run -d ... --name the-container the-image:20220704
docker rmi the-image:20220630
One notable advantage of this last approach is that it's very easy to go back to an earlier build if today's build happens to be broken; just switch the image tag back a build or two.
I've tried pushing the newly built image to ECR but it seemed that it always kept pushing the old code from cache to ECR. I've tried to clear out cache with the following commands and rebuilt the image but the issue remained:
Clear out docker
docker rmi $$(docker images | grep none | awk '{print $$3}') -f
docker system prune -a -f
Rebuild the image
docker build -t $(DOMAIN)/$(REPO_NAME):$(IMAGE_VERSION) -f docker/Dockerfile . --no-cache
pushing the image to ECR
docker push $REPO_ID.dkr.ecr.$REGION.amazonaws.com/$(DOMAIN)/$(REPO_NAME):$(IMAGE_VERSION)
I've tested the built image locally and it works with the new code. However, when I run the image remotely, it ran the old code and failed
I have no idea what was going on. Can someone help me on this please?
I have sorted this out. I would need to update the function image by the following:
aws lambda update-function-code --function-name my-lambda-func --image-uri $REPO_ID.dkr.ecr.$REGION.amazonaws.com/$(DOMAIN)/$(REPO_NAME):latest
You don't explicitly say where/how you are running that image "remotely". Could it be that you are re-using tags and those images are cached on the "remote" nodes you are deploying to (e.g. EKS cluster or ECS/EC2 cluster)? In that case, depending on the host configuration/state it may not even reach to ECR to pull the new image if the node (thinks to have) found the same image cached.
[Update] Following up from the comments, the problem occurs in a Lambda function and this blog has hints about how to update the code in the Lambda.
At first sight, the issue is that there is a mismatch in the name (a.k.a. tag) of the two images:
docker build -t $(DOMAIN)/$(REPO_NAME):$(IMAGE_VERSION) -f docker/Dockerfile . --no-cache
docker push $REPO_ID.dkr.ecr.$REGION.amazonaws.com/$(DOMAIN)/$(REPO_NAME):$(IMAGE_VERSION)
so you should probably write:
docker build -t $REPO_ID.dkr.ecr.$REGION.amazonaws.com/$(DOMAIN)/$(REPO_NAME):$(IMAGE_VERSION) -f docker/Dockerfile . --no-cache
instead.
Docker docs give:
docker image ls [OPTIONS] [REPOSITORY[:TAG]]
but this assumes you'll specify a repo.
How do you list local images just by tag?
Nice question! I was playing with docker image ls and have found that you do not have to specify repository. The following actually work:
# displays images with tag: latest
docker image ls *:latest
# displays all images but not those with: <none>
docker image ls *:*
edit:
docker image ls *:latest has some inconsistencies. I noticed that some images are not displayed whereas docker image ls | grep latest shows them. I'll get back if I find out why...
Update by Santhosh V:
It looks like a special character needs to be specified in the filter like docker image ls *\*:* for considering a \ in the image name
You can't just with the docker CLI on it's own. The CLI only supports listing all images (including or excluding intermediate layers), images matching a repo, or images matching a repo:tag.
In a shell then you can pipe to grep as has been mentioned in comments, otherwise you will have to parse the output of whichever method you use to list images.
Use:
#docker images | grep [tag]
You will use the Docker API, see the doc
https://docs.docker.com/registry/spec/api/#listing-image-tags
see also
https://gist.github.com/kizbitz/175be06d0fbbb39bc9bfa6c0cb0d4721
If the Docker API is on
http://myregistry.com
on the port 2375, then you will do something like
curl http://myregistry.com:2375/images/json
you will find in /images/json the same info as
docker images
will show
The following works on Docker version 20.10.6, build 370c289
docker images --filter=reference='*:sometag'
The official documentation can help you with more such filters.
I'm pretty new to docker and I'm a bit puzzled by the difference between tagging (--tag) an image and assigning it a name (--name).
For example, I can see that if I build my custom image out of a Docker file, I can tag it with a name:
sudo docker build --tag=tomcat-admin .
sudo docker run -it tomcat-admin
Passing the name to docker inspect produces a result:
docker inspect tomcat-admin
However it doesn't contain the same attributes of a "named" image:
docker inspect --format '{{ .NetworkSettings.IPAddress }}' tomcat-admin
Template parsing error: template: :1:19: executing "" at <.NetworkSettings.IPA...>: map has no entry for key "NetworkSettings
"
Somebody to shed some light on it?
Thanks!
I think you mixed two concepts here, which causes the confusion. On the one hand there is a Docker image which you can think of as a blueprint for starting a container. On the other hand there are containers which are running instances that are based on an image.
When you docker build -t tagname . you are creating an image and tag it with a "name:tag" format usually. So for example, you are building your image as
docker build -t myimage:1.0 .
which creates a new image that is named myimage with a version of 1.0. This is what you will see when you run docker images.
The --name parameter is then used when you create and start a new container based of your image. So for example, you run a new container using the following command:
docker run -it --name mycontainerinstance myimage
This creates a new container based of your image myimage. This container instance is named mycontainerinstance. You can see this when you run docker ps -a which will list the container with its container name mycontainerinstance.
So to better understand the difference, have a look at the docs for building an image and running a container, specifying an image. When reading the docs you will notice which commands target an image and which commands are for containers. You will also see, that there are commands that work for images and containers like docker inspect does.
Inspecting for a network address of course only works when you provide a container name, not an image. In your special case, the container got a generated name, which you can see by running docker ps -a. When you provide this name to the docker inspect command, you will likely see the ip address you wanted.
You tag an image
docker build --tag=tomcat-admin .
but you assign a name to a container
docker run -it tomcat-admin
You can assign multiple tags to images, e.g.
docker build --tag=tomcat-admin --tag=tomcat-admin:1.0 .
If you list images you get one line per tag, but they are related to the same image id:
docker images |grep tomcat
tomcat-admin 1.0 955395353827 11 minutes ago 188 MB
tomcat-admin latest 955395353827 11 minutes ago 188 MB
You can tag images also a second time, not just when you build them, so you can keep different image versions.
When you run a container based on a specific image, you can assign it a name, so you can refer it using the name instead than using the containerId.
Obviously you get different attributes by inspecting images and containers. I think it's more clear if you use different name for image tag and container name, e.g.
docker build --tag=tomcat-admin .
docker run -d -ti --name=tomcat-admin-container tomcat-admin
docker inspect tomcat-admin ==> You inspect the image
docker inspect tomcat-admin-container ==> You inspect the container
The confusing thing is that a tag consists of a name and a tag. In documentation you can see that:
--tag , -t Name and optionally a tag in the ‘name:tag’ format
So if you omit the :tag part, you actually add a name for the image. That's it.
The difference between image names and container names is explained in other's answers.
I created a new image running:
docker build -t team1/es-image2 . | tee build.log
First, the create date doesn't reflect today's date. I wasn't concerned with that at first but after running it, it sort of makes sense...the running image is from another image created previously. I ran it with this command:
docker run -i -t --rm -P team1/es-image2
I verified that the correct image was running using:
docker ps
I deleted the older image and tried running again but it still appears to be running the older image because -P showed all the older mapped ports and the working directory was also from the older image.
So, I can't understand why, the build is using the older containers even though the Dockerfile is not specifying all the items that were specified in the older image.
Thanks!
docker ps
is only to show container.
To show images you need to use
docker images
And to delete them use
docker rmi
A little clarification about image and container.
An image is the definition of a container, and a container is a part of the system isolated from the current directory tree.
You use an image to run a container. You can use the same image to run multiple container.
When building the image from the Dockerfile, you may specify --no-cache=true to exclude any intermediate builds.