AWS ECS service points to a Task Definition, which holds a name of docker image, including a tag.
So when I create a new version of my docker image I have 2 possibilities:
Update Task definition to the new version and then update Service to point to the new revision of Task Definition
Use some tag to point to the last version, let's say "current" tag will always point to the last version, Task Definition will contain "my-image:current" and then I need just restart ECS service
What is better and why?
Use a unique tag for every build, and update the task definition to point at the new tag (your first option).
The big problem you'll run into is that, in general, Docker-based systems will try to avoid pulling an image they already have. If you restart the service, but the node it restarts on sees it already has my-image:current, it will just re-run it without pulling the updated version. It seems like it can work – How does "latest" tag work in an ECS task definition and container instances pulling from ECR? includes a setup which appears to work – but it's a little hard to tell just from looking at things what exact version you have in use.
A second good reason to avoid a "current" or "latest" tag is to have the ability to roll back. If you use, for example, a timestamp-based tagging system, you deploy build 20200323, and it doesn't work, it's very easy to reset the task definition back to build 20200322 to get back to yesterday's code while your developers debug things. If it was "current" yesterday, and it's "current" today, it's a lot harder to figure out what "not-current" should be.
Related
A little while ago I set up a local repository for some of my custom containers, they all seem to work fine and when they're running if I issue a docker image ls I can see the repository image and the tag is latest
Now, I recently made a chance to one of those images, and when I pushed it to the repository I tagged it :v1.0 in an effort to have some sort of versioning.
The problem is that it seems like when I try to deploy a container based on that image name, when it defaults to latest that doesn't mean the v1.0 image, despite that one being the most recent one pushed.
Did I mess up by not tagging my original images with a version, or is there a different way to accomplish what I'm trying to do?
I'm using WebSphere Liberty inside. As WebSphere Liberty requires frequent xml editing, which is impossible with Dockerfile commands. I have to docker-commit the container from time to time, for others to make use of my images.
The command is like:
docker commit -m "updated sa1" -a "Song" $id company/wlp:v0.1
Colleges are doing similar things to the image, they continue to docker commit the container several times every day.
One day we're going to deploy the image on production.
Q1: Is the practice of frequent docker-committing advised?
Q2: Does it leave any potential problem behind?
Q3: Does it create an extra layer? I read the docker-commit document, which didn't mention if it creates another layer, I assume it means no.
I wouldn't use docker commit,
It seems like a really good idea but you can't reproduce the image at will like you can with a Dockerfile and you can't change the base image once this is done either, so makes it very hard to commit say for example a security patch to the underlying os base image.
If you go the full Dockerfile approach you can re-run docker build and you'll get the same image again. And you are able to change the base image.
So my rule of thumb is if you are creating a temporary tool and you don't care about reuse or reproducing the image at will then commit is convenient to use.
As I understand Docker every container image has two parts this is a group of read-only layers making up the bulk of the image and then a small layer which is writeable where any changes are committed.
When you run commit docker goes ahead and creates a new image this is the base image plus changes you made (the image created is a distinct image), it copies up the code to the thin writable layer. So a new read-only layer is not created it merely stores the deltas you make into the thin writeable layer.
Don't just take my word for it, take Redhats advice
For clarity that article in step 5 says:
5) Don’t create images from running containers – In other terms, don’t
use “docker commit” to create an image. This method to create an image
is not reproducible and should be completely avoided. Always use a
Dockerfile or any other S2I (source-to-image) approach that is totally
reproducible, and you can track changes to the Dockerfile if you store
it in a source control repository (git).
I don't know if this is intended behavior or bug in GCR.
Basically I tried do it like that:
Create image from local files using Docker on Windows (Linux based image).
Before creating image I delete all local images with the same name/tag.
Image is tagged like repostiory/project/name:v1
When testing locally image have correct versions of executables (docker run imageID).
Before pushing image to GCR I delete all images from GCR with the same tag/name.
When Trying to pull new image from GCR to example kubernetes it pull the first (ever) image uploaded under particular tag.
I want to reuse the same tag to not change config file with every test and I don't really need to store previous versions of images.
It sounds like you're hitting the problem described in kubernetes/kubernetes#42171.
tl;dr, the default pull policy of kubernetes is broken by design such that you cannot reuse tags (other than latest). I believe the guidance from the k8s community is to use "immutable tags", which is a bit of an oxymoron.
You have a few options:
Switch to using the latest tag, since kubernetes has hardcoded this in their default pull policy logic (I believe in an attempt to mitigate the problem you're having).
Never reuse a tag.
Switch to explicitly using the PullAlways ImagePullPolicy. If you do this, you will incur a small overhead, since your node will have to check with the registry that the tag has not changed.
Switch to deploying by image digest with the PullIfNotPresent ImagePullPolicy. A more detailed explanation is in the PR I linked, but this gets you the best of both worlds.
I have a Dockerfile something like follows:
FROM openjdk:8u151
# others here
I have 2 questions about the base image:
1. How to get the tags?
Usually, I get it from dockerhub, let's say openjdk:8u151, I can get it from dockerhub's openjdk repository.
If I could get all tags from any local docker command, then I no need to visit web to get the tags, really a little low efficiency?
2. Will the base image safe?
I mean if my base image always there?
Look at the above openjdk repo, it is an offical repo.
I found there is only 8u151 left for me to choose. But I think there should be a lots of jdk8 release during the process, so should also a lots of jdk8 images there, something like 8u101, 8u163 etc.
So can I guess the maintainer will delete some old images for openjdk?
Then if this happen, how my Dockerfile work? I should always change my base image if my upstream delete there image? Really terrible for me to maintain such kind of thing.
Even if the openjdk really just generate one release of jdk8. My puzzle still cannot be avoided, as dockerhub really afford the delete button for users.
What's the best practice, please suggest, thanks.
How to get the tags?
See "How to list all tags for a Docker image on a remote registry?".
The API is enough
For instance, visit:
https://registry.hub.docker.com/v2/repositories/library/java/tags/?page_size=100&page=2
Will the base image safe?
As long as you save your own built image in a registry (eithe rpublic one, or a self-hosted one), yes: you will be able to at least build new images based on the one you have done.
Or, even if the base image disappears, you still have its layers in your own image, and can re-tag it (provided the build cache is available).
See for instance "Is there a way to tag a previous layer in a docker image or revert a commit?".
See caveats in "can I run an intermediate layer of docker image?".
I am considering using Docker for a project, but I have one question I have not been able to find an answer to in the docs or anywhere else: is it possible to commit only a subset of the changes I make to an image?
Let's say I start a container, make a bunch of changes to the filesystem, get my thing working well and then want to commit them to a new base image. However, I only want a small subset of the changes that were actually made (maybe some of them were to log files, or other things that are unimportant). Does docker commit allow to specify that I only want changes, say, under some part of the filesystem to be committed?
You would just first remove the unnessecary files. To my knowledge, partial commits are not supported.
If you want to do this, using docker diff is really handy. You can easily see what the file system changes are that way. You can even run that against a running container.