I have on Docker an automatic build for an image based on ubuntu with some custom configurations to re-use then as base image on other specific Dockerfiles for particular projects. This works okay.
I made a change to it, committed to github which then started and did the automatic build on Docker.
From one of these other projects, I'm calling at the beginning of the Dockerfile FROM myuser/myimage but its not getting the last image with the changes, but rather it keeps caching the old one.
Shouldn't this be automatically?
You need to docker pull the latest version. Docker looks for the image from FROM locally. It doesn't notice if that tag has been updated in the registry where it came from. I have a script that runs docker pull before building images.
Related
I've gone thru multiple questions posted on the forum but didn't get a clarity regarding my requirement.
I'm building a docker image after every successful CI build, there will be hardly 1 to 2 lines of changes in Dockerfile for every successful build.
Docker Build Command:
$(docker_registry)/$(Build.Repository.Name):azul
Docker Push Command
$(docker_registry)/$(Build.Repository.Name):azul
I wanted to overwrite the current docker image with the latest one(from the latest CI build changes) but retain the same tag - azul. Does docker support this ?
Yes, docker supports it. Every line you execute results in a new layer in image, that contains the changes compared to the previous layer. After modifying the Dockerfile, new layers will be created and the same preceding layers will be reused.
If you want to clean build the whole image with no cached layers, you can use the —no-cache parameter.
Mechanically this works. The new image will replace the old one with that name. The old image will still be physically present on the build system but if you look at the docker images output it will say <none> for its name; commands like docker system prune can clean these up.
The problems with this approach are on the consumer end. If I docker run registry.example.com/image:azul, Docker will automatically pull the image only if it's not already present. This can result in you running an older version of the image that happens to be on a consumer's system. This is especially a problem in cluster environments like Kubernetes, where you need a change in the text of the image name in a Kubernetes deployment specification to trigger an update.
In a CI system especially, I'd recommend assigning some sort of unique tag to every build. This could be based on the source control commit ID, or the branch name and bind number, or the current date, or something else. You can create a fixed tag like this as a convenience to developers (an image is allowed to have multiple tags) but I'd plan to not use this for actual deployments.
I have damaged my working docker image by pushing and overwriting by a faulty docker image on hub.docker.com. How can I revert the last push?
There is no revert option that I'm aware of. If you have a good copy of your image somewhere, you can repush that to the registry. To avoid this issue in the future, follow one or more of these steps:
Avoid using the latest tag and give each build a unique tag.
Use a reproducible build process with a Dockerfile that is saved in version control which uses specific versions for all dependencies. This allows you to checkout a previous state of the Dockerfile to rerun a previous build.
Maintain a private registry of your own for your images and any dependencies you have on other images. Make sure to maintain those dependencies (updating periodically) and backup your registry.
You can use the Advanced Image Management page in docker hub to copy the digest of the good image, pull it to your system, overwrite the tag, then push back. Use these commands:
docker image pull myname/example#sha256:1234
docker tag myname/example#sha256:1234 myname/example:mytag
docker push myname/example:mytag
Assuming myname/example#sha256:1234 is your copied digest of the good image from docker hub, myname/example:mytag is what you want to tag the image as.
I learnt the basics of github and docker and both work well in my environment. On my server, I have project directories, each with a docker-compose.yml to run the necessary containers. These project directories also have the actual source files for that particular app which are mapped to virtual locations inside the containers upon startup.
My question is now- how to create a pro workflow to encapsulate all of this? Should the whole directory (including the docker-compose files) live on github? Thus each time changes are made I push the code to my remote, SSH to the server, pull the latest files and rebuild the container. This rebuilding of course means pulling the required images from dockerhub each time.
Should the whole directory (including the docker-compose files) live on github?
It is best practice to keep all source code including dockerfiles, configuration ... versioned. Thus you should put all the source code, dockerfile, and dockercompose in a git reporitory. This is very common for projects on github that have a docker image.
Thus each time changes are made I push the code to my remote, SSH to the server, pull the latest files and rebuild the container
Ideally this process should be encapsulated in a CI workflow using a tool like Jenkins. You basically push the code to the git repository,
which triggers a jenkins job that compiles the code, builds the image and pushes the image to a docker registry.
This rebuilding of course means pulling the required images from dockerhub each time.
Docker is smart enough to cache the base images that have been previously pulled. Thus it will only pull the base images once on the first build.
Our production/staging docker image build pulls the latest code from our repository and then it installs all required dependencies, this process takes a while.
This is why for development we are using a volume to map the application code to a local folder.
Is there a way to commit the local changes in the mapped volume to the actual image data storage? So that we don't have to rebuild it all the time?
Being able to put your code and label that docker image is what people use it for. You shouldn't be deploying code at run-time in production. Instead you should be building images and tagging them based on version.
You want to know what is running on your production and you also want the ability to rollback to previous version.
Now coming back to your rebuilding the code part. There are multiple ways to improve the build times
Create base images
You can create base images and just put code on top of that base image. This base image will have your necessary softwares (Node etc). And in your Dockerfile you will just copy the code.
Instead of Git Checkout Use Tag URL
You can download a specific branch/tag as a zip file instead of the whole repository and unzip it. I have seen sometimes Git repos are 100MB and the code is only 4-5 MB. This can save you time
Use multistage build
Multistage builds are not going to save you build time but they will save you image size. This is only needed if you deploy lots of containers
I want to keep the images that I used in my docker hub account while maintaining reference to the pulled image. Something like when you fork a project in github.
Currently I have tried jwilder/nginx-proxy image. Now that I am satisfied with it, I committed the working container to username/nginx-proxy image and push it.
The problem with this approach is it is like a fresh image and it doesn't show the layer from jwilder/nginx-proxy. No documentation or even Dockerfile.
If you push the image, there is no reference to the original, that behavior is normal. You can put that reference or link using your "repo info".
The Dockerfile is only shown if you did an automated build linking your github or bitbucket account and the push is automatically done based on the Dockerfile of your project.