I am building a Docker image using a command line like the following:
docker build -t myimage .
Once this command has succeeded, then rerunning it is a no-op as the image specified by the Dockerfile has not changed. Is there a way to detect if the Dockerfile (or one of the build context files) subsequently changes without rerunning this command?
looking at docker inspect $image_name from one build to another, several information doesn't change if the docker image hasn't changed. One of them is the docker Id. So, I used the Id information to check if a docker has been changed as follows:
First, one can get the image Id as follows:
docker inspect --format {{.Id}} $docker_image_name
Then, to check if there is a change after a build, you can follow these steps:
Get the image id before the build
Build the image
Get the image id after the build
Compare the two ids, if they match there is no change, if they don't match, there was a change.
Code: Here is a working bash script implementing the above idea:
docker inspect --format {{.Id}} $docker_image_name > deploy/last_image_build_id.log
# I get the docker last image id from a file
last_docker_id=$(cat deploy/last_image_build_id.log)
docker build -t $docker_image_name .
docker_id_after_build=$(docker inspect --format {{.Id}} $docker_image_name)
if [ "$docker_id_after_build" != "$last_docker_id" ]; then
echo "image changed"
else
echo "image didn't change"
fi
There isn't a dry-run option if that's what you are looking for. You can use a different tag to avoid affecting existing images and look for ---> Using cache in the output (then delete the tag if you don't want it).
Related
I have a ci/cd automation in my project with gitlab, and after push my code on master branch, gitlab runner create a new docker image on my server and set latest commit hash as tag on that image, and recreate container with new image. after a while, there is a lot of unused image, and I want to delete them automaticly.
I delete old image manually.
This is my Makefile
NAME := farvisun/javabina
TAG := $$(git log -1 --pretty=%h)
IMG := ${NAME}:${TAG}
LATEST := ${NAME}:latest
app_build:
docker build -t ${IMG} -f javabina.dockerfile . && \
docker tag ${IMG} ${LATEST}
app_up:
docker-compose -p farvisun-javabina up -d javabina
And after all of this, I want a simple bash code, or other tools, to delete unused image, for example keep 3 lastest image, or keep last 2 day past builds, and delete others.
If you are fine with keeping a single image, you can use docker image prune -f, which will remove all images but the ones associated with a container, so if you run this command while the container is running, it will remove the rest of the images.
Don't forget to run docker system prune every often as well to further reduce storage usage.
In your situation where you need to keep more than one image, you can try this:
#!/bin/bash
for tag in $(docker image ls | sed 1,4d | awk '{print $3}')
do
docker image rm -f $tag
done
The first line will list all docker images, remove from the list the first 3 ones which you want to keep, and select only the column with the image ID. Then, for each of the IDs, we remove the image.
If you want to remove more, change the 4d for another number. Notice that the first line is a header, so it must always be removed.
If you want to filter the images by a tag first, you can make your own filters.
You can schedule (e.g. once a day or once a week) in the compilation machine a "docker image prune" command.
https://docs.docker.com/engine/reference/commandline/image_prune/
Here is a way we can remove old images, after a new successful build in our pipeline
# build
- docker build -t APP_NAME:$(git describe --tags --no-abbrev) .
# docker tag
- docker tag APP_NAME:$(git describe --tags --no-abbrev) artifactory.XYZ.com/APP_NAME:latest
# remote old images
- sha=$(docker image inspect artifactory.XYZ.com/APP_NAME:latest -f '{{.ID}}')
- image_sha=$(echo $sha | cut -d':' -f2)
- image_id=$(echo $image_sha | head -c 12)
- docker image ls | grep APP_NAME | while read name tag id others; do if ! [ $id = $image_id ]; then docker image rm --force $id; fi ; done
I don't want to push a docker build image to DockerHub. Is there any way to directly deploy a docker image from CircleCI to AWS/vps/vultr without having to push it to DockerHub?
I use docker save/load commands:
# save image to tar locally
docker save -o ./image.tar $IMAGEID
# copy to target host
scp ./image.tar user#host:~/
# load into target docker repo
ssh user#host "docker load -i ~/image.tar"
# tag the loaded target image
ssh user#host "docker tag $LOADED_IMAGE_ID myimage:latest"
PS: LOADED_IMAGE_ID can be retrieved in following way:
REMOTE_IMAGE_ID=`ssh user#host"docker load -i ~/image.tar" | grep -o "sha256:.*"`
Update:
You can gzip output to make it smaller. (Don't forget unzip the image archive before load)
docker save $IMAGEID | gzip > image.tar.gz
You could setup your own registry: https://docs.docker.com/registry/deploying/
Edit: As i.bondarenko said, docker save/load are the better commands for your needs.
Disclaimer: I am the author of Dogger.
I made a blog post about it here, which allows just that: https://medium.com/#mathiaslykkegaardlorenzen/hosting-a-docker-app-without-pushing-an-image-d4503de37b89
I have looked around online and tried the obvious route (explained below) to remove an environmental variable from a docker image.
1 - I create a container from a modified ubuntu image using:
docker run -it --name my_container my_image
2 - I inspect the image and see the two environmental variables that I want to remove using:
docker inspect my_container
which yields:
...
"Env": [
"env_variable_1=abcdef",
"env_variable_2=ghijkl",
"env_variable_3=mnopqr",
...
3 - I exec into the container and remove the environmental variables via:
docker exec -it my_container bash
unset env_variable_1
unset env_variable_2
4 - I check to make sure the specified variables are gone:
docker inspect my_container
which yields:
...
"Env": [
"env_variable_3=mnopqr",
...
5 - I then commit this modified container as an image via:
docker commit my_container my_new_image
6 - And check for the presence of the deleted environmental variables via:
docker run -it --name my_new_container my_new_image
docker inspect my_new_container
which yields (drumroll please):
...
"Env": [
"env_variable_1=abcdef",
"env_variable_2=ghijkl",
"env_variable_3=mnopqr",
...
AKA the deleted variables are not carried through from the modified container to the new image in the docker commit
What am I missing out on here? Is unset really deleting the variables? Should I use another method to remove these environmental variables or another/modified method to commit the container as an image?
PS: I've confirmed the variables first exist when inside the container via env. I then confirmed they were not active using the same method after using unset my_variable
Thanks for your help!
You need to edit the Dockerfile that built the original image. The Dockerfile ENV directive has a couple of different syntaxes to set variables but none to unset them. docker run -e and the Docker Compose environment: setting can't do this either. This is not an especially common use case.
Depending on what you need, it may be enough to set the variables to an empty value, though this is technically different.
FROM my_image
ENV env_variable_1=""
RUN test -z "$env_variable_1" && echo variable 1 is empty
RUN echo variable 1 is ${env_variable_1:-empty}
RUN echo variable 1 is ${env_variable_1-unset}
# on first build will print out "empty", "empty", and nothing
The big hammer is to use an entrypoint script to unset the variable. The script would look like:
#!/bin/sh
unset env_variable_1 env_variable_2
exec "$#"
It would be paired with a Dockerfile like:
FROM my_image
COPY entrypoint.sh /
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
CMD ["same", "as", "before"]
docker inspect would still show the variable as set (because it is in the container metadata) but something like ps e that shows the container process's actual environment will show it unset.
As a general rule you should always use the docker build system to create an image, and never use docker commit. ("A modified Ubuntu image" isn't actually a reproducible recipe for debugging things or asking for help, or for rebuilding it when a critical security patch appears in six months.) docker inspect isn't intrinsically harmful but has an awful lot of useless information; I rarely have reason to use it.
Maybe you can try with this way, as in this answer:
docker exec -it -e env_variable_1 my_container bash
And then commit the container as usual.
I personally was looking to remove all environment variables to have a fresh image but without losing the contents inside the image.
The problem was that when i reused this image and reset those environment variables with new values, they were not changed, the old values were still present.
My solution was to reinitialize the image with docker export and then docker import.
Export
First, spin up a container with the image, then export the container to a tarball
docker export {container_name} > my_image.tar
Import
Import the tarball to a new image
docker import my_image.tar my_image_tag:latest
Doing this will reset the image, meaning only the contents of the container will remain.
All layers, environment variables, entrypoint, and command data will be gone.
Does everyone know a docker gitlab image that update itself when new release comes out ?
I fix the version for now because I haven't try the automatique update using the :latest tagged image.
I have tested sameersbn/gitlab and gitlab/gitlab-ce image.
Does anyone has any recommendation for updating safely ?
You can do that.
Upgrading to latest Gitlab
By using docker the upgrade process of gitlab becomes very simple. We update the image in our docker-compose.yml to use gitlab/gitlab-ce:latest
So now whenever we pull the images, it would always pull the latest version.
upgrade_gitlab.sh
#!/bin/bash
# Pull the latest image
docker-compose pull | grep -q "Image is up to date"
IMAGE_UPDATED=$?
if [ $IMAGE_UPDATED -ne 0 ]; then
echo "New image found for Gitlab"
echo "Backing up old Gitlab"
docker-compose exec gitlab gitlab-rake gitlab:backup:create
# Update docker
docker-compose up -d
# Check logs for any issues
docker-compose logs -f
else
echo "No update found for Gitlab"
fi
Now you can schedule this script or execute it using jenkins. Whatever node you like
PS: Taken from my article http://tarunlalwani.com/post/migrating-gitlab-6-mysql-to-latest-gitlab/
I have to execute the same script to two docker images.
My Dockerfile are:
FROM centos:6
...
and
FROM centos:7
...
Is it possibile to have a single file and pass a parameter, something like:
FROM centos:MYPARAMS
and during the build somethings like that:
docker build --no-cache MYPARAMS=6 .
Thank you
Just to put this in right context, it is now (since May 2017) possible to achieve this with pure docker since 17.05 (https://github.com/moby/moby/pull/31352)
Dockerfile should look like (yes, commands in this order):
ARG APP_VERSION
ARG GIT_VERSION
FROM app:$APP_VERSION-$GIT_VERSION
Then build is invoked with
docker build --build-arg APP_VERSION=1 --build-arg GIT_VERSION=c351dae2 .
Docker will try to base the build on image app:1-c351dae2
Helped me immensely to reduce logic around building images.
From my knowledge, this is not possible with Docker.
The alternative solution is to use a Dockerfile "template", and then parse it using the template library of your choice. (Or even using sed command)
At https://github.com/BITPlan/docker-stackoverflowanswers/tree/master/33351864
you'll find a bash script "build" that works the way you want.
wf#mars:~/source/docker/docker-stackoverflowanswers/33351864>./build -v 6
Sending build context to Docker daemon 3.584 kB
Step 0 : FROM centos:6
6: Pulling from library/centos
fa5be2806d4c: Pull complete
ebdbe10e9b33: Downloading 4.854 MB/66.39 MB
...
wf#mars:~/source/docker/docker-stackoverflowanswers/33351864>./build -v 7
Sending build context to Docker daemon 3.584 kB
Step 0 : FROM centos:7
The essential part is the "here" document used:
#
# parameterized dockerfile
#
dockerfile() {
local l_version="$1"
cat << EOF > Dockerfile
FROM centos:$l_version
EOF
}