I have a Java application in VSTS for which a build definition has been created to generate a number of build artifacts which include an ear file and a server configuration file. All of these build artifacts are zipped up in a final build definition task.
We now wish to create a Docker file which encapsulates the above build artifacts in another VSTS Docker build task. This will be done via a build definition commandline task and it is worth pointing out that our target docker registry is a corporate registry, not Azure.
The challenge I am now facing is how to generate the required docker image from the zipped artifact (or its contents if possible). Any ideas on how this could be achieved?
To generate a docker image from zip/tar file you can use docker load command:
docker load < test.tar.gz
Loaded image: test:latest
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
test latest 769b9341d937 7 weeks ago 2.489 MB
After that you can push the image to your private registry:
docker login <REGISTRY_HOST>:<REGISTRY_PORT>
docker tag <IMAGE_ID> <REGISTRY_HOST>:<REGISTRY_PORT>/<APPNAME>:<APPVERSION>
docker push <REGISTRY_HOST>:<REGISTRY_PORT>/<APPNAME>:<APPVERSION>
Example :
docker login repo.company.com:3456
docker tag 769b9341d937 repo.company.com:3456/test:0.1
docker push repo.company.com:3456/test:0.1
So at the end of your build pipeline add a Command Line task and run the above commands (change the values to your zip file location, username, password, registry url etc.).
Related
I am very new to docker, I have a docker file that has all instructions to build a docker image. The image has to be deployed in AWS Fargate cluster. Now I have a python script which does the below things:
1. clone a repo from GitHub
2. Locate the docker file, build the image and push the docker image to AWS ecr
3. Create a task definition and deploy into ECS fargate cluster
4. Additionally the script generate a YAML file based on certain logic that has certain parameters(not related to container config) which is required for the application
How can I make this file content available to the container before it starts running in AWS?
I was checking --entrypoint and --cmd, but here I can't change the Dockerfile
Is there any way to achieve that without changing the Dockerfile?
You can't add layers to a docker image without a Dockerfile. But you can build a new image atop the original image, COPYing the file and leaving the ENTRYPOINT CMD and other configuration as-is.
build the "intermediate" image from the upstream Dockerfile, giving it a tag you can refer to later:
docker build -t my-intermediate-image:my-version-tag .
Create your yaml file to myfile.yml
Create a new Dockerfile.final that starts FROM intermediate:
FROM my-intermediate-iamge:my-version-tag
COPY myfile.yml /expected/yaml/path
Build the final image with a tag for ECS:
docker build -f Dockerfile.final -t accountnumber.ecs.amazonaws.com:my-version .
Push the final image tag to ECS
Proceed to update the ECS task definition to point to the new image
Usually according to docs In order to build a docker Image I need to follow these steps:
Create a Dockerfile for my application.
Run docker build . Dockerfile where the . is the context of my application
The using docker run run my image into a container.
Commit my image into a container
Then using docker push push the image into a container.
Though sometimes just launching the image into a container seems like a waste of time because I can tag my images using the parameter -t into the docker build command. So there's no need to commit a container as an image.
So is neseserily to commit a running container as an image?
You don't need to run and commit. docker commit allows you to create a new image from changes made on existing container.
You do need to build and tag your image in a way that will enable you to push it.
docker build -t [registry (defaults to docker hub)]/[your repository]:[image tag] [docker file context folder]
for example:
docker build -t my-repository/some-image:image-tag .
And then:
docker push my-repository/some-image:image-tag
This will build an image from a docker file found in the current folder (where you run the docker build command). The repository in this case is my-repository, the image name is some-image and it's tag is image-tag.
Also please note that you'll have to perform docker login with your credentials to docker hub before you are able to actually push the image.
You can also tag an existing image without rebuilding it. This is useful if you want to push an existing image to a different registry or if you want to create a different image tag. for example:
docker tag my-repository/some-image:image-tag localhost:5000/my-repository/some-image:image-tag
This will add a new tag to the image from the previous example. Note the registry part added (localhost:5000). If you call docker push on that tag (docker push localhost:5000/my-repository/some-image:image-tag) the image will be pushed to a registry found on localhost:5000 (of course you need the registry up and running before trying to push).
There's no need to do so. In order to prove that you can just tag the image and push it into the registry here's an example:
I made the following Dockerfile:
FROM alpine
RUN echo "Hello" > /usr/share/hello.txt
ENTRYPOINT cat /usr/share/hello.txt
Nothing special just generates a txt file and shows its content.
Then I can build my image using tags:
docker build . -t ddesyllas/dummy:201201241200 -t ddesyllas/dummy:201201241200
And then just push them to the registry:
$ docker push ddesyllas/dummy
The push refers to repository [docker.io/ddesyllas/dummy]
1aa99de3dbec: Pushed
6bc83681f1ba: Mounted from library/alpine
201908241504: digest: sha256:93e8407b1d52620aeadd769486ef1402b9e310018cae0972760f8c1a03377c94 size: 735
1aa99de3dbec: Layer already exists
6bc83681f1ba: Layer already exists
latest: digest: sha256:93e8407b1d52620aeadd769486ef1402b9e310018cae0972760f8c1a03377c94 size: 735
And as you can see from the output you can just build the tags and push it directly, good for your ci/cd pipeline. Though, generally speaking, you may need to launch the application into a container in order to do acceptance and other type of tests (eg. end-to-end tests).
I have a web application which I want to dockerise and release using azure pipelines.
I'm already able to build and test but am struggling on releasing it.
Build Pipeline (on build server controlled with VSTS agent):
1) Build using .net core
2) Run unit tests using .net core
3) Build docker image
4) Push to registry
This then automatically triggers my Release Pipeline. The release pipeline has two stages which are create release and deploy release.
A non-docker scenario would be to create DLLs (create release) and deploy on web servers (deploy release). I'm not sure if/how this applies to docker apps.
Release Pipeline (a separate release agent running on each environment server):
Artifact (CI Build) => DEV => SIT => UAT => PROD
In each of the environments, I do:
1) stop and remove old image/container
2) pull 'latest' tag from repository
3) run new image in docker container.
Question:
How do I create a single, release specific, docker image which can then be pushed to each environment and how do I split the create release from deploy release to get this done?
Issues:
If I'm just pulling an image in each pull then I'm just getting the latest tag, I am not sure if I can tag the registry image with the release number?
The reason for tagging with a release number/name is so that I can re-deploy a previous release if necessary.
I want to prevent storing all the release images on my DEV/SIT/UAT/PROD servers. The release images need to live in the registry server so that I have a central location to pull them from.
Addressing Your Question
Your create-release job can look like this.
echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY
docker build -t $CI_REGISTRY_IMAGE -t $CI_REGISTRY_IMAGE:$VERSION .
docker push $CI_REGISTRY_IMAGE
docker push $CI_REGISTRY_IMAGE:$VERSION
Your deploy release job can look like this.
echo $DOCKER_HUB_PASSWORD | docker login -u $DOCKER_HUB_USER --password-stdin $REGISTRY_URL
docker pull $CI_REGISTRY_IMAGE
docker pull $CI_REGISTRY_IMAGE:$VERSION
docker tag $CI_REGISTRY_IMAGE $DOCKER_HUB_USER/crystal-skull
docker tag $CI_REGISTRY_IMAGE:$VERSION $DOCKER_HUB_USER/$IMAGE_NAME:$VERSION
docker push $DOCKER_HUB_USER/$IMAGE_NAME
docker push $DOCKER_HUB_USER/$IMAGE_NAME:$VERSION
Addressing Your Issues
Yes, you can tag the registry image with a particular release number. This is standard for Docker registries including Docker Hub.
I would like to build docker-image on Gitlab CI/CD with alpine. This docker has to download a website (only index.html) as a file with a date every 1 hour.
All dates/ files should be saved in the docker volume.
How to start with this? I am new in docker.
First you need to run a docker container using any image you want (alpine in your case).
Then set everything in it that you want to run (like download website)
Then create a docker image and host it on gitlab docker registry
Then you simply have to code .gitlab-ci.yaml file. After pushing that to your repository
Then you need to schedule your pipeline as mentioned here
https://docs.gitlab.com/ee/user/project/pipelines/schedules.html
I am trying to create a docker image using the below command .
docker build -t mytestapp .
My DockerFile looks like this
# Set the base image
FROM rhel7:latest
USER root
# Dockerfile author / maintainer
MAINTAINER Name <email.id#example.com>
# Update application repository list and install the Redis server.
RUN mkdir /usr/local/myapp/
ADD myapp-0.0.1-jar /usr/local/myapp/
RUN java -Dspring.profiles.active=qa -jar /usr/local/myapp/myapp-0.0.1.jar
# Expose default port
EXPOSE 8080
Questions:
1) Is it fine the way I am adding the JAR file. Will it be available inside /usr/local on the container after I prepared am image from the above build.
2) When I build the image using docker build command , is the build image is pushed to docker repository hub by default.
Since the WAR file contains credentials, I don't want to push the image to Docker Hub but we would like to push to our local Docker registry using Docker distribution and pushing with docker push.
Please clarify.
Answering your questions:
Docker recommends using the COPY instructions for adding single files into an image. It will be available inside the container at /usr/local/myapp/myapp-0.0.1-jar
When you build the image it will be available on your local docker-host. It won't leave the server unless you explicitly tell it so.
Another tip I want to give you is the recommended docker image naming convention, which is [Repository/Author]/[Imagename]:[Version].
So for your image it might be called zama/mytestapp:1.0
If you want to push it into your local registry, you'll have to name your image after the syntax [LocalRegistry:Port]/[Repository/Author]/[Imagename]:[Version].
So your image might now be called registry.example.com:5000/zama/mystestapp:1.0
If you have authentication on your registry, you need to docker login first and then simply push the image with docker push registry.example.com:5000/zama/mystestapp:1.0.