Before docker:
I used to build the Release bits and run a set of test on them. Those bits were tested from Unit Tests to Comp, Functional, E2E, etc..
With docker:
1) I have a CI pipeline to test the bits, but ...
2) With Docker file I build and push the bit on the image in the same time. So considering the non-deterministic build system this is risk.. Is any way that I can write the Dockerfile to address this concern, or what is your approach?
Dockerfile for .net core that I am using as sample:
COPY . .
COPY /service /src/service
RUN dotnet build "src/service/ConsoleApp1/ConsoleApp1.csproj" -c release -o /app
FROM build AS publish
RUN dotnet publish "src/service/ConsoleApp1/ConsoleApp1.csproj" -c release -o /app
WORKDIR /app
FROM runtime AS final
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "ConsoleApp1.dll"]
First of all, our build system should be deterministic - if it is not, you have a problem.
As for your Dockerfile: The file shown is really threeimages - the first one builds the code, the second one publishes it and the last one merely executes it.
A good pipeline usually looks like this:
Build AND Unittest - if the unittests fail, abort the build.
If unittests are green, publish the generated image to a docker registry of your choosing.
Use that image in a dev environment, e.g. a Kubernetes cluster, azure container instances etc. pp. Now run any E2E, IT tests etc you need. Only and only if all of your tests are coming back green promote the image to a production environment, e.g. move them from dev to prod. Your dev environment can look totally different depending on the solution - if you have a single service you want to deploy, run the e2e tests directly, but maybe you have a complex cluster you need to set up, so run the tests on such a test cluster.
Since the image cannot change due to the nature of docker you can safely promote the image from dev to production. But you need to make sure that you never overwrite an image in your registry, e.g. avoid the latest tag and use explicit tags or even sha256 tags.
The general practice I follow is:
First stage:
once feature branch merged to master, run tests in CICD(eg. Jenkins)
Once tests succeed, build the versioned artifact eg. (.JAR for java, .DLL for dotnet)
publish the versioned artifact to artifactory(eg Jfrog or nexus) if needed
create a git tag
Second stage:
use versioned artifact created above and create versioned container image copying only the artifact. If you don't have artifactory yet, you can simply copy the local artifact.
eg. (WARNING: is not tested)
FROM microsoft/dotnet:latest
RUN mkdir -p /usr/local/app
COPY your-service/artifact-location/artifact-name.dll /usr/local/app
ENTRYPOINT ["dotnet", "/usr/local/app/ConsoleApp1.dll"]
tag the versioned container image and push to container registry (eg. elastic container registry(ecr) from Amazon)
Third stage:
update kubernetes deployment manifest with new version of container image.
apply kubernetes deployment manifest
Here is example for java here if that helps - https://github.com/prayagupd/eccount-rest/blob/master/Dockerfile and CICD pipeline for Jenkins - https://github.com/prayagupd/eccount-rest/blob/master/Jenkinsfile
Related
I have a docker configuration that I want to run both locally and on the CI (Github).
The only difference in the Dockerfiles is the FROM directive:
local configuration uses Nexus (behind firewall)
CI configuration uses Github Container Registry (GHCR)
Rest of the configuration is exactly the same (base images are the same images, just pulled from different source).
Now the majority of Dockerfile content and files that are copied into image need to be duplicated in both env specific directories.
I'd like to have common Dockerfile configuration in case of any changes needed.
Current state example:
local/Dockerfile:
FROM nexus.example.com/myapp/app:latest
{several lines of code}
ci/Dockerfile:
FROM ghcr.io/mycompanyapp:latest
{the same several lines of code as above}
Desired:
common/Dockerfile:
{common code}
local/Dockerfile:
FROM nexus.example.com/myapp/app:latest
# INCLUDE ../common/Dockerfile
ci/Dockerfile:
FROM ghcr.io/mycompanyapp:latest
# INCLUDE ../common/Dockerfile
I am aware of existence of the edrevo/dockerfile-plus but I am looking for more official solution. It was tedious to link Dockerfile residing in different directory than build context.
Also it seems that it is not maintained actively and it may not work on Windows which is used by other team members (issue https://github.com/edrevo/dockerfile-plus/issues/27)
You can use ARGS do to this.
ARG REPO=nexus.example.com
FROM ${REPO}/app:latest
....
So the default config will be nexus.example.com and in your CI you just need to build with arg REPO=something
example:
docker build -t myapp:latest --build-args REPO=ghcr.io -f Dockerfile
For local build:
docker build -t myapp:latest -f Dockerfile
I have hosted a docker image on the gitlab repo.
I have some sensitive data in one of the image layers.
Now if someone pulls the image, can he sees the sensitive date on the intermediate layer.
Also can he know the Dockerfile commands I have used for the image.
I want the end user to only have the image and dont have any other info about its Dockerfile
But atleast i dont want him to see the intermediate files
If someone pulls the image, can he sees the sensitive date on the intermediate layer?
Yes.
Also can he know the Dockerfile commands I have used for the image.
Yes.
You should take these into account when designing your image build system. For example, these mean you should never put any sort of credentials into your Dockerfile, because anyone who has the image can easily retrieve them. You can mitigate this somewhat with #RavindraBagale's suggestion to use a multi-stage build but even so it's risky. Run commands like git clone that need real credentials from outside the Dockerfile.
The further corollary to this is, if you think your code is sensitive, Docker on its own is not going to protect it. Using a compiled language (Go, C++, Rust) will mean you can limit yourself to distributing the compiled binary, which is harder to reverse-engineer; JVM languages (Java, Scala, Kotlin) can only distribute the jar file, though IME the bytecode is relatively readable; for interpreted languages (Python, Ruby, Javascript) you must distribute human-readable code. This may influence your initial language choice.
You can use multi-stage builds,
manage secrets in an intermediate image layer that is later disposed off ,so that no sensitive data reaches the final image build.
such as in the following example:
FROM: ubuntu as intermediate
WORKDIR /app
COPY secret/key /tmp/
RUN scp -i /tmp/key build#acme/files .
FROM ubuntu
WORKDIR /app
COPY --from intermediate /app .
Another options to maintain secret are
docker secret : you can use docker secret if you are using docker swarm
secrets in docker-compose file (without swarm)
version: "3.6"
services:
my_service:
image: centos:7
entrypoint: "cat /run/secrets/my_secret"
secrets:
- my_secret
secrets:
my_secret:
file: ./super_duper_secret.txt
While I can successfully write a .gitlab-ci.yml on gitlab.com that passes my artifact to public/,
image: ruby:2.3
pages:
script:
- bundle install
- bundle exec jekyll build -d public
artifacts:
paths:
- public
I would like .gitlab-ci.yml (a container (#1) based on a docker image (#2)) to instead use a Dockerfile to build a different image (#3) to create the artifact and, again, pass the artifact from #3 to container#1/public/ so it is publicly accessible on the web.
# Dockerfile
FROM ruby:2.3
...
The rationale for this is that I might share the repo with a colleague who understands Dockerfiles but is not interested in using GitLab or .gitlab-ci.yml. So I want .gitlab-ci.yml to use Dockerfile for my purposes and my colleague can use Dockerfile in whatever way is most comfortable for them--Dockerfile is the shared, reproducible way of making the artifact.
It seems I can build Docker images in GitLab CI/CD, but I'm not sure I can do so on gitlab.com rather than self-hosted GitLab. This would then allow me to push image to a registry and use the image from the registry in a later stage (including the artifact). Although I'm not sure how I would extract it?
docker build -t myimagename . does not work in GitLab CI/CD on gitlab.com (unless I add in some services to the .gitlab-ci.yml?). Even if it did, how would I extract the artifact from the myimagename container into the GitLab CI/CD "container"?
I have a .NET Core 2.0 web api that is running on a Docker container behind a load balancer (also a docker container). I want to scale the web api by having multiple containers, one per customer. Having that in mind I had to make changes in the configuration to separate and abstract the customer details so I can have app settings per customer. That way I would have appsettings.CustomerA.config, appsettings.CustomerB.config etc.
My general Dockerfile is:
FROM microsoft/aspnetcore-build AS builder
WORKDIR /app
# copy csproj and restore as distinct layers
COPY ./Project/*.csproj ./
RUN dotnet restore
# copy everything else and build
COPY ./Project ./
RUN dotnet publish -c Release -o out
# build runtime image
FROM microsoft/aspnetcore
WORKDIR /app
COPY --from=builder /app/out ./
ENV ASPNETCORE_ENVIRONMENT Production
ENTRYPOINT ["dotnet", "Project.dll"]
That's all fine but what I don't know is whether I can have different Dockerfiles, one per customer where I specify the customer (not sure if that's a good practice or not because I would be mixing environments with customer but at the same time, the environment is for a given customer if that makes sense) or whether I should create a different config file that I copy across depending on the customer?
I took the docker template from the Microsoft docker documentation https://docs.docker.com/engine/examples/dotnetcore/#create-a-dockerfile-for-an-aspnet-core-application and added the environment myself.
Thanks for the help
It is not recommended to have a separate Dockerfile for each customer or even a separate image. This will quickly lead to big number of dockerfiles to manage and will introduce many complexities, especially when you want to upgrade all files/images when a new features demands so.
It is preferable to have one Docker image and externalize all customer specific configuration to the outside of the image.
There are multiple ways you can do that. In Docker, you can pass environment variables to the container to configure it. You can also wrap the environment vars in a config file and pass it as an env file.
Another "Docker way", to externalize stuff outside of the image is to use bind mounts. You can bind a directory from the host onto the container when running the container. In the directory, you can have config files ... that the container can pick up when starting up.
My use case is that I have multiple express micro-services that use the same middleware and I would like to create a different repo in the format of an npm module for each middleware.
Every repo is a private repo and can have a deploy key attached (can be different keys or the same)
All of this works OK locally. However when I try to use this with my docker-compose setup it fails on the npm install step, in the build stage.
Dockerfile
FROM node:alpine
RUN npm install --production
CMD npm start
docker-compose.yml
services:
node-api:
build:
context: .
dockerfile: Dockerfile
I understand this doesn't work because I don't have the deploy key I use on my local system in the Docker context.
I've looked around for a solution and none seem very easy/non hacky
Copy the key in and squash (CONS: not sure how I do this in a docker-compose file)http://blog.cloud66.com/pulling-git-into-a-docker-image-without-leaving-ssh-keys-behind/
Copy the key in on the build step and add to image. (CONS: Not very secure :( )
Use the key as a build argument. (CONS: see 2)
Dockerise something like https://www.vaultproject.io/ run that up first, add the key and use that within the node containers to get the latest key. (CONS: probably lots of work, maybe other issues?)
Use Docker secrets and docker stack deploy and store the key in docker secrets (CON: docker stack deploy has no support for docker volumes yet. See here https://docs.docker.com/compose/bundles/#producing-a-bundle unsupported key 'volumes')
My question is what is the most secure possible solution that is automated (minimal manual steps for users of the file)? Time of implementation is less of a concern. I'm trying to avoid checking in any sensitive data while making it easy for other people to run this locally.
Let's experiment with this new feature: Docker multi stage build
You can selectively copy artifacts from one stage to another, leaving behind everything you don’t want in the final image.
The idea is to build a temporary base image, then start the build again only taking what you want from the previous image. It uses multiple FROM in the same Dockerfile:
FROM node as base-node-modules
COPY your_secret_key /some/path
COPY package.json /somewhere
RUN npm install <Wich use your key>
FROM node #yes again!
...
...
COPY --from=base-node-modules /somewhere/node_modules /some/place/node_modules
...
... # the rest of your Dockerfile
...
Docker will discard everything what you don't save from the first FROM.