I'm building a docker image. I want the image to be able to pull from private repos. Per some advice I found on a blog, I have this line in the Dockerfile:
RUN git config --global url."https://<access_token_goes_here>:#github.com/".insteadOf "https://github.com/"
It works, but now I've got the access token embedded in the container and in the Dockerfile which doesn't seem secure.
What's the best practice for this?
Related
I have a Git repo with a simple Dockerfile. First row goes like this:
FROM python:3.7
My company has an internal registry with the base images. Because of this, the DevOps guys want me to change the Dockerfile to:
FROM registry.company.com:5000/python:3.7
I don't want this infrastructure detail baked in my code. URLs may change, I may want to build this image in another environment, etc. If possible, I would rather indicate the server in the pipeline, but the documentation regarding docker build has no parameter for this.
Is there a way to avoid editing the Dockerfile in this situation?
You would use a build arg for this:
ARG registry=docker.io/library
FROM ${registry}/python:3.7
Then for the build process:
docker build --build-arg registry=registry.company.com:5000 ...
Use docker.io for the registry name for the default Docker Hub, and library is the repository for official docker images, both of which you normally don't see when using the short format. Note that I usually include the library part in the local mirror so that official docker images and other repos that are mirrored can all use the same registry variable:
ARG registry=docker.io
FROM ${registry}/library/python:3.7
That means your local registry would need to have registry.company.com:5000/library/python:3.7.
To force users to specify the registry as part of the build, then don't provide a default value to the arg (or you could default the value of registry to something internal if that's preferred):
ARG registry
FROM ${registry}/python:3.7
You can work around the situation by manually pulling and re-tagging the image. docker build (and docker run) won't try to pull an image that already appears to be present locally, but that also means there's no verification that it actually matches what Docker Hub has. That means you can pull the image from your mirror, then docker tag it to look like a Docker Hub image:
docker pull registry.company.com:5000/python:3.7
docker tag registry.company.com:5000/python:3.7 python:3.7
My actual requirement is pull docker image from GitHub and build a docker image in ec2 instance and push that image to ecr. So, am just trying to clear my first step by asking help to pull image from git, very new to all this.
Let's walk through each step you're asking about in your requirements:
Pull from GitHub - You won't pull a docker image from here, however you may pull a Dockerfile from here, which would be used to build an image. The command to do this would be just like cloning any other repository: git clone <repository url>
Build the image on ec2 - First you will need to have docker installed on the ec2 instance. Assuming you're running Ubuntu on your ec2 instance, follow the good instructions on Docker's page (https://docs.docker.com/install/linux/docker-ce/ubuntu/) miror. Once docker is installed, navigate to the directory that has your Dockerfile in it (cloned from git) and type docker build . --tag mytag
Push the image to ecr - To do this, you need to have the amazon CLI installed on your box, and you need an ACCESS_KEY_ID and SECRET_ACCESS_KEY from AWS IAM. Once you have these, configure your connection by storing them as environment variables, or by typing aws configure and entering them. Once your credentials are configured, log into ECR by typing aws ecr get-login --no-include-email, and then copy/pasting the command it gives you. (you can also put ` around it to skip the copying step). This will allow you to push to ecr using docker push.
To clarify some of the points:
Github: It is a web-based hosting service for version control using git. So you can not pull docker image from Github.
To build a Docker image, you need Dockerfile. So you can fork the GitHub project which has this Dockerfile.
Then to build it on ec2, you can check out the project containing Dockerfile on ec2 server and build it using:
https://docs.docker.com/engine/reference/commandline/build/
and then you can push it to any registry using:
https://docs.docker.com/engine/reference/commandline/push/
This is my use-case. I have a node application with a lot of dependencies. One of the dependency is from another git repo. When I try to build the container it fails for obvious reasons since it does not have ssh keys to access the repository. What is the best way to pull the repository and build the docker container ?
Method #1: Put username/password in the repository URL:
git clone https://username:password#example.com/username/repository.git
Method #2: Copy the SSH key and related config file in Dockerfile:
# In Dockerfile
COPY sshkey /root/.ssh/sshkey
COPY sshconfig /root/.ssh/sshconfig
Method #3: Bind-mount the SSH key and related config file when running the container:
docker run -v sshkey:/root/.ssh/sshkey -v sshconfig:/root/.ssh/sshconfig ...
Be careful of any potential security risks.
Use a volume to "copy" the ssh keys to the place where node will look for them during the build process within the container.
I have a build time Argument with which I would like to build my Docker file; Since that argument is confidential in nature (github key of a private repo) I don't want that to end up in a docker image. This quotes is from official docker documentation regarding the build-time arguments.
Warning: It is not recommended to use build-time variables for passing
secrets like github keys, user credentials etc. Build-time variable
values are visible to any user of the image with the docker history
command.
Anybody know what is the recommended way to achieve the same?
With docker 18.09+, that will be: docker build --secret id=mysecret,src=/secret/file (using buildkit).
See PR 1288, announced in this tweet.
--secret is now guarded by API version 1.39.
Example:
printf "hello secret" > ./mysecret.txt
export DOCKER_BUILDKIT=1
docker build --no-cache --progress=plain --secret id=mysecret,src=$(pwd)/mysecret.txt -f - . <<EOF
# syntax = tonistiigi/dockerfile:secrets20180808
FROM busybox
RUN --mount=type=secret,id=mysecret cat /run/secrets/mysecret
RUN --mount=type=secret,id=mysecret,dst=/foobar cat /foobar
EOF
I'd rely on context of the Dockerfile for that. Basically, have something else (i.e. Jenkins, sub-repos) that's trusted with your Github key pull down all the necessary repos to relative locations that give your Dockerfile the context it needs. Nothing in the Docker build process itself should be managing secrets.
I can be more specific, if you specify more about your use-case. If it's just a single repo you need, you can just stick the Dockerfile in the root of that repo and rely on something else to provide credentials for cloning the repo down.
This is the point of docker secret
see for example
https://blog.docker.com/2017/02/docker-secrets-management/
I created an image for my application and uploaded it as private repository in registry.hub.docker.
Now I every time I try to pull it, I get the following error
FATA[0012] Repository not found
I have successfully authenticated myself with docker using docker login command
Command I ran
## docker login
docker login
Username (werain): werain
WARNING: login credentials saved in /Users/werain/.dockercfg.
Login Succeeded
## docker pull
docker pull werain/digitdem
Any Clue?
Use the full image name, including the tag, when pushing and pulling:
docker push werain/digitdem:latest
docker pull werain/digitdem:latest
Docker generally assumes you mean latest when you don't specify, but if you want to use your own tag or if you didn't push the same tag as you're trying to pull, then omitting the tag won't work.
Add .netrc file to the dockerfile will pass credentials inside the docker containers and helps to pull more than one private repositories to build dependencies
#vim .netrc
machine github.com
login < your github token >
Add those 2 lines and pass your github token
Then copy the .netrc file to the container by including this line
COPY .netrc /root/