I'm trying to write a Dockerfile that pulls a private repository from github. The problem is that I can't get Docker buildkit to use my SSH key properly. Even using the precise instructions and example code from their website does not work. Here is what I did:
Created a passphraseless SSH key using ssh-keygen -t ed25519 -C my_email#my_company.com
Copied the public key and added it as a Github deploy key to my repository
ssh-added the key
Ran the Dockerfile
# syntax=docker/dockerfile:1
FROM alpine
# Install ssh client and git
RUN apk add --no-cache openssh-client git
# Download public key for github.com
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
# Clone private repository
RUN --mount=type=ssh git clone git#github.com:myorg/myproject.git myproject
replacing myproject and myorg appropriately. Copy-pasting the git clone command from the Dockerfile to the terminal works. Running DOCKER_BUILDKIT=1 docker build --ssh default and DOCKER_BUILDKIT=1 docker build --ssh default=/path/to/key both fail with the error
> [4/4] RUN --mount=type=ssh git clone git#github.com/myorg/myrepository.git myrepository
#9 0.262 fatal: repository 'git#github.com/myorg/myrepository.git' does not exist
What could be going on here? I'm using Docker 20.10.12 build e91ed57 on MacOS 10.14.6.
This is a workaround rather than a solution to the problem. Instead of the --ssh option, use secrets.
# syntax=docker/dockerfile:experimental
FROM alpine
RUN apk add openssh-client git
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
ENV GIT_SSH_COMMAND="ssh -i /run/secrets/deploy_key"
RUN --mount=type=secret,id=deploy_key git clone git#github.com:myorg/myrepository.git myrepository
building with the command
DOCKER_BUILDKIT=1 docker build --no-cache --secret id=deploy_key,src=/Users/Holmes5/.ssh/deploy_key .
Related
I want to create an image from WordPress and clone a private plugin after the build. This is my dockerfile:
FROM wordpress
RUN apt-get update
RUN apt-get install -y git
RUN mkdir /root/.ssh/
ADD id_rsa /root/.ssh/id_rsa
RUN touch /root/.ssh/known_hosts
RUN chmod -R 600 /root/.ssh
RUN ssh-keyscan -T 60 mygitdomain.com >> /root/.ssh/known_hosts
ENTRYPOINT git clone git#mygitdomain.com:myuser/my-plugin.git /var/www/html/wp-content/plugins/my-plugin
the problem is WordPress is not complete, and I can not clone in a directory that does not exist (I guess)
Anyway, I want to create a dockerfile to install WordPress and, after building, clone a repo in the plugins directory. Any help?
I've followed the instructions mentioned in the Docker documentation to use SSH to access private data in builds.
I've added my SSH private key into the SSH authentication agent using ssh-add ~/.ssh/id_rsa.
Then I just created the same Dockerfile provided in the documentation (except the git repo to clone which I replaced by another one):
# syntax=docker/dockerfile:1
FROM alpine
# Install ssh client and git
RUN apk add --no-cache openssh-client git
# Download public key for github.com
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
# Clone private repository
RUN --mount=type=ssh git clone git#github.com:myorg/myproject.git myproject
If I try to build the image with the following command:
docker build --ssh default .
I get this error:
------
> [stage-0 3/4] RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts:
------
executor failed running [/bin/sh -c mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts]: exit code: 1
The instruction causing the build failure is actually ssh-keyscan github.com >> ~/.ssh/known_hosts which returns exit code 1.
I've tried it on 3 local machines: Windows, Ubuntu and Mac, and I always get the same error.
On all of these machines I'm able to use my SSH key to clone git repositories, but also on all of them the instruction ssh-keyscan github.com returns nothing.
Try to debug the issue like here: https://adamo.wordpress.com/2021/12/17/run-mounttypessh-is-not-always-easy/
RUN --mount=type=ssh env
RUN --mount=type=ssh ls -l ${SSH_AUTH_SOCK}
RUN --mount=type=ssh git clone git#github.com:repo/proj.git
I am trying to automate a docker build in Jenkins pipeline. In my dockerfile, I basically build a node application. In my npm install, I have some private git repositories which need os bindings and so have to be installed in the container. When I run this manually, I transfer my ssh keys (id_rsa) to dockerfile which is used for doing npm install. Now, my problem is when running this task in jenkins pipeline, I will be configuring a ssh-agent(Jenkins plugin). It will not be possible to extract private key from ssh-agent. How should I pass my ssh-agent to my dockerfile.
EDIT 1:
I got it partially working by this:
Docker Build Command:
DOCKER_BUILDKIT=1 docker build --no-cache -t $DOCKER_REGISTRY_URL/$IMAGE_NAME:v$BUILD_NUMBER --ssh default . &&
Then in Docker file:
This works fine:
RUN --mount=type=ssh GIT_SSH_COMMAND="ssh -vvvT -o StrictHostKeyChecking=no"
git clone git#github.com:****
Weird thing is this doesn't work:
RUN --mount=type=ssh GIT_SSH_COMMAND="ssh -vvvT -o StrictHostKeyChecking=no" npm install git+ssh//git#github.com:****
I feel this is something to do with StrictHostKeyChecking=no
I finally got it working by using ROOT user in Dockerfile and setting the npm cache to root.
The problem was that git was using the /root/.ssh folder while npm was using a different path - /home/.ssh as it's npm cache was set on /home/.ssh
For anyone still struggling, this is the config I used
Docker Build Command:
DOCKER_BUILDKIT=1 docker build --no-cache -t test --ssh default .
Dockerfile:
USER root
RUN apt-get update && \
apt-get install -y \
git \
openssh-server \
openssh-client
RUN mkdir -p -m 600 /root/.ssh && ssh-keyscan github.com >> /root/.ssh/known_hosts && echo "Host *\n StrictHostKeyChecking no" > /root/.ssh/config
RUN echo "Check ssh_config" && cat /root/.ssh/config
RUN rm -rf node_modules
RUN npm config set cache /root
RUN --mount=type=ssh GIT_SSH_COMMAND="ssh -vvvT" npm install
I'm trying to use Docker to build an image for me importing an npm package hosted in a private github repo: "mypackage": "git#github.com:myaccount/myrepo.git#v0.0.2"
This works fine locally since I have SSH access, but obviously my Docker container doesn't. I've followed the following guides to implement this using some ssh forwarding enabled in 18.09:
https://medium.com/#tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066
https://docs.docker.com/develop/develop-images/build_enhancements/#using-ssh-to-access-private-data-in-builds
Using the following docker file:
# syntax=docker/dockerfile:experimental
FROM alpine
# Install ssh client and git
RUN apk add --no-cache openssh-client git
# Download public key for github.com
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
# Clone private repository
RUN --mount=type=ssh npm install
Then, running docker build --ssh default . fails with the following error:
#13 1.309 npm ERR! Host key verification failed.
#13 1.309 npm ERR! fatal: Could not read from remote repository.
#13 1.309 npm ERR!
#13 1.309 npm ERR! Please make sure you have the correct access rights
#13 1.309 npm ERR! and the repository exists.
#13 1.310 npm ERR!
#13 1.310 npm ERR! exited with error code: 128
I'm following this documentation to the letter but am having no luck. Am I missing something? I'm on OSX, but this fails with the same error in my Travis environment as well. Help!
This has worked for me.
Dockerfile extraction:
# syntax=docker/dockerfile:experimental
...
RUN mkdir -p -m 0600 /root/.ssh
RUN ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts
COPY development/config /root/.ssh
RUN chmod 0600 /root/.ssh/config
RUN --mount=type=ssh git clone **MY_PVT_REPOSITORY**
This is the content of the development/config file you can see being copied at the third line
Host bitbucket.org
StrictHostKeyChecking no
IdentityFile **MY LOCAL PATH**/.ssh/id_rsa
The tricky thing is that you have to put the host file path to id_rsa, not the one on docker (like /home/fabio/.ssh/id_rsa and NOT /root/.ssh/id_rsa)
Then just launch
ssh-agent
export DOCKER_BUILDKIT=1
docker build --ssh default -f development/Dockerfile .
Thank you very much, Mr. Filippi!
FYI guys, you can check this article How to Set Up SSH Keys on Ubuntu 20.04
My implementation based on Fabio's comment looks like this:
mkdir -p -m 0600 ~/.ssh
echo "$KNOWNHOSTS" >> ~/.ssh/known_hosts
echo "$SSH_PRIVATE_KEY" >> ~/.ssh/id_rsa
echo "$CONFIG_SETTINGS" >> ~/.ssh/config
chmod 644 ~/.ssh/known_hosts
chmod 600 ~/.ssh/id_rsa
chmod 0600 ~/.ssh/config
In KNOWNHOSTS GitLab variable I have saved the output of "ssh-keyscan www.example.com"
In SSH_PRIVATE_KEY I am storing my SSH PRIVATE KEY for the specified user.
In CONFIG_SETTINGS I have the following:
Host www.example.com
StrictHostKeyChecking no
IdentityFile ~/.ssh/id_rsa
So I'm trying to create an image, which adds a SSH private key to /tmp, runs ssh-agent on it, does a git clone and then deletes the key again.
This is the idea I'm trying to accomplish
Dockerfile:
FROM node:4.2.4
MAINTAINER Me
CMD ["/bin/bash"]
ENV GIT_SSL_NO_VERIFY=1
ENV https_proxy="httpsproxy"
ENV http_proxy="httpproxy"
ENV no_proxy="exceptions"
ADD projectfolder/key /tmp/
RUN ssh-agent /tmp
WORKDIR /usr/src/app
RUN git clone git#gitlab.private.address:something/target.git
RUN rm /tmp/key
WORKDIR /usr/src/app/target
RUN npm install
EXPOSE 3001
Now the problem lies within the build-process. I use the following command to build:
docker build -t samprog/targetimage:4.2.4 -f projectfolder/dockerfile .
The layers up to "ADD projectfolder/key /tmp/" work just fine, though the "RUN ssh-agent /tmp" layer doesn't want to cooperate.
Error code:
Step 9 : RUN ssh-agent /tmp/temp
---> Running in d2ed7c8870ae
/tmp: Permission denied
The command '/bin/sh -c ssh-agent /tmp' returned a non-zero code: 1
Any ideas? Since I thought it was a permission issue, where the directory was already created by the parent image, I created a /tmp/temp and put the key in there. Doesn't work either, same error.
I'm using Docker version 1.10.3 on SLES12 SP1
I did it. What I did is, I got rid of ssh-agent. I simply copied the ~/.ssh- directory of my docker-host into the /root/.ssh of the image and it worked.
Do not use the ~ though, copy the ~/.ssh-directory inside the projectfolder first and then with the dockerfile inside the container.
Final dockerfile looked as follows:
FROM node:4.2.4
MAINTAINER me
CMD["/bin/bash"]
ENV GIT_SSL_NO_VERIFY=1
ENV https_proxy="httpsproxy"
ENV http_proxy="httpproxy"
ENV no_proxy="exceptions"
ADD projectfolder/.ssh /root/.ssh
WORKDIR /usr/src/app
RUN git clone git#gitlab.private.address:something/target.git
RUN rm -r /root/.ssh
WORKDIR /urs/src/app/target
RUN npm set registry http://local-npm-registry
RUN npm install
EXPOSE 3001
The dockerfile still has to be improved on efficiency and stuff, but it works! Eureka!
The image now has to be squashed and it should be safe to use, though we only use it in our local registry.
I have faced with the same problem with maven:3-alpine. It was solved when I properly installed openssh-client:
RUN apk --update add openssh-client
Then copied keys with known hosts to the image:
ADD id_rsa /root/.ssh/
ADD id_rsa.pub /root/.ssh/
ADD known_hosts /root/.ssh/
And ran git clone command inline (with ssh-agent and ssh-add):
RUN eval $(ssh-agent -s) \
&& ssh-add \
&& git clone ssh://git#private.address:port/project/project.git
Complete docker file:
FROM maven:3-alpine
RUN apk update
RUN apk add python
RUN apk add ansible
RUN apk add git
RUN apk --update add openssh-client
ADD id_rsa /root/.ssh/
ADD id_rsa.pub /root/.ssh/
ADD known_hosts /root/.ssh/
RUN eval $(ssh-agent -s) \
&& ssh-add \
&& git clone ssh://git#private.address:port/project/project.git
ADD hosts /etc/ansible/hosts
RUN ansible all -m ping --ask-pass
I had the same issue while executing any bash command when building my Dockerfile.
I solved by adding RUN chmod -R 777 ./ like suggested in the answer of this question. I think this is a workaround, I'm not sure why docker in ubuntu has permission issues when building a container.