Docker --ssh flag - Host key verification failed - docker

I'm trying to use Docker to build an image for me importing an npm package hosted in a private github repo: "mypackage": "git#github.com:myaccount/myrepo.git#v0.0.2"
This works fine locally since I have SSH access, but obviously my Docker container doesn't. I've followed the following guides to implement this using some ssh forwarding enabled in 18.09:
https://medium.com/#tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066
https://docs.docker.com/develop/develop-images/build_enhancements/#using-ssh-to-access-private-data-in-builds
Using the following docker file:
# syntax=docker/dockerfile:experimental
FROM alpine
# Install ssh client and git
RUN apk add --no-cache openssh-client git
# Download public key for github.com
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
# Clone private repository
RUN --mount=type=ssh npm install
Then, running docker build --ssh default . fails with the following error:
#13 1.309 npm ERR! Host key verification failed.
#13 1.309 npm ERR! fatal: Could not read from remote repository.
#13 1.309 npm ERR!
#13 1.309 npm ERR! Please make sure you have the correct access rights
#13 1.309 npm ERR! and the repository exists.
#13 1.310 npm ERR!
#13 1.310 npm ERR! exited with error code: 128
I'm following this documentation to the letter but am having no luck. Am I missing something? I'm on OSX, but this fails with the same error in my Travis environment as well. Help!

This has worked for me.
Dockerfile extraction:
# syntax=docker/dockerfile:experimental
...
RUN mkdir -p -m 0600 /root/.ssh
RUN ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts
COPY development/config /root/.ssh
RUN chmod 0600 /root/.ssh/config
RUN --mount=type=ssh git clone **MY_PVT_REPOSITORY**
This is the content of the development/config file you can see being copied at the third line
Host bitbucket.org
StrictHostKeyChecking no
IdentityFile **MY LOCAL PATH**/.ssh/id_rsa
The tricky thing is that you have to put the host file path to id_rsa, not the one on docker (like /home/fabio/.ssh/id_rsa and NOT /root/.ssh/id_rsa)
Then just launch
ssh-agent
export DOCKER_BUILDKIT=1
docker build --ssh default -f development/Dockerfile .

Thank you very much, Mr. Filippi!
FYI guys, you can check this article How to Set Up SSH Keys on Ubuntu 20.04
My implementation based on Fabio's comment looks like this:
mkdir -p -m 0600 ~/.ssh
echo "$KNOWNHOSTS" >> ~/.ssh/known_hosts
echo "$SSH_PRIVATE_KEY" >> ~/.ssh/id_rsa
echo "$CONFIG_SETTINGS" >> ~/.ssh/config
chmod 644 ~/.ssh/known_hosts
chmod 600 ~/.ssh/id_rsa
chmod 0600 ~/.ssh/config
In KNOWNHOSTS GitLab variable I have saved the output of "ssh-keyscan www.example.com"
In SSH_PRIVATE_KEY I am storing my SSH PRIVATE KEY for the specified user.
In CONFIG_SETTINGS I have the following:
Host www.example.com
StrictHostKeyChecking no
IdentityFile ~/.ssh/id_rsa

Related

Example of using SSH to access private data in builds provided in Docker documentation not working on my machines

I've followed the instructions mentioned in the Docker documentation to use SSH to access private data in builds.
I've added my SSH private key into the SSH authentication agent using ssh-add ~/.ssh/id_rsa.
Then I just created the same Dockerfile provided in the documentation (except the git repo to clone which I replaced by another one):
# syntax=docker/dockerfile:1
FROM alpine
# Install ssh client and git
RUN apk add --no-cache openssh-client git
# Download public key for github.com
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
# Clone private repository
RUN --mount=type=ssh git clone git#github.com:myorg/myproject.git myproject
If I try to build the image with the following command:
docker build --ssh default .
I get this error:
------
> [stage-0 3/4] RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts:
------
executor failed running [/bin/sh -c mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts]: exit code: 1
The instruction causing the build failure is actually ssh-keyscan github.com >> ~/.ssh/known_hosts which returns exit code 1.
I've tried it on 3 local machines: Windows, Ubuntu and Mac, and I always get the same error.
On all of these machines I'm able to use my SSH key to clone git repositories, but also on all of them the instruction ssh-keyscan github.com returns nothing.
Try to debug the issue like here: https://adamo.wordpress.com/2021/12/17/run-mounttypessh-is-not-always-easy/
RUN --mount=type=ssh env
RUN --mount=type=ssh ls -l ${SSH_AUTH_SOCK}
RUN --mount=type=ssh git clone git#github.com:repo/proj.git

Docker Buildkit SSH/Github woes

I'm trying to write a Dockerfile that pulls a private repository from github. The problem is that I can't get Docker buildkit to use my SSH key properly. Even using the precise instructions and example code from their website does not work. Here is what I did:
Created a passphraseless SSH key using ssh-keygen -t ed25519 -C my_email#my_company.com
Copied the public key and added it as a Github deploy key to my repository
ssh-added the key
Ran the Dockerfile
# syntax=docker/dockerfile:1
FROM alpine
# Install ssh client and git
RUN apk add --no-cache openssh-client git
# Download public key for github.com
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
# Clone private repository
RUN --mount=type=ssh git clone git#github.com:myorg/myproject.git myproject
replacing myproject and myorg appropriately. Copy-pasting the git clone command from the Dockerfile to the terminal works. Running DOCKER_BUILDKIT=1 docker build --ssh default and DOCKER_BUILDKIT=1 docker build --ssh default=/path/to/key both fail with the error
> [4/4] RUN --mount=type=ssh git clone git#github.com/myorg/myrepository.git myrepository
#9 0.262 fatal: repository 'git#github.com/myorg/myrepository.git' does not exist
What could be going on here? I'm using Docker 20.10.12 build e91ed57 on MacOS 10.14.6.
This is a workaround rather than a solution to the problem. Instead of the --ssh option, use secrets.
# syntax=docker/dockerfile:experimental
FROM alpine
RUN apk add openssh-client git
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
ENV GIT_SSH_COMMAND="ssh -i /run/secrets/deploy_key"
RUN --mount=type=secret,id=deploy_key git clone git#github.com:myorg/myrepository.git myrepository
building with the command
DOCKER_BUILDKIT=1 docker build --no-cache --secret id=deploy_key,src=/Users/Holmes5/.ssh/deploy_key .

Pass ssh-agent to dockerfile to install private repository modules

I am trying to automate a docker build in Jenkins pipeline. In my dockerfile, I basically build a node application. In my npm install, I have some private git repositories which need os bindings and so have to be installed in the container. When I run this manually, I transfer my ssh keys (id_rsa) to dockerfile which is used for doing npm install. Now, my problem is when running this task in jenkins pipeline, I will be configuring a ssh-agent(Jenkins plugin). It will not be possible to extract private key from ssh-agent. How should I pass my ssh-agent to my dockerfile.
EDIT 1:
I got it partially working by this:
Docker Build Command:
DOCKER_BUILDKIT=1 docker build --no-cache -t $DOCKER_REGISTRY_URL/$IMAGE_NAME:v$BUILD_NUMBER --ssh default . &&
Then in Docker file:
This works fine:
RUN --mount=type=ssh GIT_SSH_COMMAND="ssh -vvvT -o StrictHostKeyChecking=no"
git clone git#github.com:****
Weird thing is this doesn't work:
RUN --mount=type=ssh GIT_SSH_COMMAND="ssh -vvvT -o StrictHostKeyChecking=no" npm install git+ssh//git#github.com:****
I feel this is something to do with StrictHostKeyChecking=no
I finally got it working by using ROOT user in Dockerfile and setting the npm cache to root.
The problem was that git was using the /root/.ssh folder while npm was using a different path - /home/.ssh as it's npm cache was set on /home/.ssh
For anyone still struggling, this is the config I used
Docker Build Command:
DOCKER_BUILDKIT=1 docker build --no-cache -t test --ssh default .
Dockerfile:
USER root
RUN apt-get update && \
apt-get install -y \
git \
openssh-server \
openssh-client
RUN mkdir -p -m 600 /root/.ssh && ssh-keyscan github.com >> /root/.ssh/known_hosts && echo "Host *\n StrictHostKeyChecking no" > /root/.ssh/config
RUN echo "Check ssh_config" && cat /root/.ssh/config
RUN rm -rf node_modules
RUN npm config set cache /root
RUN --mount=type=ssh GIT_SSH_COMMAND="ssh -vvvT" npm install

Building Go apps with private modules in Docker

I'm trying to build a go project in a docker container that relies on private submodules.
I was hoping that --mount=type=ssh would pass my ssh credentials to the container and it'd work. Currently I can build locally with just make the GOPRIVATE variable set and the git config update.
Here is my relevant Dockerfile currently
# syntax = docker/dockerfile:experimental
FROM golang:1.14.3-alpine AS build
RUN apk add --no-cache git \
openssh-client \
ca-certificates
WORKDIR /src
ENV GIT_TERMINAL_PROMPT=1
ENV GOPRIVATE="gitlab.com/company_foo"
RUN git config --global url."ssh://git#gitlab.com".insteadOf "https://gitlab.com"
# Authorize SSH Host
# Skip Host verification for git
RUN mkdir -p /root/.ssh && \
chmod 0700 /root/.ssh && \
ssh-keyscan gitlab.com > /root/.ssh/known_hosts &&\
chmod 644 /root/.ssh/known_hosts && touch /root/.ssh/config \
&& echo "StrictHostKeyChecking no" > /root/.ssh/config
COPY go.mod go.sum .
RUN --mount=type=ssh mkdir -p /var/ssh && \
GIT_SSH_COMMAND="ssh -o \"ControlMaster auto\" -o \"ControlPersist 300\" -o \"ControlPath /var/ssh/%r#%h:%p\"" \
go mod download
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build go build -o api-server ./cmd/api-server
RUN --mount=type=cache,target=/root/.cache/go-build go build -o migrations ./cmd/migrations
I've also tried adding a CI_JOB_TOKEN with
RUN echo -e "machine gitlab.com\nlogin gitlab-ci-token\npassword ${CI_JOB_TOKEN}" > ~/.netrc
but this also didn't work. Perhaps I did it wrong.
All of this results in the failure:
revision v0.0.3: unknown revision v0.0.3
relating to one of our private repos.
Any advice would be appreciate.
I'm absolutely at a lost.
This workes for me.
FROM golang:1.14
ARG USERNAME=user1
ARG PASSWORD=secret
WORKDIR /app
ADD . .
ENV GOPRIVATE=private.git.local/*
RUN echo "machine private.git.local login $USERNAME password $PASSWORD" > ~/.netrc
RUN go build -o testGo main.go
CMD ["/app/testGo"]
pass your gitlab_token to docker file from gitlab_ci.yaml and do the following steps
RUN git config --global url."https://oauth2:$GITLAB_TOKEN#gitlab.com/".insteadOf "https://git#gitlab.com/"
add your repo as GO_PRIVATE
ENV GOPRIVATE=gitlab.com/*
copy .netrc file to docker root
COPY confidential/.netrc /root/.netrc
.netrc file will have the following structure
machine gitlab.com
login gitlab_user
password p#$$word

Dockerfile: Permission denied during build when running ssh-agent on /tmp

So I'm trying to create an image, which adds a SSH private key to /tmp, runs ssh-agent on it, does a git clone and then deletes the key again.
This is the idea I'm trying to accomplish
Dockerfile:
FROM node:4.2.4
MAINTAINER Me
CMD ["/bin/bash"]
ENV GIT_SSL_NO_VERIFY=1
ENV https_proxy="httpsproxy"
ENV http_proxy="httpproxy"
ENV no_proxy="exceptions"
ADD projectfolder/key /tmp/
RUN ssh-agent /tmp
WORKDIR /usr/src/app
RUN git clone git#gitlab.private.address:something/target.git
RUN rm /tmp/key
WORKDIR /usr/src/app/target
RUN npm install
EXPOSE 3001
Now the problem lies within the build-process. I use the following command to build:
docker build -t samprog/targetimage:4.2.4 -f projectfolder/dockerfile .
The layers up to "ADD projectfolder/key /tmp/" work just fine, though the "RUN ssh-agent /tmp" layer doesn't want to cooperate.
Error code:
Step 9 : RUN ssh-agent /tmp/temp
---> Running in d2ed7c8870ae
/tmp: Permission denied
The command '/bin/sh -c ssh-agent /tmp' returned a non-zero code: 1
Any ideas? Since I thought it was a permission issue, where the directory was already created by the parent image, I created a /tmp/temp and put the key in there. Doesn't work either, same error.
I'm using Docker version 1.10.3 on SLES12 SP1
I did it. What I did is, I got rid of ssh-agent. I simply copied the ~/.ssh- directory of my docker-host into the /root/.ssh of the image and it worked.
Do not use the ~ though, copy the ~/.ssh-directory inside the projectfolder first and then with the dockerfile inside the container.
Final dockerfile looked as follows:
FROM node:4.2.4
MAINTAINER me
CMD["/bin/bash"]
ENV GIT_SSL_NO_VERIFY=1
ENV https_proxy="httpsproxy"
ENV http_proxy="httpproxy"
ENV no_proxy="exceptions"
ADD projectfolder/.ssh /root/.ssh
WORKDIR /usr/src/app
RUN git clone git#gitlab.private.address:something/target.git
RUN rm -r /root/.ssh
WORKDIR /urs/src/app/target
RUN npm set registry http://local-npm-registry
RUN npm install
EXPOSE 3001
The dockerfile still has to be improved on efficiency and stuff, but it works! Eureka!
The image now has to be squashed and it should be safe to use, though we only use it in our local registry.
I have faced with the same problem with maven:3-alpine. It was solved when I properly installed openssh-client:
RUN apk --update add openssh-client
Then copied keys with known hosts to the image:
ADD id_rsa /root/.ssh/
ADD id_rsa.pub /root/.ssh/
ADD known_hosts /root/.ssh/
And ran git clone command inline (with ssh-agent and ssh-add):
RUN eval $(ssh-agent -s) \
&& ssh-add \
&& git clone ssh://git#private.address:port/project/project.git
Complete docker file:
FROM maven:3-alpine
RUN apk update
RUN apk add python
RUN apk add ansible
RUN apk add git
RUN apk --update add openssh-client
ADD id_rsa /root/.ssh/
ADD id_rsa.pub /root/.ssh/
ADD known_hosts /root/.ssh/
RUN eval $(ssh-agent -s) \
&& ssh-add \
&& git clone ssh://git#private.address:port/project/project.git
ADD hosts /etc/ansible/hosts
RUN ansible all -m ping --ask-pass
I had the same issue while executing any bash command when building my Dockerfile.
I solved by adding RUN chmod -R 777 ./ like suggested in the answer of this question. I think this is a workaround, I'm not sure why docker in ubuntu has permission issues when building a container.

Resources