I copying over my ssh key in a dockerfile for dev purposes on a windows machine, but I cannot get the SSH_AUTH_SOCK variable set,
here is my Dockerfile
FROM node:alpine
WORKDIR /usr/src/app
COPY package.json .
ADD id_rsa /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa \
&& touch /root/.ssh/known_hosts \
&& ssh-keyscan -H github.com >> /root/.ssh/known_hosts \
&& eval `ssh-agent -s` && ssh-add /root/.ssh/id_rsa
RUN yarn
CMD [ "npm", "start" ]
I tried to eval the result of ssh-agent as well in my compose file
command: >
sh -c 'eval `ssh-agent -s`
&& ssh-add /root/.ssh/id_rsa
&& echo $SSH_AUTH_SOCK'
And when I executed my simple nodejs file, the SSH_AUTH_SOCK is not set.
printenv doesn't show it as well, is there a way to do this.
I can see there are lots of ssh-agent forwarder solution for linux and osx, but it does not work on windows.
Please note that it works when passing the full path to the actual ssh key, e.g.:
$ docker build --ssh default=c:/users/MY_WINDOWS_USER_NAME/.ssh/id_ed25519 .
or
$ docker build --ssh default=c:/users/MY_WINDOWS_USER_NAME/.ssh/id_rsa .
Related
I'm trying to build a go project in a docker container that relies on private submodules.
I was hoping that --mount=type=ssh would pass my ssh credentials to the container and it'd work. Currently I can build locally with just make the GOPRIVATE variable set and the git config update.
Here is my relevant Dockerfile currently
# syntax = docker/dockerfile:experimental
FROM golang:1.14.3-alpine AS build
RUN apk add --no-cache git \
openssh-client \
ca-certificates
WORKDIR /src
ENV GIT_TERMINAL_PROMPT=1
ENV GOPRIVATE="gitlab.com/company_foo"
RUN git config --global url."ssh://git#gitlab.com".insteadOf "https://gitlab.com"
# Authorize SSH Host
# Skip Host verification for git
RUN mkdir -p /root/.ssh && \
chmod 0700 /root/.ssh && \
ssh-keyscan gitlab.com > /root/.ssh/known_hosts &&\
chmod 644 /root/.ssh/known_hosts && touch /root/.ssh/config \
&& echo "StrictHostKeyChecking no" > /root/.ssh/config
COPY go.mod go.sum .
RUN --mount=type=ssh mkdir -p /var/ssh && \
GIT_SSH_COMMAND="ssh -o \"ControlMaster auto\" -o \"ControlPersist 300\" -o \"ControlPath /var/ssh/%r#%h:%p\"" \
go mod download
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build go build -o api-server ./cmd/api-server
RUN --mount=type=cache,target=/root/.cache/go-build go build -o migrations ./cmd/migrations
I've also tried adding a CI_JOB_TOKEN with
RUN echo -e "machine gitlab.com\nlogin gitlab-ci-token\npassword ${CI_JOB_TOKEN}" > ~/.netrc
but this also didn't work. Perhaps I did it wrong.
All of this results in the failure:
revision v0.0.3: unknown revision v0.0.3
relating to one of our private repos.
Any advice would be appreciate.
I'm absolutely at a lost.
This workes for me.
FROM golang:1.14
ARG USERNAME=user1
ARG PASSWORD=secret
WORKDIR /app
ADD . .
ENV GOPRIVATE=private.git.local/*
RUN echo "machine private.git.local login $USERNAME password $PASSWORD" > ~/.netrc
RUN go build -o testGo main.go
CMD ["/app/testGo"]
pass your gitlab_token to docker file from gitlab_ci.yaml and do the following steps
RUN git config --global url."https://oauth2:$GITLAB_TOKEN#gitlab.com/".insteadOf "https://git#gitlab.com/"
add your repo as GO_PRIVATE
ENV GOPRIVATE=gitlab.com/*
copy .netrc file to docker root
COPY confidential/.netrc /root/.netrc
.netrc file will have the following structure
machine gitlab.com
login gitlab_user
password p#$$word
I have dockerfile with run tor in docker:
FROM alpine:latest
RUN apk update && apk upgrade && \
apk add tor curl && \
rm /var/cache/apk/* && \
cp /etc/tor/torrc.sample /etc/tor/torrc && \
echo "SocksPort 0.0.0.0:9050" > /etc/tor/torrc
EXPOSE 9050
USER tor
CMD /usr/bin/tor -f /etc/tor/torrc
It works. I want to run several tors in one dockerfile and open different ports (9051,9052, etc). I can create docker-compose.yml in which for every port create one docker, but it isn't a good solution in my opinion.
May be anybody know how run several tors and publish theirs ports from docker?
For me help this dockerfile:
FROM alpine:latest
RUN apk update && apk upgrade && \
apk add tor curl bash && \
rm /var/cache/apk/* && \
cp /etc/tor/torrc.sample /etc/tor/torrc
EXPOSE 9050-9060
ADD start.sh /usr/local/bin/start.sh
RUN chmod +x /usr/local/bin/start.sh
RUN echo | sed -i 's/\r$//' /usr/local/bin/start.sh
CMD /usr/local/bin/start.sh
And script start.sh:
#!/bin/bash
#making script to stop on 1st error
set -e
# Original script from
# http://blog.databigbang.com/distributed-scraping-with-multiple-tor-circuits/
# if defined TOR_INSTANCE env variable sets the number of tor instances (default 10)
TOR_INSTANCES=${TOR_INSTANCES:=10 }
# if defined TOR_OPTIONSE env variable can be used to add options to TOR
TOR_OPTIONS=${TOR_OPTIONS:=''}
base_socks_port=9050
base_control_port=11000
dir_data="/tmp/multitor.$$"
# Create data directory if it doesn't exist
if [ ! -d $dir_data ]; then
mkdir $dir_data
fi
if [ ! $TOR_INSTANCES ] || [ $TOR_INSTANCES -lt 1 ]; then
echo "Please supply an instance count"
exit 1
fi
for i in $(seq $TOR_INSTANCES)
do
j=$((i+1))
socks_port=$((base_socks_port+i))
control_port=$((base_control_port+i))
if [ ! -d "$dir_data/tor$i" ]; then
echo "Creating directory $dir_data/tor$i"
mkdir "$dir_data/tor$i" && chmod -R 700 "$dir_data/tor$i"
fi
# Take into account that authentication for the control port is disabled. Must be used in secure and controlled environments
echo "Running: tor --RunAsDaemon 1 --CookieAuthentication 0 --HashedControlPassword \"\" --ControlPort 0.0.0.0:$control_port --PidFile tor$i.pid --SocksPort 0.0.0.0:$socks_port --DataDirectory $dir_data/tor$i -f /etc/tor/torrc"
tor --RunAsDaemon 1 --CookieAuthentication 0 --HashedControlPassword "" --PidFile $dir_data/tor$i/tor$i.pid --SocksPort 0.0.0.0:$socks_port --DataDirectory $dir_data/tor$i
done
# So that the container doesn't shut down, sleep this thread
sleep infinity
Build and start:
docker build -t torone ./
docker run -d -e "TOR_INSTANCES=10" -p 9050-9060:9050-9060 --rm --name torone torone
TOR_INSTANCES - contains how many tors processes want to start.
My Dockerfile:
FROM golang:1.11.4
RUN apt-get update && apt-get install git bash curl -yqq
ENV ENV test
ENV GIT_TERMINAL_PROMPT=1
ENV GITHUB_TOKEN XXXXXXXXXXXXXXXXXX
RUN curl -Ls https://github.com/Masterminds/glide/releases/download/v0.12.3/glide-v0.12.3-linux-amd64.tar.gz | tar xz -C /tmp \
&& mv /tmp/linux-amd64/glide /usr/bin/
RUN curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
RUN mkdir -p $GOPATH/src/github.com/<Myrepo>/
COPY . $GOPATH/src/github.com/<Myrepo>/
WORKDIR $GOPATH/src/github.com/<Myrepo>/
RUN dep ensure -vendor-only
When i am building this docker file it hangs at RUN dep ensure -vendor-only
It fails to pull the dependencies which are private repos
Is there any possiblities to store git credentials inside Docker or any way to build Docker with one or more private repos of GOlang
Use some thing like this
# ensure that the private Github repo is
# accessed using SSH instead of HTTPS
RUN ssh-keyscan github.com > /root/.ssh/known_hosts
RUN echo "$SSH_KEY" > /root/.ssh/id_rsa && chmod 0600 /root/.ssh/id_rsa
RUN echo '[url "ssh://git#github.com/*your_repo*/"]' >> /root/.gitconfig && echo 'insteadOf = https://github.com/*your_repo*/' >> /root/.gitconfig
Refer this to add ssh key to your git repo
Adding .netrc file will pass credentials inside the docker containers and helps to pull more than one private repositories to build dependencies
#vim .netrc
machine github.com
login < your github token >
add those 2 lines and pass your github token
FROM golang:1.11.4
RUN apt-get update && apt-get install git bash curl -yqq
ENV ENV test
ENV GIT_TERMINAL_PROMPT=1
ENV GITHUB_TOKEN XXXXXXXXXXXXXXXXXX
RUN curl -Ls https://github.com/Masterminds/glide/releases/download/v0.12.3/glide-v0.12.3-linux-amd64.tar.gz | tar xz -C /tmp \
&& mv /tmp/linux-amd64/glide /usr/bin/
RUN curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
RUN mkdir -p $GOPATH/src/github.com/<Myrepo>/
COPY . $GOPATH/src/github.com/<Myrepo>/
COPY .netrc /root/
WORKDIR $GOPATH/src/github.com/<Myrepo>/
RUN dep ensure -vendor-only
I'm trying to create a Jenkins Docker agent that has Go.
The following is my Dockerfile.
After I build it, if I try: docker run myimage:0.0.1 go version returns the Go version, however if I try this, it doesn't find Go at all.
docker run --privileged --dns 9.0.128.50 --dns 9.0.130.50 -d -P --name slave myimage:0.0.1
docker ps ## grab the port number
ssh -p PORT_NUMBER jenkins#localhost
What am I missing in order to make Go available under the Jenkins user?
FROM golang:1.11.5-alpine
RUN apk add --no-cache \
bash \
curl \
wget \
git \
openssh \
tar
COPY ssh/*key /etc/ssh/
COPY skel/ /home/jenkins
COPY id_rsa /home/jenkins/.ssh/id_rsa
COPY id_rsa.pub /home/jenkins/.ssh/id_rsa.pub
RUN addgroup docker \
&& adduser -s /bin/bash -h /home/jenkins -G docker -D jenkins \
&& echo "jenkins ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers \
&& echo "jenkins:jenkinspass" | chpasswd \
&& chmod u+s /bin/ping \
&& chown -R jenkins:docker /home/jenkins \
&& mv /etc/profile.d/color_prompt /etc/profile.d/color_prompt.sh \
&& mv /bin/sh /bin/sh.bak \
&& ln -s /bin/bash /bin/sh
# Standard SSH port
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
If you run:
docker run myimage:0.0.1 which go
You will see that go executable in path /usr/local/go/bin/go
If you connect as jenkins user via ssh and run /usr/local/go/bin/go version all work as well.
Conclusion:
Go installation provided as root user
jenkins user added after go installed and haven't /usr/local/go/bin/go in his $PATH environment variable.
Solution:
Add /usr/local/go/bin/go to $PATH for user jenkins
Use go executable with full path.
Secret
I added a secret to drone.io using:
drone org secret add --image=* --conceal --skip-verify=true octocat SSH_KEY #/home/me/.ssh/id_rsa
Dockerfile
Because npm install needs to access private repositories, I specify an ARG in my Dockerfile, to get my private ssh_key:
FROM node:latest
ARG SSH_KEY
ENV SSH_KEY=$SSH_KEY
RUN mkdir /root/.ssh && \
echo $SSH_KEY | cut -d "\"" -f 2 > /root/.ssh/id_rsa && \
chmod 0600 /root/.ssh/id_rsa && \
eval `ssh-agent -s` && \
ssh-add /root/.ssh/id_rsa && \
echo "StrictHostKeyChecking no" >> /etc/ssh/ssh_config
RUN mkdir /app
WORKDIR /app
COPY . /app
EXPOSE 3001
CMD ["npm", "start"]
.drone.yml
And finally, in my .drone.yml pipeline, on the plugin/docker step, I use build-arg to inject the ssk_key:
pipeline:
test:
image: node:latest
commands:
- mkdir /root/.ssh && echo "$SSH_KEY" > /root/.ssh/id_rsa && chmod 0600 /root/.ssh/id_rsa
- eval `ssh-agent -s` && ssh-add /root/.ssh/id_rsa
- echo "StrictHostKeyChecking no" >> /etc/ssh/ssh_config
- npm install
- npm test
docker:
image: plugins/docker
repo: octocat/bar
tags: latest
build_args:
- SSH_KEY=${SSH_KEY}
My questions:
Is that the correct way to inject my ssh key to the Dockerfile, from drone pipeline?
the build_args are printed in the frontend logs, so is the SSK_KEY...how to avoid this?
the build args is passing my SSK_KEY + quotes around it -> "SSH_KEY", so I have to remove the quotes in my Dockerfile (by piping the string) before echoing it to /root/.ssh/id_rsa:, any way to not have these "?
Many Thanks!!
[EDIT] thanks to Adrian for suggesting a better way, remove the npm install from Dockerfile, as the node_modules can be shared through a volume between the pipeline steps.