So I'm trying to create an image, which adds a SSH private key to /tmp, runs ssh-agent on it, does a git clone and then deletes the key again.
This is the idea I'm trying to accomplish
Dockerfile:
FROM node:4.2.4
MAINTAINER Me
CMD ["/bin/bash"]
ENV GIT_SSL_NO_VERIFY=1
ENV https_proxy="httpsproxy"
ENV http_proxy="httpproxy"
ENV no_proxy="exceptions"
ADD projectfolder/key /tmp/
RUN ssh-agent /tmp
WORKDIR /usr/src/app
RUN git clone git#gitlab.private.address:something/target.git
RUN rm /tmp/key
WORKDIR /usr/src/app/target
RUN npm install
EXPOSE 3001
Now the problem lies within the build-process. I use the following command to build:
docker build -t samprog/targetimage:4.2.4 -f projectfolder/dockerfile .
The layers up to "ADD projectfolder/key /tmp/" work just fine, though the "RUN ssh-agent /tmp" layer doesn't want to cooperate.
Error code:
Step 9 : RUN ssh-agent /tmp/temp
---> Running in d2ed7c8870ae
/tmp: Permission denied
The command '/bin/sh -c ssh-agent /tmp' returned a non-zero code: 1
Any ideas? Since I thought it was a permission issue, where the directory was already created by the parent image, I created a /tmp/temp and put the key in there. Doesn't work either, same error.
I'm using Docker version 1.10.3 on SLES12 SP1
I did it. What I did is, I got rid of ssh-agent. I simply copied the ~/.ssh- directory of my docker-host into the /root/.ssh of the image and it worked.
Do not use the ~ though, copy the ~/.ssh-directory inside the projectfolder first and then with the dockerfile inside the container.
Final dockerfile looked as follows:
FROM node:4.2.4
MAINTAINER me
CMD["/bin/bash"]
ENV GIT_SSL_NO_VERIFY=1
ENV https_proxy="httpsproxy"
ENV http_proxy="httpproxy"
ENV no_proxy="exceptions"
ADD projectfolder/.ssh /root/.ssh
WORKDIR /usr/src/app
RUN git clone git#gitlab.private.address:something/target.git
RUN rm -r /root/.ssh
WORKDIR /urs/src/app/target
RUN npm set registry http://local-npm-registry
RUN npm install
EXPOSE 3001
The dockerfile still has to be improved on efficiency and stuff, but it works! Eureka!
The image now has to be squashed and it should be safe to use, though we only use it in our local registry.
I have faced with the same problem with maven:3-alpine. It was solved when I properly installed openssh-client:
RUN apk --update add openssh-client
Then copied keys with known hosts to the image:
ADD id_rsa /root/.ssh/
ADD id_rsa.pub /root/.ssh/
ADD known_hosts /root/.ssh/
And ran git clone command inline (with ssh-agent and ssh-add):
RUN eval $(ssh-agent -s) \
&& ssh-add \
&& git clone ssh://git#private.address:port/project/project.git
Complete docker file:
FROM maven:3-alpine
RUN apk update
RUN apk add python
RUN apk add ansible
RUN apk add git
RUN apk --update add openssh-client
ADD id_rsa /root/.ssh/
ADD id_rsa.pub /root/.ssh/
ADD known_hosts /root/.ssh/
RUN eval $(ssh-agent -s) \
&& ssh-add \
&& git clone ssh://git#private.address:port/project/project.git
ADD hosts /etc/ansible/hosts
RUN ansible all -m ping --ask-pass
I had the same issue while executing any bash command when building my Dockerfile.
I solved by adding RUN chmod -R 777 ./ like suggested in the answer of this question. I think this is a workaround, I'm not sure why docker in ubuntu has permission issues when building a container.
Related
I am trying to automate a docker build in Jenkins pipeline. In my dockerfile, I basically build a node application. In my npm install, I have some private git repositories which need os bindings and so have to be installed in the container. When I run this manually, I transfer my ssh keys (id_rsa) to dockerfile which is used for doing npm install. Now, my problem is when running this task in jenkins pipeline, I will be configuring a ssh-agent(Jenkins plugin). It will not be possible to extract private key from ssh-agent. How should I pass my ssh-agent to my dockerfile.
EDIT 1:
I got it partially working by this:
Docker Build Command:
DOCKER_BUILDKIT=1 docker build --no-cache -t $DOCKER_REGISTRY_URL/$IMAGE_NAME:v$BUILD_NUMBER --ssh default . &&
Then in Docker file:
This works fine:
RUN --mount=type=ssh GIT_SSH_COMMAND="ssh -vvvT -o StrictHostKeyChecking=no"
git clone git#github.com:****
Weird thing is this doesn't work:
RUN --mount=type=ssh GIT_SSH_COMMAND="ssh -vvvT -o StrictHostKeyChecking=no" npm install git+ssh//git#github.com:****
I feel this is something to do with StrictHostKeyChecking=no
I finally got it working by using ROOT user in Dockerfile and setting the npm cache to root.
The problem was that git was using the /root/.ssh folder while npm was using a different path - /home/.ssh as it's npm cache was set on /home/.ssh
For anyone still struggling, this is the config I used
Docker Build Command:
DOCKER_BUILDKIT=1 docker build --no-cache -t test --ssh default .
Dockerfile:
USER root
RUN apt-get update && \
apt-get install -y \
git \
openssh-server \
openssh-client
RUN mkdir -p -m 600 /root/.ssh && ssh-keyscan github.com >> /root/.ssh/known_hosts && echo "Host *\n StrictHostKeyChecking no" > /root/.ssh/config
RUN echo "Check ssh_config" && cat /root/.ssh/config
RUN rm -rf node_modules
RUN npm config set cache /root
RUN --mount=type=ssh GIT_SSH_COMMAND="ssh -vvvT" npm install
In the Alpine linux package site https://pkgs.alpinelinux.org/packages
NSCA packages are yet to get added. Is there an alternative to setup NSCA in Alpine Linux for passive-check?
If there is no package for it, you can always build it yourself.
FROM alpine AS builder
ARG NSCA_VERSION=2.9.2
RUN apk update && apk add build-base build-base gcc wget git
RUN wget http://prdownloads.sourceforge.net/nagios/nsca-$NSCA_VERSION.tar.gz
RUN tar xzf nsca-$NSCA_VERSION.tar.gz
RUN cd nsca-$NSCA_VERSION&& ./configure && make all
RUN ls -lah nsca-$NSCA_VERSION/src
RUN mkdir -p /dist/bin && cp nsca-$NSCA_VERSION/src/nsca /dist/bin
RUN mkdir -p /dist/etc && cp nsca-$NSCA_VERSION/sample-config/nsca.cfg /dist/etc
FROM alpine
COPY --from=builder /dist/bin/nsca /bin/
COPY --from=builder /dist/etc/nsca.cfg /etc/
Since this is using multiple stages, your resulting image will not contain development files and will still be small.
I have the following Dockerfile:
FROM ubuntu:16.04
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y \
git \
make \
python-pip \
python2.7 \
python2.7-dev \
ssh \
&& apt-get autoremove \
&& apt-get clean
ARG password
ARG username
ENV password $password
ENV username $username
RUN pip install git+http://$username:$password#org.bitbucket.com/scm/do/repo.git
I use the following commands to build the image from this Dockerfile:
docker build -t myimage:v1 --build-arg password="somepassoword" --build-arg username="someuser" .
However, in the build log the username and password that I pass as --build-arg are visible.
Step 8/8 : RUN pip install git+http://$username:$password#org.bitbucket.com/scm/do/repo.git
---> Running in 650d9423b549
Collecting git+http://someuser:somepassword#org.bitbucket.com/scm/do/repo.git
How to hide them? Or is there a different way of passing the credentials in the Dockerfile?
Update
You know, I was focusing on the wrong part of your question. You shouldn't be using a username and password at all. You should be using access keys, which permit read-only access to private repositories.
Once you've created an ssh key and added the public component to your repository, you can then drop the private key into your image:
RUN mkdir -m 700 -p /root/.ssh
COPY my_access_key /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa
And now you can use that key when installing your Python project:
RUN pip install git+ssh://git#bitbucket.org/you/yourproject.repo
(Original answer follows)
You would generally not bake credentials into an image like this. In addition to the problem you've already discovered, it makes your image less useful because you would need to rebuild it every time your credentials changed, or if more than one person wanted to be able to use it.
Credentials are more generally provided at runtime via one of various mechanisms:
Environment variables: you can place your credentials in a file, e.g.:
USERNAME=myname
PASSWORD=secret
And then include that on the docker run command line:
docker run --env-file myenvfile.env ...
The USERNAME and PASSWORD environment variables will be available to processes in your container.
Bind mounts: you can place your credentials in a file, and then expose that file inside your container as a bind mount using the -v option to docker run:
docker run -v /path/to/myfile:/path/inside/container ...
This would expose the file as /path/inside/container inside your container.
Docker secrets: If you're running Docker in swarm mode, you can expose your credentials as docker secrets.
It's worse than that: they're in docker history in perpetuity.
I've done two things here in the past that work:
You can configure pip to use local packages, or to download dependencies ahead of time into "wheel" files. Outside of Docker you can download the package from the private repository, giving the credentials there, and then you can COPY in the resulting .whl file.
pip install wheel
pip wheel --wheel-dir ./wheels git+http://$username:$password#org.bitbucket.com/scm/do/repo.git
docker build .
COPY ./wheels/ ./wheels/
RUN pip install wheels/*.whl
The second is to use a multi-stage Dockerfile where the first stage does all of the installation, and the second doesn't need the credentials. This might look something like
FROM ubuntu:16.04 AS build
RUN apt-get update && ...
...
RUN pip install git+http://$username:$password#org.bitbucket.com/scm/do/repo.git
FROM ubuntu:16.04
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install \
python2.7
COPY --from=build /usr/lib/python2.7/site-packages/ /usr/lib/python2.7/site-packages/
COPY ...
CMD ["./app.py"]
It's worth double-checking in the second case that nothing has gotten leaked into your final image, because the ARG values are still available to the second stage.
For me, I created a bash file call set-up-cred.sh.
Inside set-up-cred.sh
echo $CRED > cred.txt;
Then, in Dockerfile,
RUN bash set-up-cred.sh;
...
RUN rm cred.txt;
This is for hiding echoing credential variables.
I'm working on a Dockerfile with a multi-stage build. The general idea is to build the binary for the backend, build the javascript bundle for the frontend, and then put these two things in a final container for the app.
Here's the docker file:
# go binary
FROM golang:alpine as build-go
RUN apk --no-cache add git bzr mercurial
ENV D=/go/src/github.com/tamuhack-org/quack
RUN go get -d -v golang.org/x/net/html
RUN go get -d -v github.com/gorilla/handlers
RUN go get -d -v github.com/gorilla/mux
COPY ./main.go $D/main.go
COPY ./frontend/dist $D/frontend/dist
RUN rm -rf $D/frontend/dist/index.html
RUN rm -rf $D/frontend/dist/index.js
RUN cd $D && go build -o main && cp main /tmp/
# ui
FROM node:alpine AS build-node
RUN mkdir -p /src/ui
COPY ./frontend/package.json /src/ui/
RUN cd /src/ui && yarn install
COPY ./frontend /src/ui
# Replace the dev instance of index.html with the prod version.
RUN rm -rf /src/ui/dist/index.html
RUN mv /src/ui/dist/index-prod.html /src/ui/dist/index.html
RUN cd /src/ui && yarn build
# final
FROM alpine
RUN apk --no-cache add ca-certificates
WORKDIR /app/server/
COPY --from=build-go /tmp/main /app/server/
COPY --from=build-node /src/ui/dist /app/server/frontend/dist
EXPOSE 8080
CMD ["./main"]
What I've noticed is that when I update the frontend source code and build the docker container, the new version of the container doesn't update with the new bundle. Are there any obvious errors in the Dockerfile that may be the reason for why I'm not seeing any file changes? If I run yarn build locally, the bundle is accurate, but the docker container seems to be caching an older version. Thoughts?
Trying to build a dockerfile with alpine image and install set of directories. Here is the script below. It works fine until mkdir but does not change to /opt to download the git code.
The git code gets downloaded only to /src. Not sure whey the cd /opt command does not work.
FROM alpine
ADD . /src
WORKDIR /src
RUN apk update
RUN apk add git
RUN mkdir /opt
RUN cd /opt && git clone --recursive https://github.com/Azure/azure-iot-sdk-python.git
RUN ls -al
RUN cd azure-iot-sdk-python && ls -al build_all/linux
The current working directory is reset for every RUN to the value of the last WORKDIR.
So, because of that, this line RUN cd azure-iot-sdk-python && ls -al build_all/linux is doing it from here /src, and your cloned repo is in opt.
So both of this will work:
RUN cd /opt/azure-iot-sdk-python && ls -al build_all/linux
and:
WORKDIR /opt
RUN cd azure-iot-sdk-python && ls -al build_all/linux
Robert is right, and you could avoid this problem by following the Dockerfile best practice, and have only one RUN, see https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/