Copy files or clone via ssh in Dockerfile when building docker image - docker

Question

I have been trying for a while to copy files via ssh from a remote server (not gihub) inside the docker image I want to build, but I can't connect to host. Here is the Dockerfile up until the critical point:
FROM r-base:latest
### Install libs
RUN apt-get update && apt-get install -y \
sudo \
gdebi-core \
pandoc \
pandoc-citeproc \
openssh-server \
openssh-client \
libcurl4-gnutls-dev \
libcairo2-dev \
libxt-dev \
xtail \
wget \
libssl-dev \
libxml2 \
libxml2-dev \
libv8-dev \
curl \
gnupg \
git
COPY ./setup setup
RUN mv setup/.ssh ~/.ssh
RUN touch ~/.ssh/known_hosts
RUN chmod -R 400 ~/.ssh
RUN ssh-agent sh -c 'ssh-add /root/.ssh/id_rsa'
#RUN eval "$(ssh-agent -s)"
#RUN ssh-add -K ~/.ssh/id_rsa #This is commented out as it causes an error
RUN ssh-keyscan hostname > ~/.ssh/known_host
RUN ssh-keygen -R hostname
## THIS IS THE COMMAND WE NEED TO RUN...
RUN scp -r user#hostname:/path/to/folder ./
The owner of the folder is user. The id_rsa.pub was added to the authorized_keys file of the user user on the host, and ssh was restarted there. However I get a Failed authentication error. I tried to use my personal id_rsa which works from the command line, but it also fails inside docker. Is this a docker issuor is it solvable?

Answer 1

I finally managed to do it by generating a key with the command suggested in this post
So to reproduce my case, locally:
cd setup/.ssh/
ssh-keygen -q -t rsa -N '' -f id_rsa
Then on the server add the id_rsa.pub contents to the known hosts for the user user. Can copy the contents to clipboard using xclip: xclip -sel clip < setup/.ssh/id_rsa.pub)
Dockerfile:
I have been trying for a while to copy files via ssh from a remote server (not gihub) inside the docker image I want to build, but I can't connect to host. Here is the Dockerfile up until the critical point:
FROM r-base:latest
### Install libs
RUN apt-get update && apt-get install -y \
sudo \
gdebi-core \
pandoc \
pandoc-citeproc \
openssh-server \
openssh-client \
libcurl4-gnutls-dev \
libcairo2-dev \
libxt-dev \
xtail \
wget \
libssl-dev \
libxml2 \
libxml2-dev \
libv8-dev \
curl \
gnupg \
git
COPY ./setup setup
RUN chmod -R 600 ~/.ssh
RUN echo "IdentityFile /root/.ssh/id_rsa" >> /etc/ssh/ssh_config
RUN echo "StrictHostKeyChecking no" >> /etc/ssh/ssh_config
## THIS IS THE COMMAND WE NEED TO RUN...
RUN scp -r user#hostname:/path/to/folder ./

Answer 2

There’s no specific requirement that you must do everything inside your Dockerfile. Especially things that require remote ssh access are better done outside Docker: consider that anyone who gets your image later on can docker cp a valid ssh key out of it and potentially get access to your internal systems.
For Docker caching reasons, it’s also not a good idea to git clone or otherwise try to remotely retrieve your application from inside the Dockerfile. If you re-run docker build, and nothing else in your Dockerfile has changed, then Docker will skip over the scp step too, even if the remote content has changed.
My general recommendation would be to copy this content from outside the Dockerfile, then build it
# Using whatever credentials are in your local ssh-agent
scp -r user#hostname:/path/to/stuff dist/
# Then your Dockerfile doesn’t need scp or credentials
docker build .
Your Dockerfile then doesn’t need a bunch of extra packages that are only relevant to this path: you should be able to remove sudo openssh-server openssh-client xtail curl gnupg git without actually affecting the single main process you’re trying to run inside your container.