I am trying to write a Dockerfile to access a remote mySQL database using ssh tunneling.
Tried with the following Run command:
ssh -f -N username#hostname -L [local port]:[database host]:[remote port] StrictHostKeyChecking=no
and getting this error:
"Host key verification failed" ERROR
Assuming that the Docker container does not have access to any SSH data (i.e.: there is no ~/.ssh/known_hosts), you have two ways to handle this:
Use ssh-keyscan -t rsa server.example.com > ~/.ssh/my_known_hosts from within the container to add the remote host
Or copy the relevant line from an existing my_known_hosts or simply COPY a the whole file to the container.
Either of these approaches should do it.
Related
I have a large file on my laptop (localhost). I would like to copy this file to a docker container which is located on a remote server. I know how to do it in two steps, i.e. I first copy the file to my remote server and then I copy the file from remote server to the docker container. But, for obvious reasons, I want to avoid this.
A similar question which has a complicated answer is covered here: Copy file from remote docker container
However in this question, the direction is reversed, the file is copied from the remote container to localhost.
Additional request: is it possible that this upload can be done piece-wise or that in case of a network failure I can resume the upload from where it stopped, instead of having to upload the entire file again? I ask because the file is fairly large, ~13GB.
From https://docs.docker.com/engine/reference/commandline/cp/#corner-cases and https://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/ you would just do:
tar Ccf $(dirname SRC_PATH) - $(basename SRC_PATH) | ssh you#host docker exec -i CONTAINER tar Cxf DEST_PATH -
or
tar Ccf $(dirname SRC_PATH) - $(basename SRC_PATH) | ssh you#host docker cp - CONTAINER:DEST_PATH
Or untested, no idea if this works:
DOCKER_HOST=ssh://you#host docker cp SRC_PATH CONTAINER:DEST_PATH
This will work if you are running a *nix server and a docker with ssh server in it.
You can create a local tunnel on the remote server by following these steps:
mkfifo host_to_docker
netcat -lkp your_public_port < host_to_docker | nc docker_ip_address 22 > host_to_docker &
First command will create a pipe that you can check with file host_to_docker.
Second one is the greatest network utility of all times that is netcat. It just accepts a tcp connection and forwards it to another netcat instance, receiving and forwarding underlying ssh messages to the ssh server running on docker and writing its responses to the pipe we created.
last step is:
scp -P your_public_port payload.tar.gz user#remote_host:/dest/folder
You can use the DOCKER_HOST environment variable and rsync to archive your goal.
First, you set DOCKER_HOST, which causes your docker client (i.e., the docker CLI util) to be connected to the remote server's docker daemon over SSH. This probably requires you to create an ssh-config entry for the destination server.
export DOCKER_HOST="ssh://<your-host-name>"
Next, you can use docker exec in conjunction with rsync to copy your data into the target container. This requires you to obtain the container ID via, e.g., docker ps. Note, that rsync must be installed in the container.
#
rsync -ar -e 'docker exec -i' <local-source-path> <container-id>:/<destintaion-in-the-container>
Since rsync is used, this will also allow you to resume (if the appropriated flags are used) uploads at some point later.
Well considering I have a docker (with postgres) running I could dump the data using pg_dump using:
sudo docker exec <DOCKERNAME> pg_dump --data-only --table=some_table some_db
I could further send this to a file by adding > export.sql
sudo docker exec <DOCKERNAME> pg_dump --data-only --table=some_table some_db > export.sql
Finally this works fine in an (interactive) ssh session.
However when using ssh the file is stored on remote host, instead of in my local system, I wish to get the file locally instead of in remote. I know I can send a command directly to the ssh shell and then exporting is done to the local host:
ssh -p 226 USER#HOST 'command' > local.sql
IE:
ssh -p 226 USER#HOST 'echo test' > local.sql
However on try to combine both commands I get an error
ssh -p 226 USER#HOST 'sudo docker exec <DOCKERNAME> pg_dump --data-only --table=some_table some_db' > local.sql
sudo: no tty present and no askpass program specified
And if I dare remove sudo (which would be silly) I get: sh: docker: command not found. How do I solve this? How can I export the pg dump direct to my local pc? With a simple command? Or at least without first creating a copy of the file on the remote system?
I'd avoid sudo or docker exec for this setup.
First, make sure that your database container has a port published to the host. In Docker Compose, for example:
version: '3.8'
services:
db:
image: postgres
ports:
- '127.0.0.1:11111:5432'
The second port number must be the ordinary PostgreSQL port 5432; the first port number can be anything that doesn't conflict; the 127.0.0.1 setting makes the published port only accessible on the local system.
Then, when you connect to the remote system, you can use ssh to set up a port forward:
ssh -L 22222:localhost:11111 -N me#remote.example.com
ssh -L sets up a port forward from your local system to the remote system; ssh -N says to not run a command, just do the port forward.
Now on your local system, you can run psql and other similar client tools. Locally, localhost:22222 connects to the ssh tunnel; that forwards to localhost:11111 on the remote system; and that forwards to port 5432 in the container.
pg_dump --port=22222 --data-only --table=some_table some_db > export.sql
If you have the option of directly connecting to the database host, you could remove 127.0.0.1 from the ports: setting, and then pg_dump --host=remote.example.com --port=11111, without the ssh tunnel. (But I'm guessing it's there for a reason.)
You could forward socket connections over ssh then connect to the container from your host if you have docker installed:
ssh -n -N -T -L ${PWD}/docker.sock:/var/run/docker.sock user#host &
docker -H unix://${PWD}/docker.sock exec ...
I have a problem with connecting to my remote(DigitalOcean) docker engine. What I've done is
Made a droplet with Docker 19.03.12 on Ubuntu 20.04.
Made a new user myuser and add to docker group on the remote host.
Made a .ssh/authorized_keys for the new user it's home and set the permissions, owner etc.
Restarted both ssh and docker services.
Result
I can ssh from my Mac notebook to my remote host with myuser. (when I run ssh keychain asks for the passphrase for the id_rsa.key.)
After I logged in to remote host via ssh I can run docker ps, docker info without any problem.
Problem
Before I make a new context for the remote engine, I tried to run some docker command from my local client on my Mac laptop. Interesting part for me is none of the commands below asks for the id_rsa passphrase)
docker -H ssh://myuser#droplet_ip ps -> Error
DOCKER_HOST=ssh://myuser#droplet_ip docker ps -> Error
Error
docker -H ssh://myuser#droplet_ip ps
error during connect: Get http://docker/v1.40/containers/json: command [ssh -l myuser -- droplet_ip docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=myuser#droplet_ip: Permission denied (publickey).
What step I missed? How can I connect to a remote docker engine?
It sounds like Docker may not allow ssh to prompt for a key passphrase when connecting. The easiest solution is probably to load your key into an ssh-agent, so that Docker will be able to use the key without requesting a password.
If you want to add your default key (~/.ssh/id_rsa) you can just run:
ssh-add
You can add specific keys by providing a path to the key:
ssh-add ~/.ssh/id_rsa_special
Most modern desktop environments run an ssh-agent process by default.
I am trying to dockerize an SSH service/daemon not unlikely as described here:
https://docs.docker.com/engine/examples/running_ssh_service/#build-an-eg_sshd-image
In the container, when I try to /usr/sbin/sshd I get
Could not load host key: /etc/ssh/ssh_host_rsa_key
Could not load host key: /etc/ssh/ssh_host_dsa_key
Could not load host key: /etc/ssh/ssh_host_ecdsa_key
Could not load host key: /etc/ssh/ssh_host_ed25519_key
I tried to ssh-keygen -A and manually:
user#3df98eeeb0d7:/home/dev/$ sudo ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key -N ''
But, after failing to load host keys again after /usr/sbin/sshd, I am still not successful:
user#3df98eeeb0d7:/home/dev/$ /etc/init.d/ssh status
* sshd is not running
Since the creation of new keys is not working, I do not have a clue how to solve my problem.
Sometimes you need super user rights
sudo /usr/sbin/sshd
Solves your problem. Although on most containers your user runs all commands in root context.
I have compose file locally. How to run bundle of containers on remote host like docker-compose up -d with DOCKER_HOST=<some ip>?
After the release of Docker 18.09.0 and the (as of now) upcoming docker-compose v1.23.1 release this will get a whole lot easier. This mentioned Docker release added support for the ssh protocol to the DOCKER_HOST environment variable and the -H argument to docker ... commands respectively. The next docker-compose release will incorporate this feature as well.
First of all, you'll need SSH access to the target machine (which you'll probably need with any approach).
Then, either:
# Re-direct to remote environment.
export DOCKER_HOST="ssh://my-user#remote-host"
# Run your docker-compose commands.
docker-compose pull
docker-compose down
docker-compose up
# All docker-compose commands here will be run on remote-host.
# Switch back to your local environment.
unset DOCKER_HOST
Or, if you prefer, all in one go for one command only:
docker-compose -H "ssh://my-user#remote-host" up
One great thing about this is that all your local environment variables that you might use in your docker-compose.yml file for configuration are available without having to transfer them over to remote-host in some way.
If you don't need to run docker container on your local machine, but still on the same remote machine, you can change this in your docker setting.
On the local machine:
You can control remote host with -H parameter
docker -H tcp://remote:2375 pull ubuntu
To use it with docker-compose, you should add this parameter in /etc/default/docker
On the remote machine
You should change listen from external adress and not only unix socket.
See Bind Docker to another host/port or a Unix socket for more details.
If you need to run container on multiple remote hoste, you should configure Docker Swarm
You can now use docker contexts for this:
docker context create dev ‐‐docker “host=ssh://user#remotemachine”
docker-compose ‐‐context dev up -d
More info here: https://www.docker.com/blog/how-to-deploy-on-remote-docker-hosts-with-docker-compose/
From the compose documentation
Compose CLI environment variables
DOCKER_HOST
Sets the URL of the docker daemon. As with the Docker client, defaults to unix:///var/run/docker.sock.
so that we can do
export DOCKER_HOST=tcp://192.168.1.2:2375
docker-compose up
Yet another possibility I discovered recently is controlling a remote Docker Unix socket via an SSH tunnel (credits to https://medium.com/#dperny/forwarding-the-docker-socket-over-ssh-e6567cfab160 where I learned about this approach).
Prerequisite
You are able to SSH into the target machine. Passwordless, key based access is preferred for security and convenience, you can learn how to set this up e.g. here: https://askubuntu.com/questions/46930/how-can-i-set-up-password-less-ssh-login
Besides, some sources mention forwarding Unix sockets via SSH tunnels is only available starting from OpenSSH v6.7 (run ssh -V to check), I did not try this out on older versions though.
SSH Tunnel
Now, create a new SSH tunnel between a local location and the Docker Unix socket on the remote machine:
ssh -nNT -L $(pwd)/docker.sock:/var/run/docker.sock user#someremote
Alternatively, it is also possible to bind to a local port instead of a file location. Make sure the port is open for connections and not already in use.
ssh -nNT -L localhost:2377:/var/run/docker.sock user#someremote
Re-direct Docker Client
Leave the terminal open and open a second one. In there, make your Docker client talk to the newly created tunnel-socket instead of your local Unix Docker socket.
If you bound to a file location:
export DOCKER_HOST=unix://$(pwd)/docker.sock
If you bound to a local port (example port as used above):
export DOCKER_HOST=localhost:2377
Now, run some Docker commands like docker ps or start a container, pull an image etc. Everything will happen on the remote machine as long as the SSH tunnel is active. In order to run local Docker commands again:
Close the tunnel by hitting Ctrl+C in the first terminal.
If you bound to a file location: Remove the temporary tunnel socket again. Otherwise you will not be able to open the same one again later: rm -f "$(pwd)"/docker.sock
Make your Docker client talk to your local Unix socket again (which is the default if unset): unset DOCKER_HOST
The great thing about this is that you save the hassle of copying docker-compose.yml files and other resources around or setting environment variables on a remote machine (which is difficult).
Non-interactive SSH Tunnel
If you want to use this in a scripting context where an interactive terminal is not possible, there is a way to open and close the SSH tunnel in the background using the SSH ControlMaster and ControlPath options:
# constants
TEMP_DIR="$(mktemp -d -t someprefix_XXXXXX)"
REMOTE_USER=some_user
REMOTE_HOST=some.host
control_socket="${TEMP_DIR}"/control.sock
local_temp_docker_socket="${TEMP_DIR}"/docker.sock
remote_docker_socket="/var/run/docker.sock"
# open the SSH tunnel in the background - this will not fork
# into the background before the tunnel is established and fail otherwise
ssh -f -n -M -N -T \
-o ExitOnForwardFailure=yes \
-S "${control_socket}" \
-L "${local_temp_docker_socket}":"${remote_docker_socket}" \
"${REMOTE_USER}"#"${REMOTE_HOST}"
# re-direct local Docker engine to the remote socket
export DOCKER_HOST="unix://${local_temp_docker_socket}"
# do some business on remote host
docker ps -a
# close the tunnel and clean up
ssh -S "${control_socket}" -O exit "${REMOTE_HOST}"
rm -f "${local_temp_docker_socket}" "${control_socket}"
unset DOCKER_HOST
# do business on localhost again
Given that you are able to log in on the remote machine, another approach to running docker-compose commands on that machine is to use SSH.
Copy your docker-compose.yml file over to the remote host via scp, run the docker-compose commands over SSH, finally clean up by removing the file again.
This could look as follows:
scp ./docker-compose.yml SomeUser#RemoteHost:/tmp/docker-compose.yml
ssh SomeUser#RemoteHost "docker-compose -f /tmp/docker-compose.yml up"
ssh SomeUser#RemoteHost "rm -f /tmp/docker-compose.yml"
You could even make it shorter and omit the sending and removing of the docker-compose.yml file by using the -f - option to docker-compose which will expect the docker-compose.yml file to be piped from stdin. Just pipe its content to the SSH command:
cat docker-compose.yml | ssh SomeUser#RemoteHost "docker-compose -f - up"
If you use environment variable substitution in your docker-compose.yml file, the above-mentioned command will not replace them with your local values on the remote host and your commands might fail due to the variables being unset. To overcome this, the envsubst utility can be used to replace the variables with your local values in memory before piping the content to the SSH command:
envsubst < docker-compose.yml | ssh SomeUser#RemoteHost "docker-compose up"