Background:
I am running a Google Compute Engine VM, called host.
There is a Docker container running on the machine called container.
I connect to the VM using an account called user#gmail.com.
I need to connect through ssh from the container to the host, without being prompted for the user password.
Problem:
Minutes after successfully connecting from the container to the host, the user/.ssh/authorized_keys gets "modified" by some process from Google itself. As far as I understood this process appends some ssh keys needed to connect to the VM. In my case though, the process seems to overwrite the key that I generated from the container.
Setup:
I connect to host using Google Compute Engine GUI, pressing on the SSH button.
Then I follow the steps described in this answer on AskUbuntu.
I set the password for user on host:
user#host:~$ sudo passwd user
I set PasswordAuthentication to yes in sshd_config, and I restart sshd:
user#host:~$ sudo nano /etc/ssh/sshd_config
user#host:~$ sudo systemctl restart sshd
I enter in the Docker container using bash, I generate the key, and I copy it on the host:
user#host:~$ docker exec -it container /bin/bash
(base) root#container-id:# ssh-keygen
(base) root#container-id:# ssh-copy-id user#host
The key is successfully copied to the host, the host is added to the known_hosts file, and I am able to connect from the container to the host without being prompted for the password (as I gave it during the ssh-copy-id execution).
Now, if I detach from the host, let some time pass, and attach again, I find that the user/.ssh/authorized_keys file contains some keys generated by Google, but there is no trace of my key (the one that allows the container to connect to the host).
What puzzles me more than everything is that we consistently used this process before and we never had such problem. Some accounts on this same host have still keys from containers that no longer exist!
Does anyone has any idea about this behavior? Do you know about any solutions that let me keep the key for as long as it is needed?
It looks like the accounts daemon is doing this task. You could refer this discussion thread for more details about this.
You might find the OS Login API a easier management option. Once enabled, you can use a single gcloud command or API call to add SSH keys.
In case anyone has trouble with this even AFTER adding SSH keys to the GCE metadata:
Make sure your username is in the SSH key description section!
For example, if your SSH key is
ssh-rsa AAAA...zzzz
and your login is ubuntu, make sure you actually enter
ssh-rsa AAAA...zzzz ubuntu
since it appears Google copies the key to the authorized_keys of the user specified inside the key.
In case anyone is still looking for solution for this, I solved this issue by storing the SSH Keys in Compute Engine Metadata https://cloud.google.com/compute/docs/instances/adding-removing-ssh-keys
Related
i am setting up docker context like described here and cofigured the ssh key and the context. Unfortunately I keep getting an error from docker while i'm in the new context:
docker context use myhostcontext
docker ps
error during connect: Get "http://docker.example.com/v1.24/containers/json": command [ssh -l user -- myhost docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=ssh_askpass: exec(/usr/bin/ssh-askpass): No such file or directory
Host key verification failed.
Suprisingly when i ssh into user#myhost the connection is established as it should be.
ssh -vv user#myhost shows that it uses the given key in ~/.ssh/config
Additional Info:
Platform: Ubuntu 20.04
Docker: 20.10.23
OpenSSH_8.2p1 Ubuntu-4ubuntu0.5, OpenSSL 1.1.1f 31 Mar 2020
Here is what i've done:
I've created a docker context with
docker context create myhostcontext --docker "host=ssh://user#myhost"
I also created a new ssh keypair with ssh-keygen (tried with rsa and ecdsa),
executed ssh-add /path/to/key and ssh-copy-id -i /path/to/key user#myhost
I tried with using "id_rsa" as keyname as well as "myhost" to make sure its not just a default naming problem.
Looking at several instructions (e.g. This question) unfortunately did not help. I also checked the authorized_keys on the remote host and the public key on my local machine, they match.
My ~/.ssh/config looks like this
Host myhost
HostName myhost
User user
StrictHostKeyChecking no
IdentityFile ~/.ssh/myhost
Also removing entries from known_host did not help.
Using the remote hosts IP instead of its name did not help either.
Installing ssh-askpass just shows me, that the authenticity could not be established (default message when using ssh on a host for the first time). Since I later want to use docker context in a CI/CD environment i don't want to have any non-cli stuff.
The only other possible "issue" that comes to my mind is that the user of the remote host is different that the one i am using on the client. But - if understood correctly - that should not be an issue and also i would not know how to manage that.
Any help or suggestion is highly appreciated, since I am struggling with this for days.
Thanks in advance :)
I have a problem with connecting to my remote(DigitalOcean) docker engine. What I've done is
Made a droplet with Docker 19.03.12 on Ubuntu 20.04.
Made a new user myuser and add to docker group on the remote host.
Made a .ssh/authorized_keys for the new user it's home and set the permissions, owner etc.
Restarted both ssh and docker services.
Result
I can ssh from my Mac notebook to my remote host with myuser. (when I run ssh keychain asks for the passphrase for the id_rsa.key.)
After I logged in to remote host via ssh I can run docker ps, docker info without any problem.
Problem
Before I make a new context for the remote engine, I tried to run some docker command from my local client on my Mac laptop. Interesting part for me is none of the commands below asks for the id_rsa passphrase)
docker -H ssh://myuser#droplet_ip ps -> Error
DOCKER_HOST=ssh://myuser#droplet_ip docker ps -> Error
Error
docker -H ssh://myuser#droplet_ip ps
error during connect: Get http://docker/v1.40/containers/json: command [ssh -l myuser -- droplet_ip docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=myuser#droplet_ip: Permission denied (publickey).
What step I missed? How can I connect to a remote docker engine?
It sounds like Docker may not allow ssh to prompt for a key passphrase when connecting. The easiest solution is probably to load your key into an ssh-agent, so that Docker will be able to use the key without requesting a password.
If you want to add your default key (~/.ssh/id_rsa) you can just run:
ssh-add
You can add specific keys by providing a path to the key:
ssh-add ~/.ssh/id_rsa_special
Most modern desktop environments run an ssh-agent process by default.
I want to have a server to transparently forward an incoming ssh connection from a client to a docker container. This should include scp, git transport and so forth. This must work with keys, passwords are deactivated. The user should not see the server. Update: Yes, this really means that the user shall be unaware that there is a server. The configuration must take place entirely on the server!
client -----> server -----> container (actual connection)
client -------------------> container (what the user should see)
So, what is given is this:
user#client$ ssh user#server
user#server$ ssh -p 42 user#localhost
user#container$
But what I want is this:
user#client$ ssh user#server
user#container$
I tried using the command="ssh -p 42 user#localhost" syntax in the authorized_keys files, which kinda works, only that in the second ssh connection the user has to enter their password as the authentication is not passed (the server doesn't has the private key of user).
Further this approach doesn't work with scp even if one enters a password.
I also heard about the tunnel= command, but I don't know how to set that up (and the manpage is less than helpful).
I am using OpenSSH 7.5p1 on Arch.
Put this in your ~/.ssh/config file:
Host server-container
ProxyCommand ssh server -W localhost:42
Then simply do:
ssh server-container
As long as your usernames are consistent. If not, you can specify them as this:
Host server-container
ProxyCommand ssh server-user#server -W localhost:42
Then simply do:
ssh container-user#server-container
Just as a bonus, you can avoid to use ssh to enter into the container using docker exec. Like this:
ssh -t server docker exec -it <container-id> bash
This is the solution I came up with now. I'm a bit unhappy with the second key, as it's public part will be visible in the container's ~/.ssh/authorized_keys which very slightly breaks transparency, but other than that all other things seem to work.
user#server$ cat .ssh/authorized_keys
command="ssh -q -p 42 user#localhost -- \"$SSH_ORIGINAL_COMMAND\"",no-X11-forwarding ssh-rsa <KEYSTRING_1>
user#server$ cat .ssh/id_rsa.pub
<KEYSTRING_2>
user#container$ cat .ssh/authorized_keys
ssh-rsa <KEYSTRING_2>
The client authorises against server with their private key. Then the server jumps to the container with a dedicated key that is only there for that particular auth. I'm a bit worried that you can break out of command= by injecting some commands, but so far I found no permutation that allows to break out.
Due to passing $SSH_ORIGINAL_COMMAND, you can even do scp and ssh-copy-id and so forth.
Note: To disallow ssh-copy-id, which I want for other reasons, simply make authorized_keys non-writeable for user inside the container.
I am a newbie to Mesos. I have installed a DCOS cluster locally in one system (Centos 7).
Everything went up properly and I am able to access the GUI of DCOS but when I am trying to connect through CLI, it is asking me for password.
I have not been prompted for any kind of password during local installation through vagrant.
But when I issue the following command:
[root#blade7 dcos-vagrant]# dcos node ssh --master-proxy --leader
Running `ssh -A -t core#192.168.65.90 ssh -A -t core#192.168.65.90 `
core#192.168.65.90's password:
Permission denied, please try again.
core#192.168.65.90's password:
I don’t know the password to be given.
Kindly help me in resolving this issue
Since the local installation bases on vagrant, you can use the following convenient workaround: directly log into the virtual machines by using vagrant's ssh.
open a terminal and enter vagrant global-status to see a list of all running vagrant environments (name/id)
switch into your dcos installation directory (e.g., cd ~/dcos-vagrant), which contains the file Vagrantfile
run vagrant ssh <name or (partial) id> in order to ssh into the virtual machine. For example, vagrant ssh m1 connects to the master/leader node, which gives you essentially the same shell as dcos node ssh --master-proxy --leader would do.
Two more tips:
within the virtual machine, the directory /vagrant is mounted to the current directory of the host machine, which is nice for transferring files into/from the VM
you may try to find out the correct ssh credentials of the default vagrant user and then add these (rather than the pem file retrieved from a cloud service provider) via ssh-add to your host machine. This should give you the ability to login via dcos node ssh --master-proxy --leader --user=vagrant without a password
The command shows that you are trying to login to the server using the userid "core". If you do not know the password of user "core", I suggest reset "core" user password and try it again.
I upgraded my Mac (OS X) from an older Docker installation to Docker Toolbox, meaning that I'm now working with Docker Machine, and in the process discovered that certs I had working for push/pull with a private registry are not there, and I can't for the life of me figure out how to get them in place. At the moment when I try a test pull I get the dreaded x509: certificate signed by unknown authority error. I've searched around, looked at issues in Github, but nothing has worked for me. I even tried ssh'ing into the machine VM and manually copying them into /etc/ssl/certs, and various other things, with no luck. And I certainly don't want to get into the "insecure-registry" stuff. This used to work with boot2docker prior to moving to docker-machine.
This seems like a very simple question: I have a couple of .crt files that I need put in the right place so that I can do a push/pull. How does one do this? And secondarily, how can this not be documented anywhere? Can we wish for a docker-machine add-cert command someday?
Thanks for any help, and I hope a good answer here can stick around to assist others who run into this.
Okay so let's imagine I have a registry running at the address: 192.168.188.190:5000 and I have a proper certificate for this address.
I would now run the following commands to install the root certificate into my machine:
docker-machine scp ./dockerCA.crt $MACHINE_NAME:dockerCA.crt
docker-machine ssh $MACHINE_NAME sudo mkdir -p /etc/docker/certs.d/192.168.188.190:5000
docker-machine ssh $MACHINE_NAME sudo mv dockerCA.crt /etc/docker/certs.d/192.168.188.190:5000/dockerCA.crt
Set the variable MACHINE_NAME to whatever the name of your machine is. The machine will now trust your root certificate.
Having the same issue I read the Documentation in Docker on how to add a certificate to my computer.
As you mentioned that you are on a updated Mac OS X, proceed by doing the following:
Copy the cert file from your docker registry to your hard drive, e.g.
scp user#docker.reg.ip:/path/to/crt/domain.crt /tmp/domain.crt
Add the certificate to your trusted certificates using the following command
sudo security add-trusted-cert -d -r trustRoot \
-k /Library/Keychains/System.keychain /tmp/domain.crt
Restart your local docker handler and now you should be able to upload your local Docker images to the Docker registry.
If you are running on any other operating systems please check this site on how to add trusted root certificates.