I am trying to connect a mac to a raspberry running Jenkins, to start docker build slaves on the mac on demand. But I can't establish the connection in Jenkins' docker cloud configuration.
My setup is as follows:
raspberry pi at 192.168.2.111: Jenkins running in docker
mac mini at 192.168.2.220 using docker for mac: this should run docker containers as build slaves on demand via Jenkins docker plugin
The SSH connection from Jenkins to mac works. The mac agent is up and running.
Now I want to use the docker plugin for Jenkins and trying to configure the docker cloud.
As docker for mac does not run directly on the host machine (but on hyperkit), they say I should run socat to expose the Unix socket like this:
docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 127.0.0.1:2376:2375 alpine/socat TCP-LISTEN:2375,fork unix-connect:/var/run/docker.sock
The container is running:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
76a75c5249d7 alpine/socat "socat tcp-listen:23…" 19 seconds ago Up 17 seconds 127.0.0.1:2376->2375/tcp compassionate_feynman
Then on the Raspberry pi where the Jenkins container is running, I have also added
DOCKER_OPTS="-H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock"
at the end of the file /etc/default/docker as described here.
Then in Jenkins cloud config "docker", I set "Docker Host URI" to tcp://127.0.0.1:2376, and the Docker Hostname (The mac mini) to 192.168.2.220. Testing this setting failed.
All the tutorials I find seem to have Jenkins on the same host where the docker slaves will be run. I assume this is why the socat container is not enough in my case?
What is also interesting:
Running curl 127.0.0.1:2376/versionon the mac works (shows expected information), but running curl 192.168.2.220:2376/version from the raspberry does not work (Connection refused). Maybe this can help someone who knows more about networks...
Captain Obvious strikes back.
Just use the network IP instead of localhost when starting socat on the mac.
docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 192.168.2.220:2376:2375 alpine/socat TCP-LISTEN:2375,fork unix-connect:/var/run/docker.sock
Related
I have got a container with Jenkins. I am trying to run tests pipeline inside this container. Tests uses Testcontainers. I am struggling with error:
18:36:15.560 [testcontainers-ryuk] WARN org.testcontainers.utility.RyukResourceReaper - Cannot connect to Ryuk at 172.17.0.1:55592
java.net.ConnectException: Connection refused (Connection refused)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:609)
at org.testcontainers.utility.RyukResourceReaper.lambda$null$0(RyukResourceReaper.java:92)
at org.rnorth.ducttape.ratelimits.RateLimiter.doWhenReady(RateLimiter.java:27)
at org.testcontainers.utility.RyukResourceReaper.lambda$maybeStart$1(RyukResourceReaper.java:88)
at java.base/java.lang.Thread.run(Thread.java:829)
Jenkins container was run by command
docker run -v /var/run/docker.sock:/var/run/docker.sock -v jenkins_home:/var/jenkins_home -p 8080:8080 -p 5000:5000 -d --restart always <img>
I've tried to run Jenkins container with --network="host" but same effect.
When I am running those tests directly on my local machine by mvn clean verify, tests passed and this error didn't occur.
EDIT:
I've changed network to bridge (--network="bridge") for Jenkins container. In test I used explicite network bridge:
new GenericContainer("mongodb:1..").withNetworkMode("bridge")...
Ryuk container is created (I can see three containers on my host docker - jenkins, mongo, ryuk) and when I run docker network inspect bridge I can see three containerers are attatched to it. But when I am trying connect to Ryuk container using gateway (172.17.0.1) from Jenkins container using telenet it is NOT possible. But when I am using Ryuk IP (172.17.0.2) instead of gateway's IP I can connect. Then I've create other container on my host and tryied to connect to it form Jenkins container via gateway's IP - could do it.
SOLUTION
Just run Jenkins container with -v /var/run/docker.sock.raw:/var/run/docker.sock instead of -v /var/run/docker.sock:/var/run/docker.sock.
Thanks #Kevin Wittek!
As mentioned in one of the comments, this originates from an issue in Testcontainers for Java regarding an upstream change in Docker Desktop (see this issue).
The current workaround for Mac is to mount the raw socket when creating the container:
-v /var/run/docker.sock.raw:/var/run/docker.sock
I'm developing an Ubuntu based docker image on a windows 10 machine with Docker Desktop. The image has a server app listening for HTTP traffic on port 5000. When I run the image locally on Windows using:
docker run -d -p 5000:5000 dockerImageName /opt/serverApp
Everything works fine, I can go to http://localhost:5000 and see a web interface.
But then I push the image to a repo in Azure then pull it back down onto a Ubuntu Azure VM and execute the same command, the container appears to start correctly but executing:
curl http://localhost:5000
returns:
curl: (7) Failed to connect to localhost port 5000: Connection refused
Not that it should matter but I've also allowed incoming traffic on port 5000 in the network security group the VM is attached to.
If I execute:
sudo lsof -i:5000
Nothing is returned.
Is this an Azure related issue? Are there differences between how a Linux image is run on Windows and a VM on Azure (on Windows Docker Desktop uses a Linux VM to run containers)?
Le sigh it was this:
Docker container will automatically stop after “docker run -d”
Annoying how the behavior changes between Windows and Ubuntu when running containers.
I am trying to use the python in a docker container on a remote machine as the interpreter in Pycharm. Since that is a mouthful, here is a diagram:
There is a Jupyter Notebook running in the container, which I am able to connect to through my local browser (although this is just for testing the connection). The command I am using to launch the Docker container is
docker run --runtime=nvidia -it --rm --shm-size=2g -v /home/timo/storage:/storage -v /etc/passwd:/etc/passwd -v /etc/group:/etc/group --ulimit memlock=-1 -p 8888:8888 -p 7722:22 --ipc=host latest:latest
I can forward the port 8888 which the Jupyter notebook is running on with ssh -L 8888:0.0.0.0:8888 BBB.BBB.BBB.BBB and thus use it on the local machine. But I don't much like using Jupyter for developing and would like to use the Python interpreter in the Docker Container in Pycharm.
When I select "Add Python Interpreter" in Pycharm, I get the following options:
The documentation for Pycharm suggests using the "Add Python Interpreter/Docker" tool which looks like this:
However the documentation doesn't say how to set up the Docker container and the connections if the Docker is on a remote machine.
So my questions are: should I use a Unix or a TCP socket to connect to my remote docker? Or should I somehow forward all the relevant ports from the container and use the "SSH Interpreter" option? And if so, how do I set this all up? Am I setting up my Docker Container properly in the first place?
I think I have trawled through every forum and online resource, over the last two days, but have not come any closer to getting this to work. I have also tried to get this to work in Spyder, but to no avail either. So any advice is very appreciated!
Many thanks!
Thank you for depicting the dilemma so poignantly and clearly in your cartoon :-). My colleague and I were trying to do something similar and what ultimately worked beautifully was creating an SSH config directly to the Docker container jumping from the remote machine, and then setting it as a remote SSH interpreter so that pycharm doesn't even realize it's a Docker container. It also works well for vscode.
set up ssh service in docker container (subset of steps in https://dev.to/s1ntaxe770r/how-to-setup-ssh-within-a-docker-container-i5i, port22 stuff wasn't needed)
docker exec -it <container> bash: create admin interactive prompt for docker
apt-get install openssh-server
service ssh start
confirm with service ssh status -> * sshd is running
determine IP and test SSHing from remote machine into container (adapted from https://phoenixnap.com/kb/how-to-ssh-into-docker-container, steps 2 and 3)
from normal command prompt on remote machine (not within container): docker inspect -f "{{ .NetworkSettings.IPAddress }}" <container> to get container IP
test: ping -c 3 <container_ip>
ssh: ssh <container_ip>; should drop you into the container as your user; however, requires container to be configured properly (docker run cmd has -v /etc/passwd:/etc/passwd:ro \ etc.). It may ask for a password. note: if you do this for a different container later that is assigned the same IP, you will get a warning and may need to delete the previous key from known_hosts; just follow the instructions in the warning.
test SSH from local machine
if you don't have it set up already, set up passwordless ssh key-based authentication to the remote machine with the docker container
make SSH command that uses your remote machine as a jump server to the container: ssh -J <remote_machine> <container_ip>, as described in https://wiki.gentoo.org/wiki/SSH_jump_host; if successful you should drop into the container just as you did from the remote machine
save this setup in your ~/.ssh/config; follow the ProxyJump Example from https://wiki.gentoo.org/wiki/SSH_jump_host
test config with ssh <container_host_name_defined_in_ssh_config>; should also drop you into interactive container
configure pycharm (or vscode or any IDE that accepts remote SSH interpreter)
Preferences -> Project -> Python Interpreter -> Add -> SSH Interpreter -> New server configuration
host: <container_host_name_defined_in_ssh_config>
port: 22
username: <username_on_remote_server>
select interpreter, can navigate using the folder icon, which will walk you through paths within the docker, or you can enter the result of which python from the container
follow pycharm prompts
I have docker machine installed on windows OS and have pulled in Jenkins image from docker hub. I then run the below commands:
docker volume created myjenkins-data
docker run -p 8080:8080 -p 50000:50000 -v myjenkins-data:/var/jenkins_home jenkins
I received the admin key for Jenkins in the logs. I've confirmed that my container is still running status:
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2854c7d83879 jenkins "/bin/tini -- /usr..." About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp, 0.0.0.0:50000->50000/tcp quizzical_cray
Now, I need to log into 8080 port to view the Jenkins web app. But I do not know which host ip to use. I identified the docker host ip with the command "docker-machine ip" and got the ip address: 192.168.99.100. But using 192.168.99.100:8080 did not bring up Jenkins app. I also tried using docker inspect to get the container's ip, but port 8080 didn't work on those ips as well. Which ip address do i use to see the Jenkins app that is running in the container?
First, double-check if http://localhost:8080 is not enough.
Hyper-V (through vpnkit, if you are using Docker for Windows) should have done the port-forwarding for you.
If you are using the legacy docker toolbox (VirtualBox), then you need port-forwarding (issue 4115).
I want to be able to user docker-compose with a remote daemon. I created a local forward to the remote daemon socket like so:
#!/bin/sh
export SOCKET_DIR=$HOME/.remote-sockets
mkdir -p $SOCKET_DIR
echo "Creating SSH Dokcer Socket Tunnel"
socat "UNIX-LISTEN:/$SOCKET_DIR/docker.sock,reuseaddr,fork" \
EXEC:'ssh freebsd#107.170.216.79 socat STDIO UNIX-CONNECT\:/var/run/docker.sock'
With that script running, I export the following environment variables:
DOCKER_API_VERSION 1.19
COMPOSE_API_VERSION 1.19
DOCKER_HOST unix://$HOME/.remote-sockets/docker.sock
With those variables set, I can verify running docker ps shows me the remote containers and not my local daemon's containers. Furthermore, docker-compose ps also seems to connect to the remote daemon, as it returns an empty list. If I shut down the SSH tunnel, it fails saying it can't connect to the docker daemon.
The trouble is when I try to run docker-compose up. I've also tried docker-compose -H unix://$HOME/.remote-sockets/docker.sock up. Both commands give me the following:
ERROR: Couldn't connect to Docker daemon - you might need to run `docker-machine start default`.
Local Versions:
Docker version 1.11.0, build 4dc5990
docker-compose version 1.8.0, build 94f7016
(Gentoo Linux)
Remote Versions:
Docker version 1.7.0-dev, build 582db78
(FreeBSD 11.0-RELEASE-p1)
Why won't docker-compose up work with a different socket when the other commands seem to communicate with it fine?