Docker SMTP container to use in Jenkins-Docker container - docker

This topic is about Docker networking, which I can't get to allow dockerised Jenkins to use a dockerised SMTP server.
Here's how I run my containers and connect them to a user-defined network, so that containers' name might be use as a target host:
# Run Jenkins image, with port binding, Docker sock sharing, and configuration sharing
docker run -d -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock -v /root/jenkins_conf/:/var/jenkins_home/ --name jenkins jenkins/jenkins:lts
# Run SMTP image
docker run -d --name smtp namshi/smtp
# Create user-defined network
docker network create jenkins-net
# Connect both containers
docker network connect jenkins-net jenkins
docker network connect jenkins-net smtp
Within my jenkins container, I can reach the smtp service via the default 'bridge' network:
$ (echo >/dev/tcp/172.17.0.5/25) &>/dev/null && echo "open" || echo "close"
open
and I can also reach it via my user-defined network, both via IP and hostname:
$ (echo >/dev/tcp/172.18.0.3/25) &>/dev/null && echo "open" || echo "close"
open
$ (echo >/dev/tcp/smtp/25) &>/dev/null && echo "open" || echo "close"
open
So far, so good.
But then, in Jenkins > Manage Jenkins > Configure System > E-mail notification, trying to use the test email sending tool gives me following results:
SMTP server: 172.17.0.5 (SMTP container IP on the default 'bridge' network)
SMTP port: 25
=> the email is sent and duly received !
SMTP server: 172.18.0.3 (SMTP container IP on my user-defined 'jenkins-net' network)
SMTP port: 25
=> Failed to send out e-mail
com.sun.mail.smtp.SMTPAddressFailedException: 550 relay not permitted
SMTP server: smtp (SMTP container name on my user-defined 'jenkins-net' network)
SMTP port: 25
=> same error
Why the difference of behaviour from the SMTP server between the use of the 2 networks?
What am I missing about Docker networking?
Edit: So the quick solution was to run the smtp container with the --network option, instead of running it and then connecting it to the network. See Stefano's answer below for more details and adequacy.

The problem you described is unrelated to the docker networking. The namshi/smtp image uses exim4 as SMTP. In this specific exim4 setup, you're required to provide the neworks from where it's allowed to connect and send email.
Checking the entrypoint.sh file, I found the following command:
dc_relay_nets "$(ip addr show dev eth0 | awk '$1 == "inet" { print $2 }' | xargs | sed 's/ /:/g')${RELAY_NETWORKS}"
This means that by default, it'll accept the emails coming from the IP address associated with the interface eth0 and other possible RELAY_NETWORKS (if defined).
Since the container is not attached by default to the jenkins-net network at the creation, it won't recognize as valid the emails coming from that address.
Try to start the containers like this:
docker network create jenkins-net
SUBNET=$( docker network inspect \
-f '{{range .IPAM.Config}}{{.Subnet}}{{end}}' \
jenkins-net )
docker run -d \
--network jenkins-net \
-e RELAY_NETWORKS=":${SUBNET}" \
--name smtp \
namshi/smtp
docker run -d \
-p 8080:8080 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /root/jenkins_conf/:/var/jenkins_home/ \
--network jenkins-net \
--name jenkins \
jenkins/jenkins:lts

Related

Unable to call a service from host machine's IP hosted on docker container

I have installed docker for windows version 3.1.0(51484) and setup a container running a memcache service on it using WSL2.
'''docker run -it -d --privileged --restart always -p 11212:11211 platform/centos/memcached /usr/sbin/init'''
I'm able to call memcache from localhost (127.0.0.1) but not from my machine's IP.
Any clue?
The problem can be about the IP setting. To check it try to listen to connection to a random port
terminal 1
nc -l 29123
terminal 2
nc localhost 29123
hello there
terminal 3
nc -vv -l 0.0.0.0 29124
terminal 4
nc <your-ip> 29124
hello there
if "hello there" appears in terminal 1, but not in terminal 3, so the problem is probably your IP/internet settings
Are you sure that your machine IP constant (not change)?
Does "Machine IP" means your "Host public IP" or "Host network IP"?

Docker: ssh -L to docker container - connection refused

I'm having problems to get my ssh tunnel working for my container in a docker swarm cluster.
ssh connection on my local machine:
ssh -L 7180:test.XXX:7180 user#XXX
In my Dockerfile on the remote machine:
EXPOSE 7180
Container start:
docker -H test:2379 --tlsverify run -d -p 7180:7180 --net=my-net
I tried to connect in Firefox via:
localhost:7180
Unfortunately the connection gets refused on the remote machine:
channel 3: open failed: connect failed: Connection refused
"docker container ls" prints following for the ports:
xxx:7180->7180/tcp
Inside my container "netstat -ntlp | grep LISTEN" prints:
tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN -
I'm new to this but after all what I've read so far this should actually work. I'm using "--net=my-net" because I want to setup my own network later. I had the same issue with "--net=host". What am I doing wrong?
The ssh command should be:
ssh -L 7180:127.0.0.1:7180 user#XXX
And then from your browser, you would go to:
http://127.0.0.1:7180
I've avoided using "localhost" because some machines map this to IPv6 even if you don't have IPv6 configured.
When testing this tunnel, make sure your application is listening on the remote server by doing an ssh to that server and run a curl command directly on the server to 127.0.0.1:7180. If it doesn't work there, you would repeat your debugging with netstat inside the container and verifying the port is published in thedocker ps` output.
I got it working with
ssh -D localhost:7180 -f -C -q -N user#XXX
and using
xxx:7180
in my browser (instead of localhost).
localhost and --net=host did not work for me with ssh -L.

Connect from docker container to host elasticsearch runing instance

I have this simple setup:
A Dockerfile:
FROM centos:latest
RUN yum update -y
RUN yum install -y epel-release
RUN yum install -y java-1.8.0-openjdk
RUN yum install -y curl
RUN mkdir /var/totest
EXPOSE 9200
EXPOSE 9300
And on host machine I have a running instance of elasticsearch 5.4.2.
When I do on host curl http://localhost:9200 I get the correct response:
{
"name" : "T8apV_J",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "YFkkaM8dSJCnXbDFRFD_aw",
"version" : {
"number" : "5.4.2",
"build_hash" : "929b078",
"build_date" : "2017-06-15T02:29:28.122Z",
"build_snapshot" : false,
"lucene_version" : "6.5.1"
},
"tagline" : "You Know, for Search"
}
but, if i build the docker image and I run
docker run -p 9200:9200 -it my-simple-image /bin/bash
I get this error:
docker: Error response from daemon: driver failed programming external connectivity on endpoint infallible_brown (71c6fae1275d149b2708a1aa9b737278d340c4e7b073d858b4222eb0268ef285): Error starting userland proxy: listen tcp 0.0.0.0:9200: bind: address already in use.
How can I connect from inside the docker container to the host running elasticsearch instance?
All I need is to be able to perform from inside the container
curl http://localhost:9200/index_name/_search
You need to remove the EXPOSE 9200 directive in the Dockerfile because port number 9200 is already taken by the elastic search service.
You should curl with the ip address of your host machine.
Docker attaches containers to the bridge network to start with so you need a way to get the ip address of the host.
You may need to set an alias depending on whether or not your host is connected to a wider network. I set this alias for my bridge0 interface.
sudo ifconfig bridge0 alias <ip_address>
If your host connected to a wider network, use the inet address of assigned to your ethernet device. You can get the inet address by running:
ifconfig en0 | grep "inet " | cut -d " " -f2
You can either way pass the inet address of your network interface as an environment variable to docker:
docker run -e MY_HOST_IP=$(ip_address) -it my-simple-image /bin/bash
# or
docker run -e MY_HOST_IP=$(ifconfig en0 | grep "inet " | cut -d " " -f2) -it my-simple-image /bin/bash
curl $MY_HOST_IP:8000
See this thread for more information about your question
Elastic search uses port 9200. Now you want to publish port 9200 from a container to localhost. You can not have two applications listening on the same port of localhost.
Here is a quick hack to get it working. With Elastic accessible on your host with curl -X GET 'http://0.0.0.0:9200', I think docker run --net host --name your_app will allow the container'd app to query the Elastic instance running on your host. So, you can omit any EXPOSE or -p settings. But be aware, according to their docs:
--network="host" gives the container full access to local system services such as D-bus and is therefore considered insecure.

Open docker daemon container to outside

The docker daemon container is isolated from outside when we run it below,
$ docker run -d --name test_container ubuntu/ping \
/bin/sh -c "while true do echo hello world; sleep 1; done"
$ docker inspect test_container | grep IPAddress
[ip of test_container]
$ ping [ip of test_container]
[timeout]
$ ifconfig docker0 | grep "inet addr"
[ip of docker bridge]
$ ping [ip of docker bridge]
[ok]
$ docker exec -it test_container /bin/bash
# ping [ip of test_container]
[ok]
# ping [ip of docker bridge]
[timeout]
How to open the ip address of the docker daemon container inside out?
By default docker daemon is running on a unix socket
You can enable to listen on tcp socket by doing :
docker daemon -H tcp://validIpOnYourHost:port
By default port is 2375 if you do not provide some.
cf this page for more explanation : https://docs.docker.com/v1.11/engine/reference/commandline/daemon/
Be careful, if you expose docker throught TCP, this is not security enabled.
Probably I replied to something else, after reading your question :
could you do a :
docker network inspect bridge
and paste the json output.
I had similar issues when the attribute :
"com.docker.network.bridge.enable_ip_masquerade"
was set to false

Remote debugging JVM in docker on AWS EB

I have a docker image in which I start jstatd -p 1099 and then my Java app. I also expose port 1099 in the dockerfile.
I have deployed this docker image to AWS ElasticBeanstalk and I can see from the EB logs that the port is exposed.
/var/log/docker-ps.log
-------------------------------------
'docker ps' ran at Fri Jun 17 04:23:02 UTC 2016:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d3199a65e216 8b9c53bb10b6 "/app/run.sh" 5 minutes ago Up 5 minutes 1099/tcp, 8080/tcp jolly_carson
I would now like to profile the app using VisualVM. How can I find the correct ip to connect to? Attempts to telnet to the app's domain name on port 1099 time out.
The container's port is not bound to the instance's port, which is good because you don't want to expose your debugging interface publicly. The IP address of the container can be found with:
$ sudo docker ps
$ sudo docker inspect --format '{{ .NetworkSettings.IPAddress }}' <container_id>
Start an SSH tunnel that tunnels from port 5005 locally to that IP address and port 5005 on the box.
$ ssh ec2-user#ec2-54-204-111-222.compute-1.amazonaws.com -L 5005:<ip>:5005 -N
or you can configure port forwarding over iptables
$ sudo docker inspect --format '{{ .NetworkSettings.IPAddress }}' stupefied_swartz
172.17.0.2
$ sudo iptables -t nat -A PREROUTING -p tcp --dport 5005 -j REDIRECT --to-destination 172.17.0.2:5005

Resources