Map outgoing TCP connection to Docker container

Map outgoing TCP connection to Docker container - docker

I have a collection of IPv4 traffic recorded by tcpdump, and I am interested in mapping a given outgoing connection to its origin Docker container on the same machine. I can view the TCP source port, but it is unclear to me how to ask Docker which container is using this port. How can this be done?
Thanks

You can filter containers which publish or expose a given port:
docker ps --filter=expose=3306
or use the format switch to get a list of container IDs with the used ports:
docker ps --format "{{.ID}}: {{.Ports}}"

I had this exact problem where I wanted to know which container was the client for a particular Postgresql database query. These are the steps I took to find the container that was the source of the TCP connection I was interested in.
In postgres query pg_stat_activity to determine the client_addr and client_port of the query you care about:
SELECT * FROM pg_stat_activity;
Now you know the source ip (client_addr) you can ssh to this box and determine the network interface used by docker:
ip addr
In my case it was docker0. Now we can use the source port of the database connection (client_port) to find the container ip address:
tcpdump -i docker0 src port 52282
02:41:16.257106 IP 172.17.0.15.52282 > 192.168.2.119.postgres: Flags [P.] ...
Now we need to work out which container this source ip (172.17.0.15) belongs to, to do this we can use the docker inspect command with a filter:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container
We run this command in a loop for each of the running containers and grep for the ip we are looking for:
for name in $( docker ps --format '{{.Names}}' ); do echo -n "$name => " && docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $name; done | grep 172.17.0.15
my-container => 172.17.0.15
Now we know that the container my-container is the source of the traffic we are interested in.

Related

Docker and netstat: why is netstat not showing ports, exposed by docker containers

Related to question Docker and netstat: netstat is not showing ports, exposed by docker containers where only solutions are discussed, I would like to know why this happens?
What happens:
If I serve a service using docker (e.g. docker run --rm -p 8080:80 httpd), executing netstat -anp | grep 8080 on host shows docker listening on 8080, but it does not show open connections! I would expect to see connections also on host, because clients are connected to the host which forwards the port to the docker container.
Why is it not possible to see these connections?

netstat -a shows open socket connections. But the client is not connected to the host socket, because docker routes the packages before connection to the container by applying masquerading rules. This can be seen with iptables -L -n -t nat.
So the only way to see this connections with netstat is by executing it in the container. For example by running docker exec CONTAINER netstat -anp.

Assigning a local IP to docker containers

Is there a way to have docker automatically give it’s containers a local IP address that you can reach with it’s ports exposed?
For example, LXC has ways to do this.
lxc-create -t ubuntu -n myname
lxc-start -n myname -d
Which will then assign a local IP which you can see via lxc-ls if you have a bridge configured:
lxc-ls -f
This is super convenient for throwing up a bunch of containers for testing out deployment/configuration management like ansible.
Is it possible to do something similar in docker without much headache? I come from using LXC and I’m not familiar with the networking modes.

LXC and Docker are very similar. When LXC is installed, a random subnet is picked for configuring the IP addresses of bridge and the containers attached to it. With Docker, the default subnet is 172.17.0.0/16, which can be customized if needed. Every container stared (unless using host network or network of another container) using docker run command are assigned an IP address from the above subnet.
docker ps lists all the containers running but unfortunately it doesn't show the IP addresses.
Looks like a small trick can show the IP addresses: (based on this post)
docker ps -q | xargs docker inspect --format '{{ .Id }} - {{ .Name }} - {{ .NetworkSettings.IPAddress }}'
Also you can expose the ports using the -p option of docker run command.
Example:
docker run -itd -p 8080:80 nginx
This starts the container on the docker bridge and also exposes the nginx on the host network via port 8080.
from_host_running_container# curl http://172.17.x.y
from_not_the_host_running_container # curl http://${HOST_IP}:8080

How to get IP address of running docker container

I am using Docker for Mac. I am running a nodejs based microservice in a Docker container. I want to test node microservice through the browser. How to get IP address of running docker container?

If you don't want to map ports from your host to the container you can access directly to the docker range ip for the container. This range is by default only accessed from your host. You can check your container network data doing:
docker inspect <containerNameOrId>
Probably is better to filter:
docker inspect <containerNameOrId> | grep '"IPAddress"' | head -n 1
Usually, the default docker ip range is 172.17.0.0/16. Your host should be 172.17.0.1 and your first container should be 172.17.0.2 if everything is normal and you didn't specify any special network options.
EDIT
Another more elegant way using docker features instead of "bash tricking":
docker inspect -f "{{ .NetworkSettings.IPAddress }}" <containerNameOrId>
EDIT2
For modern docker engines, now it is this way (thanks to the commenters!):
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <containerNameOrId>

Use --format option to get only the IP address instead whole container info:
sudo docker inspect --format '{{ .NetworkSettings.IPAddress }}' <CONTAINER ID>

For modern docker engines use this command :
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id
and for older engines use :
docker inspect --format '{{ .NetworkSettings.IPAddress }}' container_name_or_id

if you want to obtain it right within the container, you can try
ip a | grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" | grep 172.17

You can start your container with the flag -P. This "assigns" a random port to the exposed port of your image.
With docker port <container id> you can see the randomly choosen port. Access is then possible via localhost:port.

For my case, below worked on Mac:
I could not access container IPs directly on Mac. I need to use localhost with port forwarding, e.g. if the port is 8000, then http://localhost:8000
See https://docs.docker.com/docker-for-mac/networking/#known-limitations-use-cases-and-workarounds
The original answer was from: https://github.com/docker/for-mac/issues/2670#issuecomment-371249949

If you want to view the IP address from within the running container, /etc/hosts file is a great place to look at. Now, to uniquely identify the entry within the hosts file, it is a good practise to run the container with the -h option. Sample commands are given below:
Run the container with -h set:
docker run -td -h guju <image name>
Log in to the running container and view the /etc/hosts file. It will show an entry like this:
172.17.0.5 guju

this will list all containers' IP addresses
while read ctr;do
sudo docker inspect --format "$ctr "'{{.Name}}{{ .NetworkSettings.IPAddress }}' $ctr
done < <(docker ps -a --filter status=running --format '{{.ID}}')

You can not access the docker's IP from outside of that host machine.
If your browser is on another machine better to map the host port to container port by passing -p 8080:8080 to run command.
Passing -p you can map host port to container port and a proxy is set to forward all traffix for said host port to designated container port.

How to connect to service in the host machine from inside a docker container?

I have a docker daemon started in the host machine listen to some ip address and port, say 10.10.10.10, and port 1234, then I start a container by invoking
sudo docker -H 10.10.10.10:1234 centos /bin/bash
Meanwhile, I have a web service runnig on the host machine, running on the port 8080. Then from inside the container, I cannot connect to this server. I tried
curl http://10.10.10.10:8080
but got an error message:
curl: (7) couldn't connect to host
But I can access the server in other machines, like http://10.10.10.11:8080
It seems that docker container cannot access the service in its own host machine? Is there anyway to fix this? Thanks

That was discussed in issue 1143 and there might be one day a --link-host option.
In the meantime, the blog post "Accessing the Docker Host Server Within a Container" lists a few option:
The Gateway Approach: netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}'
The IP Approach: boot2docker ip address is 192.168.59.103.
Although there’s no way to introspect the host’s ip address (AFAIK) you can pass this in via an environment variable:
docker#boot2docker:~$ docker run -i -t -e DOCKER_HOST=192.168.59.103 ubuntu /bin/bash
root#07561b0607f4:/# env
HOSTNAME=07561b0607f4
DOCKER_HOST=192.168.59.103
You have other options in "Network Configuration", with the virtual interface named docker0 on the host machine.

How to get the IP address of the docker host from inside a docker container [duplicate]

This question already has answers here:
From inside of a Docker container, how do I connect to the localhost of the machine?
(41 answers)
Closed 10 months ago.
As the title says, I need to be able to retrieve the IP address the docker hosts and the portmaps from the host to the container, and doing that inside of the container.

/sbin/ip route|awk '/default/ { print $3 }'
As #MichaelNeale noticed, there is no sense to use this method in Dockerfile (except when we need this IP during build time only), because this IP will be hardcoded during build time.

As of version 18.03, you can use host.docker.internal as the host's IP.
Works in Docker for Mac, Docker for Windows, and perhaps other platforms as well.
This is an update from the Mac-specific docker.for.mac.localhost, available since version 17.06, and docker.for.mac.host.internal, available since version 17.12, which may also still work on that platform.
Note, as in the Mac and Windows documentation, this is for development purposes only.
For example, I have environment variables set on my host:
MONGO_SERVER=host.docker.internal
In my docker-compose.yml file, I have this:
version: '3'
services:
api:
build: ./api
volumes:
- ./api:/usr/src/app:ro
ports:
- "8000"
environment:
- MONGO_SERVER
command: /usr/local/bin/gunicorn -c /usr/src/app/gunicorn_config.py -w 1 -b :8000 wsgi

Update: On Docker for Mac, as of version 18.03, you can use host.docker.internal as the host's IP. See aljabear's answer. For prior versions of Docker for Mac the following answer may still be useful:
On Docker for Mac the docker0 bridge does not exist, so other answers here may not work. All outgoing traffic however, is routed through your parent host, so as long as you try to connect to an IP it recognizes as itself (and the docker container doesn't think is itself) you should be able to connect. For example if you run this from the parent machine run:
ipconfig getifaddr en0
This should show you the IP of your Mac on its current network and your docker container should be able to connect to this address as well. This is of course a pain if this IP address ever changes, but you can add a custom loopback IP to your Mac that the container doesn't think is itself by doing something like this on the parent machine:
sudo ifconfig lo0 alias 192.168.46.49
You can then test the connection from within the docker container with telnet. In my case I wanted to connect to a remote xdebug server:
telnet 192.168.46.49 9000
Now when traffic comes into your Mac addressed for 192.168.46.49 (and all the traffic leaving your container does go through your Mac) your Mac will assume that IP is itself. When you are finish using this IP, you can remove the loopback alias like this:
sudo ifconfig lo0 -alias 192.168.46.49
One thing to be careful about is that the docker container won't send traffic to the parent host if it thinks the traffic's destination is itself. So check the loopback interface inside the container if you have trouble:
sudo ip addr show lo
In my case, this showed inet 127.0.0.1/8 which means I couldn't use any IPs in the 127.* range. That's why I used 192.168.* in the example above. Make sure the IP you use doesn't conflict with something on your own network.

AFAIK, in the case of Docker for Linux (standard distribution), the IP address of the host will always be 172.17.0.1 (on the main network of docker, see comments to learn more).
The easiest way to get it is via ifconfig (interface docker0) from the host:
ifconfig
From inside a docker, the following command from a docker: ip -4 route show default | cut -d" " -f3
You can run it quickly in a docker with the following command line:
# 1. Run an ubuntu docker
# 2. Updates dependencies (quietly)
# 3. Install ip package (quietly)
# 4. Shows (nicely) the ip of the host
# 5. Removes the docker (thanks to `--rm` arg)
docker run -it --rm ubuntu:22.04 bash -c "apt-get update > /dev/null && apt-get install iproute2 -y > /dev/null && ip -4 route show default | cut -d' ' -f3"

For those running Docker in AWS, the instance meta-data for the host is still available from inside the container.
curl http://169.254.169.254/latest/meta-data/local-ipv4
For example:
$ docker run alpine /bin/sh -c "apk update ; apk add curl ; curl -s http://169.254.169.254/latest/meta-data/local-ipv4 ; echo"
fetch http://dl-cdn.alpinelinux.org/alpine/v3.3/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.3/community/x86_64/APKINDEX.tar.gz
v3.3.1-119-gb247c0a [http://dl-cdn.alpinelinux.org/alpine/v3.3/main]
v3.3.1-59-g48b0368 [http://dl-cdn.alpinelinux.org/alpine/v3.3/community]
OK: 5855 distinct packages available
(1/4) Installing openssl (1.0.2g-r0)
(2/4) Installing ca-certificates (20160104-r2)
(3/4) Installing libssh2 (1.6.0-r1)
(4/4) Installing curl (7.47.0-r0)
Executing busybox-1.24.1-r7.trigger
Executing ca-certificates-20160104-r2.trigger
OK: 7 MiB in 15 packages
172.31.27.238
$ ifconfig eth0 | grep -oP 'inet addr:\K\S+'
172.31.27.238

The only way is passing the host information as environment when you create a container
run --env <key>=<value>

The --add-host could be a more cleaner solution (but without the port part, only the host can be handled with this solution). So, in your docker run command, do something like:
docker run --add-host dockerhost:`/sbin/ip route|awk '/default/ { print $3}'` [my container]
(From https://stackoverflow.com/a/26864854/127400 )

docker network inspect bridge -f '{{range .IPAM.Config}}{{.Gateway}}{{end}}'
It's possible to retrieve it using docker network inspect

The standard best practice for most apps looking to do this automatically is: you don't. Instead you have the person running the container inject an external hostname/ip address as configuration, e.g. as an environment variable or config file. Allowing the user to inject this gives you the most portable design.
Why would this be so difficult? Because containers will, by design, isolate the application from the host environment. The network is namespaced to just that container by default, and details of the host are protected from the process running inside the container which may not be fully trusted.
There are different options depending on your specific situation:
If your container is running with host networking, then you can look at the routing table on the host directly to see the default route out. From this question the following works for me e.g.:
ip route get 1 | sed -n 's/^.*src $[0-9.]*$ .*$/\1/p'
An example showing this with host networking in a container looks like:
docker run --rm --net host busybox /bin/sh -c \
"ip route get 1 | sed -n 's/^.*src $[0-9.]*$ .*$/\1/p'"
For current versions of Docker Desktop, they injected a DNS entry into the embedded VM:
getent hosts host.docker.internal | awk '{print $1}'
With the 20.10 release, the host.docker.internal alias can also work on Linux if you run your containers with an extra option:
docker run --add-host host.docker.internal:host-gateway ...
If you are running in a cloud environment, you can check the metadata service from the cloud provider, e.g. the AWS one:
curl http://169.254.169.254/latest/meta-data/local-ipv4
If you want your external/internet address, you can query a remote service like:
curl ifconfig.co
Each of these have limitations and only work in specific scenarios. The most portable option is still to run your container with the IP address injected as a configuration, e.g. here's an option running the earlier ip command on the host and injecting it as an environment variable:
export HOST_IP=$(ip route get 1 | sed -n 's/^.*src $[0-9.]*$ .*$/\1/p')
docker run --rm -e HOST_IP busybox printenv HOST_IP

TLDR for Mac and Windows
docker run -it --rm alpine nslookup host.docker.internal
... prints the host's IP address ...
nslookup: can't resolve '(null)': Name does not resolve
Name: host.docker.internal
Address 1: 192.168.65.2
Details
On Mac and Windows, you can use the special DNS name host.docker.internal.
The host has a changing IP address (or none if you have no network access). From 18.03 onwards our recommendation is to connect to the special DNS name host.docker.internal, which resolves to the internal IP address used by the host. This is for development purpose and will not work in a production environment outside of Docker Desktop for Mac.

If you want real IP address (not a bridge IP) on Windows and you have docker 18.03 (or more recent) do the following:
Run bash on container from host where image name is nginx (works on Alpine Linux distribution):
docker run -it nginx /bin/ash
Then run inside container
/ # nslookup host.docker.internal
Name: host.docker.internal
Address 1: 192.168.65.2
192.168.65.2 is the host's IP - not the bridge IP like in spinus accepted answer.
I am using here host.docker.internal:
The host has a changing IP address (or none if you have no network access). From 18.03 onwards our recommendation is to connect to the special DNS name host.docker.internal, which resolves to the internal IP address used by the host. This is for development purpose and will not work in a production environment outside of Docker for Windows.

In linux you can run
HOST_IP=`hostname -I | awk '{print $1}'`
In macOS your host machine is not the Docker host. Docker will install it's host OS in VirtualBox.
HOST_IP=`docker run busybox ping -c 1 docker.for.mac.localhost | awk 'FNR==2 {print $4}' | sed s'/.$//'`

I have Ubuntu 16.03. For me
docker run --add-host dockerhost:`/sbin/ip route|awk '/default/ { print $3}'` [image]
does NOT work (wrong ip was generating)
My working solution was that:
docker run --add-host dockerhost:`docker network inspect --format='{{range .IPAM.Config}}{{.Gateway}}{{end}}' bridge` [image]

Docker for Mac
I want to connect from a container to a service on the host
The host has a changing IP address (or none if you have no network access). From 18.03 onwards our recommendation is to connect to the special DNS name host.docker.internal, which resolves to the internal IP address used by the host.
The gateway is also reachable as gateway.docker.internal.
https://docs.docker.com/docker-for-mac/networking/#use-cases-and-workarounds

If you enabled the docker remote API (via -Htcp://0.0.0.0:4243 for instance) and know the host machine's hostname or IP address this can be done with a lot of bash.
Within my container's user's bashrc:
export hostIP=$(ip r | awk '/default/{print $3}')
export containerID=$(awk -F/ '/docker/{print $NF;exit;}' /proc/self/cgroup)
export proxyPort=$(
curl -s http://$hostIP:4243/containers/$containerID/json |
node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).NetworkSettings.Ports["DESIRED_PORT/tcp"][0].HostPort'
)
The second line grabs the container ID from your local /proc/self/cgroup file.
Third line curls out to the host machine (assuming you're using 4243 as docker's port) then uses node to parse the returned JSON for the DESIRED_PORT.

My solution:
docker run --net=host
then in docker container:
hostname -I | awk '{print $1}'

Here is another option for those running Docker in AWS. This option avoids having using apk to add the curl package and saves the precious 7mb of space. Use the built-in wget (part of the monolithic BusyBox binary):
wget -q -O - http://169.254.169.254/latest/meta-data/local-ipv4

use hostname -I command on the terminal

Try this:
docker run --rm -i --net=host alpine ifconfig

So... if you are running your containers using a Rancher server, Rancher v1.6 (not sure if 2.0 has this) containers have access to http://rancher-metadata/ which has a lot of useful information.
From inside the container the IP address can be found here:
curl http://rancher-metadata/latest/self/host/agent_ip
For more details see:
https://rancher.com/docs/rancher/v1.6/en/rancher-services/metadata-service/

This is a minimalistic implementation in Node.js for who is running the host on AWS EC2 instances, using the afore mentioned EC2 Metadata instance
const cp = require('child_process');
const ec2 = function (callback) {
const URL = 'http://169.254.169.254/latest/meta-data/local-ipv4';
// we make it silent and timeout to 1 sec
const args = [URL, '-s', '--max-time', '1'];
const opts = {};
cp.execFile('curl', args, opts, (error, stdout) => {
if (error) return callback(new Error('ec2 ip error'));
else return callback(null, stdout);
})
.on('error', (error) => callback(new Error('ec2 ip error')));
}//ec2
and used as
ec2(function(err, ip) {
if(err) console.log(err)
else console.log(ip);
})

If you are running a Windows container on a Service Fabric cluster, the host's IP address is available via the environment variable Fabric_NodeIPOrFQDN. Service Fabric environment variables

Here is how I do it. In this case, it adds a hosts entry into /etc/hosts within the docker image pointing taurus-host to my local machine IP: :
TAURUS_HOST=`ipconfig getifaddr en0`
docker run -it --rm -e MY_ENVIRONMENT='local' --add-host "taurus-host:${TAURUS_HOST}" ...
Then, from within Docker container, script can use host name taurus-host to get out to my local machine which hosts the docker container.

Maybe the container I've created is useful as well https://github.com/qoomon/docker-host
You can simply use container name dns to access host system e.g. curl http://dockerhost:9200, so no need to hassle with any IP address.

The solution I use is based on a "server" that returns the external address of the Docker host when it receives a http request.
On the "server":
1) Start jwilder/nginx-proxy
# docker run -d -p <external server port>:80 -v /var/run/docker.sock:/tmp/docker.sock:ro jwilder/nginx-proxy
2) Start ipify container
# docker run -e VIRTUAL_HOST=<external server name/address> --detach --name ipify osixia/ipify-api:0.1.0
Now when a container sends a http request to the server, e.g.
# curl http://<external server name/address>:<external server port>
the IP address of the Docker host is returned by ipify via http header "X-Forwarded-For"
Example (ipify server has name "ipify.example.com" and runs on port 80, docker host has IP 10.20.30.40):
# docker run -d -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock:ro jwilder/nginx-proxy
# docker run -e VIRTUAL_HOST=ipify.example.com --detach --name ipify osixia/ipify-api:0.1.0
Inside the container you can now call:
# curl http://ipify.example.com
10.20.30.40

On Ubuntu, hostname command can be used with the following options:
-i, --ip-address addresses for the host name
-I, --all-ip-addresses all addresses for the host
For example:
$ hostname -i
172.17.0.2
To assign to the variable, the following one-liner can be used:
IP=$(hostname -i)

Another approach is based on traceroute and it's working on a Linux host for me, e.g. in a container based on Alpine:
traceroute -n 8.8.8.8 -m 4 -w 1 | awk '$1~/\d/&&$2!~/^172\./{print$2}' | head -1
It takes a moment, but lists the first hop's IP that does not start with 172. If there is no successful response, try increasing the limit on the tested hops using -m 4 argument.

With https://docs.docker.com/machine/install-machine/
a) $ docker-machine ip
b) Get the IP address of one or more machines.
$ docker-machine ip host_name
$ docker-machine ip host_name1 host_name2

Develop Reference

ios ruby-on-rails asp.net-mvc docker delphi jenkins grails google-sheets machine-learning dart