I'm trying to setup azurite using these instructions, I see a number of others have been successful with this. I need to configure SSL (and eventually oauth) for my client app testing. The azurite container works fine without SSL, but when SSL is activated my client can't connect because the container isn't exposing the certificate.
I used mkcert to create the certificate. This is my docker-compose file. I'm mounting /certs and /data from my host.
version: '3.9'
services:
azurite:
image: mcr.microsoft.com/azure-storage/azurite
container_name: "azurite"
hostname: azurite
restart: always
ports:
- "10000:10000"
- "10001:10001"
- "10002:10002"
command: "azurite --oauth basic --cert /certs/127.0.0.1.pem --key /certs/127.0.0.1-key.pem --debug /logs/azurite-debug.log"
volumes:
- ./azurite-store:/data
- ./certs:/certs
- ./azurite-logs:/logs
Using openssl inside the container shows;
That's the cert I expect from mkcert & it's mounted as per the compose file.
From my laptop, openssl shows the following;
And there ends the fun! Why is the cert visible on the url inside the container, but not from the outside? I can't see anything in the compose file that would control if a cert is being exposed or not - I'm reasonably sure docker doesn't work like that - it's only exposing the tcp/ip layer to my laptop.
If I stop the container, port 10000 isn't reachable, start it and it opens so I don't think it's another process that I'm connecting to by mistake. Also, the fact that I get a connection means that it's not a connectivity issue.
Anyone got any thoughts on this one - it's weird! "Cert filtering" if I can call it that is certainly a new one!?
A little time away from the keyboard always helps
Looks like this is an application binding issue. Looks like node had bound to loopback (with the cert) and while docker had mapped the port out to my host, at an application layer node wasn't listening with a cert on that port. Changing blobHost to 0.0.0.0 allowed node to bind to all ip addresses on the container which in turn meant the cert was visible on the mapped port.
The docker-compose "command" becomes;
command: "azurite --oauth basic -l /data --cert /certs/127.0.0.1.pem --key /certs/127.0.0.1-key.pem --debug /logs/azurite-debug.log --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0"
I also found that despite mapping /data to a local volume I was loosing the blob containers on a container restart. Adding "-l /data" solved that one too.
Related
I have an easy apache docker setup defined in a docker-compose.yml:
services:
apache:
image: php:7.4-apache
command: /bin/bash -c "/var/www/html/startup.sh && exec 'apache2-foreground'"
volumes:
- ./:/var/www/html
- /c/Windows/System32/drivers/etc/hosts:/tmp/hostsfile
ports:
- "80:80"
From the startup.sh script I want to modify the hosts file from the host OS through the volume. Here I want to dynamically add an entry to resolve the hostname test.local to the ip address of the docker web application like so:
<ip-address> test.local
This way I should be able to open the application with the specified hostname http://test.local in my local browser.
Before writing the startup.sh script I wanted to try to manually open the application at http://172.19.0.2 via the containers IP address I got from
docker inspect apache_test
But the page won't open: ERR_CONNECTION_TIMED_OUT
Shouldn't I be able to access the application from that IP? What am I missing? Do I use the wrong IP address?
BTW: I am using Docker Desktop for Windows with the Hyper-V backend
Edit: I am able to access the application via http://localhost but since I want to add a new entry to the hosts file this is not the solution.
Apparently in newer versions of Docker Desktop for Windows you can't access (ping) your linux containers by IP.
Now there is some really dirty workaround for this that involves changing a .ps1 file of your Docker installation to get back the DockerNAT interface on windows. After that you need to add a new route in your windows routing table as described here:
route /P add <container-ip> MASK 255.255.0.0 <ip-found-in-docker-desktop-settings>
Then you might be able to ping your docker container from the windows host. I didn't test it though...
I found a solution to my original issue (resolution of test.local to container IP via hosts file) while reading one of the threads linked above here
That involves setting a free loopback IP in the 127.0.0.0/8 IP range in your ports section of the docker-compose.yml:
ports:
- "127.55.0.1:80:80"
After that you add the following to your hosts file:
127.55.0.1 test.local
And you can open your application at http://test.local
To do that for other dockerized applications too just choose another free loopback address.
I have the feeling that I am overlooking something obvious as my solutions/ideas so far seem too cumbersome. I have searched intensively for a good solution, but so far without success - probably because I do not know what to look for.
Question:
How do you interact with the graphical interfaces of web servers running in different containers (within the same Docker Network) on a remote server, given URL redirections between these containers?
Initial situation:
I have two containers (a Flask web application and a Tomcat server with OpenAM running on it) running on my docker host (Azure-VM).
On the VM I can output the content of both containers via the ports that I have opened.
Using ssh port forwarding I can interact with the graphical components of both containers on my local machine.
Both containers were created with the same docker-compose and can be accessed via their domain name without additional network settings.
So far I have configured OpenAM on my local machine using ssh port forwarding.
Problem:
The Flask web app references OpenAM by its domain name defined in docker-compose and vice versa. I forward to my local machine the port of the Flask container. The Flask application is running and I can interact with it in my browser.
The system fails as soon as I am redirected from Flask to OpenAM on my local machine because the reference to the OpenAM container used by Flask is specific to the Docker network. Also, the port of the OpenAM Container is different.
In other words, the routing between the two networks is nonexistent.
Solutions Ideas:
Execute the requests on the VM using command-line tools.
Use a container with a headless browser that automatically executes the requests.
Use Network Setting 'Host' and execute the headless browser on the VM instead.
Route all requests through a single container (similar to a VPN) and use ssh port forwarding.
Simplified docker-compose:
version: "3.4"
services:
openam:
image: openidentityplatform/openam
ports:
- 5001:8080
command: /usr/local/tomcat/bin/catalina.sh run
flask:
build: ./SimpleHTTPServer
ports:
- 5002:8000
command: python -m http.server 8000
Route all requests through a single container - This is the correct approach.
See API gateway pattern
The best solution that I could find so far. It does not serve for production. However, for prototyping or if simply trying to emulate a server structure by using containers it is an easy setup.
General Idea:
Deploy a third VNC container running a Webbrowser and forward the port of this third container to your local machine. As the third container is part of the docker network it can naturally resolve the internal domain names and the VNC installation on your local machine enables you to interact with the GUIs.
Approach
Add the VNC to the docker-compose of the original question.
Enable X11 forwarding on the server and client-side.
Forward the port of the VNC container using ssh.
Install VNC on the client, start a new session, and enter the predefined password.
Try it out.
Step by Step
Add the VNC container (inspired by creack's post on stackoverflow) to the docker-compose file from the original question:
version: "3.4"
services:
openam:
image: openidentityplatform/openam
ports:
- 5001:8080
command: /usr/local/tomcat/bin/catalina.sh run
flask:
build: ./SimpleHTTPServer
ports:
- 5002:8000
command: python -m http.server 8000
firefoxVnc:
container_name: firefoxVnc
image: creack/firefox-vnc
ports:
- 5900:5900
environment:
- HOME=/
command: x11vnc -forever -usepw -create
Run the docker-compose: docker-compose up
Enable X11 forwarding on the server and client-side.
On client side $ vim ~/.ssh/config and add the following lines:
Host *
ForwardAgent yes
ForwardX11 yes
On server-side run $ vim /etc/ssh/sshd_config and edit the following lines:
X11Forwarding yes
X11DisplayOffset 10
Forward the port of the VNC container using ssh
ssh -v -X -L 5900:localhost:5900 gw.example.com
Make sure to include the -X flag for X11. The -v flag is just for debugging.
Install VNC on the client, start a new session and enter the predefined password.
Install VNC viewer on your local machine
Open the installed viewer and start a new session using the forwarded address localhost:59000
When prompted type in the password 1234 which was set in the original Dockerfile of the VNC dicker image (see creack's post linked above).
You can now either go to openam:8080/openam/ or apache:80 within the browser of the VNC localhost:5900 session.
An even better solution that is clean, straightforward, and also works perfectly when running parts of the application on different virtual machines.
Setup and Use an SSH SOCKS Tunnel
For Google Chrome and macOS:
Set your network settings to host within the Dockerfile or docker-compose.
Start an SSH tunnel:
$ ssh -N -D 9090 [USER]#[SERVER_IP]
Add the SwitchyOmega proxy addon to your Chrome browser.
Configure SwitchyOmega by going to New Profile > Proxy Profile, clicking create, entering the same server IP as for the ssh command and the port 9090.
Open a new terminal tap and run:
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
--user-data-dir="$HOME/proxy-profile" \
--proxy-server="socks5://localhost:9090"
A new Crome session will open up in which you can simply browse your docker applications.
Reference | When running Linux or Windows | Using Firefox (no addon needed)
The guide How to Set up SSH SOCKS Tunnel for Private Browsing explains how to set up an SSH SOCKS Tunnel running Mac, Windows, or Linux and using Google Chrome or Firefox. I simply referenced the setup for macOS and Crome in case the link should die.
The goal: To deploy on docker swarm a set of services, one of which is only available for me when I am connected to the OpenVPN server which has also been spun up on docker swarm.
How can I, step by step, only connect to a whoami example container, with a domain in the browser, when I am connected to a VPN?
Background
The general idea would be have, say, kibana and elasticsearch running internally which can only be accessed when on the VPN (rather like a corporate network), with other services running perfectly fine publicly as normal. These will all be on separate nodes, so I am using an overlay network.
I do indeed have OpenVPN running on docker swarm along with a whoami container, and I can connect to the VPN, however it doesn't look like the IP is changing and I have no idea how to make it so that the whoami container is only available when on the VPN, especially considering I'm using an overlay network which is multi-host. I'm also using traefik, a reverse proxy which provides me with a mostly automatic letsencrypt setup (via DNS challenge) for wildcard domains. With this I can get:
https://traefik.mydomain.com
But I also want to connect to vpn.mydomain.com (which I can do right now), and then be able to visit:
https://whoami.mydomain.com
...which I cannot. Yet. I've posted my traefik configuration in a different place in case you want to take a look, as this thread will grow too big if I post it here.
Let's start with where I am right now.
OpenVPN
Firstly, the interesting thing about OpenVPN and docker swarm is that OpenVPN needs to run in privileged mode because it has to make network interfaces changes amongst other things, and swarm doesn't have CAP_ADD capabilities yet. So the idea is to launch the container via a sort of 'proxy container' that will run the container manually with these privileges added for you. It's a workaround for now, but it means you can deploy the service with swarm.
Here's my docker-compose for OpenVPN:
vpn-udp:
image: ixdotai/swarm-launcher:latest
hostname: mainnode
environment:
LAUNCH_IMAGE: ixdotai/openvpn:latest
LAUNCH_PULL: 'true'
LAUNCH_EXT_NETWORKS: 'app-net'
LAUNCH_PROJECT_NAME: 'vpn'
LAUNCH_SERVICE_NAME: 'vpn-udp'
LAUNCH_CAP_ADD: 'NET_ADMIN'
LAUNCH_PRIVILEGED: 'true'
LAUNCH_ENVIRONMENTS: 'OVPN_NATDEVICE=eth1'
LAUNCH_VOLUMES: '/etc/openvpn:/etc/openvpn:rw'
volumes:
- '/var/run/docker.sock:/var/run/docker.sock:rw'
networks:
- my-net
deploy:
placement:
constraints:
- node.hostname==mainnode
I can deploy the above with: docker stack deploy --with-registry-auth --compose-file docker/docker-compose.prod.yml my-app-name and this is what I'm using for the rest. Importantly I cannot just deploy this as it won't load yet. OpenVPN configuration needs to exist in /etc/openvpn on the node, which is then mounted in the container, and I do this during provisioning:
// Note that you have to create the overlay network with --attachable for standalone containers
docker network create -d overlay app-net --attachable
// Create the config
docker run -v /etc/openvpn:/etc/openvpn --log-driver=none --rm ixdotai/openvpn ovpn_genconfig -u udp://vpn.mydomain.com:1194 -b
// Generate all the vpn files, setup etc
docker run -v /etc/openvpn:/etc/openvpn --log-driver=none --rm ixdotai/openvpn bash -c 'yes yes | EASYRSA_REQ_CN=vpn.mydomain.com ovpn_initpki nopass'
// Setup a client config and grab the .ovpn file used for connecting
docker run -v /etc/openvpn:/etc/openvpn --log-driver=none --rm ixdotai/openvpn easyrsa build-client-full client nopass
docker run -v /etc/openvpn:/etc/openvpn --log-driver=none --rm ixdotai/openvpn ovpn_getclient client > client.ovpn
So now, I have an attachable overlay network, and when I deploy this, OpenVPN is up and running on the first node. I can grab a copy of client.ovpn and connect to the VPN. Even if I check "send all traffic through the VPN" though, it looks like the IP isn't being changed, and I'm still nowhere near hiding a container behind it.
Whoami
This simple container can be deployed with the following in docker-compose:
whoami:
image: "containous/whoami"
hostname: mainnode
networks:
- ${DOCKER_NETWORK_NAME}
ports:
- 1337:80
deploy:
placement:
constraints:
- node.hostname==mainnode
I put port 1337 there for testing, as I can visit my IP:1337 and see it, but this doesn't achieve my goal of having whoami.mydomain.com only resolving when connected to OpenVPN.
I can ping a 192.168 address when connected to the vpn
I ran the following on the host node:
ip -4 address add 192.168.146.16/24 dev eth0
Then when connected to the VPN, I can resolve this address! So it looks like something is working at least.
How can I achieve the goal stated at the top? What is required? What OpenVPN configuration needs to exist, what network configuration, and what container configuration? Do I need a custom DNS solution as I suggest below? What better alternatives are there?
Some considerations:
I can have the domains, including the private one whoami.mydomain.com public. This means I would have https and get wildcard certificates for them easily, I suppose? But my confusion here is - how can I get those domains only on the VPN but also have tls certs for them without using a self-signed certificate?
I can also run my own DNS server for resolving. I have tried this but I just couldn't get it working, probably because the VPN part isn't working properly yet. I found dnsmasq for this and I had to add the aforementioned local ip to resolve.conf to get anything working locally for this. But domains would still not resolve when connected to the VPN, so it doesn't look like DNS traffic was going over the VPN either (even though I set it as such - my client is viscosity.
Some mention using a bridge network, but a bridge network does not work for multi-host
Resources thus far (I will update with more)
- Using swarm-launcher to deploy OpenVPN
- A completely non-explanatory answer on stackexchange which I have seen referenced as basically unhelpful by multiple people across other Github threads, and one of the links is dead
So I was banging my head head against a brick wall about this problem and just sort of "solved" it by pivoting your idea:
Basically I opened the port of the vpn container to its host. And then enable a proxy. This means that I can reach that proxy by visiting the ip of the pc in which the vpn resides (AKA the Docker Host of the VPN container/stack).
Hang with me:
I used gluetun vpn but I think this applies also if you use openvpn one. I just find gluetun easier.
Also IMPORTANT NOTE: I tried this in a localhost environment, but theoretically this should work also in a multi-host situation since I'm working with separated stacks. Probably, in a multi-host situation you need to use the public ip of the main docker host.
1. Create the network
So, first of all you create an attachable network for this docker swarm stacks:
docker network create --driver overlay --attachable --scope swarm vpn-proxy
By the way, I'm starting to think that this passage is superfluous but need to test it more.
2. Set the vpn stack
Then you create your vpn stack file, lets call it stack-vpn.yml:
(here I used gluetun through swarm-launcher "trick". This gluetun service connects through a VPN via Wireguard. And it also enables an http proxy at the port 8888 - this port is also mapped to its host by setting LAUNCH_PORTS: '8888:8888/tcp')
version: '3.7'
services:
vpn_launcher:
image: registry.gitlab.com/ix.ai/swarm-launcher
volumes:
- '/var/run/docker.sock:/var/run/docker.sock:rw'
networks:
- vpn-proxy
environment:
LAUNCH_IMAGE: qmcgaw/gluetun
LAUNCH_PULL: 'true'
LAUNCH_EXT_NETWORKS: 'vpn-proxy'
LAUNCH_PROJECT_NAME: 'vpn'
LAUNCH_SERVICE_NAME: 'vpn-gluetun'
LAUNCH_CAP_ADD: 'NET_ADMIN'
LAUNCH_ENVIRONMENTS: 'VPNSP=<your-vpn-service> VPN_TYPE=wireguard WIREGUARD_PRIVATE_KEY=<your-private-key> WIREGUARD_PRESHARED_KEY=<your-preshared-key> WIREGUARD_ADDRESS=<addrs> HTTPPROXY=on HTTPPROXY_LOG=on'
LAUNCH_PORTS: '8888:8888/tcp'
deploy:
placement:
constraints: [ node.role == manager ]
restart_policy:
condition: on-failure
networks:
vpn-proxy:
external: true
Notice that either the swarm-launcher and the gluetun containers are using the network previously created vpn-proxy.
3. Set the workers stack
For the time being we will set an example with 3 replicas of alpine image here (filename stack-workers.yml):
version: '3.7'
services:
alpine:
image: alpine
networks:
- vpn-proxy
command: 'ping 8.8.8.8'
deploy:
replicas: 3
networks:
vpn-proxy:
external: true
They also use the vpn-proxy overlay network.
4. Launch our stacks
docker stack deploy -c stack-vpn.yml vpn
docker stack deploy -c stack-workers workers
Once they are up you can access any worker task and try to use the proxy by using the host ip where the proxy resides.
As I said before, theoretically this should work on a multi-host situation, but probably you need to use the public ip of the main docker host (although if they share the same overlay network it could also work with the internal ip address (192...) ).
I have Consul running via docker using docker-compose
version: '3'
services:
consul:
image: unifio/consul:latest
ports:
- "8500:8500"
- "8300:8300"
volumes:
- ./config:/config
- ./.data/consul:/data
command: agent -server -data-dir=/data -ui -bind 0.0.0.0 -client 0.0.0.0 -bootstrap-expect=1
mongo:
image: mongo
ports:
- "27017:27017"
- "27018:27018"
volumes:
- ./.data/mongodb:/data/db
command: mongod --bind_ip_all
and have a nodejs service running on port 6001 which exposes a /health endpoint for health checks.
I am able to register the service via this consul package.
However, visiting the consul UI I can see that the service has a status of failing because the health check is not working.
The UI show this message:
Get http://127.0.0.1:6001/health: dial tcp 127.0.0.1:6001: getsockopt: connection refused
Not sure why is not working exactly, but I kind of sense that i may have misconfigured consul.
Any help would be great.
Consul is running in your docker container. When you use 127.0.0.1 in this container, it refers to itself, not to your host.
You need to use a host IP that is known to your container (and of course make sure your service is reachable and listening on this particular IP).
In most cases, you should be able to contact your host from a container through the default docker0 bridge ip that you can get with ip addr show dev docker0 from your host as outlined in this other answer.
The best solution IMO is to discover the gateway that your container is using which will point to the particular bridge IP on your host (i.e. the bridge created for your docker-compose project when starting it). There are several methods you can use to discover this ip from the container depending on the installed tooling and your linux flavor.
While Zeitounator's answer is perfectly fine and answers your direct question,
the "indirect" solution to your problem would be to manage the nodejs service
through docker-compose.
IMHO it's a good idea to manage all services involved using the same tool,
as then their lifecycles are aligned and also it's easy to configure them to talk
to each other (at least that's the case for docker-compose).
Moreover, letting containers access services on the host is risky in terms of security.
In production environments you usually want to shield host services from containers,
as otherwise the containers lose their "containment" role.
So, in your case you would need to add the nodejs service to docker-compose.yml:
services:
(...)
nodejs-service:
image: nodejs-service-image
ports:
- "6001:6001" [this is only required if you need to expose the port on the host]
command: nodejs service.js
And then your Consul service would be able to access nodejs-service
through http://nodejs-service:6001/health.
I've following coder-compose configuration:
version: '2'
services:
nginx:
build: ./nginx
links:
- tomcat1:tomcat1
- tomcat2:tomcat2
- tomcat3:tomcat3
ports:
- "80:80"
tomcat1:
build: ./tomcat
ports:
- "8080"
tomcat2:
build: ./tomcat
ports:
- "8080"
tomcat3:
build: ./tomcat
ports:
- "8080"
So, the question is, how to get access to the host network from the linked container(s):tomcat1, tomcat2, tomcat3. Here is the diagram:
Update
Seems, my diagram doesn't help much. Nginx is a load balancer, Tomcat 1-3 are application nodes. Deployed web. app needs to get access to internet resource.
Internet access is by default active on all containers (in bridge mode). All you need to check is if the http(s)_proxy variables are set if you are behind a proxy.
If your question if how to access docker host from a container (and not the reverse: access a container from the local docker host), then you would need to inspect the routing table of a container: see "From inside of a Docker container, how do I connect to the localhost of the machine?"
export DOCKER_HOST_IP=$(route -n | awk '/UG[ \t]/{print $2}')
There is a recent (June 2016) effort to add a adding a dockerhost as an entry in /etc/hosts of all running containers: issue 23177.
Update March 2020: this issue has been closed, and redirect to PR 40007: "Support host.docker.internal in dockerd on Linux"
This PR allows containers to connect to Linux hosts by appending a special string "host-gateway" to --add-host e.g. "--add-host=host.docker.internal:host-gateway" which adds host.docker.internal DNS entry in /etc/hosts and maps it to host-gateway-ip
This PR also add a daemon flag call host-gateway-ip which defaults to
the default bridge IP
Docker Desktop will need to set this field to the Host Proxy IP so DNS requests for host.docker.internal can be routed to VPNkit
This will be in Docker for Linux (and Docker Desktop, which runs the Linux daemon, although inside a lightweight VM).
Difference between this and the current implementation on Docker Desktop is that;
the current Docker Desktop implementation is in a part of the code-base that's proprietary (i.e., part of how Docker Desktop is configured internally)
this code could be used by the Docker Desktop team in future as well (to be discussed)
this PR does not set up the "magic" host.docker.internal automatically on every container, but it can be used to run a container that needs this host by adding docker run --add-host host.docker.internal:host-gateway
(to be discussed); setting that "magic" domain automatically on containers that are started could be implemented by adding an option for this in the ~/.docker/config.json CLI configuration file.