In order to debug and setup a pair of docker stacks (one is a client and other a server along with their own private services they each require) using docker compose, I'm running them locally to make sure they're functioning correctly.
They will eventually be communicating across the internet with a nginx server on the server side to act as a reverse proxy. But for now, i'm specifying the client use the 172.19.0.3:1234 address of the server container.
I'm able to curl/ping both the client container and server container from the host machine, but running an interactive session and trying to curl the server's 172.19.0.3:1234 address just times out.
I feel the 172.x is being used incorrectly here. Is their some obvious issue with what I've described so far? What is the better approach for what I'm trying to do.
Seems that after doing some searching, I am in a similar situation to this question: Communicating between Docker containers in different networks on the same host.
I've decided to use docker network connect to connect the client to the server's network for my purposes.
Related
I have a legacy system. It contains a number of servers running on Linux and a number of GUI clients running on Windows. All the components (servers and clients) are in the same network and they communicate with each other directly. They are identified by ip and port number.
For development purpose, I now run the servers in containers using compose on a Linux host. The servers communicate with each other within the docker network without any issues. However, I have trouble to make the client work with servers. Port mapping doesn't work here since a client needs to talk to many servers with different or same port. What I am asking is if it is possible to treat the Windows client as part of the docker network. I read about tools such as weave net, etc., but haven't found anything useful. Any suggestions?
I'm using GKE for deployments.
Edit: I need to access a customer's API endpoint which is only accessible when using their VPN. So far I can run a container which connects to this VPN and I can cURL the endpoint successfully.
For the above, I have configured a Debian docker image which successfully connects to a VPN (specifically, using Kerio Control VPN) when deployed. Whenever I make a net request from this container, it runs through the VPN connection, as expected.
I have another image which runs a .NET Core program which makes necessary HTTP requests.
From this guide I know it is possible to run a container's traffic through another using pure docker. Specifically using the --net=container:something option (trimmed the example):
docker run \
--name=jackett \
--net=container:vpncontainer \
linuxserver/jackett
However, I have to use Kubernetes for this deployment so I think it would be good to use a 2-container pod. I want to keep the VPN connection logic and the program separated.
How can I achieve this?
Each container in pod have shared network resources. If you run vpn client in one container them all containers in this pod will have access to network via vpn.
Based on your comment I think I can advise you two methods.
Private GKE Cluster with CloudNAT
In this setup, you should you use Private GKE cluster with CloudNAT for external communication. You would need to to use manual externalIP.
This scenario is using specific externalIP for VPN connection, but it's required from your customer to whitelist access for this IP.
Site to site VPN using CloudVPN
You can configure your VPN to forward packets to your cluster. For details you should check other Stackoverflow threads:
Google Container Engine and VPN
Why can't I access my Kubernetes service via its IP?
I'm using a similar approach. I have a Django app for whose static files to be served files I need nginx. I want the app to be accessible through VPN for which I'm using OpenVPN.
Both the nginx container and the django container are in the same pod. My limited understanding is that it would be enough to run VPN in the background in the nginx container and it should successfully route requests to the backend using localhost because they're in the same pod.
But this doesn't seem to be working. I get a 504 Time-Out in the browser and the nginx logs confirm that the upstream timed out. Have you done anything extra to make this work in your case?
I have been looking everywhere for this answer. To me it seems like an obvious question, however, the answer has eluded me.
My current setup is, I have redis, mongodb and two api servers on the same bridge network. The first server serves as a gateway api that does all the auth, and exposes certain api calls. The backend api is the one that handles all the db interactions and data munging. If I hit the backend (inner) api alone, I am able to see the contents (this api would not be exposed in real production environment). However, if I make the same request from within the gateway api, I am not able to hit the backend (inner) api that is also part of the bridged network I created.
Below is a diagram of the container interactions.
I still use legacy linking, but I'm a little bit familiar with this. I think the problem is that you are trying to hit "localhost" from inside your gateway container. The inner API container cannot be resolved as "localhost" inside of the gateway API container. You are able to hit "localhost:8099" from the host machine or externally because of the port mapping, but none of your other containers will be able to resolve that address/port because they 'think' it's a remote machine.
Here's a way to test what I'm thinking. In your host's shell, run the bridge inspect command shown here. Copy the IP address from Containers.<inner-api-hash>.IPV4. Then open a shell in the gateway container with docker exec -it <gateway-id> /bin/bash and then use curl or wget to see if you can hit that IP address you copied.
If my thinking is correct, you will see that you must use your inner-API node's Docker assigned IP address from the other containers. Amongst other options, you can start containers with a static IP address as shown here.
This is starting to escape the scope of my knowledge, but you can also configure a container DNS. Configure container DNS.
I have some docker containers running on Bluemix using private IP addresses. I would now like to setup a tunnel from my laptop (running linux) to access the private network on Bluemix.
I had first created a container running an ssh-server. Using ssh -D I was able to setup a SOCKS5 proxy connection. This worked fine with Chrome but not all applications support a SOCKS proxy.
(google-chrome --proxy-server=socks5://localhost:<tunnel port>)
So I tried to create a container with an OpenVPN server. Unfortunately this does not work on Bluemix as the containers are not running privileged and thus can not create a tun device.
Bluemix also has a VPN and a Secure Gateway service, which sound promising but so far I could not figure out how to get those working.
Does anybody know if it is possible to make the private docker network available locally and how to connect to that?
Generally speaking containers should be used to implement services available to external applications (an APIs service, or a runtime, or a dbms, or something like that).
According to this, what you could achieve is a set of services available for you on different containers, and a single container working as SSH tunnel gateway, making your local environment connected to it using SSH and defining a set of local and remote SSH ports forwarding, with different policies according to the service/port and the IP of the service.
It should work for all the services, and you haven't to use a socks proxy to forward requests to different hosts: using remote SSH forwarding your SSH endpoint will redirect your requests to the right service inside the local/private lan.
I found that this guide describes correctly how to work with local&remote port forwarding.
http://www.debianadmin.com/howto-use-ssh-local-and-remote-port-forwarding.html
About the OpenVPN solution, as you already know it is not possible to use software requiring privileged mode on containers, because it couldn't be allowed on Bluemix due to security reasons: if you wish to have this kind of solution I strongly suggest you to use OpenVPN on a VM on Bluemix UK region (still beta but an architecture expected to be the final architecture as soon as VM service will become GA service)
I think that these options are the ones available on Bluemix to achieve what you describe without using the VPN service suggested by #bill-wentworth
I am setting up a simple cluster using docker on several hosts. Before using docker the processes were simply started with a argument giving the address to a config server. The first thing each process does is to connect to the config server, get the addresses (host and port) of all the other services as well as register itself with host (and several different ports, one for each the services it provides).
However, it does not seem to be possible to dockerize this workflow? Since a process in a container seems not to be able to get the address and ports on the host (based on for example How to get the IP address of the docker host from inside a docker container) it does not know what to register itself as. Is this really not possible?
If not, are there any alternative ways this sort of setup is intended to be run using docker?