I have a docker container which has Flask application running.
I want to use this API for connection. I am very new to Kerberos and I have very little practical knowledge on it.
My host system will have a Kerberos client installed which can connect to hadoop successfully. Can I somehow use the host system TGT to connect to Hadoop? I have looked at almost all the resources available online and I understand how Kerberos and Hadoop interact. I would just like to know how do I make it work from inside the docker container.
Related
I have been able to set up containerised RabbitMQ server, and reach into it with basic .NET Core clients and check message send and receive working using management portal on http://localhost:15672/.
But I am having real frustrations when I also Containerise my Sender/Receiver .NET Core clients, on being able to establish a connection. I have set up an explicit "shipnetwork", so all containers in the following docker-compose deployment should see each other.
This is the Error I get in the sender attempting the connection:
My SendRabbit .NET core App is as follows. This code was working on my local Windows 10 development machine, with a host of 'localhost' against the RabbitMQ server running as a container. But when I change this to a [linux] docker project, and set the host to "rabbitmq", to correspond to the service name in the docker compose. Now I just get Endpoint Connection errors exceptions within my Sender container.
I have also attempted the same RabbitMQ server and Sender Image with the same docker-compose on a Google Cloud Linux Virtual Machine, and get the same errors. So I do not think it is the Windows 10 docker hosting VM environment hassles.
I thought docker was going to make development and deployment of microservices, but setting up a basic RabbitMQ connections is proving to be a real pain.
I have thought that maybe the rabbitmq server is not up and running, so perhaps ambitious to put in the same docker-compose. But I have checked running my SendRabbit container
$docker run --network shipnetwork sendrabbit
some minutes later. But I still get the same connection error
docker networks **** networks !
When I checked the actual docker networks, I had:
bridge
host
shipnetwork
rabbitship_shipnetwork
The docker compose was actually creating the 'new' network: rabbitship_shipnetwork every time it was spun up, and placing the rabbimq server on that network. The netwrok is named from appending the directory name, with the name in the compsos yaml. So I was using the wrong network in my senders. So I should have been using
$docker run --network rabbitship_shipnetwork sendrabbit
This works fine, and creates messages into the rabbitmq server
So I don't feel that docker-compose is actually very helpful in creating networks, since it is sensitive to the directory name it is run in ! Its unlikely that I can build an app .docker files, and deploy all Apps from a single directory, especially when rabbitmq has to be started separately, before senders and receivers can use it.
docker-compose 0
I am a newbie to docker. When I go through docker tutorial, I saw that "Docker client can communicate with more than one daemon". What does that mean exactly?
By default, the Docker daemon listens on a Unix socket, /var/run/docker.sock. However, Docker can also be configured to listen on a TCP socket. In fact, it is often configured this way on Mac and Windows systems because Docker is actually running inside a virtual machine and the default Docker socket is not available on the host filesystem.
Because there are different ways of connecting to Docker, you must be able to configure the Docker client to connect to a Docker daemon at a specific location. You can do this using the DOCKER_HOST environment variable. You can point this at a network location:
export DOCKER_HOST=tcp://192.168.99.101:2376
Or at an alternate socket location:
export DOCKER_HOST=unix:///tmp/docker.sock
If you have Docker configured to listen for tcp connections, you can use the Docker client on a single machine to communicate with Docker on multiple hosts (but if you decide to do something like this, read through "Protect the Docker daemon socket").
Per the Docker Documentation,
The Docker client can communicate with more than one daemon.
This means that the command-line utility docker can connect to different services that run in the background,
Docker uses a client-server architecture. The Docker client talks to the Docker daemon, which does the heavy lifting of building, running, and distributing your Docker containers.
So for example, you could configure the daemon to run on a separate machine and connect to it from your workstation.
I am writing a small application with flask which is meant to interact with the docker api in order to run containers on demand. I would like to deploy this application within a docker container. However, I understood that it is relatively bad to mount the docker socket, as it has root privilege on the local host.
Is there a proper method to access the docker api within a container in order to avoid this caveat ?
Why is mounting the Docker socket to an unprivileged container a bad idea?
In order to mount the unix socket to your Docker container, you would need to change the permissions of the Docker daemon socket. This, obviously, could give non-root users the ability to access the Docker daemon, which might be a problem if you are worried about privilege escalation attacks. (source)
Do I really need to secure the Docker socket?
This depends on your usecase. If you have many users on your server, and are particularly worried about a non-privileged user affecting your app, then definitely secure the socket. If this is a virtual machine that is completely dedicated to the app, insecure might be easier.
How do I interact with the socket insecurely?
Just change the permissions (described here) and then mount the socket to the container. It's that simple.
How do I interact with the socket securely?
I think there are two good ways of doing this:
Restart the Docker Daemon with TLS Authentication enabled. Rather than accessing the unix socket, access it using HTTPS with a signed SSL key. More instructions on setting that up can be found here.
Use an Authorization Plugin on the unix socket as described here.
When people talk about the 'Docker Engine' do they mean both the Client and the Daemon? Or is it something else entirely?
As I see it there is a Docker Client, a Docker Daemon. The Client runs locally and connects to the Daemon which does the actual running of the containers. The Client can connect to a remote Daemon. Are these both together the Engine? thanks
The Docker Engine is the Docker Daemon running on a single host, installed with the Docker Client CLI. Here are the docs that answer this specific question.
On top of that, you can have a Swarm running that joins multiple hosts to horizontally scale and provide fault tolerance. And there are numerous other projects from Docker, like their Registry, Docker Cloud, and Universal Control Plane, that are each separate from the engine.
Docker engine is a client-server application which comprises of 3 components.
1. Client: Docker CLI or the command line window that helps us to interact.
2. REST API: Client communicate with the server with REST API, the commands issued by the client is sent to the server in the form of REST API, it is this reason our server can either be in the local or remote machine.
3. Server: Server here is either the local or remote machine or host machine which has a daemon process running in it which receives the commands and creates, manages and destroys the docker objects like images, containers, volumes etc.
I have some docker containers running on Bluemix using private IP addresses. I would now like to setup a tunnel from my laptop (running linux) to access the private network on Bluemix.
I had first created a container running an ssh-server. Using ssh -D I was able to setup a SOCKS5 proxy connection. This worked fine with Chrome but not all applications support a SOCKS proxy.
(google-chrome --proxy-server=socks5://localhost:<tunnel port>)
So I tried to create a container with an OpenVPN server. Unfortunately this does not work on Bluemix as the containers are not running privileged and thus can not create a tun device.
Bluemix also has a VPN and a Secure Gateway service, which sound promising but so far I could not figure out how to get those working.
Does anybody know if it is possible to make the private docker network available locally and how to connect to that?
Generally speaking containers should be used to implement services available to external applications (an APIs service, or a runtime, or a dbms, or something like that).
According to this, what you could achieve is a set of services available for you on different containers, and a single container working as SSH tunnel gateway, making your local environment connected to it using SSH and defining a set of local and remote SSH ports forwarding, with different policies according to the service/port and the IP of the service.
It should work for all the services, and you haven't to use a socks proxy to forward requests to different hosts: using remote SSH forwarding your SSH endpoint will redirect your requests to the right service inside the local/private lan.
I found that this guide describes correctly how to work with local&remote port forwarding.
http://www.debianadmin.com/howto-use-ssh-local-and-remote-port-forwarding.html
About the OpenVPN solution, as you already know it is not possible to use software requiring privileged mode on containers, because it couldn't be allowed on Bluemix due to security reasons: if you wish to have this kind of solution I strongly suggest you to use OpenVPN on a VM on Bluemix UK region (still beta but an architecture expected to be the final architecture as soon as VM service will become GA service)
I think that these options are the ones available on Bluemix to achieve what you describe without using the VPN service suggested by #bill-wentworth