Creating HazelCast cluster running on multiple docker containers - docker

Does anyone know what configuartion needs to be done in Hazelcast.xml if we want to form Hazelcast cluster between its instances running on multiple docker containers. Should we provide 127.0.0.1 as address of member or the address should be that of docker host? Also does local.localAddress property needs to point to docker host address?
Edit :
We did some changes by setting the public ip and is able to form cluster but with the limitation that while defining port mapping in docker run command port of host should be same as port of container if we set different port of host and map it to hazelcast port like 8047:5071 it doesn't work it has to be 5701:5701, any idea why such a behaviour

you can set the public-addressproperty in hazelcast config as the ip of the host machine . This will allow the nodes to join the cluster.
<network>
<public-address>host-machine-ip</public-address>
</network>

Related

Can we use a DNS name for a service running on a Docker Swarm on my local system?

Say I have a Swarm of 3 nodes on my local system. And I create a service say Drupal with a replication of 3 in this swarm. Now, say each of the node has one container each running Drupal. Now when I have to access this in my browser I will have to use the IP address of one of the nodes <IP Address>:8080 to access Drupal.
Is there a way I can set a DNS name for this service and access it using DNS name instead of having to use IP Address and port number?
You need to configure the DNS server that you use on the host making the query. So if your laptop queries the public DNS, you need to create a public DNS entry that would resolve from the internet (on a domain you own). This should resolve to the docker host IPs running the containers, or an LB in front of those hosts. And then you publish the port on the host to the container you want to access.
You should not be trying to talk directly to the container IP, these are not routeable from outside of the docker host. And the docker DNS used for service discovery is for container to container communication. This is separate from communication outside of docker that goes through a published port.

How docker containers expose services?

I'm deploying a stack of services through the command:
docker stack deploy -c <docker-compose.yml> <stack-name>
And I'm mapping ports of one of these services on docker compose with ports: 8000:8000.
The network driver being used is overlay.
I can access these services via localhost:8000, via Peers IP(?).
When I inspect the network created, I can see the local IPs of each container (for instance, 10.0.1.2). But Where is the external IP of container (the one like 172.0. ...) ?
I am running these docker container on a virtual machine ubuntu.
How can I access the services running on containers from other nodes running on other networks? Isn't possible to access via hostIP:port?
If so, how do I get the host IP? When I do docker-machine IP I get "host is not running".
[EDIT: I wasn't doing port mapping between the host and the VM in virtualbox. Now it works!]
Whats the best way to communicate between containers on the same swarm?
Thanks
Whats the best way to communicate between containers on the same swarm? Through name discovery?
In general if you communicate between containers you should use the container/service name.
And for your other problem you probably wan't a reverse proxy like nginx or traefik.

Can we have two or more container running on docker at the same time

I have not done any practical with the docker and container, But as per my knowledge.
As per the documents available online I did not get the details about the running two or more containers at the same time.
Docker allows container to map port address of container to the host machine.
Now, the question is can we run multiple container at the same time on docker? if yes then if two containers are mapped to same port number then how does the port is handled in this case?
Also out of curiosity, can two containers on docker communicate with each other?
Yes you can run multiple containers on a single host; docker is designed for exactly that.
You cannot map two containers of different images to the same port number; you get an error response if you try. However, if your containers run the same image (e.g.2 instances of a webapp) you could run them as a service, and have them exposed on the same port. Docker will load-balance the requests. You can read more about services here or follow the Get Started (Part 3, services) here
Yes, the containers on a single host can communicate with each other, by container name. For example if you have one container running MongoDB called mongo, and another one running Node.js called webserver, the webserver container can connect to the database by using the name mongo e.g. db.Connect("mongodb://mongo:27017/testdb").
We can run more one than one Docker at a time in a host but yes we will hit the limitation of binding the same port to the docker; so to resolve this we need to bind different port in the host to docker that is if you are running mongo-db then its default port is 27017 so we can run two mongo-db as -p 27017:27017 for Docker D1 and -p 27018:27017 for Docker D2 and 5000:27017 for docker D3; Like this you can bind different host port to map to 27017 for mongo-db port; Now your question is how to manage this ports from host then I would recommend you to use nginx for port managing in the host machine.
Coming to your next question all dockers are connected to default docker0 bridge network so we can connect to any of the dockers connected to default bridge 'docker0' network; If I am right it will come with ipaddress of 172.x.x.x network. Get inside to the docker and run 'ip addr' to see the ip-address assigned to the dockers and you can test connection by running ping command.
Yes two containers can run same time, they can also communicate with each other also, you can define your own network and they can communicate with each other. if two containers have their private ports, they are their internal ports, one container port does not collide with another container port. if you want to expose the port to host, then you have to publish the port(s).

docker containers static IP to communicate two different hosts

is it possible to change the ip of docker0 or provide a static IP to docker containers, because by default docker containers have the ip range of 172.17.0.2/16 but my network is 192.168.X.X/24 in this situation on the server container is running there all the containers is able to communicate within servers but from other server this failed to connect.
How do you set up your cluster? Do you use Swarm? If so, you need to use a k/v storage backend to enable communication between two containers hosted on different hosts. Is this what you aim to do, or do you want the host to communicate with the container on the other host?
Anyway, the solution is similar.
I re-writing a tuto for Docker Swarm to pull request it into their Swarm doc, you may want to take a look: https://www.auzias.net/en/docker-network-multihost/
Have a nice day!
problem can be fix by using --network=host
this will allow your container to use the host machine network. for direct accessing your container you can change the ssh port of the container and access your container with the specific port number.
I answered a similar question here
https://stackoverflow.com/a/35359185/4094678
The difference in your case would be to create a netowrk with subnet 192.168.X.X/24 and then assign desired ip addr to container with --ip
Here we can't able to change docker0 Ip address, but we have option to create multiple networks.
Solution 1:
can be by using start container with host network --network=host
Solution2:
we can also start the container by exposing the cluster required port and from another node we can communicate it.
-p hostport:serviceport
Or, Solution3:
We can deploy cluster over docker swarm.

Hazelcast TCP/IP discovery on DCOS/ Marathon and docker

I have a dockerized dropwizard service deployed on marathon. I am using Hazelcast as a distributed cache which I start has part of my dropwizard service. I have placed a constraint to ensure that each container is started on a unique Host.
"constraints": [
[
"hostname",
"UNIQUE"
]
],
I have exposed 2 ports on my docker container, 10012 for my service and 10013 for Hazelcast. I am using Zookeeper for my Dropwizard service discovery. Thus when I startup my Hazelcast instance I have access to the hostnames of all the machines on which my docker containers are running and I add all of them as below.
TcpIpConfig tcpIpConfig = join.getTcpIpConfig();
// finder is a handle to a service discovery service and the following gets me all the hosts on which my docker containers will run.
List<ServiceNode<ShardInfo>> nodes = finder.getAllNodes();
nodes.stream()
.peek(serviceNode -> log.info("Adding " + serviceNode.representation() + " to hazelcast."))
.map(serviceNode -> serviceNode.getHost())
.forEach(host -> tcpIpConfig.addMember(host));
tcpIpConfig.setRequiredMember(null).setEnabled(true);
Now issues:
If I use network type as BRIDGE while deploying on Marathon, then I don't know the docker container host and thus my 2 docker containers don't know each other. It looks something like this:
ip-10-200-2-219.ap-southeast-1.compute.internal (docker host) - 172.12.1.18 (docker container ip)
ip-10-200-2-220.ap-southeast-1.compute.internal (docker host) - 172.12.1.20 (docker container ip)
From zookeeper I get the docker host IPs but not the docker container IPs.
If I use network type as HOST then everything works but an issue is that I then have to make sure that ports which my docker containers are running always have port 1001 and 10013 available. (With BRIDGE the docker container ports are bound to a random ports).
Analysis:
The two docker containers are inside their own network localized to the slaves. They need to recognise each other using the public IP of the slave and the bridged port to which 5701 (or whatever hazelcast port you are using).
Solution
In the TCP/IP configuration set the Public Address and Port when starting the instance. All instances will do this and they will talk to each other using the marathon slave IP and the randomized port used to it.
Use the HOST and PORT_5701 variables provided by marathon and available inside the container to do this.
Config hzConfig = new Config();
hzConfig.getNetworkConfig().setPublicAddress(
String.format("%s:%s",
System.getenv("HOST"),
System.getenv("PORT_5701")));
Refer to hazelcast network config documentation to understand a bit more about the public address option.
You can use TCP/IP Discovery mechanism and make sure hazelcast nodes bind to public ip of docker container. Although this solution is might help if you know docker container IPs, before your deployment.
<hazelcast>
...
<network>
...
<join>
<multicast enabled="false">
</multicast>
<tcp-ip enabled="true">
<member>docker-host1</member>
<member>docker-host2</member>
<member>172.12.1.20</member>
<member>192.168.1.21</member>
</tcp-ip>
...
</join>
...
</network>
...
</hazelcast>
Getting hazelcast members to discover each other and form a cluster is tricky. I ended up following advice of #bitsofinfo -
https://github.com/hazelcast/hazelcast/issues/9219
#santanu 's answer is correct. Public address needs to be set properly for hazelcast members to be able to discover each other. Here's a parameterized way of doing this : https://github.com/gagangoku/hazelcast-docker

Resources