I have some configuration where users can create nodered instances using docker containers , but i've used one docker container for each instance and i've used nginx as reverse proxy.
Thus where i need to know how much containers can be created in one network and if the number is limited how can i increase it ?
There are a few possible answers to your question:
I dont think Nginx will limit the amount of NodeRed instances you can have.
If you are working on 1 machine with 1 IP address you can change the port number for every NodeRed instance so the limit would be at around 65,535 instances (a little lower as a few ports are already used)
If you are using multiple machines (lets say virtual machines) with only 1 port for NodeRed instances, you are limited to the amount of IP addresses in your subnet.
In a normal /24 subnet (255.255.255.0) there would be 254 IP addresses.
3.1. You can change your subnet in your local network.
https://www.freecodecamp.org/news/subnet-cheat-sheet-24-subnet-mask-30-26-27-29-and-other-ip-address-cidr-network-references/
If you are using multiple machines and are using a wide range of available ports, you have nearly no limit on how many instances you can deploy. The limit would be your hardware i think.
Related
Am trying to implement a cluster of containerised applications in the production using docker in the swarm mode.
Let me describe a very minimalist scenario.
All i have is just 5 aws-ec2 instances.
None of these nodes have a public IP assigned and all have private IPs assigned part of a subnet.
For example,
Manager Nodes
172.16.50.1
172.16.50.2
Worker Nodes
172.16.50.3
172.16.50.4
172.16.50.5
With the above infrastructure, have created a docker swarm with the first node's IP (172.16.50.1) as the --advertise-addr so that the other 4 nodes join the swarm as manager or worker with their respective tokens.
I didn't want to overload the Manager Nodes by making them doing the role of worker nodes too. (Is this a good idea or resource under-utilization?).
Being the nodes are 4 core each, am hosting 9 replicas of my web application which are distributed in the 3 worker nodes each running 3 containers hosting my web app.
Now with this setup in hand, how should i go about exposing the entire docker swarm cluster with a VIP (virtual IP) to the external world for consumption?
please validate my below thoughts:
1. Should I have a classic load-balancer setup like keeping a httpd or nginx or haproxy based reverse proxy which has a public IP assigned
and make it balance the load to the above 5 nodes where our
docker-swarm is deployed?
One downside I see here is that the above reverse-proxy would be Single Point of Failure? Any ideas how this could be made fault-tolerant/hightly available? should I try a AnyCast solution?
2. Going for a AWS ALB/ELB which would route the traffic to the above 5 nodes where our swarm is.
3. If keeping a separate Load Balancer is the way to go, then what does really docker-swarm load-balancing and service discovery is all
about?
what is docker swarm's answer to expose 1 virtual IP or host name to the external clients to access services in the swarm cluster?
Docker-swarm touts a lot about overlay networks but not sure how it
relates to my issue of exposing the cluster via VIP to clients in the
internet. Should we always keep the load balancer aware of the IP
addresses of the nodes that join the docker swarm later?
please shed some light!
On further reading, I understand that the Overlay Network we are creating in the swarm manager node only serves inter container communication.
The only difference from the other networking modes like bridge, host, macvlan is that the others enables communication among containers with in a single host and while the Overlay network facilitates communication among containers deployed in different subnets too. i.e., multi-host container communication.
with this knowledge as the headsup, to expose the swarm to the world via a single public IP assigned to a loadbalancer which would distribute requests to all the swarm nodes. This is just my understanding at a high level.
This is where i need your inputs and thoughts please...explaining the industry standard on how this is handled?
I am going through the Docker tutorials and I'm a bit confused why containers might have different IP addresses than the nodes containing them in a swarm. My confusion is based on the below diagram, from this page in the tutorial.
The bigger green boxes are the nodes in the swarm; they each have their own IP and load balancer, and externally they're listening at port 8080. I believe that the yellow boxes are containers/tasks in the my-web service. They are listening on port 80 and I guess the service is setup to map port 80 from each container to port 8080 externally.
That much I understand more or less, but I don't see why the container/task would have/need a different IP address from the node that it is running on. Can anybody explain this?
If I had to guess, it would be because each container is basically a VM and VMs need their own IP addresses and no two VMs can have the same IP address, therefore the container cannot have the same IP as the node. But I'm not sure if that explanation is correct.
I'm still fairly new to docker/containers myself, but it's my understanding that you're referring to internal IPs and external IPs. Namely that the 192.168.99.100-102 would be externally addressable (aka publicly available), whereas the 10.0.0.1-2 address are for internal addressing only.
The reason for the internal addressing is so that you can have a larger pool of ip addresses to work with for your containers, which is why the 10.0.0.0/8 address space is used. These nodes still need to be addressable, so that your load balancer can correctly distribute the load. And according to the wikipedia entry, you've got 16,777,216 available IPs which allows your swarm to scale to many many containers if you needed it. Whereas you only have a limited number of external IP addresses for your services to be hit on.
I have some experience with Docker Compose and container linking. In a non-swarm environment, you could easily connect from, e.g, the web container to the db_mysql container using its name (for example, in PHP I can configure the MySQL connection to be:
$dsn = 'mysql:host=db_mysql;
I am having a hard time understanding how that works with Docker in Swarm mode, especially considering the "replicas" and "load balancing" mechanisms.
Let's say I have 5 different Docker Machines, each having a different public IP, participating in a Swarm. I also have a web service and a db service that's replicated across these 5 different machines (1 instance per each machine).
My question is: how do I make any of the 5 web containers, communicate to any of the 5 db_mysql containers without forcing these web containers to have knowledge of any Docker Machine public IPs or the fact that these containers live within a Swarm?
You use the service name. This will resolve in DNS to either a VIP or the 5 ip addresses (one for each replica) of the service. Under the covers, the VIP uses IPVS to round robin to one of the healthy replicas without suffering from stale DNS issues. You can also get all the replica IP addresses using service_name.tasks even if you use the default VIP.
In Docker's DNS implementation, you can resolve the container name, and any network alias. The network alias includes the service name with DNSRR (used by docker-compose without swarm). Or the service name resolves to a VIP in swarm mode. The hostname of the container does not resolve, likely because it can change outside of the control (and therefore knowledge) of the docker engine.
Using Docker version 19.03.5 the correct DNS name to query in order to obtain all the IP addresses of the replica of a service is the following:
tasks.<service-name>
all. I'm learning Docker. But still cannot find any documentations about how Docker ingress network connect several separated hosts.
I have 2 VMs in different datacenters and want create swarm cluster on them.
Is it possible that default installed ingress network makes containers on vm1 visible for containers on vm2 inside some overlay network? Or both vm1 and vm2 should be in same local network?
In general, it's not recommended to span datacenters within a Swarm. You can span availability zones (datacenters in same geo area that are ~10ms or less latency) but between regions should be their own Swarms. This is 100% a latency issue of inter-virtual-network traffic (overlay driver) and the Raft consensus traffic between Swarm managers. There is no hard limit on latency, but you likely don't want the complexity in a single Swarm of trying to prevent traffic in your apps from hopping back and forth between datacenters... unless the datacenters are very low latency.
For more data on this look at the Docker Success site (search swarm overlay and filter to reference), as the Docker EE requirements for Swarm are the same as Docker CE generally.
The other requirement between nodes in a Swarm is that they have ports open between each other's public IP's. Ideally, there is no NAT between nodes.
If both hosts are part of the same docker swarm cluster then from perspective of docker it does not matter that they are in different data centers. Routing between services will just work. For example service1 on host1 will be able to access service2 in another data center. You might however need to account for any possibly large latencies that would occur because of physical distance of hosts.
It is also the same story with the ingress network. It does not care that there are 2 data centers. Any swarm cluster node will participate in it and route incoming requests to the correct service/host.
Elasticsearch is designed to run in cluster mode, all I have to do is to define the relevant node IPs in the cluster via environment variable and as long as network connectivity is available it will connect and join the other nodes to the cluster.
I have 3 nodes, 1 is acting as the docker swarm manager and the other two are workers. I have initialized the manager and joined the worker nodes and everything looks ok from that standpoint.
Now I'm trying to run the elasticsearch container in a way that will allow me to join all nodes to the same elasticsearch cluster, however, I want the nodes to join using their overlay network interface and that means that I need to know the container internal IP addresses at the time of running the docker service create command, how can I do this? Do I have to use something like consul to achieve this?
Some clarifications:
I need to know, at the time of service creation the IP addresses (or DNS names) for all Elasticsearch participants so I could start the cluster correctly. This has to be at the time of creation and not afterwards. Also, as I understand, I can expose ports 9200/9300 for all services and work with the external machine IPs and get it to work, but I would like to use the overlay network to do all these communications (I thought this is what swarm mode is for).
Only a partial solution here.
So, when attaching your services to a custom overlay network, you indeed have access to Docker's custom Service discovery feature. I'll detail the networking feature of Docker Swarm mode, before trying to tie it to your problem.
I'll be using the different term of services and tasks, in which a service could be elasticsearch, whereas a task is a single instance of that elasticsearch service.
Docker networking
The idea is that for each services you create, docker assigns a Virtual IP (VIP), and a custom dns alias. You can retrieve this VIP using the docker service inspect myservice command.
But, there is two modes to attach a service to an overlay network dnsrr and VIP. You can select these options using the --endpoint-mode options of docker service create.
The VIP mode (I believe it is the default one, or at least the most used), affects the virtual ip to the service's dns alias. This means that doing an nslookup servicename would return to you a single vip, that behind the scenes, would be linked to one of your container in a round robin fashion. But, there is also a special dns alias that lets you access all of your instances ips (all of your tasks ips) : tasks.myservice.
So in VIP mode you can retrieve all of your tasks ips using a simple nslookup tasks.myservice, where myservice is a service name.
The other mode is dnsrr. This mode simply gets rid of the VIP, and connects the dns alias to the different tasks (=service instances), in a round robin way. This way, you simply have to do a nslookup myservice to retrieve the different service instances ip.
Elasticsearch clustering
Ok so first of all I'm not really familiar with the way elasticsearch lets you cluster. From what I understood from your question, you need when running the elasticsearch binary, give it as a parameter, the adress of all of the other nodes it needs to cluster with.
So what I would do, is to create a custom Elasticsearch image, probably based on the one from the default library, to which I would add a custom Entrypoint that would firstly run a script to retrieve the other tasks ip.
I'd believe that staying in VIP mode is suitable for you, since there is the tasks.myservice dns alias. You'll then need to parse the output to retrieve the tasks ip (and probably remove yours). Then you'll be able to save them in a config file environment variable, or use them as a runtime option for your elasticsearch binary.
Edit: To create a custom overlay network, you will need to use the docker network create command, and use the --network option of docker service create
This is answer is mainly based on the Swarm mode networking documentation