We have a multiservice architecture consisting of HAProxy front end ( we can change this to another proxy if required), a mongodb database, and multiple instances of a backend app running under Docker Swarm.
Once an initial request is routed to an instance ( container ) of the backend app we would like all future requests from mobile clients to be routed to the same instance. The backend app uses TCP sockets to communicate with a VoIP PBX.
Ideally we would like to control the number of instances of the backend app using the replicas key in the docker-compose file. However if a container died and was recreated we would require mobile clients continue routing to the same container. The reason for this is each container is holding state info.
Is this possible with Docker swarm? We are thinking each instance of the backend app when created gets an identifier which is then used to do some sort of path based routing.
HAproxy has what you need. This article explains all.
As a conclusion of the article, you may choose from two solutions:
IP source affinity to server and Application layer persistence. The latter solution is stronger/better than the first but it requires cookies.
Here is an extras from the article:
IP source affinity to server
An easy way to maintain affinity between a user and a server is to use user’s IP address: this is called Source IP affinity.
There are a lot of issues doing that and I’m not going to detail them right now (TODO++: an other article to write).
The only thing you have to know is that source IP affinity is the latest method to use when you want to “stick” a user to a server.
Well, it’s true that it will solve our issue as long as the user use a single IP address or he never change his IP address during the session.
Application layer persistence
Since a web application server has to identify each users individually, to avoid serving content from a user to an other one, we may use this information, or at least try to reproduce the same behavior in the load-balancer to maintain persistence between a user and a server.
The information we’ll use is the Session Cookie, either set by the load-balancer itself or using one set up by the application server.
What is the difference between Persistence and Affinity
Affinity: this is when we use an information from a layer below the application layer to maintain a client request to a single server
Persistence: this is when we use Application layer information to stick a client to a single server
sticky session: a sticky session is a session maintained by persistence
The main advantage of the persistence over affinity is that it’s much more accurate, but sometimes, Persistence is not doable, so we must rely on affinity.
Using persistence, we mean that we’re 100% sure that a user will get redirected to a single server.
Using affinity, we mean that the user may be redirected to the same server…
Affinity configuration in HAProxy / Aloha load-balancer
The configuration below shows how to do affinity within HAProxy, based on client IP information:
frontend ft_web
bind 0.0.0.0:80
default_backend bk_web
backend bk_web
balance source
hash-type consistent # optional
server s1 192.168.10.11:80 check
server s2 192.168.10.21:80 check
Session cookie setup by the Load-Balancer
The configuration below shows how to configure HAProxy / Aloha load balancer to inject a cookie in the client browser:
frontend ft_web
bind 0.0.0.0:80
default_backend bk_web
backend bk_web
balance roundrobin
cookie SERVERID insert indirect nocache
server s1 192.168.10.11:80 check cookie s1
server s2 192.168.10.21:80 check cookie s2
Related
I'm new to AWS.
I started learning about ALB and NLB. I know ALB working in Layer 7 protocols and NLB working in layer 4 protocols.
Can anyone please explain the real time example of ALB and NLB?? When to use ALB and NLB??
Even though all the web application will use TCP protocols for making connection between server and client.
So Is ALB use the TCP (layer 4) protocols??
Then what is the different between them? Can anyone please explain briefly???
In summary: an NLB only knows about TCP, while an ALB knows everything about the request.
An NLB can only route a request based on IP addresses and other TCP-package info.
An ALB can route a request by looking at the content of it: what protocol is it using (HTTP, HTTPS)? What path is it trying to query (/api/v1, /api/v2)? What content-type is it requesting?
So, if you want requests for the v1 API endpoint to be routed to an autoscaling group of EC2 instances and requests for the v2 API endpoint routed to another group of instances, then your best option is the ALB because it allows you to configure rules that make your desired routing possible.
On the other hand, if you just want that clients coming from Germany are routed to one autoscaling group and clients from the USA to another group, the NLB should be sufficient because you can set up rules that match the IP addresses of those countries.
TL;DR To load balance HTTP requests, use an ALB. For TCP/UDP load balancing, use an NLB.
An ALB (Application Load Balancer) understands HTTP. If you need to do HTTP-based routing (e.g., routing to different targets depending on the request path) you need to use an ALB.
Unique features of ALBs include:
HTTP path-based routing
HTTP header-based routing
Redirects
Lambda functions as targets
An NLB (Network Load Balancer) operates at the transport level (TCP/UDP). NLBs are more performant than ALBs because they don't need to parse HTTP messages.
NLBs support some unique features too:
Static IP
Elastic IP addresses
Preserving the source IP
You can see a full comparison of features on the Elastic load balancing features page.
I am trying to setup load balanced environment having 2 application server instances. I am unable to make spring web socket relay messages to all instances. Let's take a example to describe my problem better:
Server 1 : Responsible for job executions and 35% user load.
Server 2 : 100% user load.
Both are connected to same database schema so job request can come for any server instance but will get executed on Server 1.
Now, I have used spring web socket plugin for my GRAILS application and I push messages to browser using
brokerMessagingTemplate.convertAndSend(user.notificationChannel, ((notification.toMap(user) as JSON)).toString())
It was working fine on single server setup. But on multi-server setup,
notifications are only received on Server 1 as that is the one calling the code block, if reverse the scenario, then vice-versa result is observed.
How can I push same notification to all server instances, so that user always gets the notification no matter what server instance he is on?
I initially thought of utilising a common queue like RabitMQ but that will add to system requirements and will get disapproved by client.
NOTE: Third party service solutions won't work in my case as applications are on intranet and don't have internet access.
websockets by default point to a hostname/ip address - whilst you could setup a dns record / hostname that points to multiple different ip's / servers. This itself would break communication flow of the websockets if it sent handshake to one and the message to another.
The most simplest approach would be to think of some db table that is shared across both and as each instance comes up/alive it records its local ip / socket port to a db table - each instance can then read this table and work out at any point which are the hosts to transmit a socket message to - (this table would need to managed somehow - upon a brand new bootup ) it would be empty and would popuplate as instances came up - something again to manage when a host is taken down shutdown.
Each instance would then be running an ws internal client. When a message is sent the ws client would be triggered attempting to find all alive websocket servers "from the db" to each using the ws client it would attempt to connect and send the message on. Each would then get the message and either broadcast to all connected users or if it is from user x meant for user x then like per chat plugin it would relay it only to user x if found on server y and so on.
this then keeps it all inline with 1 technology controlling the entire process websocket server that has its own client which relay to the end multiple instance ws socket server
This seems to have something to do with the subnet/availability zone, but I'm new to using a VPC and it's eluding me.
VPC: 10.80.0.0/16
subnet: 10.80.1.0/24 (us-east-1b)
subnet: 10.80.2.0/24 (us-east-1a)
All instances are Windows Server 2012.
I have an internet facing ELB created within my VPC (10.80.0.0/16). There is one instance added from AZ us-east-1a, which is on subnet 10.80.2.0/24. The instance is running IIS 7.5, with an app running on port 80 and /health.aspx set up for use as the ELB health check.
Internal traffic on the VPC is flowing normally (unrestricted). I can request health.aspx from this instance from another instance in us-east-1b (10.80.1.0/24). I can also copy files from one instance to another.
Outbound traffic is unrestricted. I can RDP to the instance (when connected to our VPN) and open a browser and request a web page and get it.
The ELB says the instance is healthy and I can see the requests to health.aspx in the IIS logs. Both the ELB and the instance are configured with a security group that allows 80 and 443.
But if I try to request {elb-url}/health.aspx over the open internet the request just times out. Similarly, with an elastic IP associated to the instance, a request to {elastic-ip}/health.aspx times out.
#Chris, thanks for the response...as it happens, I've already worked it out with some help from a friend. I'll post my findings here for posterity (in case anybody else was similarly confused about how ELB works).
This would be more clear with a diagram. But the summary is that in each availability zone, you need to create both a public and a private subnet. When you add availability zones to your ELB, you need to select the public subnet for the zone. This had already been done in us-east-1b before I got to this setup, and I had simply missed this nuance of ELB configuration. So for the new availability zone, I had to do this...
us-east-1c
private subnet 10.1.3.0/24 (using nat instance as default route)
public subnet 10.1.4.0/24 (using internet gateway as default route)
Then my instance goes in the private subnet as expected.
And the lynch pin of this whole thing is (drum roll....)
When I add us-east-1c to my ELB, I have to select the public subnet...10.1.4.0. Otherwise the instances will pass the health check (since the ELB can communicate with any instance within my entire VPC) but the responses from the servers cannot make it back out to the public internet.
This is what is so confusing. And I still don't fully understand it. The instance can make a request for, say, www.google.com. I can RDP to it and open a browser and get the web page. But a request from a host (like my laptop at my house) will die. strange.
PS: another note...make sure you are using enough NAT instance for your load. I think we ran into an issue where our NAT instance simply ran out of ports because too many web servers were trying to route outbound connections to 3rd party APIs through it. Quite honestly, I'm not good enough at this level of network/OS troubleshooting to be sure. But my theory is that our 8 instances of IIS were holding too many connections open to the NAT instance. We were also abusing the NIC on that micro instance. I upped us to two large instances, one per AZ and things smoothed back out. Both NAT instances are humming and we're not seeing the hung processes in IIS anymore.
Debugging this kind of issue is always a challenge. I have a few ideas to suggest based on what you have written (and generally apply to trying to solve this problem) that come from dealing with this a number of times.
Have you checked both the security groups and network ACLs? Bear in mind that all network ACLs need to be specified in both directions, as they are stateless. Also bear in mind that ELBs are a bit unique in this regard. While they are associated with your VPC, they sometimes need extra rules to ensure connectivity. In the past I have debugged this by opening all network ACLs on all ports, then removing these rules until it has stopped working in order to identify where the block was.
Security groups should be checked too. They are stateful but ensure that your load balancer has permissions to be hit from the web.
Have you checked this isn't an application configuration problem? I don't know how IIS comes out of the box but I would check it is setup to respond to all hostnames.
Check the ELB isn't an internal one, as that wouldn't be publically addressable.
You say the ELB is configured with the health check, but it's worth checking you also have the listener setup for port 80? It's in a separate tab on the dashboard and you will need this in addition to the health check for connectivity through the ELB.
Hope one of these tips is useful to you.
Struts 2 support stop double-submission of forms by generate a unique random token and store it in the session, and use token tag pass the token to the client form then verify the tokens from session and form.
As far as i know, this solution can only work in a single JVM because the session is separated from each other. I cannot find something useful about how to use this solution in a distributed Java environment. We use Nginx proxy HTTP requests to multiple JVMs and Nginx does not guarantee to proxy the same request to the same JVM every time.
Can someone give me some help?
BTW, i am trying to use this solution to stop CSRF attack.
You have two choices (neither of which really has anything to do with Struts 2, but has everything to do with session management in a distributed environment):
Use Session Affinity - so when a user creates a session, Nginx remembers which backend server the user went to, and that session is bound to that server for all subsequent requests. (This is the more typical solution). might get you started.
Depending on your application server, there may be the possibility of sharing the session data between servers. For example, in Tomcat 6, the configuration directions are.
I am creating a blogging-like application where we allow our customers to use their own custom domain names such as domainexample.com, so each different domain serves the same application but with different content.
However I am struggling to figure out how to set this up on a production server. If my production server has a static IP then I can surely just set an a-record on each domain to the ip of the production server.
But what if the production server does not have a static IP. For example if we want to host it on heroku or engineyard? I have seen a few solutions online that require using rewrite rules but they require server restarts and cant really dynamically add and remove new domains as new users sign up. Does anyone know any good solutions to let multiple domains hit one rails app?
Heroku isn't your only option. If you can anticipate your customer's domains, have a look at this. If you can't, Rails routes constraints and a combination of the accepted answer to the question linked above should get you where you need to be going. Sounds like you wouldn't want to restart your server--so no editing of the routes. You might also make domains part of your models, or distinguish at the controller level or use URL rewriting in your web-server layer.
The problem, as I see it, is that Rails breaks its mantra of opinion over configuration here. There are many ways of serving up from multiple domains. That might be an intrinsic complexity, but the Rails Guides could at least document one possible solution.
If your customers just CNAME to your domain or create the A record to your IP and you don't handle TLS termination for these custom domains, your app will not support HTTPS, and without it, your app won't work in modern browsers on these custom domains.
You need to set up a TLS termination reverse proxy in front of your webserver. This proxy can be run on a separate machine but you can run it on the same machine as the webserver.
CNAME vs A record
If your customers want to have your app on their subdomain, e.g. app.customer.com they can create a CNAME app.customer.com pointing to your proxy.
If they want to have your app on their root domain, e.g. customer.com then they'll have to create an A record on customer.com pointing to your proxy's IP. Make sure this IP doesn't change, ever!
How to handle TLS termination?
To make TLS termination work, you'll have to issue TLS certificates for these custom domains. You can use Let's Encrypt for that. Your proxy will see the Host header of the incoming request, e.g. app.customer1.com or customer2.com etc., and then it will decide which TLS certificate to use by checking the SNI.
The proxy can be set up to automatically issue and renew certificates for these custom domains. On the first request from a new custom domain, the proxy will see it doesn't have the appropriate certificate. It will ask Let's Encrypt for a new certificate. Let's Encrypt will first issue a challenge to see if you manage the domain, and since the customer already created a CNAME or A record pointing to your proxy, that tells Let's Encrypt you indeed manage the domain, and it will let you issue a certificate for it.
To issue and renew certificates automatically, I'd recommend using Caddyserver, greenlock.js, OpenResty (Nginx).
tl;dr on what happens here;
Caddyserver listens on 443 and 80, it receives requests, issues, and renews certificates automatically, proxies traffic to your backend.
How to handle it on my backend
Your proxy is terminating TLS and proxying requests to your backend. However, your backend doesn't know who is the original customer behind the request. This is why you need to tell your proxy to include additional headers in proxied requests to identify the customer. Just add X-Serve-For: app.customer.com or X-Serve-For: customer2.com or whatever the Host header is of the original request.
Now when you receive the proxied request on the backend, you can read this custom header and you know who is the customer behind the request. You can implement your logic based on that, show data belonging to this customer, etc.
More
Put a load balancer in front of your fleet of proxies for higher availability. You'll also have to use distributed storage for certificates and Let's Encrypt challenges. Use AWS ECS or EBS for automated recovery if something fails, otherwise, you may be waking up in the middle of the night restarting machines, or your proxy manually.
Alternatively, there have been a few services like this recently that allow you to add custom domains to your app without running the infrastructure yourself.
If you need more detail you can DM me on Twitter #dragocrnjac