We are using MarkLogic Rest Api in order to send/update the data on to marlogic database server. Also we are using transactions for multi-statement commit/rollback. So when we pass the request via a load balancer to marklogic to get the transaction id and associate that transaction id with the subsequent requests and finally we are trying to commit the request with the same transaction id but load balancer throws a error Bad request (Load Balancer doesn't recognize the request as it fails).
We are creating a transactions by calling
http://host:port/version/transactions API via POST method and this
will return transaction id in the Location response header.
Then we will associate a transaction id with the document
insert/update via eval (http://host:port/version/eval) service api.
After that commit a transaction created by making a POST request to
the /transactions service, send a POST request to the
/transactions/{txid} service with a URL of the form:
http://host:port/version/transactions/txid?result=outcome
How to get the session cookie from load balancer ?
How to pass that to subsequent requests via c# code ?
The general pattern for configuring the load balancer is documented here:
https://docs.marklogic.com/guide/rest-dev/transactions#id_42381
Questions on configuring a specific load balancer are best directed to the vendor or community for that load balancer.
On an unrelated note, the eval endpoint should be used only as a convenience during development and not in production because of the security risk.
As an alternative, consider either installing a main module and using the invoke endpoint or using the out-of-the-box documents endpoint.
Hoping that helps,
You need to enable Session Affinity (or session stickiness) in your load balancer for the SessionID cookie, and ensure that it is maintained and used by the (c#) client.
For your ELB, an example is provided in the Resources Declaration section of the MarkLogic Server on Amazon Web Services (AWS) Guide
ElasticLoadBalancer is the Load Balancer for all of the ASGs. For details on the AWS::ElasticLoadBalancing::LoadBalancer type, see http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-elb.html.
ElasticLoadBalancer:
Type: 'AWS::ElasticLoadBalancing::LoadBalancer'
DependsOn:
- VpcStack
- ElbSecurityGroup
Properties:
AppCookieStickinessPolicy:
- CookieName: SessionID
PolicyName: MLSession
SecurityGroups:
- !Ref ElbSecurityGroup
Subnets:
- !GetAtt [VpcStack, Outputs.PublicSubnet1Id]
- !If [MultiZone, !GetAtt [VpcStack, Outputs.PublicSubnet2Id], !Ref 'AWS::NoValue']
- !If [MultiZone, !GetAtt [VpcStack, Outputs.PublicSubnet3Id], !Ref 'AWS::NoValue']
ConnectionDrainingPolicy:
Enabled: 'true'
Timeout: '60'
CrossZone: 'true'
Transaction Management When Using a Load Balancer
This section applies only to client applications that use multi-statement transactions and interact with a MarkLogic Server cluster through a load balancer. For additional general-purpose load balancer guidelines, see Connecting Through a Load Balancer.
When you use a load balancer, it is possible for requests from your application to MarkLogic Server to be routed to different hosts, even within the same session. This has no effect on most interactions with MarkLogic Server, but operations that are part of the same multi-statement transaction need to be routed to the same host within your MarkLogic cluster. This consistent routing through a load balancer is called session affinity.
Most load balancers provide a mechanism that supports session affinity. This usually takes the form of a session cookie that originates on the load balancer. The client acquires the cookie from the load balancer, and passes it on any requests that belong to the session. The exact steps required to configure a load balancer to generate session cookies depends on the load balancer. Consult your load balancer documentation for details.
To the load balancer, a session corresponds to a browser session, as defined in RFC 2109 (https://www.ietf.org/rfc/rfc2109.txt). However, in the context of a Java Client API application using multi-statement transactions, a session corresponds to a single multi-statement transaction.
The Java Client API leverages a session cookie to preserve host affinity across operations in a multi-statement transaction in the following way. This process is transparent to your application; the information is provided to illustrate the expected load balancer behavior.
When you create a transaction using DatabaseClient.openTransaction, the Java Client API receives a transaction id from MarkLogic and, if the load balancer is properly configured, a session cookie from the load balancer. This information is cached in the Transaction object.
Each time you perform a Java Client API operation that includes a Transaction object, the Java Client API attaches the transaction id and the session cookie to the request(s) it sends to MarkLogic. The session cookie causes the load balancer to route the request to the same host in your MarkLogic cluster that created the transaction.
When MarkLogic receives a request, it ignores the session cookie (if present), but uses the transaction id to ensure the operation is part of the requested transaction. When MarkLogic responds, the load balancer again adds a session cookie, which the Java Client API caches on the Transaction object.
When you commit or roll back a transaction, any cookies returned by the load balancer are discarded since the transaction is no longer valid. This effectively ends the session from the load balancer's perspective because the Java Client API will no longer pass the session cookie around.
Any Java Client API operation that does not include a Transaction object will not include a session cookie (or transaction id) in the request to MarkLogic, so the load balancer is free to route the request to any host in your MarkLogic cluster.
Related
I am attempting to use FreeRADIUS to perform a local DB lookup (MAC address), if no results are found return a CoA to the device to provision a captive portal, however if results are found to then manipulate the attributes on the request (add username/password from DB) and proxy the request onto another RADIUS server
Is this advanced logic possible? I have found some examples to pre/post process a proxy request, but not control if the request should be processed locally or proxied etc
in authorize you should be able to set the realm (used for proxy) after preprocess and before your realm module gets processed.
https://networkradius.com/doc/3.0.10/raddb/sites-available/default.html
We have a multiservice architecture consisting of HAProxy front end ( we can change this to another proxy if required), a mongodb database, and multiple instances of a backend app running under Docker Swarm.
Once an initial request is routed to an instance ( container ) of the backend app we would like all future requests from mobile clients to be routed to the same instance. The backend app uses TCP sockets to communicate with a VoIP PBX.
Ideally we would like to control the number of instances of the backend app using the replicas key in the docker-compose file. However if a container died and was recreated we would require mobile clients continue routing to the same container. The reason for this is each container is holding state info.
Is this possible with Docker swarm? We are thinking each instance of the backend app when created gets an identifier which is then used to do some sort of path based routing.
HAproxy has what you need. This article explains all.
As a conclusion of the article, you may choose from two solutions:
IP source affinity to server and Application layer persistence. The latter solution is stronger/better than the first but it requires cookies.
Here is an extras from the article:
IP source affinity to server
An easy way to maintain affinity between a user and a server is to use user’s IP address: this is called Source IP affinity.
There are a lot of issues doing that and I’m not going to detail them right now (TODO++: an other article to write).
The only thing you have to know is that source IP affinity is the latest method to use when you want to “stick” a user to a server.
Well, it’s true that it will solve our issue as long as the user use a single IP address or he never change his IP address during the session.
Application layer persistence
Since a web application server has to identify each users individually, to avoid serving content from a user to an other one, we may use this information, or at least try to reproduce the same behavior in the load-balancer to maintain persistence between a user and a server.
The information we’ll use is the Session Cookie, either set by the load-balancer itself or using one set up by the application server.
What is the difference between Persistence and Affinity
Affinity: this is when we use an information from a layer below the application layer to maintain a client request to a single server
Persistence: this is when we use Application layer information to stick a client to a single server
sticky session: a sticky session is a session maintained by persistence
The main advantage of the persistence over affinity is that it’s much more accurate, but sometimes, Persistence is not doable, so we must rely on affinity.
Using persistence, we mean that we’re 100% sure that a user will get redirected to a single server.
Using affinity, we mean that the user may be redirected to the same server…
Affinity configuration in HAProxy / Aloha load-balancer
The configuration below shows how to do affinity within HAProxy, based on client IP information:
frontend ft_web
bind 0.0.0.0:80
default_backend bk_web
backend bk_web
balance source
hash-type consistent # optional
server s1 192.168.10.11:80 check
server s2 192.168.10.21:80 check
Session cookie setup by the Load-Balancer
The configuration below shows how to configure HAProxy / Aloha load balancer to inject a cookie in the client browser:
frontend ft_web
bind 0.0.0.0:80
default_backend bk_web
backend bk_web
balance roundrobin
cookie SERVERID insert indirect nocache
server s1 192.168.10.11:80 check cookie s1
server s2 192.168.10.21:80 check cookie s2
Suppose there is a web server that hosts arbitrary user-controlled content under some paths - public IPFS gateways are the example that got me thinking about this. Is it possible for that server to prevent pages that it serves from installing service workers on clients (and thus spoofing content for non-user-controlled paths)?
There's some helpful info in the service worker specification:
An HTTP request to fetch a service worker's script resource will
include the following header:
Service-Worker Indicates this request is a service worker's script
resource request.
Note: This header helps administrators log the requests and detect
threats.
If you'd like to make sure that your web server doesn't allow any service worker registrations, one approach would be to check for the Service-Worker header on incoming requests and have your web server return an appropriate HTTP error response (anything 4xx or 5xx would work—maybe 403 or 412?) whenever you detect that.
We are using Amazon ELB for our servers that provide REST APIs, and we want our client applications to be able to use the ELB application stickiness policy to process some REST APIs. Some of the APIs must be done as a transaction (multiple steps or APIs involved) and therefore, must be handled by the same server behind ELB. We are not able to change the responses sent by the server to include a session ID. How can we handle this since ELB only provides its AWSELB cookie if the server includes a "set-cookie" with the session ID cookie name specified in ELB?
We do have control of the client applications. We can modify the HTTP requests they make to the servers (through ELB).
"The load balancer only inserts a new stickiness cookie if the application response includes a new application cookie" is only applicable to application-controlled session stickiness.
Configure your balancer for duration-based session stickiness.
I have a website that is 100% https and will only work as https. My site is an asp.net mvc application running on IIS 7.5.
It is on multiple servers with traffic distributed via a load balancer.
I am not in control of the hardware.
For http requests, I was hoping that it could be stopped at the load balancer and a redirect to https at this point.
However the hardware company wont do this for me, and instead I need to do the redirect from http to https within IIS on the server. Therefore unencrypted traffice can enter the inner network with redirect being at the server level. I would feel more comfortable with such a transfer happening at the load balancer.
Do I have valid concerns?
Threat model:
HTTP request:
Attacker
| Security Boundary
V V
Client -- http request --> Load Balancer
|
Client <-- redirect -----------+
Threats which occur from allowing HTTP redirects regardless of methodology:
Spoofing: Client could connect to MITM spoofed HTTP server which does not pass through the redirect, but instead proxies connections to the actual HTTPS server
Tampering: Client could receive redirect URL from MITM spoofed server which directs them to another action (E.g.: client receives redirect to https://yoursite.com/login.aspx?redirect=/deleteAllDocuments)
Information Disclosure: Initial HTTP request is disclosed, any information in POST or GET is available unencrypted to eavesdroppers.
Arguments for performing redirect on server other than target server:
Firewall can be limited to HTTPS data, limiting risk of unencrypted data due to misconfiguration
Configuration and liability could become "Someone else's problem", from at least a political perspective
Vulnerabilities in HTTP server would be isolated and could not be used to attack HTTPS server or underlying application
Arguments for performing redirect on something other than the load balancer:
Load balancers are not servers
Load balancers are not servers, and therefore might have lightly used code paths when used as servers, which could be more prone to undiscovered bugs or performance problems
Configuration is not available to you, but you (or your company) is still probably liable for any misconfigurations which occur (from a legal perspective)
In light of the above analysis, for highest security with lowest risk:
I would not put the redirect on the target server, nor the load balancer, but instead on a VM which only serves to redirect pages. A minimal Linux or windows box should be able to be tightly locked down to limit exposure.
I would not allow redirects with a query-string or POST data (E.g.: show 404 for any non GET / HTTP/1.1 request)
I would call the possibility of spoofing and tampering an acceptable risk, or show a page to the user explaining that the site must be accessed using HTTPS instead of using a redirect
But, if you can assume the following conditions are met, placing the HTTP server co-resident with the HTTPS server should not reduce security.
Any bug in the HTTP server is present in the HTTPS server
The HTTP server is correctly configured to disallow access to protected resources (set up as a separate site in IIS, for example. Secure site still has no HTTP binding)
No other application is able to create a server on HTTP (netsh urlacl only has IIS, for example)
Configuration is audited to ensure the above configurations are properly maintained (Periodic pen tests, manual configuration review, configuration change management, and an IDS or IPS system)
In some cases, the reduced complexity may even be easier to secure than a separate server. Additionally, if the administrator is unfamiliar with the load balancer's configuration, they may be more prone to make a critical error in configuration than if they were to make the same configuration in a product they know well.
Do I have valid concerns?
Do you have a valid concern about the initial connection being over HTTP? Sure. The initial request can be intercepted and the response spoofed in a MitM attack. The attacker can then either prevent the user from using HTTP (adding ssl/tls between the attacker and your server and relaying to the victim in the clear) or can create an imposter SSL session with the client that terminates at the attacker before being re-encrypted on it's way to you (using various spoofing techniques to make the attack less obvious to the casual user).
However, if such an attack were launched, I would be far more worried about the transit from the client to your load balancer, not between your load balancer and IIS. If you suspect that you have malicious systems behind your load balancer, you have an entirely different set of problems.
See my answer over on security.so for some relevant information regarding redirects from HTTP to HTTPS.