Causes for Application Gateway Connection Timeout - azure-application-gateway

Greeetings,
I have the following setup
Application Gateway -- Private Endpoints -- App Services
The application gateway is in its own resource group along with the virtual network in the 1.0.0.0 subnet.
The private endpoints and app services have a resource group per resource. So for me that would be 5 Services plus the main group with the AG.
I created all elements via the Portal.
After setting up the backend pool (for starters just for one service) and using the basic settings on all the elements (listeners, rules) I wanted to connect to the gateway public ip or dns name, however all I receive is a timeout, without any hint whatsoever in the monitoring as to what could cause the problem.
The application gateway does not even register a request.
Does anyone know what could be the cause of this? Could the Public IP be faulty for some reason? I even tried disabling the private endpoint on one of the services for debugging purposes but to no avail, seems like that is not the cause.
Any help is appreciated :)

Connection Timeout means that one or more than one server could not
complete your request within some period and does not receive a timely
response from another one that works as a gateway or proxy i.e; did
not complete within alloted time.
Initially try to reload the page in different browser or even on different devices.Clean the site from spam and cookies.
Please check if any of the below possible causes:
(main cause)REQUEST TIMEOUT : The number of seconds that
the application gateway will wait to receive a response from the
backend pool before it returns a “connection timed out” error
message.
When a user request is received, the application gateway applies the
configured rules to the request and routes it to a back-end pool
instance. It waits for a configurable interval of time for a response
from the back-end instance. By default, this interval is 20 seconds.
Solution: Reference >> Try Setting request-timeout :Application Gateway allows you to configure this setting via the
BackendHttpSetting,
Ex:
New-AzApplicationGatewayBackendHttpSettings -Name 'Setting01' -Port 80 -Protocol Http -CookieBasedAffinity Enabled -RequestTimeout 60
Also see App gateway
-troubleshoot-app-service-redirection-app-service-url.
NOTE: If your connections are getting dropped at less than the request
time out seconds set, then we need to find which connection is
trigged. And may need to Contact azure support
In addition to server timeouts, there are other causes
See if it due to default health check probe:like 1) Back-end VMs or
instances of virtual machine scale set are not responding to the
default health probe. 2) Invalid or improper configuration of custom
health probes. 3) Azure Application Gateway's back-end pool is not
configured or empty.
Troubleshoot problems-with-default-health-probe and custom healtH probe : Application gateway automatically configures a
default health probe using properties of the BackendHttpSetting but
Custom health probes allow additional flexibility to the default
probing behavior where you can configure the probe interval, the URL, the path to test, and how many failed responses to accept
before marking the back-end pool instance as unhealthy.
Also check the app service time outs : see appgw-timeouts and app
service time out setting
Other causes to check
Slow server, Problems with the firewall,network connection
If NSG, UDR or Custom DNS blocking access to backend pool members.
If VMs or instances in virtual machine scale set are healthy.
For those : Check the logs and DNS records and try by disabling the proxy or temporarily disabling the CDN
References:
Azure application gateway throws 502 when application sends 401 -
Stack Overflow
Azure Application Gateway error 502 when using application gateway
-Stack Overflow

Related

Application Gateway with Web App Backend (domain not resolving)

I have a both a multi and single site Application Gateway with a Web App backend pool. Settings at the backend pool point to port 80 (http).
I am using a personal domain of example: www.mydomain.club and using either web host or DNS. When I enter www.mydomain.club into the browser, it resolves to https://mywebapp.azurewebsites.net:80 in the browser bar, with error message on page indicating that I cannot reach the site.
It seems like the DNS/host recognizes the domain resolving required, but tries push for port 443 (https), but indicates there is a port 80, while returning the web app address (even though I'm going through the application gate). If I restrict access the web app, I get a 401 error.
I have tried:
CNAME pointing to the DNS name for Application Gateway IP - mydns.eastus.cloudapp.azure.com
A recording using both # and WWW pointing to the Gateway IP.
Overriding the host name in the backend settings.
• As seen from the URL, i.e., ‘https://mywebapp.azurewebsites.net:80’, the Azure webapp website points to ‘HTTPS’ port 443 but you might have configured the webapp’s website incorrectly in the backend pool for ‘HTTP’ port 80. Due to which when you are trying to access the ‘www.mydomain.club’ domain name over the internet, it is redirecting you to the ‘https://mywebapp.azurewebsites.net:80’ URL as you have configured it so in the application gateway’s backend pool.
Thus, would suggest you to please reconfigure the ‘https://mywebapp.azurewebsites.net:80’ to only ‘https://mywebapp.azurewebsites.net’ in the application gateway’s backend pool. This will ensure that the website ‘www.mydomain.club’ is redirected correctly to the above said website only and the pages hosted in that webapp will be displayed correctly.
• Secondly, please do ensure that you have created an ‘A’ host record for ‘www’ in the public DNS zone for ‘mydomain.club’ domain which will ensure that the website is hosted on a publicly available IP address and domain name. Also, since you want the website domain name, i.e., ‘www.mydomain.club’ to be redirected to the application gateway URI/FQDN, i.e., ‘xyz.eastus.cloudapp.net’, then create a CNAME record for ‘www.mydomain.club’ address with the alias as ‘xyz.eastus.cloudapp.net’. This will ensure that when trying to access the domain name ‘www.mydomain.club’, it will be redirected to the application gateway’s URL and will be displaying the backend site web page accordingly as defined in the routing rules and the listener rules.
This will ensure that, you will not encounter 401 error when accessing the website DNS, i.e., ‘www.mydomain.club’ and it opens correctly.
Please find the below documentation link for more details: -
https://learn.microsoft.com/en-us/answers/questions/379193/application-gateway-with-a-custom-domain-name.html

Configuring multiple gateways in a batman-adv mesh network

I have been able to set up a wireless mesh network using batman-adv running on a few Raspberry Pis. The Pis are able to detect each other on the mesh, communicate between each other, and share an internet connection if any Pi on the network is acting as a gateway.
The problem I am facing is that if multiple gateways exist on the network, the clients (devices which are receiving the internet connection by the gateway) will never switch to getting their internet connection from any other gateway than the one they initially obtained it from, even if the routing algorithm detects that they have a better connection to another gateway.
Each gateway is running a DHCP server to assign IP addresses to the clients accessing the internet through them. The DHCP server I am using is dnsmasq.
The expected behaviour is either:
the clients send a DHCP request to the gateway capable of providing the better connection and end their lease with their current DHCP server, or
the clients keep their assigned IP from the DHCP server and (magically?) start accessing the internet through the gateway capable of providing the better connection.
Neither of these scenarios are happening. If for whatever reason the gateway providing the internet to a client drops, the client will not attempt to create a new connection until the DHCP lease times out. However this is contradictory to what is claimed on the wiki for this package: https://www.open-mesh.org/projects/batman-adv/wiki/Gateways
Please advise on how to configure this mesh network so that either one of the expected behaviours will happen.
This is exactly the expected behavior at the IP layer.
If a DHCP server provides a default gateway, that gateway remains in place for the life of the lease. The DHCP client, by design, cannot definitively determine if the default gateway is down.
DHCP is a poll-based protocol; a DHCP server cannot proactively change the parameters of a lease until the DHCP client requests an update. The instructions in the link of your question indicates that (emphasis added):
To inform clients possessing a valid DHCP lease about a gateway
change (the link quality to the gateway could have dropped or the
client is roaming around) batman-adv will also inspect incoming DHCP
renewal packets. If their destination is not the currently selected
gateway and below a certain TQ threshold (currently defaulting to a TQ
of 50), the DHCP renewal packet is not forwarded, thereby forcing the
client to request a new DHCP lease from a better-connected gateway.
Note that the change happens by inspecting incoming renewal packets. Due to the nature of DHCP, you are stuck waiting for a lease renewal request to update the gateway.
You can set low lease times to increase the polling rate, but if you need real-time reaction to failed gateways, you will need to employ an actual routing protocol.

Azure Application Gateway exit IPs

We're looking to use Application Gateway as a frontend\proxy for a website we host onsite. The website has a public IP but we'd like to restrict access to the site to only traffic coming from the Application Gateway. Is there a way to determine what IP the traffic would be coming from as it exits Azure? Is it like other Azure traffic where it could come from any IP subnet they have assigned to that specific region? I haven't seen this question asked and it's not directly referenced in any of MS documentation that I can find.
Thanks!
Not sure why you want to restrict access to the site to only traffic coming from the Application Gateway because if you configure it and add the website in the backend of Application Gateway. The traffic from a client will always reach the website through the Application gateway as the Application Gateway works as an application proxy by accepting traffic and based on rules that are defined with it, routes the traffic to the appropriate back-end instances.
You may want to know to restrict access on the application gateway subnet via NSG. Then the inbound or outbound traffic in the Application Gateway subnet will be filtered via NSG.
Network Security Groups (NSGs) are supported on the application
gateway subnet with the following restrictions:
Exceptions must be put in for incoming traffic on ports 65503-65534
for the Application Gateway v1 SKU and ports 65200 - 65535 for the v2
SKU. This port-range is required for Azure infrastructure
communication. They are protected (locked down) by Azure certificates.
Without proper certificates, external entities, including the
customers of those gateways, will not be able to initiate any changes
on those endpoints.
Outbound internet connectivity can't be blocked.
Traffic from the AzureLoadBalancer tag must be allowed.
Hope this will help, let me know if you have any other concerns.
Update
If you just want to whitelist the Azure service on the firewall, you can read the Azure Datacenter IP Ranges. You can figure out which datacenters your service are located in then narrow down the IP ranges.

How to fix System.Net.Sockets.SocketException

I'm working on an ASP.NET MVC website, it has been installed on three different servers (the same code version), it works on two of the servers i.e the user can login, but on one server after the user has submitted their username and password, the following server error is displayed in the browser:
Security Exception
System.Security.SecurityException: Token not found
The event viewer for the environment the error is occuring on shows the following two error messages at the point where th euser attempts to login:
Could not init pool.
System.Net.Sockets.SocketException (0x80004005):
A connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond (server ip address)
And this error:
Error: System.Security.SecurityException: Token not found
Which is the server error displayed in the browser. This has in the past been resolved by deleting an authentication cookie, but that doesn't work for this error.
I have tried setting the trust level in the web.config to full which didn't work. I have tried restarting IIS.
Has anyone else come accross a similar problem? Could this problem be caused because a service of some kind isn't running? Any suggestions will be appreciated.
Most of the time these are connectivity timeouts due to different IP protocols (IPV4/IPV6) between the two server/computers trying to communicate or extra authentication rules setup on one of the computers for outgoing or incoming connectivity.
Ways to troubleshoot the issue:
Review IIS logs
Review EventLogs
Try adjusting TCP/IP parameters in the registry to increase the time allowed to connect
http://msdn.microsoft.com/en-us/library/aa560610(d=printer,v=bts.20).aspx
Even though the above article references Windows 2003, I've discovered it applies to some 2008 environments.
I was getting 0x80004005 & The requested name is valid, but no data of the requested type was found. My complete error
Check You have proper internet connection are there on your machine or not. And you are able to ping the remote server or not. I solved by checking both.
We were getting this error when one of a third party API hosted on of our server(Azure) was not able to communicate with one of our web services on a different machine(iNetU). Options were to open ports so that these machines could talk or to move the web service to the machine hosting the third party API.

Custom domains in a Rails App

I want users of my service to be able to add their own custom domains. For example, www.[their domain].com should be able to access their application's index and show pages. My service is implemented in Rails 3.
I've seen apps like Tumblr offer this functionality for their front facing blogs. Although I have seen apps for Rails that implement sub domains in the way that Basecamp does, I have not found a resource for fully custom domains.
They'll need to create an A record in their DNS to point to your app servers IP. You'll need to know what domain they have pointed to your server and log it against their account, and also set your web servers config in such a way as to channel the requests from other domains to your app. You can then use the request object to look up their account in your application_controller.
I just answered a similar question so I decided to chime in here too.
#Codebeef gave a good answer but this won't work anymore in the world where HTTPS is a must in most modern browsers.
This is the full picture of how to handle custom domains for your app.
If your customers just CNAME to your domain or create the A record to your IP and you don't handle TLS termination for these custom domains, your app will not support HTTPS, and without it, your app won't work in modern browsers on these custom domains.
You need to set up a TLS termination reverse proxy in front of your webserver. This proxy can be run on a separate machine but you can run it on the same machine as the webserver.
CNAME vs A record
If your customers want to have your app on their subdomain, e.g. app.customer.com they can create a CNAME app.customer.com pointing to your proxy.
If they want to have your app on their root domain, e.g. customer.com then they'll have to create an A record on customer.com pointing to your proxy's IP. Make sure this IP doesn't change, ever!
How to handle TLS termination?
To make TLS termination work, you'll have to issue TLS certificates for these custom domains. You can use Let's Encrypt for that. Your proxy will see the Host header of the incoming request, e.g. app.customer1.com or customer2.com etc., and then it will decide which TLS certificate to use by checking the SNI.
The proxy can be set up to automatically issue and renew certificates for these custom domains. On the first request from a new custom domain, the proxy will see it doesn't have the appropriate certificate. It will ask Let's Encrypt for a new certificate. Let's Encrypt will first issue a challenge to see if you manage the domain, and since the customer already created a CNAME or A record pointing to your proxy, that tells Let's Encrypt you indeed manage the domain, and it will let you issue a certificate for it.
To issue and renew certificates automatically, I'd recommend using Caddyserver, greenlock.js, OpenResty (Nginx).
tl;dr on what happens here;
Caddyserver listens on 443 and 80, it receives requests, issues, and renews certificates automatically, proxies traffic to your backend.
How to handle it on my backend
Your proxy is terminating TLS and proxying requests to your backend. However, your backend doesn't know who is the original customer behind the request. This is why you need to tell your proxy to include additional headers in proxied requests to identify the customer. Just add X-Serve-For: app.customer.com or X-Serve-For: customer2.com or whatever the Host header is of the original request.
Now when you receive the proxied request on the backend, you can read this custom header and you know who is the customer behind the request. You can implement your logic based on that, show data belonging to this customer, etc.
More
Put a load balancer in front of your fleet of proxies for higher availability. You'll also have to use distributed storage for certificates and Let's Encrypt challenges. Use AWS ECS or EBS for automated recovery if something fails, otherwise, you may be waking up in the middle of the night restarting machines, or your proxy manually.
If you need more detail you can DM me on Twitter #dragocrnjac

Resources