It appears that your reverse proxy set up is broken - Jenkins IIS - jenkins

I cant seem to correctly setup Reverse Proxy for my Jenkins with IIS.
Everything works correctly however I cannot get that message to disappear! Running the "administrativeMonitor/hudson.diagnosis.ReverseProxySetupMonitor/testForReverseProxySetup" is not returning a 200.. but in fact a 404.
I also use the plugin Blue Ocean, I noticed that when running pipelines when accessing Jenkins via my Reverse Proxy, I cannot see the live updates!! I dug into this and found out that it usually is from a bad Revery Proxy setup. If I access jenkins via localhost, I can see Blue Ocean Pipeline updates live, so something must be setup wrong!
What I got:
Windows Server 2019 Datacenter
IIS 10
I have HTTPS setup, so my aim is: Go to jenkins.example.com, it will redirect to https://jenkins.example.com which in turn is a reverse proxy loading jenkins.
My web.config:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<rewrite>
<rules useOriginalURLEncoding="false">
<!-- Enforces redirection of all HTTP traffic to HTTPS -->
<rule name="Enforce HTTPS" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{HTTPS}" pattern="off" />
</conditions>
<action type="Redirect" url="https://{HTTP_HOST}/{R:1}" redirectType="Permanent" />
</rule>
<rule name="ReverseProxyToLocalJenkinsRule" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{HTTPS}" pattern="on" />
</conditions>
<action type="Rewrite" url="http://jenkins.example.com:7070/{R:1}" />
<serverVariables>
<set name="X-Forwarded-Proto" value="https" />
<set name="X-Forwarded-Port" value="443" />
<set name="X-Forwarded-Host" value="{HTTP_HOST}" />
</serverVariables>
</rule>
</rules>
</rewrite>
<security>
<requestFiltering allowDoubleEscaping="true" />
</security>
</system.webServer>
<system.web>
<httpRuntime maxUrlLength="4096" />
</system.web>
</configuration>
This seems to be the best working reverse proxy I can obtain, following jenkins tutorials usually leads me to login issues (URL doubling up, i.e: example.com/https://example.com)
P.s, this is the latest windows server image out there and is a clean base build.
I ran through many tutorials however nothing has solved this issue, any advice is muchly appreciated.
I also want to point out that I have one server with Jenkins installed and IIS, Jenkins is accessible at localhost:7070 on the server locally. I also have edited my hosts file on the server to:
127.0.0.1 jenkins.example.com
Where the jenkins.example.com is an actual subdomain/domain pointing to the public IP of my server.
So when I ping: jenkins.domain.com on the server locally.. it returns 127.0.0.1. If I ping outside of that server, I.e.. on my PC, I get the public IP.
EDIT:
I have Application Request Routing installed and below is my config:

I just spent 6 hours of my life debugging this one, and I hope if some poor soul finds this page, as I did, my answer will be of some help. Almost identical situation to the original question, but for the sake of being thorough, I'll detail my exact situation, and my exact solution:
I am running a Windows Server 2019 Datacenter on AWS EC2
My EC2 instance lives under an Application Load Balancer which automatically redirects all http traffic to https, but forwards requests to the server using http only. That is, the user is always connected to the load balancer using https, but the load balancer connects to the server using only http
On that server, I am running Jenkins locally at localhost:8080
I want to reverse-proxy jenkins.mydomain.com to localhost:8080 without configuring SSL at the IIS level
I have read through ALL of the tutorials available on Jenkins' website, other websites with similar "IIS Jenkins Reverse Proxy" tutorials, and all related StackOverflow articles. NONE have solved my issue, and most want you to configure an SSL cert at the IIS level.
Also, I was getting the same mysterious error of the URL doubling up like jenkins.mydomain.com/"http://jenkins.mydomain.com:80"
Here is my exact configuration that solved the issue:
Obviously, you need to configure everything on AWS correctly. If you're not using AWS, make sure that your configuration is comparable.
EC2 Instance with security policy allowing incoming connections on port 80
Application Load Balancer redirects http requests to https, and forwards https requests to your EC2 instance on port 80
Application Load Balancer configured with SSL certificate
DNS configured to direct jenkins.mydomain.com to your Application Load Balancer
As with other tutorials, you must have ARR installed for IIS
UNLIKE other tutorials, do NOT add 127.0.0.1 jenkins.mydomain.com to your hosts file. If you have added it, remove it.
In IIS, Application Request Routing Cache > Server Proxy Settings
Enable Proxy: checked
Reverse rewrite host in response headers: checked
THIS DIFFERS FROM OTHER TUTORIALS, AND IS ULTIMATELY THE KEY THAT CAUSED MY SETUP TO WORK
In IIS, Default Website > Request Filtering > Edit Feature Settings
Allow Double Escaping: checked
In IIS, Default Website > URL Rewrite > Add Rule(s) > Blank Rule
Name: Jenkins Rewrite
Pattern: (.*)
Conditions: {HTTP_HOST} matches .*jenkins.mydomain.com.*
Server Variables: None
Action: Rewrite http://localhost:8080{UNENCODED_URL}
Note that THERE IS NO NEED to include an http to https redirect rule here. If you have one, I don't think it'll hurt anything, but don't get caught up by thinking this is important, as other tutorials suggest.
In IIS, Configuration Editor > system.webServer/rewrite/rules
useOriginalURLEncoding: False
In Jenkins (which you may need to access via loacalhost:8080 at first), Manage Jenkins > Configure System
Jenkins URL: https://jenkins.mydomain.com/
In Jenkins, Manage Jenkins > Configure Global Security
Enable Proxy Compatibility: checked
And that's it! It's possible that some of those steps (especially 9 and 10) may be unnecessary, but in the process of getting to where I am now, those values were ALREADY set, and once it started working I didn't want to change ANYTHING!
Again, the major breakthrough for me was to ENABLE "Reverse rewrite host in response headers".
Edit: Also, by the way, once you get everything working, feel free to just hit the "Disable" button on the Jenkins warning about your reverse proxy configuration being messed up. I think that this situation, and the resulting configuration to make it work, is so non-standard that Jenkins doesn't recognize that it is, in fact, working.

Related

500 - Internal server error with no details

Client gave me a server with public ip. I could see default IIS page in browser locally. Then I did following steps
I published solution (Asp.Net MVC) on my file system.
Copied published folder on server.
Created new website in IIS.
Stopped default web site as it was using port 80.
Assigned port 80 to newly created website.
Opened ip again in browser but it gave me 500 - Internal Server Error.
I made sure following configuration is there in web.config file
<configuration>
<system.webServer>
<httpErrors errorMode="Detailed" />
<asp scriptErrorSentToBrowser="true"/>
</system.webServer>
<system.web>
<customErrors mode="Off"/>
<compilation debug="true"/>
</system.web>
</configuration>
also added following line in config
<validation validateIntegratedModeConfiguration="false" />
still it showed no details for error even on the server.
I connected same publish folder to my local IIS and it is working fine.
Did I miss anything?
The reason for 500 error can be complicated.
So we have to check several things first.
1.Please ensure IIS asp.net feature has been installed on the client server's side. You can check these configurations by reviewing modules and handler mapping in IIS manager.
2.Please check whether the application works fine when you publish to Client's local folder.
3.Please ensure IUSR, authenticated user and application pool identity has been added to the security group of your folder.
4.Please Check whether special sub-status code or win-32 status code can be found in your IIS log.
5.Please ensure your .net framework and your clients' are the same.
6.Please check whether your application pool has selected the correct mode and runtime version.
With more information, we may find out the root cause of the 500 error.

Jenkins reverse proxy on IIS

I am configuring reverse proxy for jenkins on IIS 10. I get 500 - Internal server error.URL Rewrite Module Error. and Jekins shows the message "It appears that your reverse proxy set up is broken". I have added inbound and outbound rules with "localhost", IP Address and with host name as well. Is there anything else specific to IIS I need to configure?
I also have "It appears that your reverse proxy set up is broken", but everything works fine (i.e. I can access my Jenkins instance from another computer without any issues).
My setup is as follows:
Path Prefix
In jenkins.xml I have provided the prefix I want to have for my Jenkins instance:
<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\jenkins.war" --httpPort=8080 --webroot="%BASE%\war" --prefix=/jenkins</arguments>
IIS configuration
web.config for the web site that should be used as a reverse proxy looks the following:
<configuration>
<system.webServer>
<rewrite>
<rules>
<rule name="ReverseProxyInboundRule1" stopProcessing="true">
<match url="(jenkins.*)" />
<action type="Rewrite" url="http://localhost:8080/{R:1}" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>
This assumes that Jenkins runs on its default port: 8080.
Not sure, but I think that Configure -> Jenkins location -> Jenkins URL should point to something like http://your_host_name/jenkins.
Of course, Web site bindings should include your hostname to access Jenkins from another machine.

How to use Domain name with https instead of localhost for debugging website in IIS

I am implementing Single Sign On(using OKTA) on one of our MVC based website where the OKTA takes a Response URL to throw its response back to my code once the user has entered his credentials on OKTA login page
OKTA takes an HTTPS based response URL to throw its response.
Things I have done:
1) I have specified a dummy Hostname to my machine using etc/hosts file like:
127.0.0.1 www.abc.com
2) Enabled https:// on www.abc.com on my local machine
3) Specified a route {{controller}}/{{Action}} to receive the response in my code.
4) I have tried specifying Bindings to my local applicationConf file:
<bindings>
<binding protocol="http" bindingInformation="*:58565:localhost" />
<binding protocol="https" bindingInformation="*:443:www.abc.com" />
</bindings>
My VS project configuration looks like this:
Is it possible to receive the response on the local MVC route (hit the breakpoint) when the URL (where OKTA throws response) is like https:// www.abc.com/Home/Auth
Problem is when I write localhost:58565/Home/Index or http:// www.abc.com/Home/Index with VS debugging on, it hits the breakpoint but when I type https:// www.abc.com/Home/Index (with https) it does not hit the breakpoint in Visual Studio. With https it says 404 Not found.
Am I missing something?
Thanks
I got the issue fixed. Posting the solution so that it may be helpful for others.
Steps I followed:
1) Created a Self signed certificate with Powershell(make sure you run Powershell command line with Admin Privileges)
New-SelfSignedCertificate -DnsName www.mydomain.com -CertStoreLocation cert:\LocalMachine\My
2) Have correct bindings in my ApplicationConf.host file(this file is now at Project level now since VS 2015 rather than at IIS level)
like the one below:
<bindings>
<binding protocol="http" bindingInformation="*:58565:localhost" />
<binding protocol="https" bindingInformation="*:44311:localhost" />
</bindings>
You can add more bindings if using Hostname instead of Localhost. Make sure ports are different for each binding. Also, Https ports should start from 443--ending at 44399.
3) Remove the existing https binding from IIS Default Website and add it back with the certificate created in Step 1 and correct hostname used in Step 1.
Now you should be able to run and debug your code on local machine with self signed SSL.

Azure Deployment Slot swap redirects to same domain

My scenario is as follows.
I have my website.azurewebsites.net main site which redirects to https://website.dk (I bought this domain) and created another deployment slot called website-dev.azurewebsites.net. However when I want to visit the 'dev' one it is still being redirected to https://website.dk which has the changes from website.azurewebsites.net.
How can I access website-dev.azurewebsites.net without being redirected to the main one?
Edit: Maybe it is the redirect in the web.config file, but I have commented out the redirect part and published?
Commenting out the redirection is not a good practice as it could cause issues if you swap your dev environment to your production environment and you forget to add the redirection again.
You should add conditions to your rules so they are only executed in the production environment. I recommend you to follow the steps described in https://gist.github.com/alindgren/999c542a6625ddfb41f1b0293878dcad
I have found the solution, turns out I had to comment the entire <rewrite> block in the web.config.
<!--<rewrite>
<rules>
<rule name="Force HTTPS" enabled="true">
<match url="(.*)" ignoreCase="false" />
<conditions>
<add input="{HTTPS}" pattern="off" />
</conditions>
<action type="Redirect" url="https://website.dk/" appendQueryString="true" redirectType="Permanent" />
</rule>
</rules>
</rewrite>-->
For me, this symptom occurred due to the load balancer routing rules with the new Deployment slots (Preview) functionality. This allow allows you to redirect a proportion of traffic from one slot (i.e., production) to other slots (i.e., staging). You can also route traffic from staging back to production, so there is a matrix of reroutings.
This configuration sends all traffic to the expected site
TO prod stag
FROM
prod 100% 0%
stag 0% 100%
This is a sensible configuration for AB testing, with 5% of traffic prod going to staging, and all staging traffic going to staging
TO prod stag
FROM
prod 95% 5%
stag 0% 100%
I slipped up and had this setup, with 100% of staging going to production
TO prod stag
FROM
prod 100% 0%
stag 100% 0%
The UI is a little unintuitive.
To fix the problem,
for each web app (production, staging, dev etc) in your web service,
click on "Deployment slots (Preview)"
set the proportion of traffic as appropriate for that web app.
If you are not doing any A/B testing or similar, you should direct 100% of traffic to that same app (i.e., 100% in the greyed box, 0% in all the others.)
There also appears to be an issue with Azure with the same symptoms as described in the question, although this was not my problem.
Azure load balancer redirects staging slots traffic to production, unable to access staging

Windows Azure cannot perform a VIP swap between deployments that have a different number of endpoints

I'm trying to perform a VIP swap via the azure portal and I'm getting the error:
Windows Azure cannot perform a VIP swap between deployments that have a different number of endpoints.
I looked closer and I DO see difference in # of endpoints (2 in production vs 3 in staging).
Production:
Input Endpoints
OUR.API:168.62.21.50:80
OUR.API:168.62.21.50:3389
Staging:
Input Endpoints
OUR.API:168.62.22.55:80
OUR.API:168.62.22.55:3389
OUR.API:168.62.22.55:8172
Port 80 is web and 3389 is remote desktop. So far so good. Where is that additional port, 8172, coming from? Nothing in the application listens to anything other than port 80. Plus the applications in the staging and production areas are almost identical - so it's gotta be the framework. Any steps in narrowing this down?
[edit]
Also, my role's ServiceDefinition.csdef has just one endpoint defined:
<Sites>
<Site name="Web">
<Bindings>
<Binding name="Endpoint1" endpointName="Endpoint1" />
</Bindings>
</Site>
</Sites>
<Endpoints>
<InputEndpoint name="Endpoint1" protocol="http" port="80" />
</Endpoints>
"Where is that additional port, 8172, coming from?"
It is from WebDeploy being enabled in the publishing settings
Long answer:
I parsed the entire configuration one by one between the staging and production, here is what I found:
Your publishing settings are saved to
<azurerole>\Profiles\<yourprofilename>.azurePubxml
My production deployment had
<AzureEnableWebDeploy>True</AzureEnableWebDeploy> while my staging deployment had <AzureEnableWebDeploy>False</AzureEnableWebDeploy>
The Azure infrastructure then looks that up and opens port 8172 to enable WebDeploy on the staging roles. So that's why the endpoints are different despite no new endpoint defined in the ServiceDefinition.csdef file.
I'm not sure why having different number of endpoints should prevent an Azure Publication itself.
For me the issue was that my PROD instances had RDP enabled and the STAGING did not. So "RDP" was the endpoint that was different.

Resources