The Scenario
I have an application where we took the good old query string URL structure:
?x=1&y=2&z=3&a=4&b=5&c=6
and changed it into a path structure:
/x/1/y/2/z/3/a/4/b/5/c/6
We're using ASP.NET MVC and (naturally) ASP.NET routing.
The Problem
The problem is that our parameters are dynamic, and there is (theoretically) no limit to the amount of parameters that we need to accommodate for.
This is all fine until we got hit by the following train:
HTTP Error 400.0 - Bad Request
ASP.NET detected invalid characters in the URL.
IIS would throw this error when our URL got past a certain length.
The Nitty Gritty
Here's what we found out:
This is not an IIS problem
IIS does have a max path length limit, but the above error is not this.
Learn dot iis dot net
How to Use Request Filtering
Section "Filter Based on Request Limits"
If the path was too long for IIS, it would throw a 404.14, not a 400.0.
Besides, the IIS max path (and query) length are configurable:
<requestLimits
maxAllowedContentLength="30000000"
maxUrl="260"
maxQueryString="25"
/>
This is an ASP.NET Problem
After some poking around:
IIS Forums
Thread: ASP.NET 2.0 maximum URL length?
http://forums.iis.net/t/1105360.aspx
it turns out that this is an ASP.NET (well, .NET really) problem.
The heart of the matter is that, as far as I can tell, ASP.NET cannot handle paths longer than 260 characters.
The nail in the coffin in that this is confirmed by Phil the Haack himself:
Stack Overflow
ASP.NET url MAX_PATH limit
Question ID 265251
The Question
So what's the question?
The question is, how big of a limitation is this?
For my app, it's a deal killer. For most apps, it's probably a non-issue.
What about disclosure? No where where ASP.NET Routing is mentioned have I ever heard a peep about this limitation. The fact that ASP.NET MVC uses ASP.NET routing makes the impact of this even bigger.
What do you think?
I ended up using the following in the web.config to solve this problem using Mvc2 and .Net Framework 4.0
<httpRuntime maxUrlLength="1000" relaxedUrlToFileSystemMapping="true" />
Http.sys service is coded with default maximum of 260 characters per Url segment.
An "Url segment" in this context is the content between "/" characters in the Url. For example:
http://www.example.com/segment-one/segment-two/segment-three
The max allowed Url segment length can be changed with registry settings:
Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\HTTP\Parameters
Value: UrlSegmentMaxLength
Type: REG_DWORD
Data: (Your desired new Url segment maximum allowed length, e.g. 4096)
More about http.sys settings:
http://support.microsoft.com/kb/820129
The maximum allowed value is 32766. If a larger value is specified, it will be ignored. (Credit: Juan Mendes)
Restarting the PC is required to make a change to this setting take effect. (Credit: David Rettenbacher, Juan Mendes)
To solve this, do this:
In the root web.config for your project, under the system.web node:
<system.web>
<httpRuntime maxUrlLength="10999" maxQueryStringLength="2097151" />
...
In addition, I had to add this under the system.webServer node or I got a security error for my long query strings:
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxUrl="10999" maxQueryString="2097151" />
</requestFiltering>
</security>
...
OK so part of the reason I posted this was also because we have found a work around.
I hope this will be useful to someone in the future :D
The workaround
The workaround is quite simple, and it's quite nice too.
Since we know which parts of the site will need to use dynamic parameters (and hence will have a dynamic path and length), we can avoid sending this long url to ASP.NET routing by intercepting it before it even hits ASP.NET
Enter IIS7 Url Rewriting (or any equivalent rewrite module).
We set up a rule like this:
<rewrite>
<rules>
<rule>
<rule name="Remove Category Request Parameters From Url">
<match url="^category/(\d+)/{0,1}(.*)$" />
<action type="Rewrite" url="category/{R:1}" />
</rule>
</rules>
</rewrite>
Basically, what we're doing is just keeping enough of the path to be able to call the correct route downstream. The rest of the URL path we are hacking off.
Where does the rest of the URL go?
Well, when a rewrite rule is fired, the IIS7 URL Rewrite module automagically sets this header in the request:
HTTP_X_ORIGINAL_URL
Downstream, in the part of the app that parses the dynamic path, instead of looking at the path:
HttpContext.Request.Url.PathAndQuery
we look at that header instead:
HttpContext.Request.ServerVariables["HTTP_X_ORIGINAL_URL"]
Problem solved... almost!
The Snags
Accessing the Header
In case you need to know, to access the IIS7 Rewrite Module header, you can do so in two ways:
HttpContext.Request.ServerVariables["HTTP_X_ORIGINAL_URL"]
or
HttpContext.Request.Headers["X-ORIGINAL-URL"]
Fixing Relative Paths
What you will also notice is that, with the above setup, all relative paths break (URLs that were defined with a "~").
This includes URLs defined with the ASP.NET MVC HtmlHelper and UrlHelper methods (like Url.Route("Bla")).
This is where access to the ASP.NET MVC code is awesome.
In the System.Web.Mvc.PathHelper.GenerateClientUrlInternal() method, there is a check being made to see if the same URL Rewrite module header exists (see above):
// we only want to manipulate the path if URL rewriting is active, else we risk breaking the generated URL
NameValueCollection serverVars = httpContext.Request.ServerVariables;
bool urlRewriterIsEnabled = (serverVars != null && serverVars[_urlRewriterServerVar] != null);
if (!urlRewriterIsEnabled) {
return contentPath;
}
If it does, some work is done to preserve the originating URL.
In our case, since we are not using URL rewriting in the "normal" way, we want to short circuit this process.
We want to pretend like no URL rewriting happened, since we don't want relative paths to be considered in the context of the original URL.
The simplest hack that I could think of was to remove that server variable completely, so ASP.NET MVC would not find it:
protected void Application_BeginRequest()
{
string iis7UrlRewriteServerVariable = "HTTP_X_ORIGINAL_URL";
string headerValue = Request.ServerVariables[iis7UrlRewriteServerVariable];
if (String.IsNullOrEmpty(headerValue) == false)
{
Request.ServerVariables.Remove(iis7UrlRewriteServerVariable);
Context.Items.Add(iis7UrlRewriteServerVariable, headerValue);
}
}
(Note that, in the above method, I'm removing the header from Request.ServerVariables but still retaining it, stashing it in Context.Items. The reason for this is that I need access to the header value later on in the request pipe.)
Hope this helps!
I was having a similar max URL length issue using ASP.NET Web API 4, which generated a slightly different error:
404 Error
The fix for me was described above by updating the Web.config with BOTH of the following tags:
<system.web>
<httpRuntime maxUrlLength="10999" maxQueryStringLength="2097151" />
and
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxUrl="10999" maxQueryString="2097151" />
</requestFiltering>
</security>
I think you're trying to hard to use GET. Try changing the request method to POST and put those query string parameters into the request body.
Long URL does not help SEO as well, does it?
It appears that the hard-coded max URL length has been fixed in .NET 4.0. In particular, there is now a web.config section with:
<httpRuntime maxRequestPathLength="260" maxQueryStringLength="2048" />
that let you expand the range of allowed URLs.
Related
I have code that i used a year or two ago to create an activation link that is send by email, using a code as a parameter, i tried so many different ways, encoding, using other syntax, adding web.config settings <pages validateRequest="false" />
requestValidationMode="2.0"
, annotations [AllowHtml], literally tried dozen of post on the internet but none of them worked.
So i am overseeing something here, that i am sure but i can't find the solution.
The error i get is:
A potentially dangerous Request.Path value was detected from the client (?)
The format i use is
Url.Action("action","controller", new { Id = guidValue }, Request.Url.Scheme)
My routing is the Default so this should work.
The Url is like this in the address bar once clicked:
http://localhost:52641/Account/Login?ReturnUrl=%2FAccount%2FAccountActivation%2F%3Fid%3Dfc39f53f-6fa7-43d2-b30a-8b4e20f0f237
While it should give me:
http://localhost:52641/Account/AccountActivation?id=fc39f53f-6fa7-43d2-b30a-8b4e20f0f237
What is happening here?
Thank you for any feedback!
I removed the complete built in Authentication (NuGet packages & classes)
from my solution (since i didn't needed them) and this made the above case work as it was. I can't elaborate more on the what's and how's (i am sure other people can) but i think it has to do with routing that comes with the authentication.
I am running MVC 5 and have a search API that produces the following link: /SearchedItem.?format=json where SearchedItem. is the user's input into search. This obviously causes a famous 404 due to a dot character. I've looked into all of the following solutions:
Dot character '.' in MVC Web API 2 for request such as api/people/STAFF.45287
Dots in URL causes 404 with ASP.NET mvc and IIS
ApiController returns 404 when ID contains period
However, neither adding a slash (tried both /SearchedItem./?format=json and /SearchedItem.?format=json/) nor RAMMFAR worked.
Looking for any new suggestions.
You have to change your web.config, the trailing dot let's iis think you are accessing an image.
Add the following within the system.webServer / handlers ( web.config)
<add name="ApiURIs-ISAPI-Integrated-4.0"
path="/api/*"
verb="GET,HEAD,POST,DEBUG,PUT,DELETE,PATCH,OPTIONS"
type="System.Web.Handlers.TransferRequestHandler"
preCondition="integratedMode,runtimeVersionv4.0" />
Another suggest would be to set RunAllManagedModulesForAllRequests to true, but i wouldn't recommend that. All static assets would be handled through the .net code then :)
I see in your question that you checked related links. But are you sure? Because i have came accross this in the past and above was my solution...
Most suitable way is to encode once you route to url and decode in the respective action you can use HttpUtility to perform encoding and decoding
If you don't want to encode and decode then try adding relaxedUrlToFileSystemMapping config as explained Here
I have a multi-tenanted MVC site where all the MapRoutes start as follows:
http://foo.com/{organisation}/{branch}/{controller}/{action}
Each organisation can have several branches. For example:
The Metro organisation has two branches:
http://foo.com/metro/north/
http://foo.com/metro/south/
The Gold organisation has three branches:
http://foo.com/gold/usa/
http://foo.com/gold/eu/
http://foo.com/gold/asia/
There are further Controllers and Actions below these but EVERY MapRoute / url follows this format.
My organisations don't like the generic URLs and want to personalize them. Is it possible to change from this:
http://foo.com/metro/north/
http://foo.com/metro/south/
To this (implying that {branch} = 'metro')
http://www.metrowebsite.com/north/
http://www.metrowebsite.com/south/
Using some kind of URL mapping or URL re-direction without changing the MVC website?
I'm not even sure of the correct terminology to use to search for help on this.
Thanks in anticipation!
I guess you could map all the DNS entries for those URLs to point to the same webserver instance and then use a URL rewriting configuration to translate that into the 'standard' path that your application expects.
I have used URL rewrite myself (http://www.iis.net/downloads/microsoft/url-rewrite) to do something similar, so I know in concept it works. I just can't 100% say that it recieves the full url (i.e. http://www.metrowebsite.com/north/) and not just the path (i.e. /north). You may need to verify this part.
Essentially the module would take incoming URls, i.e. http://www.metrowebsite.com/north/
and could use a regular expression to adapt it into the format http://www.metrowebsite.com/north/
Here is an example of one that I did:
<rewrite>
<rules>
<rule name="MyRewriteRul" stopProcessing="true">
<match url="Some/Sort/Of/Friendly/Url/$" />
<action type="Rewrite" url="MyControllerName/MyActionName" appendQueryString="true" />
</rule>
</rewrite>
This configuration goes directly into your Web.config file - see the documentation for more details on this.
Very basic example that rewrites incoming Urls matching 'Some/Sort/Of/Friendly/Url/' and rewrites it into the format my system expects, i.e. 'MyControllerName/MyActionName'.
This is something you could build upon I think.
In regards to your routes, I would recommend modifying them slightly, introducing some 'static' components to the urls so that you don't get unnecessary path match conflicts. I also did a similar multi-tenant pathing exercise.
Consider doing the following:
http://foo.com/o/{organisation}/b/{branch}/{controller}/{action}
You can then have a match rule as such:
routes.MapRoute(
"Organisation.Default",
"o/{organisation}/b/{branch}/{controller}/{action}/{id}",
new { siteView = "Public", controller = "Home", action = "Index", id = "0" });
Which won't conflict with your default routes, as the /o and /b parts clearly signify that its route for an organisation. You can obviously modify this as you like, just something to consider. :)
I have this route:
{controller}/{id}/{action}
Because I think it makes more sense from RESTful perspective.
The problem is that id can contain slashes (/) and those are treated as route separators even when encoded as "%2F". Even when I have this Web.config section in place:
<uri>
<schemeSettings>
<add name="http" genericUriParserOptions="DontUnescapePathDotsAndSlashes" />
<add name="https" genericUriParserOptions="DontUnescapePathDotsAndSlashes" />
</schemeSettings>
</uri>
Because I have id in the middle I can't employ {*id} approach which captures the rest of the route including the action.
It looks like my only option is to encode / into an RFC compliant character like !, however I do not want to do it using ad-hoc custom code inside controller. I want controller to receive id intact, already decoded. And I want my Url.Action to generate properly encoded URL. Is that too much to ask from MVC, or do I need to scatter ActionFilters and custom URL helpers around?
The only way I could find is to throw in a custom IRouteConstraint to manipulate the RouteValueDictionary it receives. That sounds like a dirty hack though: a constraint manipulating its input. God knows its side effects. Do you think this is a sane enough idea, or is there a better mechanism in ASP.NET MVC allowing that?
EDIT: This workaround only works when parsing the route, not when generating one.
What you are trying to do cannot be done; this has been answered several times on SO - especially of late in relation to RavenDb which uses "/" in the Id field by default (though this can be changed)
http://site.com/page%3fcharacter
This URL will return the following error:
Illegal characters in path.
I'm already put this in web.config:
<system.web>
<httpRuntime requestValidationMode="2.0" requestPathInvalidCharacters="" />
<pages validateRequest="false">
...
How can I fix this error?
If you want to allow the request through, you need to add requestPathInvalidCharacters and set it to an empty string:
<system.web>
<httpRuntime requestPathInvalidCharacters="" />
</system.web>
Edit You should leave your original question in place, because now my answer does not make sense.
But in answer to your second question, that it's because %3f corresponds to '?' which is not allowed in file names on Windows. You can set the relaxedUrlToFileSystemMapping property to true to change this behaviour:
<system.web>
<httpRuntime requestPathInvalidCharacters=""
relaxedUrlToFileSystemMapping="true" />
</system.web>
You might want to look through all of the properties in the HttpRuntimeSection class to see if there's any others that might apply.
You can also implement a sub class of RequestValidator and set up your web.config to use your subclass (that will presumably allow all URLs through?). Personally, I wouldn't bother and just let the built-in classes handle it. It's unlikely that a normal user is every going to accidentally type in "%3f" in a path, and why bother going to so much trouble to improve the use-case for malicious users?
This, by the way, is actually a new feature in ASP.NET 4, which is why Stack Overflow doesn't spit out an error: it's running on .NET 3.5.
Here's a nice article by Hanselman explaining all the nooks and crannies related to your issue:
Experiments in Wackiness: Allowing percents, angle-brackets, and other naughty things in the ASP.NET/IIS Request URL
Probably because that looks a lot like a malformed url.
& is used as a separator for the query string parameters i.e. site.com/page?some=20&another=15