Background:
Foobar.htm form uses this:
<form action="http://rawurl-here.gotdns.org" method="POST">
[...]
</form>
rawurl-here.gotdns.org is a Dynamic DNS url that redirects the user to:
http://currentsite001.mysite.org
Question:
Is there a way to ensure that the POST parameters sent by Foobar.htm always reach the final target, regardless of the passthru from rawurl-here.gotdns.org?
No, POST requests cannot be redirected. The HTTP spec says that any attempt to redirect a non-GET/HEAD request must be confirmed by the user. However, as noted in the text for the 302 redirect, most browsers ignore this and simply change the POST to a GET instead at which point your parameters are gone.
rawurl-here.gotdns.org is a Dynamic DNS url that redirects
You need a dynamic DNS service that doesn't redirect, but just points the DNS A record directly to your IP address. Set your box up to respond to requests for rawurl-here.gotdns.org and now you don't need a redirect.
DNS redirect and framing services suck anyhow.
I normally use DynDNS and haven't problems with POST data.
Do you have problems? Or just want ensure if the data are sent for your target?
[]'s,
And Past
Related
Context
I've been running an intranet admin panel in Symfony 3.x for several years. The users login with google oauth and the system checks if the email matches a validated one in a lookup-list. The oauth client handling is done with the "HWI OAuth Bundle".
In order to start a clean way to migrate this admin panel into SF4 and later to SF5 we've started breaking our monolyth into microservices running in docker.
Moving to docker behind a reverse proxy
Today we were moving this admin panel into a docker. Then we are having the public apache2 doing a ProxyPass towards the docker running the admin panel. Let's imagine the docker runs in http://1.2.3.4:7540 Let's assume the public address is https://admin-europe.example.com
What happens is that the symfony application has a relative URL, as the route google_login configured in the routing.yml and in the service configuration defined in the security.yml:
routing:
# Required by the HWI OAuth Bundle.
hwi_oauth_redirect:
resource: "#HWIOAuthBundle/Resources/config/routing/redirect.xml"
prefix: /connect
hwi_oauth_connect:
resource: "#HWIOAuthBundle/Resources/config/routing/connect.xml"
prefix: /connect
hwi_oauth_login:
resource: "#HWIOAuthBundle/Resources/config/routing/login.xml"
prefix: /login
# HWI OAuth Bundle route needed for each resource provider.
google_login:
path: /login/check-google
logout:
path: /logout
security:
firewalls:
# disables authentication for assets and the profiler, adapt it according to your needs
dev:
pattern: ^/(_(profiler|wdt)|css|images|js)/
security: false
secured_area:
anonymous: true
logout:
path: /logout
target: /
handlers: [ admin.security.logout.handler ]
oauth:
resource_owners:
google: "/login/check-google"
login_path: /
use_forward: false
failure_path: /
oauth_user_provider:
service: admin.user.provider
So when the application was not dockerized, it run properly because the route requested to be the "redirect route" to google was https://admin-europe.example.com/login/check-google.
Nevertheless, now that it's inside the docker when the HWI bundle is building the data block to send to google it requests for this http://1.2.3.4:7540/login/check-google to be authorised as the "redirect URI" but of course it should not. Of course the redirect URI should continue to be https://admin-europe.example.com/login/check-google.
I naturally get this error message:
The reverse proxy
We already have in the reverse proxy the ProxyPassReverse and, in fact, the very same configuration has been working hassle-free for over a month with another microservice we already successfully moved (but that service did not need auth, was a public site).
This is natural, as ProxyPassReverse will tackle into http data but the google-oauth info-block is not handled by the ProxyPassReverse, as it's natural.
The problem
The problem here is not to have this address validated (put a domain alias into the private IP address, etc.)
The problem here is how to generate the "proper public URL" from inside the docker without creating a hard-dependency for the container contents in function of the environment it's going to run. Doing so would be an anti-pattern.
Exploring solutions
Of course the "easy" solution would be to "hardcode" the "external route" inside the container.
But this has a flaw. If I also want the same docker to be accessed from, say, https://admin-asia.example.com/ (note the -asia instead of the -europe), I'll run into problems as the asia users will be redirected to the europe route. This is a mere example, don't care about the specific europe-asia thing... the point is that the container should not be conscious of the sorrounding architecture. Or at least, conscious to "interact" but definitively not to have "hardcoded" inside the container things that depend on the environment.
Ie: Forget about the -europe and -asia thing. Imagine the access is admin-1111. It does not make sense that I have to "recompile" and "redeploy" the container if one day I want it to be accessible as admin-2222.
Temporal solution
I think it would solve the problem to point both the route in the rounting.yml and the config in the security.yml to a "parameter" (in 3.x in parameters.yml) and then move that into an Environment Variable when updating to SF4, but I'm unsure on how the cache compiler of the symfony would behave with a route that does not have a value, but a route that "changes dynamically".
Then pass the value of the redirecion when the container is started. This would solve the problem only partially: All the container would be bound to a redirect route set at the time of start, but it still would not solve the case of the same container instance accessed via different names thus needing multiple redirect routes. Instead when running non-dockerized that works as it just takes the "hostname" to build the absolute path on a relative-path definition.
Investigation so far
When accessing, the browser shows I'm going to
https://accounts.google.com/o/oauth2/auth
?response_type=code
&client_id=111111111111-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.apps.googleusercontent.com
&scope=email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fplus.profile.emails.read+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fplus.login
&redirect_uri=http%3A%2F%2Fmy.nice.domain.example.com%3A7040%2Fapp_dev.php%2Flogin%2Fcheck-google
Here we see that the redirect_uri parameter is the place where we'll return after passing the control to google momentarily.
So somebody needs to be building this URL.
I seeked for "redirect_uri" within the source code and I found that the involved classes are GoogleResourceOwner which extends GenericOAuth2ResourceOwner.
Both classes seem to belong to the domain as per the tests passing the $redirectUri as a string which needs to be already built by the caller.
The involved method is public function getAuthorizationUrl($redirectUri, array $extraParameters = array()). This receives the redirect URI and builds the auth URI with the redurect URI encoded as a parameter.
So, who are the consumers/clients of getAuthorizationUrl()?
I only found one single client usage in OAuthUtils, in a line that says return $resourceOwner->getAuthorizationUrl($redirectUrl, $extraParameters); within the function public function getAuthorizationUrl(Request $request, $name, $redirectUrl = null, array $extraParameters = array())
I see that mainly this OAuthUtils is acting as an adapter between the Symfony Request and the OAuth domain model. Within this method we mainly find code to create the $redirectUri.
The cleanest solution for me would be to create a child class OAuthUtilsBehindProxy inheriting from OAuthUtils, overwriting the method getAuthorizationUrl() and having it interpret the X-FORWARDED-* headers of the request, and then have the dependency injection to autowire my class everywhere the OAuthUtils is used with the hope that noone is doing a new OAuthUtils and every user of this class is getting it passed on the constructor.
This would be clean and woul work.
But frankly it seems an overkill to me. I'm pretty sure someone before me has put an app that needs Google OAuth made with HWI behind a reverse proxy and I wonder if there's a "config option" that I'm missing or really I have to re-code all this and inject it via D.I.
So, question
How do I have HWI-OAuth bundle to behave properly when running in a docker container behind a reverse proxy in regards on how to build the "redirect route" for the google-oauth service?
Is there any way to tell either the HWI bundle or either symfony to add a "full-host" prefix IN FUNCTION of the the X-FORWARDED-* headers "if available"? This would leave the docker image "fixed" and would run in "any" environment.
The underlying reason is the way Symfony generated the full-addresses from a relative path or route name.
Here's the investigation:
The method HWI/OAuthUtils::getAuthorizationUrl() is the one that generates the OAUth auth URI and consumes the method Symfony/HttpUtils::generateUri() to get the absolute URI of the redirect_to callback that will be encoded inside the Auth URI.
The method Symfony/HttpUtils::generateUri() generates an absolute URI (that in our case will be the callback) and to do so, the method handles 3 general cases:
The parameter is already an absolute URI (the return is the parameter without further processing)
The parameter is a relative URL (the function calls the Request class to build the proto + host + port + project-path prefix to prepend to the relative URI)
The parameter is a route name (the funcion calls the Router class to build the absolute URI)
In my example I was configuring a relative URL (google: "/login/check-google") in the security.yml so HttpUtils was delegating into the Request class.
Looking at the source of the Request class we observe:
The Request class is able to use proxy headers to build the absolute class.
But for security, by default symfony does not trust that a proxy exists merely because there are X-FORWARDED-* headers in it.
Indeed it's more secure plus more flexible.
There are 2 levels of security:
Somewhere we need to tell the Request class what is the list of trusted IPs that are proxies accessing the application.
Somewhere else we need to tell the Request class what specific proxy headers are trusted and what headers are not, even it supports different standards headers (RFC headers, non-RFC apache headers, etc)
Stated here https://symfony.com/blog/fixing-the-trusted-proxies-configuration-for-symfony-3-3 is that you need to configure the trusted proxies in the front-controller by calling the static method Request::setTrustedProxies();
So adding those couple of lines in the front-controller one killing non-nee4ded headers and the other with the IP ranges of the proxies, solved the problem:
# app.php
<?php
use Symfony\Component\HttpFoundation\Request;
$loader = require __DIR__.'/../app/autoload.php';
include_once __DIR__.'/../var/bootstrap.php.cache';
$kernel = new AppKernel('prod', false);
$kernel->loadClassCache();
Request::setTrustedHeaderName( Request::HEADER_FORWARDED, null ); # <-- Kill unneeded header.
Request::setTrustedProxies( [ '192.168.104.0/24', '10.0.0.0/8' ] ); # <-- Trust any proxy that lives in any of those two private nets.
$request = Request::createFromGlobals();
$response = $kernel->handle($request);
$response->send();
$kernel->terminate($request, $response);
With this change:
Symfony Request is able to build correct public absolute addresses from relative addresses if called thru a proxy, by deducting the host from HTTP_X_FORWARDED_HOST and HTTP_X_FORWARDED_PORT instead of HTTP_HOST and SERVER_PORT.
Symfony HttpUtils also, as it was delegating to Request.
HWI is in turn able to build a correct absolute callback redirect_to.
HWI can set the proper callback encoded inside the AuthUri.
The AuthURI that contains the proper absolute URI taking in account the proxy effect is sent to google.
Google sees the "public URI" as the one registered in the google configuration.
The workflow completes and the login process can end successfully.
Apparently I was under the misconception that GET and POST methods differ in the sense that the query parameters are put in plaintext as a part of the URL in GET method and the query parameters are THERE IN THE URL IN ENCODED(ENCRYPTED) FORM .
However , I realize that this was a grave misconception . And after going through :
https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9
and after writing a simple socket server in python and sending it both GET and POST (through form submission) and printing the request in server side
I got to know that only in GET the parameters are there in the URL but in POST the parameters are there in the request body .
I went through the following question as well so as to see if there is any difference in sending a GET and POST at lower level (C-Level) :
Simple C example of doing an HTTP POST and consuming the response
So as in the above question above I saw that there is no special encryption being applied to the POST request .
As such I would like to confirm the following :
1.The insecurities associated with GET and POST are only because of the GET method attaching the parameters in the URL .
For somebody who can have the whole request body , both the GET and POST methods are equally vulnerable .
Over the network , both the GET and POST body are sent with the equal degree of encryption applied to them .
Looking forward to comments and explanations.
Yes. The server only gets to know about the URL the user entered/clicked on because it's sent as the data of the request, after (transport) security has been negotiated so it's not inherently insecure:
you type into a browser: https://myhost.com/a.page?param=value
browser does DNS lookup of myhost.com
browser connects to https port 443 of retrieved ip
browser negotiates security, possibly including myhost.com if the server is using SNI certificates
connection is now encrypted, browser sends request data over the link:
GET /a.page?param=value HTTP/1.1
Host: my host.com
(other headers)
//Probably no body data
---- or ----
POST /a.page HTTP/1.1
Host: my host.com
(other headers)
param=value //body data
You can see it's all just data sent over an encrypted connection, the headers and the body are separated by a blank line. A GET doesn't have to have a body but is not prevented from having one. A POST usually has a body, but the point I'm making is that the data sent (param=value) that is relevant to the request (the stuff the user typed in, potentially sensitive info) is included somewhere in the request - either in the headers or the body - but all of it is encrypted
The only real difference from a security perspective is that the browser history tends to retain the full URL and hence in the case of a GET request would show param=value in the history to the next person reading it. The data in transit is secure for either GET or POST, but the tendency to put sensitive data on a POST centres on the "data at rest" concept in the context of the client browser's history. If the browser kept no history (and the address bar didn't show the parameters to shoulder surfers) then either method would be approximately equivalent to the other
Securing the connection between browser and server is quite simple and then means the existing send/receive data facilities all work without individual attention, but it's by no means the only way of securing connection. It would be conceivably possibly not to have the transport do it but instead for the server to send a piece of JavaScript and a public part of a public/private key pair on the page somewhere, then every request the page [script causes the browser to] makes could have its data individually encrypted and even though an interim observer could see most of the request, the data could be secured that way. It is only decryptable by the server because the server retains the private part of the key pair
I have requirement of changing GET to POST redirection to external URL.
Currently, we are using checkFormRedirect(url,req,res) to redirect to external URL which by default uses GET as per my understanding. I want to change this request to POST.
One way is we can use HTTPClient API for re-direction.
Is there any way ATG out of box provide some thing to POST redirection. Please help.
If you submitted a form in JSP as you are using checkFormRedirect(). It is already a POST request and you can get data in your handlerXXX method.
You can use this method to control redirects. The API call of this method looks somewhat like:-
public boolean checkFormRedirect(pSuccessURL, pFailureURL, pRequest, pResponse);
Now, this method redirects to pSuccessURL if no form errors are found in the form. Otherwise, it redirects to pFailureURL.
Suppose I have URL as
http://someurl.com/Search?q=a&page=8
(Above mentioned URL is getting called throug AJAX, in MVC4.paging)
What I want is to show only upto http://someurl.com/Search?q=a
I want to hide my second parameter which is page=8
Is this possible?
EDIT: More confusion to add.
<a data-ajax="true" data-ajax-loading="#divLoading" data-ajax-method="POST" data-ajax-mode="replace" data-ajax-success="successPaging" data-ajax-update="#searchresults" href="/Search?q=a&page=1" title="Go to first page"><<</a>
Is button of Next in my Paging, it is making an AJAX request, So I don't know how to change GET to POST for this.
The URL isn't there just for looks; it's telling the server what resource is being requested, and in the case of a query string, that's information the server needs to return a response. http://someurl.com/Search?q=a is a completely different resource than http://someurl.com/Search?q=a&page=8. With a GET request, all you have is the URL, so all the information the server needs must be in the URL. What others in the comments are telling you to do is use a POST request, which among other things includes a post body. In other words, you can pass information to the server both in the URL and in the post body. That allows you to remove the page parameter from the URL and include it in the post body instead. That's the only way you can achieve what you want.
That said, strictly speaking, a POST is inappropriate for fetching a resource like this. POST should be used to update or modify a resource or to call some atomic method in an API scenario. It can also be used for the creation of resources, although PUT is more appropriate there. GET is supposed to be used to return a resource which is not variable. For example, any request to http://someurl.com/Search?q=a&page=8 should always return the same response no matter what client requests it. And, it's even less important what URL is actually being used because the user does not see it at all, since you're requesting it via AJAX (it won't show in the navigation bar). Just keep it as a GET request and leave the parameters as they are.
our front end guy needs to form a url containing the hash, (i.e, http://blah/#some-link.) when we hit this on the browser and inspect the http traffic using fiddler, we saw that everything after blah/ gets removed, so the request is really just http://blah/. we also confirmed this on our server eclipse debug log.
the request gets redirected to the correct login page by Spring security(because user hasn't logged in), but the url on the browser now shows:
http://blah/some-link (the hash got removed) but the url on the browser should really be http://blah/log-in.
any idea why this is? any fix or workaround? thanks in advance.
URI part after # is called a fragment:
URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
Scheme and hier-part identify the location of a document, and fragment helps the browser to identify a location inside this document.
Fragment is stripped from URI by client software before it is sent as a part of request.
From RFC3986:
the fragment identifier is not used in the scheme-specific
processing of a URI; instead, the fragment identifier is separated
from the rest of the URI prior to a dereference, and thus the
identifying information within the fragment itself is dereferenced
solely by the user agent, regardless of the URI scheme. Although
this separate handling is often perceived to be a loss of
information, particularly for accurate redirection of references as
resources move over time, it also serves to prevent information
providers from denying reference authors the right to refer to
information within a resource selectively.
Content after the # is only used on the client side, per HTTP specification. If you require that information on the server, you can either use a different separator, or you can submit it via ajax after the page has loaded by reading it on the client with javascript.
The part of the URI including and after the hash (#) is never sent to the server as part of the HTTP request.
The reason is that the hash identifier was originally designed to point at references within the given web page and not to new resources on the server.
If you want to get the hash identifier, you'll have to use some client-side JavaScript to grab the value and submit it with the form.
Hashmark is removed from URL when the back button is clicked in IE9, IE10 or IE11
In IE10 , first time on clicking the HREF link leads to the correct below url:
http://www.example.com/yy/zz/ff/paul.html#20007_14
If back button is clicked again the, then it comes to the below url:
http://www.example.com/yy/zz/ff/paul.html
Solution :
Please change the url with https
It works for me
you can do this with javascript
<script>
if(window.location.hash) {
console.log(window.location.hash);
window.location.hash = window.location.hash;
}
</script>