I am getting occasional errors of this type in production and staging:
Unsafe redirect to "https://${ip}:${port}/businesses/new", pass allow_other_host: true to redirect anyway.
It is being caused by Curl requests from a random IP:
User-Agent: "curl/7.64.1"
Accept: "*/*"
Host: "${ip}:${port}"
Version: "HTTP/1.1"
I do not get it with any other URLs processed within the app.
When I try the same curl request in development I get this (which is correct):
Rails - [ActionDispatch::HostAuthorization::DefaultResponseApp] Blocked host: "${ip}:${port}"
I cannot find where the difference is that makes this throw an exception in production.
Any insights on this issue would be appreciated.
I am using the authentication code mode of Huawei account kit to login users to my app. To check the app server to account server behavior, I use the cURL command shown bellow to obtain the access token from the authorization code. But the following command would return an error.
curl -v -H "Content-Type:application/x-www-form-urlencoded" -d #body.txt -X POST https://oauth-login.cloud.huawei.com/oauth2/v3/token
the "body.txt" file contains the required information for the request:
grant_type=authorization_code&
code=DQB6e3x9zFqHIfkHR2ctp7htDs5tG5p6jXTkTCeoAAULtuS69PntuuD9pwqHrdXyvrlezuRc/aq+zuDU7OnQdRpImnvZcEX+RIOijYMXYu1j+zxpQ+W/J50Z7pY1qhyxZtavqkELY+6o2jSifaiIxC/MJc7KgqKV3jGn9kUIEZovSnM&
client_id=my_id&
client_secret=my_secrete&
redirect_uri=hms://redirect_uri
The command returns:
> POST /oauth2/v3/token HTTP/1.1
> Host: oauth-login.cloud.huawei.com
> User-Agent: curl/7.64.0
> Accept: */*
> Content-Type:application/x-www-form-urlencoded
> Content-Length: 430
>
* upload completely sent off: 430 out of 430 bytes
< HTTP/1.1 400 Bad Request
< Date: Mon, 23 Nov 2020 03:38:21 GMT
< Content-Type: application/json
< Content-Length: 67
< Connection: keep-alive
< Cache-Control: no-store
< Pragma: no-cache
< Server: elb
<
* Connection #0 to host oauth-login.cloud.huawei.com left intact
{"sub_error":20152,"error_description":"invalid code","error":1101}
What should I do to get this API call working using cURL as expected?
Authentication code must be urlencoded before sent. The command in the question used that code without urlencoding non-letter characters. Please use the same command with encoded authorization code as parameter to "code" to perform the request to acquire access token
Encoding could be done inline by if doing so is desired
curl --data-urlencode "para1=value1"
Please refer to: Link or using online tool such as : Link
Using other tools to acquire access token is possible as long as the parameters are properly encoded with %2x format.
According to the error information {"sub_error":20152,"error_description":"invalid code","error":1101}, the problem is caused by incorrect code parameters.
It is recommended that you can check whether the value of code in the request is the same as the Authorization Code obtained by the mobile app.
FOR Details,see docs.
How can I configure uwsgi to pass in the request path unmodified as PATH_INFO? I.e. if there is a request https://example.com/foo%5F/../bar?x=y, I want PATH_INFO to be literally /foo/../%5Fbar, and not /_bar.
The uWSGI documentation says uWSGI is able to rewrite request variables in lot of advanced ways, but I am unable to find any way to set individual request variables, at least not without modifying the source code of uwsgi.
The reason I want to do is that I have a frontend application which takes user input and then sends a request to http://backend.app/get/USER_INPUT. Trouble is, there is an uwsgi in between, and when the user input is ../admin/delete-everything, the request goes to http://backend.app/admin/delete-everything!
(This uwsgi change I desire will not be the only fix; the frontend app should certainly validate user input, and the backend app should not offer /admin to the frontend app in the first place. But as a measure of defense-in-depth, I'd like my requests to pass uwsgi unmodified.)
I am running bare uWSGI without nginx, i.e. uwsgi --http 0.0.0.0:8000 --wsgi-file myapp/wsgi.py --master --processes 8 --threads 2.
For what it's worth, the backend app that looks into PATH_INFO is Django.
My previous answer holds true for the clients which do url parsing at the source. This answer is applicable, when you can actually get the correct request.
The wsgi.py is run by uwsgi and the application object is called as callable. This in case of Django is WSGIHanlder, which has below code
class WSGIHandler(base.BaseHandler):
request_class = WSGIRequest
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.load_middleware()
def __call__(self, environ, start_response):
set_script_prefix(get_script_name(environ))
signals.request_started.send(sender=self.__class__, environ=environ)
print(environ)
request = self.request_class(environ)
response = self.get_response(request)
response._handler_class = self.__class__
status = '%d %s' % (response.status_code, response.reason_phrase)
response_headers = [
*response.items(),
*(('Set-Cookie', c.output(header='')) for c in response.cookies.values()),
]
start_response(status, response_headers)
if getattr(response, 'file_to_stream', None) is not None and environ.get('wsgi.file_wrapper'):
response = environ['wsgi.file_wrapper'](response.file_to_stream)
return response
I created a sample view to test the same
from django.http import HttpResponse
def index(request, **kwargs):
return HttpResponse("Hello, world. You're at the polls index. " + request.environ['PATH_INFO'])
def index2(request, **kwargs):
return HttpResponse("Hello, world. You're at the polls index2. " + request.environ['PATH_INFO'])
and registered them using below code
from django.urls import include, path
from polls.views import index2, index
urlpatterns = [
path('polls2/', index2, name='index2'),
path('polls2/<path:resource>', index2, name='index2'),
path('polls/', index, name='index'),
path('polls/<path:resource>', index, name='index'),
]
So what you need is overriding this class. Below is an example
import django
from django.core.handlers.wsgi import WSGIHandler
class MyWSGIHandler(WSGIHandler):
def get_response(self, request):
request.environ['ORIGINAL_PATH_INFO'] = request.environ['PATH_INFO']
request.environ['PATH_INFO'] = request.environ['REQUEST_URI']
return super(MyWSGIHandler, self).get_response(request)
def get_wsgi_application():
"""
The public interface to Django's WSGI support. Should return a WSGI
callable.
Allows us to avoid making django.core.handlers.WSGIHandler public API, in
case the internal WSGI implementation changes or moves in the future.
"""
django.setup()
return MyWSGIHandler()
application = get_wsgi_application()
After this can you can see the below results
$ curl --path-as-is "http://127.0.0.1:8000/polls/"
Hello, world. You're at the polls index. /polls/
$ curl --path-as-is "http://127.0.0.1:8000/polls2/"
Hello, world. You're at the polls index2. /polls2/
$ curl "http://127.0.0.1:8000/polls2/../polls/"
Hello, world. You're at the polls index. /polls/
$ curl --path-as-is "http://127.0.0.1:8000/polls2/../polls/"
Hello, world. You're at the polls index. /polls2/../polls/%
As you can see the change to PATH_INFO doesn't change which view is picked. As polls2 still picks index function
After digging a bit more, I realised there is another path and path_info variable. The class for the same is picked using path_info
So we update our function like below
class MyWSGIHandler(WSGIHandler):
def get_response(self, request):
request.environ['ORIGINAL_PATH_INFO'] = request.environ['PATH_INFO']
request.environ['PATH_INFO'] = request.environ.get('REQUEST_URI', request.environ['ORIGINAL_PATH_INFO'])
request.path = request.environ['PATH_INFO']
request.path_info = request.environ.get('REQUEST_URI', request.environ['PATH_INFO'])
return super(MyWSGIHandler, self).get_response(request)
After this change, we get the desired results
$ curl --path-as-is "http://127.0.0.1:8000/polls2/../polls/"
Hello, world. You're at the polls index2. /polls2/../polls/
So your problem has mostly nothing to do with uwsgi or Django as such. To demonstrated the issue, I created a simple flask app with a catch all handler
from flask import Flask
app = Flask(__name__)
#app.route('/', defaults={'path': ''})
#app.route('/<path:path>')
def catch_all(path):
return 'You want path: %s' % path
if __name__ == '__main__':
app.run()
Now when you run this and make a curl request
$ curl -v http://127.0.0.1:5000/tarun/../lalwani
* Rebuilt URL to: http://127.0.0.1:5000/lalwani
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 5000 (#0)
> GET /lalwani HTTP/1.1
> Host: 127.0.0.1:5000
> User-Agent: curl/7.54.0
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 22
< Server: Werkzeug/0.15.2 Python/3.7.3
< Date: Fri, 26 Jul 2019 07:45:16 GMT
<
* Closing connection 0
You want path: lalwani%
As you can see that the server never had a chance to even know we requested this. Now lets do it again and ask curl not to tamper the url
$ curl -v --path-as-is http://127.0.0.1:5000/tarun/../lalwani
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 5000 (#0)
> GET /tarun/../lalwani HTTP/1.1
> Host: 127.0.0.1:5000
> User-Agent: curl/7.54.0
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 31
< Server: Werkzeug/0.15.2 Python/3.7.3
< Date: Fri, 26 Jul 2019 07:48:17 GMT
<
* Closing connection 0
You want path: tarun/../lalwani%
Now you can see that my app did receive the actual path. Now let's see the same case in a browser, with app not even running
As you can even though my service is not even running but the browser itself refactored the call to /lalwani instead of /tarun/../lalwani. So there is nothing that could have been done at your end to even correct the issue, until unless you are using a client which supports disabling the url parsing at source
After deploying my rails 3 app in production I noticed that paths are not always shown in the browser window. For example going to login or my_profile links would still only show http://my_app.com instead of expected http://my_app.com/login or http://my_app.com/my_profile. The views did change and were functional. I could also see the database being hit and views being rendered from the logs (which led me to believe it was not a simple browser cache issue). Going directly to http://my_app.com/login worked, however, using the links in the app would take me to the expected place while leaving the login url displayed. I tried it in several browsers (firefox, opera and chrome) and got the same behavior. The app was deployed under nginx + passenger and later nginx + thin cluster. My question is, whats going on? Could it be nginx settings or my production environment settings? I am not sure where to start.
Running curl -v my_app.com shows
* About to connect() to my_app.com port 80 (#0)
* Trying xx.xx.xx.xx... connected
* Connected to my_app.com (xx.xx.xx.xx) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.21.1 (x86_64-apple-darwin10.4.0) libcurl/7.21.1 OpenSSL/1.0.0a zlib/1.2.5 libidn/1.19
> Host: my_app.com
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Set-Cookie: ARPT=PKKIKIS10.0.81.64CKILJ; path=/
< Content-Type: text/html; charset=utf-8
< Status: 200
< X-Powered-By: Phusion Passenger (mod_rails/mod_rack) 2.2.5
< ETag: "fce6dec3543058bec16175466020a906"
< X-Runtime: 7
< Content-Length: 787
< Cache-Control: private, max-age=0, must-revalidate
< Server: nginx/0.7.62 + Phusion Passenger 2.2.4 (mod_rails/mod_rack)
< P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT"
< X-Cache: MISS from server.com
< Via: 1.0 server.com:8080
< Connection: close
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN"
"http://www.w3.org/TR/html4/frameset.dtd">
<html>
<head>
<title>http://my_app.com/</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=EmulateIE7" />
<meta name="generator" content="Hover Redirect Service">
</head>
<frameset framespacing="0" rows="100%,*" cols="100%" frameborder="no" border="0">
<frame name="DDIRECTXYZZY2" scrolling="auto" src="http://xxx.xx.xxx.xxx" noresize>
<frame name="DDIRECTXYZZY" scrolling="no" noresize>
<noframes>
<h1>http://my_app.com/</h1>
<p>Please click here to view the non-framed versi
on.</p>
</noframes>
</frameset>
</html>
So that is clearly a problem. The whole thing gets framed by the DNS redirect? The setup is not Phusion Passenger + nginx. It was originally, but now its thin + nginx. Also, when going directly to ip address of the app, things are just fine. When going to the domain name, I get the framed version. curl -v response of just the ip address also looks normal (like its loading the entire page).
The problem is almost certainly your frames. The inner frame is loading the correct content but because you're (presumably) not targeting 'top' in your links (or whatever it is), the browser is still showing the URL of the outermost frame.
Targeting frames: http://www.w3.org/TR/html4/present/frames.html#h-16.3
If you were not expecting to see frames in your response at all then it's likely they're the responsibility of a shoddy "DNS" service. Get a real DNS address pointed to your server and you'll be singing.
Did you try using another client computer to do your tests? If I follow your explication, that's one option you haven't investigated.
Can you send us your nginx configuration maybe?
My Ruby on Rails application uses the following controller code to generate a sitemap.xml file:
class SitemapController < ApplicationController
layout nil
def index
headers['Content-Type'] = 'application/xml'
last_post = Post.last
if stale?(:etag => last_post, :last_modified => last_post.updated_at.utc)
respond_to do |format|
format.xml { #posts = Post.sitemap } # sitemap is a named scope
end
end
end
end
My understanding is that the stale? method should ensure a HTTP 304 Not Modified response if the content hasn't changed. However, whenever I test this using curl or a web browser I always get an HTTP 200:
$ curl --head localhost:3000/sitemap.xml
HTTP/1.1 200 OK
Connection: close
Date: Mon, 13 Apr 2009 15:50:00 GMT
Last-Modified: Wed, 08 Apr 2009 16:52:07 GMT
X-Runtime: 100
ETag: "5ff2ed60ddcdecf291e7191e1ad540f6"
Cache-Control: private, max-age=0, must-revalidate
Content-Type: application/xml; charset=utf-8
Content-Length: 29318
Am I using the stale? method correctly? Is it even possible to test this locally?
it is likely that your Rails code is just fine but curl is not sending the If-Modified-Since header when you perform your test. From the curl docs:
TIME CONDITIONS
HTTP allows a client to specify a time
condition for the document it
requests. It is If-Modified-Since or
If-Unmodified-Since. Curl allow you to
specify them with the -z/--time-cond
flag.
For example, you can easily make a
download that only gets performed if
the remote file is newer than a local
copy. It would be made like:
curl -z local.html
http://remote.server.com/remote.html
Or you can download a file only if the
local file is newer than the remote
one. Do this by prepending the date
string with a '-', as in:
curl -z -local.html
http://remote.server.com/remote.html
You can specify a "free text" date as
condition. Tell curl to only download
the file if it was updated since
yesterday:
curl -z yesterday
http://remote.server.com/remote.html
Curl will then accept a wide range of
date formats. You always make the date
check the other way around by
prepending it with a dash '-'.