I made an ajax request to rails. The URL is:
/learners/-638284588?is_combined_page=true&email=test0221k+staging#tt.com&type=ld"
In the log, I see these entries:
Started GET "/learners/-638284588?is_combined_page=true& email=test0221k+staging#rosettastone.com&type=lcd" for 127.0.0.1 at Fri Feb 23 09:31:56 -0500 2018
Processing by Extranet::LearnersController#show as HTML
Parameters: {"type"=>"ld", "email"=>"test0221k staging#tt.com", "is_combined_page"=>"true", "id"=>"-638284588"}
[WARNING] Audit logging has been enabled for Account
parameter ----------------> {"type"=>"lcd", "email"=>"test0221k staging#tt.com", "controller"=>"extranet/learners", "is_combined_page"=>"true", "action"=>"show", "id"=>"-638284588"} test0221k staging#tt.com
In the URL, the email parameter appears as:
email="test0221k+staging#tt.com"
which has a + sign. But when I read the parameter from params[:email], it is printed as:
"email"=>"test0221k staging#tt.com"
in which the + is replaced by a space.
Why does rails overwrite + with a space?
How can I avoid this "test0221k staging#tt.com" and get "test0221k+staging#tt.com"?
This is defined by the URL specification. Rails is just confirming to that spec.
Parameters need to be encoded before putting them in the URL. The easy way to do this is to call encodeURIComponent() in your JavaScript.
Forms do this encoding for you automatically.
Related
Please bare with me for a question for which it's nearly impossible to create a reproducible example.
I have an API setup with FastAPI using Docker, Serverless and deployed on AWS API Gateway. All routes discussed are protected with an api-key that is passed into the header (x-api-key).
I'm trying to accomplish a simple redirect from one route to another using fastapi.responses.RedirectResponse. The redirect works perfectly fine locally (though, this is without api-key), and both routes work perfectly fine when deployed on AWS and connected to directly, but something is blocking the redirect from route one (abc/item) to route two (xyz/item) when I deploy to AWS. I'm not sure what could be the issue, because the logs in CloudWatch aren't giving me much to work with.
To illustrate my issue let's say we have route abc/item that looks like this:
#router.get("/abc/item")
async def get_item(item_id: int, request: Request, db: Session = Depends(get_db)):
if False:
redirect_url = f"/xyz/item?item_id={item_id}"
logging.info(f"Redirecting to {redirect_url}")
return RedirectResponse(redirect_url, headers=request.headers)
else:
execution = db.execute(text(items_query))
return convert_to_json(execution)
So, we check if some value is True/False and if it's False we redirect from abc/item to xyz/item using RedirectResponse(). We pass the redirect_url, which is just the xyz/item route including query parameters and we pass request.headers (as suggested here and here), because I figured we need to pass along the x-api-key to the new route. In the second route we again try a query in a different table (other_items) and return some value.
I have also tried passing status_code=status.HTTP_303_SEE_OTHER and status_code=status.HTTP_307_TEMPORARY_REDIRECT to RedirectResponse() as suggested by some tangentially related questions I found on StackOverflow and the FastAPI discussions, but that didn't help either.
#router.get("/xyz/item")
async def get_item(item_id: int, db: Session = Depends(get_db)):
execution = db.execute(text(other_items_query))
return convert_to_json(execution)
Like I said, when deployed I can successfully connect directly to both abc/item and get a return value if True and I can also connect to xyz/item directly and get a correct value from that, but when I pass a value to abc/item that is False (and thus it should redirect) I get {"message": "Forbidden"}.
In case it can be of any help, I try debugging this using a "curl" tool, and the headers I get returned give the following info:
Content-Type: application/json
Content-Length: 23
Connection: keep-alive
Date: Wed, 27 Jul 2022 08:43:06 GMT
x-amzn-RequestId: XXXXXXXXXXXXXXXXXXXX
x-amzn-ErrorType: ForbiddenException
x-amz-apigw-id: XXXXXXXXXXXXXXXX
X-Cache: Error from cloudfront
Via: 1.1 XXXXXXXXXXXXXXXXXXXXXXXXX.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: XXXXX
X-Amz-Cf-Id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
So, this is hinting at a CloudFront error. Unfortunately I don't see anything slightly hinting at this API when I look into my CloudFront dashboard on AWS, there literally is nothing there (I do have permissions to view the contents though...)
The API logs in CloudWatch look like this:
2022-07-27T03:43:06.495-05:00 Redirecting to /xyz/item?item_id=1234...
2022-07-27T03:43:06.495-05:00 [INFO] 2022-07-27T08:43:06.495Z Redirecting to /xyz/item?item_id=1234...
2022-07-27T03:43:06.496-05:00 2022-07-27 08:43:06,496 INFO sqlalchemy.engine.Engine ROLLBACK
2022-07-27T03:43:06.496-05:00 [INFO] 2022-07-27T08:43:06.496Z ROLLBACK
2022-07-27T03:43:06.499-05:00 END RequestId: 6f449762-6a60189e4314
2022-07-27T03:43:06.499-05:00 REPORT RequestId: 6f449762-6a60189e4314 Duration: 85.62 ms Billed Duration: 86 ms Memory Size: 256 MB Max Memory Used: 204 MB
I have been wondering if my issue could be related to something I need to add to somewhere in my serverless.yml, perhaps in the functions: part. That currently looks like this for these two routes:
events:
- http:
path: abc/item
method: get
cors: true
private: true
request:
parameters:
querystrings:
item_id: true
- http:
path: xyz/item
method: get
cors: true
private: true
request:
parameters:
querystrings:
item_id: true
Finally, it's probably good to note that I have added custom middleware to FastAPI to handle the two different database connections I need for connecting to other_items and items tables, though I'm not sure how relevant this is, considering this functions fine when redirecting locally. For this I implemented the solution found here. This custom middleware is the reason for the redirect in the first place (we change connection URI based on route with that middleware), so I figured it's good to share this bit of info as well.
Thanks!
As noted here and here, it is mpossible to redirect to a page with custom headers set. A redirection in the HTTP protocol doesn't support adding any headers to the target location. It is basically just a header in itself and only allows for a URL (a redirect response though could also include body content, if needed—see this answer). When you add the authorization header to the RedirectResponse, you only send that header back to the client.
A suggested here, you could use the set-cookie HTTP response header:
The Set-Cookie HTTP response header is used to send a cookie from the
server to the user agent (client), so that the user agent can send it back to
the server later.
In FastAPI—documentation can be found here and here—this can be done as follows:
from fastapi import Request
from fastapi.responses import RedirectResponse
#app.get("/abc/item")
def get_item(request: Request):
redirect_url = request.url_for('your_endpoints_function_name') #e.g., 'get_item'
response = RedirectResponse(redirect_url)
response.set_cookie(key="fakesession", value="fake-cookie-session-value", httponly=True)
return response
Inside the other endpoint, where you are redirecting the user to, you can extract that cookie to authenticate the user. The cookie can be found in request.cookies—which should return, for example, {'fakesession': 'fake-cookie-session-value-MANUAL'}—and you retrieve it using request.cookies.get('fakesession').
On a different note, request.url_for() function accepts only path parameters, not query parameters (such as item_id in your /abc/item and /xyz/item endpoints). Thus, you can either create the URL in the way you already do, or use the CustomURLProcessor suggested here, here and here, which allows you to pass both path and query parameters.
If the redirection takes place from one domain to another (e.g., from abc.com to xyz.com), please have a look at this answer.
I am trying to access a private API endpoint that accepts a phone number, including the country code, in the URL query parameters. Because the phone number has to include the "+", the phone number has to be encoded (i.e. the '+' becomes '%2B')
When I call this:
RestClient.get("mytestserver.com/get_info?phone=%2B11231231234") in ruby, i get back an invalid response, that indicates its hitting the server, but the value isn't being read correctly.
The same call in the browser, works.
I was able to get it to work using the Net::HTTP class, by doing the following:
uri = URI("http://mytestserver.com/get_info")
uri.query = URI.encode_www_form({phone: "+11231231234"})
response = Net::HTTP.get_response(uri)
But I wanted to see if there was a way to do it via RestClient?
It works like this.
phone = CGI.escape('+11231231234')
RestClient.get("localhost:3000/test?phone=#{phone}")
logs on server:
Started GET "/test?phone=%2B11231231234" for ::1 at 2020-09-25 11:13:13 +0100
Processing by TestersController#index as */*
Parameters: {"phone"=>"+11231231234"}
Rendering testers/index.html.erb within layouts/application
I have some ajax that is POSTing data to a URL, and I can see Chrome sending off the data :
Request URL:http://ubuntu:3000/groups?authenticity_token=EjAsrE07jziAMLt918Mgid4PSpRFfjIaz%2Bd9ZCxmbTo%3D
Request Payload
{"object":{"name":"test","description":"testing","is_era":false,"unique_ui_identifier":"db3772f13e8c5ec958105c72f11b7b89"}}
I then look at the dev logs, and the server shows the following:
Started POST "/groups?authenticity_token=EjAsrE07jziAMLt918Mgid4PSpRFfjIaz%2Bd9ZCxmbTo%3D" for 192.168.222.1 at Mon Aug 26 10:46:19 +0930 2013
Processing by GroupsController#create as */*
Parameters: {"group"=>{}, "authenticity_token"=>"EjAsrE07jziAMLt918Mgid4PSpRFfjIaz+d9ZCxmbTo="}
So, the server doesn't seem to be receiving the POST data. i.e. the 'object' with name/description/etc.
A few things I find strange:
It seems to be receiving an empty 'group' hash, but I have no idea where this would be coming from.
Is this strange: 'Processing by GroupsController#create as /'...what is the / ?
If anyone can help me, I would be very appreciative.
I've got a production app and occasionally I see this error appear in the logs.. I can't seem to duplicate it on my end, so it seems like there is a user out there somewhere with some specific settings / cookies / etc that are causing this problem. I am not sure... The log shows that this user was redirected to this callback url: "users/auth/facebook/callback?code=AQCayaAoFOruFgwbfg1D682j8DbxOt0CZYNH3Vv5RtYKlQgSzISyN8ygTn25W_RTl3fu35cS1-tl5ArZ9B_XylwORP0hGU6st8P6TyTYUzfiR1m0poaSRkX-KBeWiBvT6IUsm-Af0VJcUNTQPg-dM1F9y5CgJ2bTJEJqhCE9wYlvkUY3kguwcl3TQ48FTT4-PhA///"
The actual error is:
RuntimeError: #<OAuth2::Response:0x1da7fae0 #error=#<OAuth2::Error: OAuth2::Error>, #options={:parse=>:query}, #parsed={"{\"error\":{\"message\":\"Invalid verification code format.\",\"type\":\"OAuthException\"}}"=>nil}, #response=#<Faraday::Response:0x1da8fa80 #on_complete_callbacks=[], #env={:response=>#<Faraday::Response:0x1da8fa80 ...>, :request_headers=>{"Content-Type"=>"application/x-www-form-urlencoded"}, :body=>"{\"error\":{\"message\":\"Invalid verification code format.\",\"type\":\"OAuthException\"}}", :status=>400, :url=>#<Addressable::URI:0xed52bf0 URI:https://graph.facebook.com/oauth/access_token>, :request=>{:proxy=>nil}, :parallel_manager=>nil, :response_headers=>{"expires"=>"Sat, 01 Jan 2000 00:00:00 GMT", "access-control-allow-origin"=>"*", "content-type"=>"text/javascript; charset=UTF-8", "connection"=>"close", "www-authenticate"=>"OAuth \"Facebook Platform\" \"invalid_code\" \"Invalid verification code format.\"", "date"=>"Thu, 13 Oct 2011 15:58:29 GMT", "content-length"=>"81", "cache-control"=>"no-store", "x-fb-rev"=>"457598", "x-fb-server"=>"10.65.13.60", "pragma"=>"no-cache"}, :ssl=>{:ca_file=>"/etc/pki/tls/certs/ca-bundle.crt"}, :method=>:post}>>
and the parameters were:
{"code"=> "AQCayaAoFOruFgwbfg1D682j8DbxOt0CZYNH3Vv5RtYKlQgSzISyN8ygTn25W_RTl3fu35cS1-tl5ArZ9B_XylwORP0hGU6st8P6TyTYUzfiR1m0poaSRkX-KBeWiBvT6IUsm-Af0VJcUNTQPg-dM1F9y5CgJ2bTJEJqhCE9wYlvkUY3kguwcl3TQ48FTT4-PhA///",
"action"=>"",
"controller"=>""}
I know if I manually go to "users/auth/facebook/callback?code=blah" it will trigger this same error because the callback code parameter is obviously bogus, but it doesn't look like there's anything suspicious about the actual code in the user's params hash-- so I am wondering why it would be an invalid format?
Has anyone else experienced this?
I think the following link provides the answer: http://developers.facebook.com/docs/authentication/#authenticating-users-in-a-web-application
You first need to call https://graph.facebook.com/oauth/authorize with your client_id and redirect_uri. This will then redirect you back to the redirect_uri, with a verification code in the query string that you can pass to your https://graph.facebook.com/oauth/access_token call (via the code parameter) to exchange for an oAuth access token.
Good luck! :)
In my Rails log file, I see lots of
Started GET "/" for 63.148.78.244 at Fri Sep 24 19:03:39 +0000 2010
Processing by ProductsController#index as HTML
I understand this means Rails is serving up an HTML page. However, what does this mean?
Started GET "/" for 63.148.78.244 at Fri Sep 24 18:05:51 +0000 2010
Processing by ProductsController#index as */*
Completed in 0ms
Why the */*?
It depends on the HTTP_ACCEPT header that is sent by the browser. The common scenario is that the browser sends list of all MIME types that can process and server returns the result in one of them - typically HTML.
But in some cases it's not this way. For example if you use wget without any other parameters.
Try
wget http://yourserver
and you will see in your log file * / * which means that the "browser" accepts anything you will send back (it's quite obvious that wget can accept anything as it is just storing it into the file).