Our app provides an API that people can use to submit URLs like this:
curl -X POST http://app.local/resource -d'url=http://news.google.com/newshl=en&q=obama&um=1&ie=UTF-8&output=rss'
Unfortunately, it seems that Rails messes up with this param. Any idea on how to fix this?
See the log below :
Processing ApplicationController#index (for 127.0.0.1 at 2010-06-08 19:03:09) [POST]
Parameters: {"um"=>"1", "url"=>"http://news.google.com/newshl=en", "output"=>"rss", "q"=>"obama", "ie"=>"UTF-8"}
I would expect the following :
Parameters: {"url"=>"hhttp://news.google.com/newshl=en&q=obama&um=1&ie=UTF-8&output=rss"}
What exactly Rails messes up?
If you are referring to the fact that it didn't get complete Google URL (i.e. separated it to output, q and other params) that's because you need to encode '&' character if you want to use it as a part of a value. Something like:
curl -X POST http://app.local/resource -d'url=http://news.google.com/newshl=en%26q=obama%26um=1%26ie=UTF-8%26output=rss'
Related
When running my own mediawiki on localhost I run into a problem with the api endpoint for editing a page. The api works fine otherwise. For instance when querying tokens I get the following output:
$ curl "http://localhost/api.php?action=query&meta=tokens&type=createaccount|csrf|login&format=json"
{"batchcomplete":"","query":{"tokens":{"createaccounttoken":"1e5c2ce3f9e12fdab05a0e6e6352da3162644214+\\","csrftoken":"+\\","logintoken":"35ee2e6ccbbd654bcfada9cddab07f7662634214+\\"}}}
Interestingly the csrftoken has the strange value '+\\', which also seems to be appended to the logintoken.
When posting an edit action (for an existing page called Alice) with this token I get the reponse that the token is missing from the post body. But it's not missing, or is it?
$ curl -X POST "http://localhost/api.php?action=edit&title=Alice&summary=test&text=article&baserevid=0&token=+\\&format=json"
{"error":{"code":"mustpostparams","info":"The following parameter was found in the query string, but must be in the POST body: token.","*":"See http://localhost/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes."}}
Then I followed this advice and added the cookie-jar option to make the curl request from the same session, but it gave the same error response. Any ideas?
By the way I used the docker.io/bitnami/mediawiki:1 docker image together with a mariadb to set up my mediawiki.
The following parameter was found in the query string, but must be in the POST body
I'm not very familiar with curl but it seems that you are sending a post request with the params in the query part. Try sending them in the post body as the error suggests:
curl --data "action=edit&title=Alice&summary=test&text=article&baserevid=0&token=+\\&format=json" http://localhost/api.php
I'm having an odd problem with Ruby on Rails globbing. My route looks like this:
get 'people/info_from_url/*url', to: 'people#info_from_url'
So from the frontend, I have get requests to URLs like:
my-api.com/people/info_from_url/youtube.com/XXX
my-api.com/people/info_from_url/twitter.com/XXX
my-api.com/people/info_from_url/tinyurl.com/XXX
...
These all work as expected - I get a parameter in the people#info_from_url controller action called url that contains the full URL that was sent.
However, for one particular kind of URL (XXX.carrd.co), the last part gets cut off. In the frontend, I send a get request to my-api.com/people/info_from_url/XXX.carrd.co. From the backend logs:
INFO -- : Started GET "/people/info_from_url/XXX.carrd.co/"
INFO -- : Processing by PeopleController#info_from_url as
INFO -- : Parameters: {"url"=>"XXX.carrd"}
Somewhere, the .co gets dropped. I'm not sure why this could be happening or how to debug it, since the change happens before I'm able to access the params hash. I could deal with this manually by just checking if it's a carrd link, but I'd like to know why it's happening and if any other kind of link might experience this issue. Thanks for any help!
Routes have an implicit optional (.:format) segment at the end. You can use this option to prevent that:
get 'people/info_from_url/*url', to: 'people#info_from_url', format: false
EDIT: you can check the last part here in the docs explaining that too https://guides.rubyonrails.org/routing.html#route-globbing-and-wildcard-segments
I'm trying to pass a URL as a param to my Rails app:
Started DELETE "/images/0?s3_filepath=https://s3.amazonaws.com/buildinprogresstest/uploads/blllbyq5k3qinl4l/uploads_2F9qggxxf5dvlsor-667601c8f38d8d41af07828accbf3147_2F2014-07-13%252B18.44.29.jpg" for 127.0.0.1 at 2016-08-31 11:52:09 -0400
The s3_filepath is not being properly parsed in the params:
Parameters: {"s3_filepath"=>"https://s3.amazonaws.com/buildinprogresstest/uploads/blllbyq5k3qinl4l/uploads_2F9qggxxf5dvlsor-667601c8f38d8d41af07828accbf3147_2F2014-07-13%2B18.44.29.jpg", "id"=>"0"}
If you look closely, the filename includes the sequence "252B18" but the params seems to remove the numbers "52"
I'm at a loss as to why this is happening. Any ideas?
Normally parameters are url-encoded and decoded on rails side. %25 is decoded to %, that's why it is removed from your input. You need to properly encode this url.
In Ruby you can use CGI.escape
CGI.escape "https://s3.amazonaws.com/buildinprogresstest/uploads/blllbyq5k3qinl4l/uploads_2F9qggxxf5dvlsor-667601c8f38d8d41af07828accbf3147_2F2014-07-13%252B18.44.29.jpg"
=> "https%3A%2F%2Fs3.amazonaws.com%2Fbuildinprogresstest%2Fuploads%2Fblllbyq5k3qinl4l%2Fuploads_2F9qggxxf5dvlsor-667601c8f38d8d41af07828accbf3147_2F2014-07-13%25252B18.44.29.jpg"
If you send this request via javascript you can use escape function in javascript
escape("https%3A%2F%2Fs3.amazonaws.com%2Fbuildinprogresstest%2Fuploads%2Fblllbyq5k3qinl4l%2Fuploads_2F9qggxxf5dvlsor-667601c8f38d8d41af07828accbf3147_2F2014-07-13%25252B18.44.29.jpg")
I issued a HTTP request using CURL.
curl -X GET \
http://localhost:3000/api/v1/user_api/35/edit?old_password=aa&new_password=bb&password_confirmation=cc
My rails server only recognise the first parameter
Started GET "/api/v1/user_api/35/edit?old_password=aa" for 127.0.0.1 at 2013-08-01 14:12:52 +0100
Processing by Api::V1::UserApiController#edit as */*
Parameters: {"old_password"=>"aa", "id"=>35}
Any idea what is wrong here?
-------------------------------------Edit-----------------------------------------------------
WARNING! This is a very bad wild insane example of handling password reset.
The question is only aimed for topic of curl and query parsing issues.
Thank you for all your comments
Any idea what is wrong here?
You're passing passwords in plain text in the query string.
How about you start by not doing that?
Also, you need to put your URL in quotes as the & otherwise needs to escaped
curl -X GET \
"http://localhost:3000/api/v1/user_api/35/edit?old_password=aa&new_password=bb&password_confirmation=cc"
But really, please don't use GET to send password data
Hitting a bit of a brick wall here. I'm trying to send a string containing line breaks (\n, turned to \u000a by JSON.stringify) as part of a JSON object over to a Rails app:
{"bob":{"id":46,"notes":"foo\u000abar\u000abaz"}}
This goes over the wire as this, with \u000a escaped as %5Cu000a:
http://localhost/bobs/46?draft=true&%7B%22bob%22%3A%7B%22id%22%3A46%2C%22notes%22%3A%22foo%5Cu000abar%5Cu000abaz%22%7D%7D=
But the second the request hits Rubyland, the newlines disappear in a puff of ether, turning into spaces:
Processing Api::BobsController#update (for 127.0.0.1 at 2011-05-19 11:01:43) [PUT]
Parameters: {"draft"=>"true", "action"=>"update", "id"=>"46", "controller"=>"api/bobs", "bob"=>{"notes"=>"foo bar baz", "id"=>46}
And it's not just some logging artifact, but they're going into the database that way as well:
ree-1.8.7-2010.02 > Bob.find_by_id(46)
=> #<Bob id: 46, notes: "foo bar baz"...>
If I send eg. "\\n" instead of "\n", they come through fine:
Processing Api::BobsController#update (for 127.0.0.1 at 2011-05-19 11:01:43) [PUT]
Parameters: {"draft"=>"true", "action"=>"update", "id"=>"46", "controller"=>"api/bobs", "bob"=>{"notes"=>"foo\\nbar\\nbaz", "id"=>46}
What's going on, and why?
Update: A colleague vaguely recalls hearing that Passenger has been suspected of dropping some special chars, but he can't find a reference to back this up, and neither can I...?
This could be nothing, but aren't PUT methods meant to be POST'ed in RESTful Rails? GET-ing any URL should be repeatable w/o any change to the database.
If you changed your AJAX call to post you could also indicate proper content-type of application/json so Rails knows how to handle it.