When hitting an API in a Rails app, it seems that certain headers fail to be parsed. For example, this works:
Accept: application/json
But this doesn't:
Accept: application/json; charset=utf-8
Is failing to understand this 2nd header a legitimate problem with the header? Even if it is, how can I make Rails understand it, or at least understand that it's a request for Json?
The issue is that, in Rails 3.2.14, the Mime::Type.parse method does not support specifying a charset parameter in the Accept header; in fact it only supports the "q" parameter (see the Q_SEPARATOR_REGEXP constant).
In Rails 4, however, the method has been updated to support arbitrary parameters (see this commit), so upgrading to Rails 4 should fix this problem.
If you can't upgrade to Rails 4, I would suggest doing a temporary hack in some Rack middleware to strip out the charset (this assumes you don't actually intend to honour the acceptable charset):
class AcceptCharsetStripper
def initialize(app)
#app = app
end
def call(env)
env["HTTP_ACCEPT"].gsub!(/;\s*charset=\S+/, "")
#app.call(env)
end
end
Related
This question already has answers here:
Trigger Rack middleware on specific Rails routes
(3 answers)
Closed 7 years ago.
I'm integrating a 3rd party API, and at one step they post JSON data into our server.
The content type is application/json, but the payload is actually gzipped. Due to the content type, rails is throwing an ActionDispatch::ParamsParser::ParseError exception when trying to parse the data.
I plan to write a rack middleware to catch the exception and try and uncompress the payload, but I only want to do it for that particular action, is there a way to execute the middleware only for specific routes?
Update
I'd already thought about pattern matching the path, but ideally I'd like something that is a little more robust in case paths change in the future.
Update
The issue regarding compression is better approached by matt's comment below, and the routing is answered in another question that this is now marked a duplicate of.
You can easily to do a regex match on the path, and only execute uncompress code when the path matches. In this case, you are not skipping the middleware for all other routes, but processing the check should be very lightweight. Place this in lib/middleware/custom_middleware.rb:
class CustomMiddleware
def initialize(app)
#app = app
end
def call(env)
#req = Rack::Request.new(env)
if should_use_this?
# Your custom code goes here.
end
end
def should_use_this?
#req.path =~ /^matchable_path/ # Replace "matchable_path" with the path in question.
end
end
Then put the following in your config/environments/production.rb
config.middleware.use "CustomMiddleware"
I have a custom rack middleware used by my Rails 4 application. The middleware itself is just here to default Accept and Content-Type headers to application/json if the client did not provide a valid information (I'm working on an API). So before each request it changes those headers and after each request it adds a custom X-Something-Media-Type head with a custom media type information.
I would like to switch to Puma, therefore I'm a bit worried about the thread-safety of such a middleware. I did not play with instances variables, except once for the common #app.call that we encounter in every middleware, but even here I reproduced something I've read in RailsCasts' comments :
def initialize(app)
#app = app
end
def call(env)
dup._call(env)
end
def _call(env)
...
status, headers, response = #app.call(env)
...
Is the dup._call really useful in order to handle thread-safety problems ?
Except that #app instance variable I only play with the current request built with the current env variable :
request = Rack::Request.new(env)
And I call env.update to update headers and forms informations.
Is it dangerous enough to expect some issues with that middleware when I'll switch from Webrick to a concurrent web server such as Puma ?
If yes, do you know a handful way to make some tests en isolate portions of my middleware which are non-thread-safe ?
Thanks.
Yes, it's necessary to dup the middleware to be thread-safe. That way, anything instance variables you set from _call will be set on the duped instance, not the original. You'll notice that web frameworks that are built around Rack work this way:
Pakyow
Sinatra
One way to unit test this is to assert that _call is called on a duped instance rather than the original.
I've written functional tests for API endpoints built in Rails using shoulda testing framework.
An example looks like the following:
setup do
authenticated_xml_request('xml-file-name')
post :new
end
should respond_with :success
authenticated_xml_request is a test helper method that sets
#request.env['RAW_POST_DATA'] with XML content.
After upgrading the app from rails 2.3.3 to rails 2.3.8, the functional tests are failing because the XML content received is not merged in the params hash.
I'm setting the request with the correct mime type via #request.accept =
"text/xml"
I'm able to inspect the content of the request using request.raw_post but i'd like to keep the current setup working.
Also, running a test from the terminal using cURL or any other library (rest_http) in development mode, the API works perfectly well.
Any tips or help is much appreciated.
Now it's simpler:
post "/api/v1/users.xml", user.to_xml, "CONTENT_TYPE" => 'application/xml'
Note that you have to specify appropriate "CONTENT_TYPE". In other case your request will go as 'application/x-www-form-urlencoded' and xml won't be parsed properly.
I solved the issue by adding a custom patch to rails (test_process.rb file) to convert incoming xml to hash then merge into parameters hash.
on line 439:
parameters ||= {}
parameters.merge!(Hash.from_xml(#request.env['RAW_POST_DATA'])) if #request.env['RAW_POST_DATA'] && #request.env['CONTENT_TYPE']=='application/xml'
I'm trying to find the best way to make paperclip urls secure, but only for secure pages.
For instance, the homepage, which shows images stored in S3, is http://mydomain.com and the image url is http://s3.amazonaws.com/mydomainphotos/89/thisimage.JPG?1284314856.
I have secure pages like https://mydomain.com/users/my_stuff/49 that has images stored in S3, but the S3 protocol is http and not https, so the user gets a warning from the browser saying that some elements on the page are not secure, blah blah blah.
I know that I can specify :s3_protocol in the model, but this makes everything secure even when it isn't necessary. So, I'm looking for the best way to change the protocol to https on the fly, only for secure pages.
One (probably bad) way would be to create a new url method like:
def custom_url(style = default_style, ssl = false)
ssl ? self.url(style).gsub('http', 'https') : self.url(style)
end
One thing to note is that I'm using the ssl_requirement plugin, so there might be a way to tie it in with that.
I'm sure there is some simple, standard way to do this that I'm overlooking, but I can't seem to find it.
If anyone stumbles upon this now: There is a solution in Paperclip since April 2012! Simply write:
Paperclip::Attachment.default_options[:s3_protocol] = ""
in an initializer or use the s3_protocol option inside your model.
Thanks to #Thomas Watson for initiating this.
If using Rails 2.3.x or newer, you can use Rails middleware to filter the response before sending it back to the user. This way you can detect if the current request is an HTTPS request and modify the calls to s3.amazonaws.com accordingly.
Create a new file called paperclip_s3_url_rewriter.rb and place it inside a directory that's loaded when the server starts. The lib direcotry will work, but many prefer to create an app/middleware directory and add this to the Rails application load path.
Add the following class to the new file:
class PaperclipS3UrlRewriter
def initialize(app)
#app = app
end
def call(env)
status, headers, response = #app.call(env)
if response.is_a?(ActionController::Response) && response.request.protocol == 'https://' && headers["Content-Type"].include?("text/html")
body = response.body.gsub('http://s3.amazonaws.com', 'https://s3.amazonaws.com')
headers["Content-Length"] = body.length.to_s
[status, headers, body]
else
[status, headers, response]
end
end
end
Then just register the new middleware:
Rails 2.3.x: Add the line below to environment.rb in the beginning of the Rails::Initializer.run block.
Rails 3.x: Add the line below to application.rb in the beginning of the Application class.
config.middleware.use "PaperclipS3UrlRewriter"
UPDATE:
I just edited my answer and added a check for response.is_a?(ActionController::Response) in the if statement. In some cases (maybe caching related) the response object is an empty array(?) and hence fails when request is called upon it.
UPDATE 2:
I edited the Rack/Middleware code example above to also update the Content-Length header. Otherwise the HTML body will be truncated by most browsers.
Use the following code in a controller class:
# locals/arguments/methods you must define or have available:
# attachment - the paperclip attachment object, not the ActiveRecord object
# request - the Rack/ActionController request
AWS::S3::S3Object.url_for \
attachment.path,
attachment.options[:bucket].to_s,
:expires_in => 10.minutes, # only necessary for private buckets
:use_ssl => request.ssl?
You can of course wrap this up nicely into a method.
FYI - some of the answers above do not work with Rails 3+, because ActionController::Response has been deprecated. Use the following:
class PaperclipS3UrlRewriter
def initialize(app)
#app = app
end
def call(env)
status, headers, response = #app.call(env)
if response.is_a?(ActionDispatch::BodyProxy) && headers && headers.has_key?("Content-Type") && headers["Content-Type"].include?("text/html")
body_string = response.body[0]
response.body[0] = body_string.gsub('http://s3.amazonaws.com', 'https://s3.amazonaws.com')
headers["Content-Length"] = body_string.length.to_s
[status, headers, response]
else
[status, headers, response]
end
end
end
And make sure that you add the middleware in a good place in the stack (I added it after Rack::Runtime)
config.middleware.insert_after Rack::Runtime, "PaperclipS3UrlRewriter"
I have a old-stupid service making request to my app that fails when the Content-Type include the charset line
Content-Type text/html; charset=utf-8
and I don't know how to remove it from my rails response. Every time that I override the headers forcing just the first part (Content-Type text/html) Rails adds the charset to the header...
For Rails 3/4, the code that handles this is in ActionDispatch::Response.assign_default_content_type_and_charset! in actionpack/lib/action_dispatch/http/response.rb.
Setting response.headers['Content-Type'] instead of response.content_type should eliminate the charset. Chubas' solution does this for all responses.
For Rails 2, the code that handles this is in content_type= and charset= in actionpack/lib/action_controller/response.rb.
As Carson's solution describes, setting ActionController::Base.default_charset = nil should eliminate the charset.
This worked for me:
class MyController
after_filter :remove_charset
def remove_charset
headers['Content-type'] = "text/html"
end
end
If you're working on development, make sure you clear your browser's cache.
There is this method, but didn't work for me. I don't know why, it may even be a bug.
The only way I was able to get it to work is by setting the default charset
ActionController::Base.default_charset = nil
Also, setting the Content-Transfer-Encoding header to binary will turn off the charset.
Putting this in the controller did it for me:
ActionDispatch::Response::default_charset = nil
I put in in my base controller to remove it from all responses.