I'm trying to find the best way to make paperclip urls secure, but only for secure pages.
For instance, the homepage, which shows images stored in S3, is http://mydomain.com and the image url is http://s3.amazonaws.com/mydomainphotos/89/thisimage.JPG?1284314856.
I have secure pages like https://mydomain.com/users/my_stuff/49 that has images stored in S3, but the S3 protocol is http and not https, so the user gets a warning from the browser saying that some elements on the page are not secure, blah blah blah.
I know that I can specify :s3_protocol in the model, but this makes everything secure even when it isn't necessary. So, I'm looking for the best way to change the protocol to https on the fly, only for secure pages.
One (probably bad) way would be to create a new url method like:
def custom_url(style = default_style, ssl = false)
ssl ? self.url(style).gsub('http', 'https') : self.url(style)
end
One thing to note is that I'm using the ssl_requirement plugin, so there might be a way to tie it in with that.
I'm sure there is some simple, standard way to do this that I'm overlooking, but I can't seem to find it.
If anyone stumbles upon this now: There is a solution in Paperclip since April 2012! Simply write:
Paperclip::Attachment.default_options[:s3_protocol] = ""
in an initializer or use the s3_protocol option inside your model.
Thanks to #Thomas Watson for initiating this.
If using Rails 2.3.x or newer, you can use Rails middleware to filter the response before sending it back to the user. This way you can detect if the current request is an HTTPS request and modify the calls to s3.amazonaws.com accordingly.
Create a new file called paperclip_s3_url_rewriter.rb and place it inside a directory that's loaded when the server starts. The lib direcotry will work, but many prefer to create an app/middleware directory and add this to the Rails application load path.
Add the following class to the new file:
class PaperclipS3UrlRewriter
def initialize(app)
#app = app
end
def call(env)
status, headers, response = #app.call(env)
if response.is_a?(ActionController::Response) && response.request.protocol == 'https://' && headers["Content-Type"].include?("text/html")
body = response.body.gsub('http://s3.amazonaws.com', 'https://s3.amazonaws.com')
headers["Content-Length"] = body.length.to_s
[status, headers, body]
else
[status, headers, response]
end
end
end
Then just register the new middleware:
Rails 2.3.x: Add the line below to environment.rb in the beginning of the Rails::Initializer.run block.
Rails 3.x: Add the line below to application.rb in the beginning of the Application class.
config.middleware.use "PaperclipS3UrlRewriter"
UPDATE:
I just edited my answer and added a check for response.is_a?(ActionController::Response) in the if statement. In some cases (maybe caching related) the response object is an empty array(?) and hence fails when request is called upon it.
UPDATE 2:
I edited the Rack/Middleware code example above to also update the Content-Length header. Otherwise the HTML body will be truncated by most browsers.
Use the following code in a controller class:
# locals/arguments/methods you must define or have available:
# attachment - the paperclip attachment object, not the ActiveRecord object
# request - the Rack/ActionController request
AWS::S3::S3Object.url_for \
attachment.path,
attachment.options[:bucket].to_s,
:expires_in => 10.minutes, # only necessary for private buckets
:use_ssl => request.ssl?
You can of course wrap this up nicely into a method.
FYI - some of the answers above do not work with Rails 3+, because ActionController::Response has been deprecated. Use the following:
class PaperclipS3UrlRewriter
def initialize(app)
#app = app
end
def call(env)
status, headers, response = #app.call(env)
if response.is_a?(ActionDispatch::BodyProxy) && headers && headers.has_key?("Content-Type") && headers["Content-Type"].include?("text/html")
body_string = response.body[0]
response.body[0] = body_string.gsub('http://s3.amazonaws.com', 'https://s3.amazonaws.com')
headers["Content-Length"] = body_string.length.to_s
[status, headers, response]
else
[status, headers, response]
end
end
end
And make sure that you add the middleware in a good place in the stack (I added it after Rack::Runtime)
config.middleware.insert_after Rack::Runtime, "PaperclipS3UrlRewriter"
Related
I need to get a request uuid or time when server get a request. It's easy in Rails, but I'm working on a gem and I would like it to be more generic. So I would like it to work also with Sinatra and every other Ruby application which works in a http server.
This is another problem, it's a gem. I can't put Time.now at the beggining of my application controller. I need it to be generic, so it should work with different frameworks.
What would you propose?
You can implement a Rack middleware which you can use independently from your actual application framework (as long as it used rack, which is true for at least Rails, Sinatra, Padriono and most other Ruby web frameworks).
Rails already includes a middleware for adding a unique ID to a request of required in ActionDispatch::RequestId. Another alternative could be the rack-request-id gem.
A minimal versions of this midleware could look like this:
class RequestIdMiddleware
def initialize(app)
#app = app
end
def call(env)
env['request_id'] = env['HTTP_X_REQUEST_ID'] || SecureRandom.uuid
env['request_started_at'] = Time.now
#app.call(env)
end
end
You can then use this middleware in your config.ru or by adding this to your application.rb in Rails:
config.middleware.use RequestIdMiddleware
I have a custom rack middleware used by my Rails 4 application. The middleware itself is just here to default Accept and Content-Type headers to application/json if the client did not provide a valid information (I'm working on an API). So before each request it changes those headers and after each request it adds a custom X-Something-Media-Type head with a custom media type information.
I would like to switch to Puma, therefore I'm a bit worried about the thread-safety of such a middleware. I did not play with instances variables, except once for the common #app.call that we encounter in every middleware, but even here I reproduced something I've read in RailsCasts' comments :
def initialize(app)
#app = app
end
def call(env)
dup._call(env)
end
def _call(env)
...
status, headers, response = #app.call(env)
...
Is the dup._call really useful in order to handle thread-safety problems ?
Except that #app instance variable I only play with the current request built with the current env variable :
request = Rack::Request.new(env)
And I call env.update to update headers and forms informations.
Is it dangerous enough to expect some issues with that middleware when I'll switch from Webrick to a concurrent web server such as Puma ?
If yes, do you know a handful way to make some tests en isolate portions of my middleware which are non-thread-safe ?
Thanks.
Yes, it's necessary to dup the middleware to be thread-safe. That way, anything instance variables you set from _call will be set on the duped instance, not the original. You'll notice that web frameworks that are built around Rack work this way:
Pakyow
Sinatra
One way to unit test this is to assert that _call is called on a duped instance rather than the original.
I have been struggling with a problem for the past days in a Ruby on Rails App I'm currently working on. I have different countries and for each country we use different Amazon S3 buckets. Amazon S3 key credentials are stored as constants in config/environments/environment_name.rb(ex:demo.rb) There is no way for me to determine which country we are operating from the config file. I can determine which country we are operating from the controllers,models,views,etc but not from the config file. Is there a Ruby meta programming or some other kind of magic that I'm not aware of so that I want to say if we are working on UK as a country in the app, use UK's bucket credentials or Germany as a country, use Germany's bucket credentials? I can't think of a way to pass parameters to environment files from the app itself. Thank you very much in advance for all your helps.
Rather than actually pass the configuration details to whichever S3 client you're using at launch, you should probably select the relevant credentials for each request. Your config file can define them all in a hash like so:
# config/s3.rb
S3_BUCKETS => {
:us => 'our-files-us',
:gb => 'our-files-gb',
:tz => 'special-case'
}
Then you can select the credentials on request like so (in maybe your AppController):
bucket_name = S3_BUCKETS[I18n.locale]
# pass this info to your S3 client
Make sense?
Write a little middleware if you want to keep the knowledge of the per-country configuration out of the main application.
A middleware is extremely simple. A do-nothing middleware looks like this:
class DoesNothing
def initialize(app, *args)
#app = app
end
def call(env)
#app.call(env)
end
end
Rack powers applications through chaining a series of middlewares together... each one is given a reference to #app, which is the next link in the chain, and it must invoke #call on that application. The one at the end of the chain runs the app.
So in your case, you can do some additional configuration in here.
class PerCountryConfiguration
def initialize(app)
#app = app
end
def call(env)
case env["COUNTRY"]
when "AU"
Rails.application.config.s3_buckets = { ... }
when "US"
Rails.application.config.s3_buckets = { ... }
... etc
end
#app.call(env)
end
end
There are several ways to use the middleware, but since it depends on access to the Rails environment, you'll want to do it from inside Rails. Put it in your application.rb:
config.middleware.use PerCountryConfiguration
If you want to pass additional arguments to the constructor of your middleware, just list them after the class name:
config.middleware.use PerCountryConfiguration, :some_argument
You can also mount the middleware from inside of ApplicationController, which means all of the initializers and everything will have already been executed, so it may be too far along the chain.
I am working on a Ruby on Rails app that communicates with RackSpace cloudfiles (similar to Amazon S3 but lacking some features).
Due to the lack of the availability of per-object access permissions and query string authentication, downloads to users have to be mediated through an application.
In Rails 2.3, it looks like you can dynamically build a response as follows:
# Streams about 180 MB of generated data to the browser.
render :text => proc { |response, output|
10_000_000.times do |i|
output.write("This is line #{i}\n")
end
}
(from http://api.rubyonrails.org/classes/ActionController/Base.html#M000464)
Instead of 10_000_000.times... I could dump my cloudfiles stream generation code in there.
Trouble is, this is the output I get when I attempt to use this technique in Rails 3.
#<Proc:0x000000010989a6e8#/Users/jderiksen/lt/lt-uber/site/app/controllers/prospect_uploads_controller.rb:75>
Looks like maybe the proc object's call method is not being called? Any other ideas?
Assign to response_body an object that responds to #each:
class Streamer
def each
10_000_000.times do |i|
yield "This is line #{i}\n"
end
end
end
self.response_body = Streamer.new
If you are using 1.9.x or the Backports gem, you can write this more compactly using Enumerator.new:
self.response_body = Enumerator.new do |y|
10_000_000.times do |i|
y << "This is line #{i}\n"
end
end
Note that when and if the data is flushed depends on the Rack handler and underlying server being used. I have confirmed that Mongrel, for instance, will stream the data, but other users have reported that WEBrick, for instance, buffers it until the response is closed. There is no way to force the response to flush.
In Rails 3.0.x, there are several additional gotchas:
In development mode, doing things such as accessing model classes from within the enumeration can be problematic due to bad interactions with class reloading. This is an open bug in Rails 3.0.x.
A bug in the interaction between Rack and Rails causes #each to be called twice for each request. This is another open bug. You can work around it with the following monkey patch:
class Rack::Response
def close
#body.close if #body.respond_to?(:close)
end
end
Both problems are fixed in Rails 3.1, where HTTP streaming is a marquee feature.
Note that the other common suggestion, self.response_body = proc {|response, output| ...}, does work in Rails 3.0.x, but has been deprecated (and will no longer actually stream the data) in 3.1. Assigning an object that responds to #each works in all Rails 3 versions.
Thanks to all the posts above, here is fully working code to stream large CSVs. This code:
Does not require any additional gems.
Uses Model.find_each() so as to not bloat memory with all matching objects.
Has been tested on rails 3.2.5,
ruby 1.9.3 and heroku using unicorn, with single dyno.
Adds a GC.start at every 500 rows, so as not to blow the heroku dyno's
allowed memory.
You may need to adjust the GC.start depending on your Model's memory footprint. I have successfully used this to stream 105K models into a csv of 9.7MB without any problems.
Controller Method:
def csv_export
respond_to do |format|
format.csv {
#filename = "responses-#{Date.today.to_s(:db)}.csv"
self.response.headers["Content-Type"] ||= 'text/csv'
self.response.headers["Content-Disposition"] = "attachment; filename=#{#filename}"
self.response.headers['Last-Modified'] = Time.now.ctime.to_s
self.response_body = Enumerator.new do |y|
i = 0
Model.find_each do |m|
if i == 0
y << Model.csv_header.to_csv
end
y << sr.csv_array.to_csv
i = i+1
GC.start if i%500==0
end
end
}
end
end
config/unicorn.rb
# Set to 3 instead of 4 as per http://michaelvanrooijen.com/articles/2011/06/01-more-concurrency-on-a-single-heroku-dyno-with-the-new-celadon-cedar-stack/
worker_processes 3
# Change timeout to 120s to allow downloading of large streamed CSVs on slow networks
timeout 120
#Enable streaming
port = ENV["PORT"].to_i
listen port, :tcp_nopush => false
Model.rb
def self.csv_header
["ID", "Route", "username"]
end
def csv_array
[id, route, username]
end
It looks like this isn't available in Rails 3
https://rails.lighthouseapp.com/projects/8994/tickets/2546-render-text-proc
This appeared to work for me in my controller:
self.response_body = proc{ |response, output|
output.write "Hello world"
}
In case you are assigning to response_body an object that responds to #each method and it's buffering until the response is closed, try in in action controller:
self.response.headers['Last-Modified'] = Time.now.to_s
Just for the record, rails >= 3.1 has an easy way to stream data by assigning an object that respond to #each method to the controller's response.
Everything is explained here: http://blog.sparqcode.com/2012/02/04/streaming-data-with-rails-3-1-or-3-2/
Yes, response_body is the Rails 3 way of doing this for the moment: https://rails.lighthouseapp.com/projects/8994/tickets/4554-render-text-proc-regression
This solved my problem as well - I have gzip'd CSV files, want to send to the user as unzipped CSV, so I read them a line at a time using a GzipReader.
These lines are also helpful if you're trying to deliver a big file as a download:
self.response.headers["Content-Type"] = "application/octet-stream"
self.response.headers["Content-Disposition"] = "attachment; filename=#{filename}"
In addition, you will have to set the 'Content-Length' header by your self.
If not, Rack will have to wait (buffering body data into memory) to determine the length.
And it will ruin your efforts using the methods described above.
In my case, I could determine the length.
In cases you can't, you need to make Rack to start sending body without a 'Content-Length' header.
Try to add into config.ru "use Rack::Chunked" after 'require' before the 'run'. (Thanks arkadiy)
I commented in the lighthouse ticket, just wanted to say the self.response_body = proc approach worked for me though I needed to use Mongrel instead of WEBrick to succeed.
Martin
Applying John's solution along with Exequiel's suggestion worked for me.
The statement
self.response.headers['Last-Modified'] = Time.now.to_s
marks the response as non-cacheable in rack.
After investigating further, I figured one could also use this :
headers['Cache-Control'] = 'no-cache'
This, to me, is just slightly more intuitive. It conveys the message to any1 else who may be reading my code. Also, in case a future version of rack stops checking for Last-Modified , a lot of code may break and it may be a while for folks to figure out why.
I'm maintaining a Ruby on Rails site and I'm confused as to how to perform redirects to relative URLs using the https protocol.
I can successfully create a redirect to a relative URL using http, for example:
redirect_to "/some_directory/"
But I cannot discern how to create a redirect to a URL using the https protocol. I have only been able to do so by using absolute URLS, for example:
redirect_to "https://mysite.com/some_directory/"
I would like to keep my code clean, and using relative URLs seems like a good idea. Does anyone know how to achieve this in Rails?
The ActionController::Base#redirect_to method takes an options hash, one of the parameters of which is :protocol which allows you to call:
redirect_to :protocol => 'https://',
:controller => 'some_controller',
:action => 'index'
See the definition for #redirect_to and #url_for for more info on the options.
Alternatively, and especially if SSL is to be used for all your controller actions, you could take a more declarative approach using a before_filter. In ApplicationController you could define the following method:
def redirect_to_https
redirect_to :protocol => "https://" unless (request.ssl? || request.local?)
end
You can then add filters in your those controllers which have actions requiring SSL, e.g:
class YourController
before_filter :redirect_to_https, :only => ["index", "show"]
end
Or, if you require SSL across your entire app, declare the filter in ApplicationController:
class ApplicationController
before_filter :redirect_to_https
end
If you want your entire application to be served over https then since Rails 4.0 the best way to do this is to enable force_ssl in the configuration file like so:
# config/environments/production.rb
Rails.application.configure do
# [..]
# Force all access to the app over SSL, use Strict-Transport-Security,
# and use secure cookies.
config.force_ssl = true
end
By default this option is already present in config/environments/production.rb in in newly generated apps, but is commented out.
As the comment says, this will not just redirect to https, but also sets the Strict-Transport-Security header (HSTS) and makes sure that the secure flag is set on all cookies. Both measures increase the security of your application without significant drawbacks. It uses ActionDispatch:SSL.
The HSTS expire settings are set to a year by default and doesn't include subdomains, which is probably fine for most applications. You can configure this with the hsts option:
config.hsts = {
expires: 1.month.to_i,
subdomains: false,
}
If you're running Rails 3 (>=3.1) or don't want to use https for the entire application, then you can use the force_ssl method in a controller:
class SecureController < ApplicationController
force_ssl
end
That's all. You can set it per controller, or in your ApplicationController. You can force https conditionally using the familiar if or unless options; for example:
# Only when we're not in development or tests
force_ssl unless: -> { Rails.env.in? ['development', 'test'] }
You're probably better off using ssl_requirement and not caring if a link or redirect is or isn't using https. With ssl_requirement, you declare which actions require SSL, which ones are capable of SSL and which ones are required not to use SSL.
If you're redirecting somewhere outside of your Rails app, then specifying the protocol as Olly suggests will work.
If you want to globally controll the protocol of urls generated in controllers, you can override the url_options method in you application controller. You could force the protocol of the generated urls depending on the rails env like so :
def url_options
super
#_url_options.dup.tap do |options|
options[:protocol] = Rails.env.production? ? "https://" : "http://"
options.freeze
end
end
this example works in rails 3.2.1, i'm not exactly sure for earlier or future versions.
This answer is somewhat tangential to the original question, but I record it in case others end up here in similar circumstances to myself.
I had a situation where I needed to have Rails use https proto in url helpers etc. even though the origin of all requests is unencrypted (http).
Now, ordinarily in this situation (which is normal when Rails is behind a reverse proxy or load balancer etc.), the x-forwarded-proto header is set by the reverse proxy or whatever, so even though requests are unencrypted between the proxy & rails (probably not advisable in production by the way) rails thinks everything is in https.
I needed to run behind an ngrok tls tunnel. I wanted to have ngrok terminate the tls with letsencrypt certificates I specified. However when it does so, ngrok does not offer the ability to customize headers, including setting x-forwarded-proto (although this feature is planned at some point in the future).
The solution turned out to be quite simple: Rails does not depend on either the protocol of the origin or whether x-forwarded-proto is set directly, but on the Rack env var rack.url_scheme. So I just needed to add this Rack middleware in development:
class ForceUrlScheme
def initialize(app)
#app = app
end
def call(env)
env['rack.url_scheme'] = 'https'
#app.call(env)
end
end
In Rails 4 one can use the force_ssl_redirect before_action to enforce ssl for a single controller. Please note that by using this method your cookies won't be marked as secure and HSTS is not used.
If you want to force ALL traffic via https, then the best way in Rails 6 is to configure production.rb with:
config.force_ssl = false
If you need a more flexible solution, you can handle it with a simple before_action filter:
class ApplicationController < ActionController::Base
include SessionsHelper
include LandingpageHelper
include ApplicationHelper
include UsersHelper
include OrganisationHelper
before_action :enforce_ssl, :except => [:health]
def enforce_ssl
if ENV['ENFORCE_SSL'].to_s.eql?('true') && !request.ssl?
redirect_to request.url.gsub(/http/i, "https")
end
end
end
If you run your application on AWS ECS Fargate with health checks, then you need a more flexible solution because the health check from the AWS target group is not invoked via https. Of course, you want the health check to work and at the same time, you want to force SSL for all other controller methods.
The ENFORCE_SSL is just an environment variable that turns this feature on/off.
Add protocol to ..._url:
redirect_to your_url(protocol: 'https')
or with subdomain:
redirect_to your_url(protocol: 'https', subdomain: 'your_subdomain')
Relative URLs, by definition, use the current protocol and host. If you want to change the protocol being used, you need to supply the absolute URL. I would take Justice's advice and create a method that does this for you:
def redirect_to_secure(relative_uri)
redirect_to "https://" + request.host + relative_uri
end
Open the class that has redirect_to and add a method redirect_to_secure_of with an appropriate implementation. Then call:
redirect_to_secure_of "/some_directory/"
Put this method in the lib directory or somewhere useful.