I am using Rackspace Cloud Files as File Storage server for my application. The files that users upload must be authorized from within my application, then from a controller it would redirect to the correct Rackspace Cloud Files CDN URL. I am trying to do authorization using Rackspace Cloud Files' Referrer ACL.
So let me just add a very simple snippet to clarify what I am trying to accomplish.
class FilesController < ApplicationController
def download
redirect_to(some_url_to_a_file_on_cloud_files_url)
end
end
The URL the user would access to get to this download action would be the following:
http://a-subdomain.domain.com/projects/:project_id/files/:file_id/download
So with the CloudFiles gem I have set up an ACL Referrer regular expression that should work.
http\:\/\/.+\.domain\.com\/projects\/\d+\/files\/\d+\/download
When the user clicks on a link in the web UI, it routes them to the above URL and depending on the parameters, it will from the download action redirect the user to the correct Rackspace Cloud Files File URL.
Well, what I get is an error, saying that I am unauthorized (wrong http referrer). I have a hunch that because I am doing a redirect from the download action straight to cloud files, that it doesn't "count" as a HTTP Referrer and, rather than use this URL as a referrer, I think it might be using this URL:
http\:\/\/.+\.domain\.com\/projects\/\d+\/files
Since this is the page you are on when you want to click on the "download" link, that directs the user to the download action in the FilesController.
When I set the HTTP Referrer for Rackspace ACL to just this:
http\:\/\/.+\.domain\.com\/projects\/\d+\/files
And then click on a link, I am authorized to download. However, this isn't safe enough since then anyone could for example just firebug into the html and inject a raw link to the file and gain access.
So I guess my question is, does anyone have any clue how or why, what I am trying to accomplish is not working, and have any suggestions/ideas? As I said I think it might be that when a user clicks the link, that the referrer is being set to the location of which the file is being clicked, not the url where the user is being redirected to the actual file on cloud files.
Is something like this possible?
class FilesController < ApplicationController
def download
# Dynamically set a HTTP Referrer here before
# redirecting the user to the actual file on cloud files
# so the user is authorized to download the file?
redirect_to(some_url_to_a_file_on_cloud_files_url)
end
end
Any help, suggestions are much appreciated!
Thanks!
Generally Micahel's comment is more than enough to explain why S3 tops rackspace for this matter, but if you'd really like to add some special HTTP headers to your Rackspace request - do an HTTP request of your own and fetch the file manually:
class DownloadsController < ApplicationController
def download
send_data HTTParty.get(some_url_to_a_file_on_cloud_files_url, :headers => {"x-special-headers" => "AWESOME" }), :file_name => "myfile.something"
end
end
Yes, you can code this example better but it's the general idea.
Although there is still no 'Referer' check, you can create temp urls (signed urls) with the current version of Rackspace CloudFiles.
The following code is taken from Rackspace documentation site.
require "openssl"
unless ARGV.length == 4
puts "Syntax: <method> <url> <seconds> <key>"
puts ("Example: GET https://storage101.dfw1.clouddrive.com/v1/" +
"MossoCloudFS_12345678-9abc-def0-1234-56789abcdef0/" +
"container/path/to/object.file 60 my_shared_secret_key")
else
method, url, seconds, key = ARGV
method = method.upcase
base_url, object_path = url.split(/\/v1\//)
object_path = '/v1/' + object_path
seconds = seconds.to_i
expires = (Time.now + seconds).to_i
hmac_body = "#{method}\n#{expires}\n#{object_path}"
sig = OpenSSL::HMAC.hexdigest("sha1", key, hmac_body)
puts ("#{base_url}#{object_path}?" +
"temp_url_sig=#{sig}&temp_url_expires=#{expires}")
end
Related
I am trying to build a website in Rails 4 to track users redirects and site element views.
I decided to use session ids which I believe are quite unique in the short term but I'm having a strange issue.
Example procedure:
user follows a redirect, the system stores this action with a Session ID, let's say xxx
user reaches destination page, which contains a tracker, the system stores this action with ANOTHER Session ID, yyy
user reaches another page which also contains a tracker, the system stores this action with Session ID yyy
After the second action is stored, the session ID stays the same yyy for every request after that, but I need to have the same session ID every time.
In session I also store a SecureRandom.hex generated code, which also changes from the first to the second request (which is not a surprise, since the session ID changes).
I also tried using a cookie, same result.
Please notice that these redirects are external, but all the requests are then made to the same domain (exactly the same, without www and in https).
Any idea?
Thanks in advance.
Update
this is the source code responsible for managing redirects:
before_action :load_redirect, :only => [:http_redirect]
def http_redirect
raise ActionController::RoutingError.new('Redirect has been disabled') unless #redir.enabled
ua = UserAction.create(
:session_id => session.id,
:user_agent => request.user_agent,
:trackable => #redir,
:ip_address => request.remote_ip,
:referer => request.referer
)
redirect_to #redir.destination_url
end
private
def load_redirect
#redir = Redirect.find(params[:id])
end
UPDATE:
Since you are using an iframe (per comment discussion below) for tracking code, the issue is likely that on the external site cookies are not being passed from parent page to the iframe because the iframes origin (domain) is different from the parent page.
OLD ANSWER:
(Still could be helpful for others debugging similar issues)
Source code would help. Without that, here are a few things to try:
Try disabling CSRF protection for the external tracking link action (I'm assuming it POSTs or PUTs data from an external source). CSRF protection could be creating a new or null session for those requests. Put this in the controller that contains the action accepting data from the external source:
protect_from_forgery :except => [:your_action]
The redirect (especially if it's a 301) could be cached in the browser you are using, hence having a different cookie and session than the request your tracking code makes. The stale cookie would be part of the cached redirect.
Try putting cache control headers on your controller action that does the redirect.
response.headers['Cache-Control'] = 'no-cache, no-store, must-revalidate'
response.headers['Pragma'] = 'no-cache'
response.headers['Expires'] = '0'
Your browser may not support setting cookies on a redirect, or possibly third-party cookies. Try in a different modern browser?
There could be a bug in your code. If these solutions don't work, maybe post it?
I have an issue with wanting to use session across domains (not subdomain). Eg, I have .co.uk, .com.au, and .com all for the same address.
I know for subdomains I can use something like:
SomeApp::Application.config.session_store :cookie_store, key: '_some_app_session', domain => :all, :tld_length => 2
But I would like my solution to work between actually domains to have one set of sessions/cookies.
As your default session store is 'cookie_store'
You could just do it the same way as when you might send an email link with an authentication token. Check to verify that the cookie is correct on example.org and, if it is, redirect them to:
http://example.com?token=
and then check to make sure the token matches the one you have in the DB when they arrive. If the token does match, create the session cookie for the example.com domain and then change the token in the database.
This will successfully transfer from one domain to another while providing persistent login on the new domain (via cookie) and shutting the door behind them by changing the authentication token in the DB.
EDIT
To answer your question below, I don't think you need middleware or anything fancy. You could do a simple before filter in the application controller of example.org, something like:
before_filter :redirect_to_dot_com
...
def redirect_to_dot_com
url = "http://example.com" + request.fullpath
url= destination + (url.include?('?') ? '&' : '?') + "token=#{current_user.token}" if signed_in?
redirect_to url, status: 301
end
That will redirect the user either way, and append the token to the query if the user is signed in on the .org site.
Go to more details on Persisting user sessions when switching to a new domain name (Ruby on Rails)
I wouldn't use the PHP style routings which pass ?php=bad style variables via :get especially if you're already using sessions. And also since then you'd have to parse the original URL and a bunch of other work.
Instead of using session[:edition_id] = 'UK' you can use:
cookies[:edition_id] = { value: 'UK', domain: 'some-app.com', expires: 1.year.from_now }
# or if you want to be google 10.years.from_now
When you use session[:edition_id] = 'UK' the value will be encrypted by rails and stored in the _myapp_session cookie. But in your case that probably doesn't matter much.
If you set the cookie explicitly on the domain you want to read it from, it will work without having to set odd ball variables via get and then trying to interpret them again on redirect.
In Rails, how can I make an http request to a page, like "http://google.com" and set the response to a variable?
Basically I'm trying to get the contents of a CSV file off of Amazon S3:
https://s3.amazonaws.com/datasets.graf.ly/24.csv
My Rails server needs to return that content as a response to an AJAX request.
Get S3 bucket
Access the file and read it
Render its contents (so the ajax request receives it)
A few questions have suggested screen scraping, but this sounds like overkill (and probably slow) for simply taking a response and pretty much just passing it along.
API
Firstly, you need to know how you're accessing the data
The problems you've cited are only valid if you just access someone's site through HTTP (with something like CURL). As you instinctively know, this is highly inefficient & will likely get your IP blocked for continuous access
A far better way to access data (from any reputable service) is to use their API. This is as true of S3 as Twitter, Facebook, Dropbox, etc:
AWS-SDK
#GemFile
gem "aws-sdk-core", "~> 2.0.0.rc2"
#config/application.rb
Aws.config = {
access_key_id: '...',
secret_access_key: '...',
region: 'us-west-2'
}
#config/initializers/s3.rb
S3 = Aws::S3.new
S3 = Aws.s3
Then you'll be able to use the API resources to help retrieve objects:
#controller
# yields once per response, even works with non-paged requests
s3.list_objects(bucket:'aws-sdk').each do |resp|
puts resp.contents.map(&:key)
end
CORS
If you were thinking of xhring into a server, you need to ensure you have the correct CORS permissions to do so
Considering you're wanting to use S3, I would look at this documentation to ensure you set the permissions correctly. This does not apply to the API or an HTTP request (only Ajax)
To do as you asked:
the open-uri solution from
How make a HTTP request using Ruby on Rails?
(to read from https in the simplest way possible), and
the set headers solution from in rails, how to return records as a csv file,
and
and a jquery library to decode the csv, eg http://code.google.com/p/jquery-csv/ -
Alternatively decode the csv file in rails and pass a json array of arrays back:
decode the csv as suggested in Rails upload CSV file with header
return the decoded data with the appropriate type
off the top of my head it should be something like:
def get_csv
url = 'http://s3.amazonaws.com/datasets.graf.ly/%d.csv' % params[:id].to_i
data = open(url).read
# set header here
render :text => data
end
Here's a story:
User A should be able to upload an image.
User A should be able to set a privacy. ("Public" or "Private").
User B should not be able to access "Private" images of User A.
I'm planning to user Paperclip for dealing with uploads.
If I store the images under "RAILS_ROOT/public/images", anyone who could guess the name of the files might access the files. (e.g., accessing http://example.com/public/images/uploads/john/family.png )
I need to show the images using img tags, so I cannot place a file except public.
How can I ensure that images of a user or group is not accessible by others?
(If I cannot achieve this with Paperclip, what is a good solution?)
You may make your rails server output the contents of image files. This is done via a controller action (most of actions print HTML, but this one will print JPG, for example).
Then you may use your authorization system to restrict access on controller level!
class ImagesController
#Default show Image method streams the file contents.
#File doesn't have to be in public/ dir
def show
send_file #image.filename, :type => #image.content_type,
:disposition => 'inline'
end
# Use your favorite authorization system to restrict access
filter_access_to :show, :require => :view, :attribute_check => :true
end
In HTML code you may use:
<img src="/images/show/5" />
I would have Paperclip use S3 on the back-end, set uploaded files to private, and then use "Query String Request Authentication Alternative" to generate the URLs for my image tags.
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/index.html?RESTAuthentication.html
Here's how I did this in a similar application.
Store your images on Amazon S3 instead of the local file system. Paperclip supports this.
Set your :s3_permissions to "private" in your Paperclip options
In your Image model, define a method that let's you output an authorized, time-limited url for the image.
Mine looks like this:
def s3_url(style = :original, time_limit = 30.minutes)
self.attachment.s3.interface.get_link(attachment.s3_bucket.to_s, attachment.path(style), time_limit)
end
You can then show images to people only if they're authorized to see them (implement that however you like)–and not have to worry about people guessing/viewing private images. It also keeps them from passing URLs around since they expire (the URL has a token in it).
Be warned that it takes time for your app to generate the authorized urls for each image. So, if you have several images on a page, it will affect load time.
If you want to host files yourself, you can perform authentication at the controller level as has been suggested. One of my applications has an AssetController that handles serving of files from the 'private' directory, for example.
One thing I wanted to add is that you should review this guide for setting up X-Sendfile, which will let your application tell the web server to handle actually sending the files. You'll see much better performance with this approach.
To make a long explanation short, suffice it to say that my Rails app allows users to upload images to the app that they will want to keep in the app (meaning, no hotlinking).
So I'm trying to come up with a way to obfuscate the image URLs so that the address of the image depends on whether or not that user is logged in to the site, so if anyone tried hotlinking to the image, they would get a 401 access denied error.
I was thinking that if I could route the request through a controller, I could re-use a lot of the authorization I've already built into my app, but I'm stuck there.
What I'd like is for my images to be accessible through a URL to one of my controllers, like:
http://railsapp.com/images/obfuscated?member_id=1234&pic_id=7890
If the user where to right-click on the image displayed on the website and select "Copy Address", then past it in, it would be the SAME url (as in, wouldn't betray where the image is actually hosted).
The actual image would be living on a URL like this:
http://s3.amazonaws.com/s3username/assets/member_id/pic_id.extension
Is this possible to accomplish? Perhaps using Rails' render method? Or something else? I know it's possible for PHP to return the correct headers to make the browser think it's an image, but I don't know how to do this in Rails...
UPDATE: I want all users of the app to be able to view the images if and ONLY if they are currently logged on to the site. If the user does not have a currently active session on the site, accessing the images directly should yield a generic image, or an error message.
S3 allows you to construct query strings for requests which allow a time-limited download of an otherwise private object. You can generate the URL for the image uniquely for each user, with a short timeout to prevent reuse.
See the documentation, look for the section "Query String Request Authentication Alternative". I'd link directly, but the frame-busting javascript prevents it.
Should the images be available to only that user or do you want to make it available to a group of users (friends)?
In any case if you want to stop hotlinking you should not store the image files under DocumentRoot of your webserver.
If the former, you could store the image on the server as MD5(image_file_name_as_exposed_to_user + logged_in_username_from_cookie). When the user requests image_file_name_as_exposed_to_user, in your rails app, construct the image filename as previously mentioned and then open the file in rails app and write it out (after first setting Content-Type in response header appropriately). This is secure by design.
If the image could be shared with friends, then you should not incorporate username in constructed filename but rest of the advice should work.
This is late in the day to be answering, but another option altogether would be to store the files in MongoDB's GridFS, served through a bit of Rack Middleware that requires auth to be passed. Pretty much as secure as you like, and the URLs don't even need obfuscation.
The other benefit of this is in the availability of the files and the future scalability of the system.
Thanks for your responses, but I'm still skeptical as to whether or not "timing out" the URL from Amazon is a very effective way to go.
I've updated my question above to be a little more clear about what I'm trying to do, and trying to prevent.
After some experimentation, I've come up with a way to do what I want to do in my Rails App, though this solution is not without downsides. Effectively what I've done is to construct my image_tag with a URL that points to a controller, and takes a path parameter. That controller first tests whether or not the user is authorized to see the image, then it fetches the content of the image in a separate request, and stores the content in an instance variable, which is then passed to a repond_to view to return the image, successfully obfuscating the actual image's URL (since that request is made separately).
Cons:
Adds to request time (I feel that the additional time it takes to do this double-request is acceptable considering the privacy this method gives me)
Adds some clutter to views and routes (a small amount, maybe a bit more than I'd like)
If the user is authorized, and tries to access the image directly, the image is downloaded immediately rather than displayed in the browser (anyone know how to fix this? Modify HTTP headers? Only seems to do this with the jpg, though...)
You have to make a separate view for each file format you intend to serve (two for me, jpg and png)
Are there any other cons or considerations I should be aware of with this method? So far what I've listed, I can live with...
(Refactoring welcome.)
application_controller.rb
class ApplicationController < ActionController::Base
def obfuscate_image
respond_to do |format|
if current_user
format.jpg { #obfuscated_image = fetch_url "http://s3.amazonaws.com/#{Settings.bucket}/#{params[:path]}" }
else
format.png { #obfuscated_image = fetch_url "#{root_url}/images/assets/profile/placeholder.png" }
end
end
end
protected
# helps us fetch an image, obfuscated
def fetch_url(url)
r = Net::HTTP.get_response(URI.parse(url))
if r.is_a? Net::HTTPSuccess
r.body
else
nil
end
end
end
views/application/obfuscate_image.png.haml & views/application/obfuscate_image.jpg.haml
= #obfuscated_image
routes.rb
map.obfuscate_image 'obfuscate_image', :controller => 'application', :action => 'obfuscate_image'
config/environment.rb
Mime::Type.register "image/png", :png
Mime::Type.register "image/jpg", :jpg
Calling an obfuscated image
= image_tag "/obfuscate_image?path=#{#user.profile_pic.path}"
The problem you have is that as far as I know you need the images on S3 to be World-readable for them to be accessible. At some point in the process an HTTP GET is going to have to be performed to retrieve the image, which is going to expose the real URL to tools that can sniff HTTP, such as Firebug.
Incidentally, 37signals don't consider this to be a huge problem because if I view an image in my private Backpack account I can see the public S3 URL in the browser address bar. Your mileage may vary...