Rails: Read the contents from another site - ruby-on-rails

Question

In Rails, how can I make an http request to a page, like "http://google.com" and set the response to a variable?
Basically I'm trying to get the contents of a CSV file off of Amazon S3:
https://s3.amazonaws.com/datasets.graf.ly/24.csv
My Rails server needs to return that content as a response to an AJAX request.
Get S3 bucket
Access the file and read it
Render its contents (so the ajax request receives it)
A few questions have suggested screen scraping, but this sounds like overkill (and probably slow) for simply taking a response and pretty much just passing it along.

Answer 1

API
Firstly, you need to know how you're accessing the data
The problems you've cited are only valid if you just access someone's site through HTTP (with something like CURL). As you instinctively know, this is highly inefficient & will likely get your IP blocked for continuous access
A far better way to access data (from any reputable service) is to use their API. This is as true of S3 as Twitter, Facebook, Dropbox, etc:
AWS-SDK
#GemFile
gem "aws-sdk-core", "~> 2.0.0.rc2"
#config/application.rb
Aws.config = {
access_key_id: '...',
secret_access_key: '...',
region: 'us-west-2'
}
#config/initializers/s3.rb
S3 = Aws::S3.new
S3 = Aws.s3
Then you'll be able to use the API resources to help retrieve objects:
#controller
# yields once per response, even works with non-paged requests
s3.list_objects(bucket:'aws-sdk').each do |resp|
puts resp.contents.map(&:key)
end
CORS
If you were thinking of xhring into a server, you need to ensure you have the correct CORS permissions to do so
Considering you're wanting to use S3, I would look at this documentation to ensure you set the permissions correctly. This does not apply to the API or an HTTP request (only Ajax)

Answer 2

To do as you asked:
the open-uri solution from
How make a HTTP request using Ruby on Rails?
(to read from https in the simplest way possible), and
the set headers solution from in rails, how to return records as a csv file,
and
and a jquery library to decode the csv, eg http://code.google.com/p/jquery-csv/ -
Alternatively decode the csv file in rails and pass a json array of arrays back:
decode the csv as suggested in Rails upload CSV file with header
return the decoded data with the appropriate type
off the top of my head it should be something like:
def get_csv
url = 'http://s3.amazonaws.com/datasets.graf.ly/%d.csv' % params[:id].to_i
data = open(url).read
# set header here
render :text => data
end