I am using Nginx server with a RoR webapplication (version 6.1.4).
I have several audio files around the site and I want to restrict direct access to them.
The page have publicly accessible part and another for registered members. Here they can upload and share mp3's through the platform.
I added the following lines to nginx configuration:
location ~* \.mp3 {
valid_referers server_names;
if ($invalid_referer) {
return 403;
}
}
This one is working fine for the hardcoded audios and prevents direct access.
But if someone logs in and traces the html for the sourcefiles of uploaded audios, it is still accessible for them. I am using ActiveStorage for managing file uploads and it is on a s3 storage.
Appreciate any ideas!
Did not check, but customize this and try:
location ~* \.mp3 { # location for .mp3 files
if (-f $request_filename) { # if file actually exists
return 301 $scheme://$server_name/RoR_APP_URI_with_auth_check/$request_uri;
}
}
This could be a good opportunity to use the Proxy design pattern. You could create a controller/action that handles user authentication and then either redirects to the appropriate url or directly serves the file using send_file. There are pro's and con's to this approach but it would be a way to authenticate requests and restrict access to paywalled content.
Here's an example from a production app I'm working on:
def avatar_proxy
if Rails.env.development?
tmp_file = open(current_user.avatar.path)
else
url = current_user.avatar.url
tmp_file = open(url)
end
send_file tmp_file, :type => current_user.avatar.content_type, disposition: 'inline'
end
Related
Using url_for() on a file stored in active storage returns a url that leads to the application and then redirects to the actual location. Because of a bug in firefox with CORS, the redirect breaks my application.
Is there any way to get the direct link to the file with ActiveStorage?
You can do this
record.active_storage_object.blob.service_url
Found here https://github.com/rails/rails/blob/master/activestorage/app/controllers/active_storage/blobs_controller.rb
I had to dig through the rails source to create this so I have no idea how recommended it is but this works for disk storage at least.
ActiveStorage::Current.host = "yourhostname"
attachment_blob = ActiveStorage::Attachment.find_by(record_type: "YourModel", record_id: record.id).blob
direct_url = ActiveStorage::Blob.service.url(
attachment_blob.key,
expires_in: 20000,
disposition: "attachment",
filename: attachment_blob.filename,
content_type: attachment_blob.content_type
)
For me, rails_blob_url(#blog.pdf) (if you're trying to get the file stored as #blog.pdf) worked best.
In Rails, what is the best (ie. efficient, elegant) way to download a file from a public HTTP url, upload to Amazon S3 and delete the file in the server.
I am using Heroku so I have the additional restriction of a ephemeral file system.
If reading into memory isn't an option.
You can use the /tmp directory. As long as this is in the same process using that shouldn't be a problem.
The answer is to read your image into memory instead of storing it on the filesystem. Here's an example.
s3 = Aws::S3::Resource.new
obj = s3.bucket(your_bucket_name).object(your_object_key)
s3_put_url = URI.parse(obj.presigned_url(:put))
image_url = 'http://www.google.com/google.jpg'
image_file = open(image_url).read
Net::HTTP.start(s3_put_url.host) { |http| http.send_request('PUT', s3_put_url.request_uri, image_file); }
# Let's get the URL
s3.bucket(your_bucket_name).object(your_object_key).presigned_url(:get)
So Im messing around with developing a spotify app, trying to get it to talk to my local rails application API. I cant get anything other than a req.status 0 when I try it.
I think its either a problem with the spotify manifest.json file, not allowing the port:3000 to go on the url you set in required permissions, but it also says the following in their documentation.
https://developer.spotify.com/technologies/apps/tutorial/
If you need to talk to an outside web API you're welcome to, as long as you abide by the rules set in the Integration Guidelines. Please note that when talking with a web API, the requests will come from the origin sp://$APPNAME (so sp://tutorial for our example) - make sure the service you are talking to accepts requests from such an origin.
So, Im not sure if rails is set to not allow this sort of thing, or if its an issue with the putting the port into the required permissions, but my request
var req = new XMLHttpRequest();
req.open("GET", "http://127.0.0.1:3000/api/spotify/track/1.json", true);
console.log(req);
req.onreadystatechange = function() {
console.log(req.status);
console.log(req.readyState);
if (req.readyState == 4) {
if (req.status == 200) {
console.log("Search complete!");
console.log(req.responseText);
}
}
};
req.send();
Always returns status 0 where as their example:
var req = new XMLHttpRequest();
req.open("GET", "http://ws.audioscrobbler.com/2.0/?method=geo.getevents&location=" + city + "&api_key=YOUR_KEY_HERE", true);
req.onreadystatechange = function() {
console.log(req.status);
if (req.readyState == 4) {
console.log(req);
if (req.status == 200) {
console.log("Search complete!");
console.log(req.responseText);
}
}
};
req.send();
Will return a 403 response at least. its like the request is not being made or something?
Anyone have any idea what might be going on?
Much appreciated!
When talking to external services from a Spotify App, even if they're running on your local machine, you need to make sure that two things are in place correctly:
The URL (or at least the host) is in the RequiredPermissions section of your manifest. Port doesn't matter. http://127.0.0.1 should be fine for your case.
The server is allowing the sp://your-app-id origin for requests, as noted in the documentation you pasted in your question. This is done by setting the Access-Control-Allow-Origin header in your service's HTTP response. People often set it to Access-Control-Allow-Origin: * to allow anything to make requests to their service.
Thanks for help, I got it figured out, I think it was multiple things, with one main Im an idiot moment for not trying that earlier
First off, I had to run rails on port 80, as obviously if Im accessing my site from 127.0.0.1:3000, thats not going to work if spotify app is requesting 127.0.0.1 unless I can load that directly in the browser, which you cannot unless you run on 80. That is done via
rvmsudo rails server -p 80
Need to use rvmsudo because changing port requires permissions.
Next I had to set access controll allow origin as noted above, that can be done in rails 3 by adding before filter to your app controller as follows.
class ApplicationController < ActionController::Base
logger.info "I SEE REQUEST"
before_filter :cor
def cor
headers["Access-Control-Allow-Origin"] = "*"
headers["Access-Control-Allow-Methods"] = %w{GET POST PUT DELETE}.join(",")
headers["Access-Control-Allow-Headers"] = %w{Origin Accept Content-Type X-Requested-With X-CSRF-Token}.join(",")
head(:ok) if request.request_method == "OPTIONS"
end
end
Finally, and most importantly (sigh), you cant just righclick and reload your spotify app when you make changes to your manifest file, exit spotify completely and restart it!
In my app, I have a requirement that is stumping me.
I have a file stored in S3, and when a user clicks on a link in my app, I log in the DB they've clicked the link, decrease their 'download credit' allowance by one and then I want to prompt the file for download.
I don't simply want to redirect the user to the file because it's stored in S3 and I don't want them to have the link of the source file (so that I can maintain integrity and access)
It looks like send_file() wont work with a remote source file, anyone recommend a gem or suitable code which will do this?
You would need to stream the file content to the user while reading it from the S3 bucket/object.
If you use the AWS::S3 library something like this may work:
send_file_headers!( :length=>S3Object.about(<s3 object>, <s3 bucket>)["content-length"], :filename=><the filename> )
render :status => 200, :text => Proc.new { |response, output|
S3Object.stream(<s3 object>, <s3 bucket>) do |chunk|
output.write chunk
end
}
This code is mostly copied form the send_file code which by itself works only for local files or file-like objects
N.B. I would anyhow advise against serving the file from the rails process itself. If possible/acceptable for your use case I'd use an authenticated GET to serve the private data from the bucket.
Using an authenticated GET you can keep the bucket and its objects private, while allowing temporary permission to read a specific object content by crafting a URL that includes an authentication signature token. The user is simply redirected to the authenticated URL, and the token can be made valid for just a few minutes.
Using the above mentioned AWS::S3 you can obtain an authenticated GET url in this way:
time_of_exipry = Time.now + 2.minutes
S3Object.url_for(<s3 object>, <s3 bucket>,
:expires => time_of_exipry)
Full image download method using temp file (tested rails 3.2):
def download
#image = Image.find(params[:image_id])
open(#image.url) {|img|
tmpfile = Tempfile.new("download.jpg")
File.open(tmpfile.path, 'wb') do |f|
f.write img.read
end
send_file tmpfile.path, :filename => "great-image.jpg"
}
end
You can read the file from S3 and write it locally to a non-public directory, then use X-Sendfile (apache) or X-Accel-Redirect (nginx) to serve the content.
For nginx you would include something like the following in your config:
location /private {
internal;
alias /path/to/private/directory/;
}
Then in your rails controller, you do the following:
response.headers['Content-Type'] = your_content_type
response.headers['Content-Disposition'] = "attachment; filename=#{your_file_name}"
response.headers['Cache-Control'] = "private"
response.headers['X-Accel-Redirect'] = path_to_your_file
render :nothing=>true
A good writeup of the process is here
Is anyone else having a difficult time getting Twitters oAuth's callback URL to hit their localhost development environment.
Apparently it has been disabled recently. http://code.google.com/p/twitter-api/issues/detail?id=534#c1
Does anyone have a workaround. I don't really want to stop my development
Alternative 1.
Set up your .hosts (Windows) or etc/hosts file to point a live domain to your localhost IP. such as:
127.0.0.1 xyz.example
where xyz.example is your real domain.
Alternative 2.
Also, the article gives the tip to alternatively use a URL shortener service. Shorten your local URL and provide the result as callback.
Alternative 3.
Furthermore, it seems that it works to provide for example http://127.0.0.1:8080 as callback to Twitter, instead of http://localhost:8080.
I just had to do this last week. Apparently localhost doesn't work but 127.0.0.1 does Go figure.
This of course assumes that you are registering two apps with Twitter, one for your live www.mysite.example and another for 127.0.0.1.
Just put http://127.0.0.1:xxxx/ as the callback URL, where xxxx is the port for your framework
Yes, it was disabled because of the recent security issue that was found in OAuth. The only solution for now is to create two OAuth applications - one for production and one for development. In the development application you set your localhost callback URL instead of the live one.
Callback URL edited
http://localhost:8585/logintwitter.aspx
Convert to
http://127.0.0.1:8585/logintwitter.aspx
This is how i did it:
Registered Callback URL:
http://127.0.0.1/Callback.aspx
OAuthTokenResponse authorizationTokens =
OAuthUtility.GetRequestToken(ConfigSettings.getConsumerKey(),
ConfigSettings.getConsumerSecret(),
"http://127.0.0.1:1066/Twitter/Callback.aspx");
ConfigSettings:
public static class ConfigSettings
{
public static String getConsumerKey()
{
return System.Configuration.ConfigurationManager.AppSettings["ConsumerKey"].ToString();
}
public static String getConsumerSecret()
{
return System.Configuration.ConfigurationManager.AppSettings["ConsumerSecret"].ToString();
}
}
Web.config:
<appSettings>
<add key="ConsumerKey" value="xxxxxxxxxxxxxxxxxxxx"/>
<add key="ConsumerSecret" value="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"/>
</appSettings>
Make sure you set the property 'use dynamic ports' of you project to 'false' and enter a static port number instead. (I used 1066).
I hope this helps!
Use http://smackaho.st
What it does is a simple DNS association to 127.0.0.1 which allows you to bypass the filters on localhost or 127.0.0.1 :
smackaho.st. 28800 IN A 127.0.0.1
So if you click on the link, it will display you what you have on your local webserver (and if you don't have one, you'll get a 404). You can of course set it to any page/port you want :
http://smackaho.st:54878/twitter/callback
I was working with Twitter callback url on my localhost. If you are not sure how to create a virtual host ( this is important ) use Ampps. He is really cool and easy. In a few steps you have your own virtual host and then every url will work on it. For example:
download and install ampps
Add new domain. ( here you can set for example twitter.local) that means your virtual host will be http://twitter.local and it will work after step 3.
I am working on Win so go under to your host file -> C:\Windows\System32\Drivers\etc\hosts and add line: 127.0.0.1 twitter.local
Restart your Ampps and you can use your callback. You can specify any url, even if you are using some framework MVC or you have htaccess url rewrite.
Hope This Help!
Cheers.
Seems nowadays http://127.0.0.1 also stopped working.
A simple solution is to use http://localtest.me instead of http://localhost it is always pointing to 127.0.0.1 And you can even add any arbitrary subdomain to it, and it will still point to 127.0.0.1
See Website
When I develop locally, I always set up a locally hosted dev name that reflects the project I'm working on. I set this up in xampp through xampp\apache\conf\extra\httpd-vhosts.conf and then also in \Windows\System32\drivers\etc\hosts.
So if I am setting up a local dev site for example.com, I would set it up as example.dev in those two files.
Short Answer: Once this is set up properly, you can simply treat this url (http://example.dev) as if it were live (rather than local) as you set up your Twitter Application.
A similar answer was given here: https://dev.twitter.com/discussions/5749
Direct Quote (emphasis added):
You can provide any valid URL with a domain name we recognize on the
application details page. OAuth 1.0a requires you to send a
oauth_callback value on the request token step of the flow and we'll
accept a dynamic locahost-based callback on that step.
This worked like a charm for me. Hope this helps.
It can be done very conveniently with Fiddler:
Open menu Tools > HOSTS...
Insert a line like 127.0.0.1 your-production-domain.com, make sure that "Enable remapping of requests..." is checked. Don't forget to press Save.
If access to your real production server is needed, simply exit Fiddler or disable remapping.
Starting Fiddler again will turn on remapping (if it is checked).
A pleasant bonus is that you can specify a custom port, like this:
127.0.0.1:3000 your-production-domain.com (it would be impossible to achieve this via the hosts file). Also, instead of IP you can use any domain name (e.g., localhost).
This way, it is possible (but not necessary) to register your Twitter app only once (provided that you don't mind using the same keys for local development and production).
edit this function on TwitterAPIExchange.php at line #180
public function performRequest($return = true)
{
if (!is_bool($return))
{
throw new Exception('performRequest parameter must be true or false');
}
$header = array($this->buildAuthorizationHeader($this->oauth), 'Expect:');
$getfield = $this->getGetfield();
$postfields = $this->getPostfields();
$options = array(
CURLOPT_HTTPHEADER => $header,
CURLOPT_HEADER => false,
CURLOPT_URL => $this->url,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_SSL_VERIFYHOST => false
);
if (!is_null($postfields))
{
$options[CURLOPT_POSTFIELDS] = $postfields;
}
else
{
if ($getfield !== '')
{
$options[CURLOPT_URL] .= $getfield;
}
}
$feed = curl_init();
curl_setopt_array($feed, $options);
$json = curl_exec($feed);
curl_close($feed);
if ($return) { return $json; }
}
I had the same challenge and I was not able to give localhost as a valid callback URL. So I created a simple domain to help us developers out:
https://tolocalhost.com
It will redirect any path to your localhost domain and port you need. Hope it can be of use to other developers.
set callbackurl in twitter app : 127.0.0.1:3000
and set WEBrick to bind on 127.0.0.1 instead of 0.0.0.0
command : rails s -b 127.0.0.1
Looks like Twitter now allows localhost alongside whatever you have in the Callback URL settings, so long as there is a value there.
I struggled with this and followed a dozen solutions, in the end all I had to do to work with any ssl apis on local host was:
Go download: cacert.pem file
In php.ini * un-comment and change:
curl.cainfo = "c:/wamp/bin/php/php5.5.12/cacert.pem"
You can find where your php.ini file is on your machine by running php --ini in your CLI
I placed my cacert.pem in the same directory as php.ini for ease.
These are the steps that worked for me to get Facebook working with a local application on my laptop:
goto apps.twitter.com
enter the name, app description and your site URL
Note: for localhost:8000, use 127.0.0.1:8000 since the former will not work
enter the callback URL matching your callback URL defined in TWITTER_REDIRECT_URI your application
Note: eg: http://127.0.0.1/login/twitter/callback (localhost will not work).
Important enter both the "privacy policy" and "terms of use" URLs if you wish to request the user's email address
check the agree to terms checkbox
click [Create Your Twitter Application]
switch to the [Keys and Access Tokens] tab at the top
copy the "Consumer Key (API Key)" and "Consumer Secret (API Secret)" to TWITTER_KEY and TWITTER_SECRET in your application
click the "Permissions" tab and set appropriately to "read only", "read and write" or "read, write and direct message" (use the least intrusive option needed for your application, for just and OAuth login "read only" is sufficient
Under "Additional Permissions" check the "request email addresses from users" checkbox if you wish for the user's email address to be returned to the OAuth login data (in most cases check yes)