I have a website hosted on Heroku, and using Ruby on Rails with the paperclip gem.
I am trying to prevent hotlinking to all my files in my S3 bucket, so I have everything on private and only allow user to access using an expiring URL
I want to provide a more user-friendly page when user tries to reuse an expired URL. Currently it is showing the message below:
<Error>
<Code>AccessDenied</Code>
<Message>Request has expired</Message>
<X-Amz-Expires>300</X-Amz-Expires>
<Expires>2016-04-15T19:41:33Z</Expires>
<ServerTime>2016-04-15T19:41:39Z</ServerTime>
<RequestId>D5DD935553A2CF88</RequestId>
<HostId>
55+rFtFbksDMyBWf5cWwgJ+aWvJKwe5umSXgTEWYKgfoT5QR5sbJY9fRNFIiBAqd35OR2MoiCzQ=
</HostId>
</Error>
Is there a way to customize the error page on S3?
S3 offers custom error pages through the web site endpoints -- but not the REST endpoints... but signed URLs only work on the REST endpoints, and not the web site endpoints.
So, no, there is not a way to directly solve this using only S3.
One option is to use CloudFront, which offers the ability to replace the standard error pages with a custom static page, but the error content is lost and all you have is a static page. You also have to use the CloudFront URL signing mechanism, which is different than S3 (though it also has some advantages, such as wildcard support in a signed URL).
In this answer to a question that is similar, but not a complete duplicate I demonstrated the way I've used an XSL transform to "style" the S3 error XML, by modifying the XML returned to the browser, injecting a link to the XSL stylesheet, and letting the browser do the rest of the work... see the screen shots.
I'm quite pleased with the solution, though it has what some people would consider a drawback -- it requires all of the S3 requests be served via a proxy server running HAProxy in EC2. There's a small additional cost for the EC2 instance, but no added cost for the bandwidth, since the transfer from S3 into EC2 is free, and the transfer from EC2 to the Internet is the same price as transfer from S3 to the Internet. With this setup, the S3 signed URLs still work. The additional advantages in my application us that this allows me to use my SSL certs with S3 static content (although this capability is also available through CloudFront), and the fact that the proxy's access logs are in real-time.
Related
I have a rails app that is running on heroku and am using Cloudflare Pro with their Full SSL to encrypt traffic between: User <-SSL-> Cloudflare <-SSL-> Heroku, as detailed in: http://mikecoutermarsh.com/adding-ssl-to-heroku-with-cloudflare/ .
I am also using the rack-ssl-enforcer gem to force all http requests to go through https.
This is working properly, except I have the following issues, by browser:
1) Firefox. I have to add a security exception the first visit to the site, getting the "This site is not trusted" warning. Once on the site, I also have the warning in the address bar:
2) Chrome: page loads first time, but the lock in the address bar has a warning triangle on it, when clicked displays:
Your connection is encrypted with 128-bit encryption. However, this
page includes other resources which are not secure. These resources
can be viewed by others while in transit, and can be modified by an
attacker to change the look of the page. The connection uses TLS 1.2.
The connection is encrypted and authenticated using AES_128_GCM and
uses ECDHE_RSA as the key exchange mechanism.
Safari: initially loads with https badge, but it immediately drops off
Is there a way to leverage Cloudflare SSL + piggyback of Heroku native SSL without running into these security warnings? If not, I don't see much value in the configuration.
My apologies for slinging erroneous accusations against Cloudflare and Heroku :-)
Turns out the issue was not the fault of either, but instead that images on the app (being served from AWS S3) were being served up without https.
If anyone runs into this situation, lessons learned across a wasted day:
S3 only lets you serve up content via https if you serve from your bucket's dedicated url: s3.amazonaws.com/your-bucket-name/etc..
a) I tried setting the bucket up for static website hosting, so I could use the url "your-bucket-name.your-url.s3-website-us-east-1.amazonaws.com/etc...", and then set up a CNAME within my DNS that sends "your-bucket-name.your-url" to "your-bucket-name.your-url.s3-website-us-east-1.amazonaws.com/etc...", to pretty up urls
b) this works, but AWS only lets you serve via https with your full url (s3.amazonaws.com/your-bucket-name/etc..) or *.s3-website-us-east-1.amazonaws.com/etc...", which doesnt work if you have a dot in your bucket name (your-bucket-name.your-url), which was required for me to do the CNAME redirect
If you want to use AWS CDN with https, on your custom domain, AWS' only option is CloudFront with a SSL certificate, which they charge $600/mo, per region. No thanks!
In the end, I sucked it up and have ugly image URLs that looks like: https://s3-website-us-east-1.amazonaws.com/mybucketname...", and using paperclip, I specify https: with ":s3_protocol => :https," in my model. Other than that all is working properly now.
On the PayPal payment page customization you can insert your own URI for the logo and header background. Obviously it's bad news to insert a non 443 (https) URL, but it also appears that insertion of a secure S3 link is also a major fail.
Initially I inserted my own cname alias to it cdn.mydomain.com and then after seeing an error message from Firefox, thought I needed to pass the actual AWS S3 full path to the bucket which is: https://cdn.mydomain.com.s3.amazonaws.com/paypal/my-logo.png
Even after doing this, I'm seeing the same problem with Firefox:
I am figuring that there is a different way to refer to a bucket via a path vs the subdomain for the bucket, but I'm not readily finding the answer. Or is there an altogether different way to handle this so that there's no certificate issue?
Use a subdomain/bucketname that doesn't have a dot in it. The cert is only good for *.s3.amazonaws.com, not *.*.s3.amazonaws.com.
Is there a way to create a CloudFront signed url that limits the number of times that a file can be downloaded?
According to this post Controlling number of downloads on Amazon S3, you can get the number of file downloads via the cloudfront api (but it cant find any reference to this on the amazon site)
Has anyone managed to achieve this via CloudFront?
Yes, with CloudFront you can serve Private Content.
Basically you can protect your content in two ways:
Require that your users use special CloudFront signed URLs to access your content, not the standard CloudFront public URLs.
Require that your users access your Amazon S3 content using CloudFront URLs, not Amazon S3 URLs.
When you create signed URLs for your objects, you can specify:
An ending date and time, after which the URL is no longer valid.
(Optional) The date and time that the URL becomes valid.
(Optional) The IP address or range of addresses of the computers that can be used to access your content.
I would like to know if Google App Engine can be used as a Content delivery network like aws S3. I'm running a RoR app on Heroku and I would like store my uploaded files on GAE instead of s3.
If it's possible what would be the best way to do it?
http://24ways.org/2008/using-google-app-engine-as-your-own-cdn
It won't be able to host files over 1MB though.
Make sure to read through the comments on that blog post as well, some have concerns about the terms of service.
GAE in itself isn't meant to be a CDN... that doesn't, however, stop you from writing a CDN application on top of it. The only limit you'll need to worry about is the 50 MB limit on the size of the blobstore. Such an app will have to provide a URL that you can hit to get the upload URL, which could then be used to upload the file. The download url can also be generated with the upload URL, and used to access the content.
Can one of the Amazon services (their S3 data service, or otherwise) be used to offload server of static files for a Ruby on Rails app, but still support the app's authentication & authorization?
That is such that when the user browser downloaded the initial HTML for one page of the Ruby on Rails application, when it went back for static content (e.g. an image or CSS file), that this request would be:
(a) routed directly to the Amazon service (no RoR cycles used to serve it, or bandwidth), BUT
(b) the browser request for this item (e.g. an image) would still have to go through an authentication/authorization layer based on the user model in the Ruby on Rails application - in other words to ensure not just anyone could get the image...
thanks
The answer is a yes with a but. You can use a feature of S3 that allows you to create links to secure S3 objects that has a small time to live, default is 5 minutes. This will work for any S3 object that is uploaded as private. This means that the browser will only have X seconds or whatever to request the file from S3. Example code from docs for the AWS gem:
S3Object.url_for('beluga_baby.jpg', 'marcel_molina')
You can also specify an expires_in or expires option per file. The bad thing is that you would need to create a helper for your stylesheet, image, and js links to create the proper S3 URLs.
I would recommend that you setup a domain name for your S3 bucket, like "examples3.amazonaws.com" and put all your standard image files and CSS there as public. Then set that as the asset host in your rails config. Then, only use the secure links for static files that really need it.