Imagine the following use case:
You have a basecamp style application hosting files with S3. Accounts all have their own files, but stored on S3.
How, therefore, would a developer go about securing files so users of account 1, couldn't somehow get to files of account 2?
We're talking Rails if that's a help.
S3 supports signed time expiring URLs that mean you can furnish a user with a URL that effectively lets only people with that link view the file, and only within a certain time period from issue.
http://www.miracletutorials.com/s3-amazon-expiring-urls/
If you want to restrict control of those remote resources you could proxy the files through your app. For something like S3 this may defeat the purpose of what you are trying to do, but it would still allow you to keep the data with amazon and restrict access.
You should be careful with an approach like this as it could cause your ruby thread to block while it is proxying the file, which could become a real problem with the application.
Serve the files using an EC2 Instance
If you set your S3 bucket to private, then start up an EC2 instance, you could serve your files on S3 via EC2, using the EC2 instance to verify permissions based on your application's rules. Because there is no charge for EC2 to transfer to/from S3 (within the same region), you don't have to double up your bandwidth consumption costs at Amazon.
I haven't tackled this exact issue. But that doesn't stop me from having an opinion :)
Check out cancan:
http://github.com/ryanb/cancan
http://railscasts.com/episodes/192-authorization-with-cancan
It allows custom authorization schemes, without too much hassle.
Related
So I have a standard Rails app running on ec2 that needs access to s3. I am currently doing it with long-term access keys, but rotating keys is a pain, and I would like to move away from this. It seems I have two alternative options:
One, tagging the ec2 instance with a role with proper permissions to access the s3 bucket. This seems easy to setup, yet not having any access keys seems like a bit of a security threat. If someone is able to access a server, it would be very difficult to stop access to s3. Example
Two, I can 'Assume the role' using the ruby SDK and STS classes to get temporary access keys from the role, and use them in the rails application. I am pretty confused how to set this up, but could probably figure it out. It seems like a very secure method, however, as even if someone gets access to your server, the temporary access keys make it considerably harder to access your s3 data over the long term. General methodology of this setup.
I guess my main question is which should I go with? Which is the industry standard nowadays? Does anyone have experience setting up STS?
Sincere thanks for the help and any further understanding on this issue!
All of the methods in your question require AWS Access Keys. These keys may not be obvious but they are there. There is not much that you can do to stop someone once they have access inside the EC2 instance other than terminating the instance. (There are other options, but that is for forensics)
You are currently storing long term keys on your instance. This is strongly NOT recommended. The recommended "best practices" method is to use IAM Roles and assign a role with only required permissions. The AWS SDKs will get the credentials from the instance's metadata.
You are giving some thought to using STS. However, you need credentials to call STS to obtain temporary credentials. STS is an excellent service, but is designed to for handing out short term temporary credentials to others - such as the case where your web server is creating credentials via STS to hand to your users for limited case use such as accessing files on S3 or sending an email, etc. The fault in your thinking about STS is that once the bad guy has access to your server, he will just steal the keys that you call STS with, thereby defeating the need to call STS.
In summary, follow best practices for securing your server such as NACLs, security groups, least privilege, minimum installed software, etc. Then use IAM Roles and assign the minimum privileges to your EC2 instance. Don't forget the value of always backing up your data to a location that your access keys CANNOT access.
Each tenant will have their own photo gallery which stores photos on Amazon S3. Seeing as S3 is relatively new to me I'm looking for some advice and best practices on how to manage this in terms of buckets, IAM groups/users, security, usage reporting, and possibly billing.
The way I see it is I have two options.
Option 1:
One master bucket. Each tenant has a sub-directory where their photos are stored. I would have one IAM group for the whole application and create a new IAM user for each tenant with access to only their sub-directory. In the future if I want to know how much S3 space a tenant is using will it be easy to report on? Would I want to have a unique AWS access key and secret key for each tenant even though they are going to the same bucket?
Option 2:
Each tenant gets their own bucket. Each tenant would get their own IAM user with access only to their bucket. Is this option better for reporting on usage?
General questions:
Are there any major drawbacks to either option?
Is there another option I'm unaware of?
Can I report on storage via an IAM user's activity or does it happen
at the bucket level?
I think you're trying to turn your S3 account into a multi-user thing, which it's not.
Each tenant gets their own bucket
You are limited to 100 buckets, so this is probably not what you want. (Unless it's a very exclusive web service :)
One master bucket
OK
IAM user for each tenant
Um, I think there's a limit for IAM users too.
if I want to know how much S3 space a tenant is using will it be easy to report on?
You can write a script easy enough.
billing
You can use DevPay buckets, in which case you can have 100 buckets per user. But this requires each user sign up for AWS and other complications.
Can I report on storage via an IAM user's activity or does it happen at the bucket level?
IAM is only checked at "ingress". After that, it's all just "your account". So the files don't have different "owners".
Is there another option I'm unaware of?
The usual way is to have a thin EC2 service that controls the security:
You write a web app and run it on EC2. It knows how to authenticate your users.
When a user wants to upload, they either POST it to EC2 (and it copies to S3, and probably resizes it anyway). Or you generate a signed POST/PUT URL for the browser to directly upload into S3 (really easy to do once you understand.)
When a user wants to view a file, they hit your service to get a signed URL that allows them access to their file. But that access times out after a while. That's OK, since they are only accessing the files via your EC2 webpage.
The upshot is that your EC2 box can be small because it's just creating URLs for the browser.
Im struggling to find an answer to this. I have a website that is deployed in a shared hosting environment. I want to allow people to upload files to my azure blob storage account.
I have this working locally, using the storage emulator, however when I publish the site I get a Security Exception.
Is this actually possible under a shared hosting envrionment ?
Cheers
A bit more detail would help, in understanding how these uploads are taking place. That said, I'll make the assumption that people are uploading directly to Blob Storage, and not through your Website (or Web Service).
To allow direct uploads, you need to provide either a public blob or container (which everyone in the world can see), or create a temporary Shared Access Signature (SAS) on a specific blob or container, that grants access for a short time window.
If your app is Silverlight, then you are probably running into a cross-domain issue (and you'll need to correct that with an access policy).
If you provide more details around the way uploads are being sent, as well as the client and server technology, I can edit my answer to be more specific.
just recently I was thinking and wondered, how does Dropbox upload my files to its S3 storage and how might that one be organized?
Let's just completely forget about the sync aspect for a second and scale the problem down to one S3 bucket.
Say, in that bucket's root directory you have lots of folders, each belonging to an arbitrary user.
Now if that user wants to upload a file to his folder... how does that happen internally? I mean, Dropbox can't just store the Amazon S3 access credentials/keys hard-coded into the application (be it on ios or windows) as it might get reverse-engineered and thus exposed.
Any thoughts on this?
Thanks!
Some guys from EADS did reengineering on Dropbox, the presentation slides are available for download: A CRITICAL ANALYSIS OF
DROPBOX SOFTWARE SECURITY
In the same way websites don't allow users to directly access their databases but rather provide interfaces that can control permissions and handle authentication, I'm sure Dropbox has some kind of application that the client on your computer interacts with. Their server daemon will have permissions to write to the disk, but your computer has to go through it (and it's security procedures) before anything your computer sends is written.
I have a Rails application that I want to add file upload to, so that the users have access to a "resources" section where they can upload and share (although not publicly) any type of file. I know I could build a solution using paperclip and S3 for example, but to try and avoid the admin overhead of all that I'm looking at API interfaces to drop.io and box.net. Does anyone have any experience of these? I've got a basic demo working rather well to drop.io, but I was just wondering if anyone had any better ideas or experiences.
Many thanks
D
I use attachment_fu with S3 backend. For User Interface goodness, I use YUI's file uploader.
Some of the files are uploaded with world read access, others with no public read access.
I use Attachement_fu to create self-signed urls to enable clients to access the private S3 files.
I did write some small helper routines for the S3 library for re-connecting after a timeout, handling various errors that the S3 library can raise, etc.
Building your own library for drop.io and/or box.net
Your idea of using the API for a commercial service is interesting but I haven't run into any problems with the above config. And the price for direct S3 access is very low.
If you do decide to go this route, you may want to open source your code. You'd benefit by getting testing, ideas, and possible code contributions from the community.
Note that if you have a lot of uploads, you can end up with a performance issue if the uploads are synchronous with the Rails thread--the rails process is busy uploading and can't do anything else until the upload is done.
HTH,
Larry