I am planning to create a webapp which user can upload mp3 files.So it required large server space.So weather I have to buy the whole server space or can i use any webservice(which giving spaces like amazon-s3).I am using Ruby on Rails .Please give me advices.
I would use Amazon S3 for something like this.
I manage a Web Service developed in Python where images are uploaded, and S3 works a treat. Needing large server space is no issue here either as I don't think there are any limits as to how much you can store on S3.
Day 1 costs $0, then you are billed monthly per byte used.
There is a tutorial for Ruby and S3 here.
Related
We are using "image_path" a lot of times in one file.
In skylight.io I see that the rendering of this image_paths takes a lot of resources and a lot of time.
Does anybody know a workaround?
Your server now handles the rendering of this image because this file might be stored on your server. This is why it takes a lot of resources because the server has to send down a lot of bytes to the client.
You should consider using an image CDN such as Cloudinary or Amazon CloudFront or S3 if you using AWS to move this load away from your server.
In my application I have around 400 images that need to be displayed at various times. There will be no user uploaded imagery. In other words, I control all the pictures being used within my application.
I'm wondering what the recommended route is. Would it be best to put all the images in app/assets/images or would it be better to upload all of them to a 3rd party service like AWS?
The application will eventually be living through Heroku. Thanks.
From this question (and first comment), your total compiled code and assets cannot exceed 100MB. As long as you keep under this, you'll be fine with Heroku. However, if you exceed that, or the number of files will change dramatically or consistently, I'd recommend Cloudinary, which gives you 500MB of FREE (file)storage and is available as a Heroku Add-on.
I've been having this issue for sometime now. On fillim.com (indie film distribution, so large files) we're using using this fork of the s3_swf_upload gem for rails. We're getting everyone complaining that it will fail sometimes 3-4 times before it will fully upload the file, like almost everyone.
We're on Heroku, and we're then of course needing to do direct uploads to S3.
We're not getting any errors generated, in our logs or in the browser, and we just can not for the life of us find the cause.
Has anyone had these issues before? Does anyone know of alternatives? If anyone knows of an alternative that supports files larger than 2GB, that would be even better.
If You are trying to upload files on amazon s3, Then use AWS::S3 a Ruby Library for uploading files.
http://amazon.rubyforge.org/
I thing default size
:fileSizeLimit (integer = 524288000)
Individual file size limit in bytes (default is 512 MB)
you need to increase your filesizelimit
The repeated failures is unsurprising. If you're going to upload files that large, you want to leverage S3's "multipart upload" support. Essentially, the file is broken-up into pieces, sent in parts, then reassembled on the S3-side.
The official AWS SDK for Ruby supports this feature, but you'd have to implement it into your gem. I don't know whether or not that's outside the scope of what you were looking for.
Also, am I correct in understanding that you're wanting to allow users to upload files > 2GB from their web browsers?
I need to write an application, which allows the user to upload large videos. Afaik, PHP stores the entire uploaded file into memory (at least per default), so you get problems with large files. Has Rails similar problems? I need to receive files up to 2 GB.
My setup will be:
Ruby 1.8.7
Rails 3.0
Passenger 3.0
Apache 2.2
Unless you recommend something else, I would give Paperclip a try.
Regards, Johannes
It's possible, and we have an commercial website which is currently handling uploads ~3GBs for long HD videos just fine with CarrierWave - a great alternative to Paperclip.
So long as you have Apache setup correctly to accept requests that large, you probably won't have the same issues that PHP applications and the like traditionally do with configuration hell needed to set the maximum request size and whatnot.
Read this for the caveats, though: http://www.therailsway.com/2009/4/23/uploading-files
Edit: For what it's worth, we're using Nginx + upload module (see https://github.com/vkholodkov/nginx-upload-module for info) to do this and avoid the issues in the above article; afaik Rails loads the entire uploaded file into memory when handling uploads normally, which means you're going to need to have a significant amount of memory unless you're using something like the mod_porter plugin mentioned in the above article.
Since Heroku is a read-only filesystem I can't use paperclip to store a small quantity of files on the server. Database image storage is an option, but not particularly ideal since that may crank my client's DB size up from a few hundred KB to over the 5 MB 'free' shared DB limit (depending on size of images).
That leaves Amazon S3 as a likely solution. I understand that Heroku is hosted on EC2 (I believe?). Amazon's pricing wording was a little bit confusing when referring to S3-EC2 file transfers. If I have my client setup an S3 account and let them do file transfers to and from there, what is the pricing going to look like?
Is it cheaper from an S3 point-of-view to to both upload and download data in the rails controllers, and then feed the data to the browser using send_file? Or would it make more sense to just link straight to the image or pdf from the browser like normal?
Would my client have to pay anything at all since heroku is hosted on Amazon? I was looking for other questions related to this but there weren't any really straight answers concerning which parts of the file transfer would be charged for.
I guess the storage would cost a little (hardly anything), but what about the bandwidth? Thanks :)
Is it cheaper from an S3 point-of-view
to to both upload and download data in
the rails controllers, and then feed
the data to the browser using
send_file? Or would it make more sense
to just link straight to the image or
pdf from the browser like normal?
From an S3 standpoint, yes, this would be free, because Heroku would be covering your transfer costs. HOWEVER: Heroku only lets a script run for 30 seconds, and during that time, other clients wont be able to load the site, so this is really a terrible idea. Your best bet is to serve the files out of S3 directly, in which case, yes your customer would be transfer between S3 and the end user.
Any interaction you have with the file from Heroku (i.e. metadata and what not) will be free because it is EC2->S3.
For most cases, your pricing would be identical to what it would be if you were not using heroku. The only case where this would change would be if your app is constantly accessing the data directly on S3 (to read metadata/load files)
You can use Paperclip on Heroku - just not the local file system for storage. Fortunately Paperclip can use s3 for storage. Heroku has a tech article here that covers it.
Also when an asset that's been uploaded is displayed on a page (lookup asset_host) the image would be loaded directly from your s3 buckets URL so you will pay Amazon for a get request to the image and then for data transfer involved but also for storing the assets on s3. Have you looked at the s3 calculator to get indicative costs?