I have used carrierwave for my users to upload files in my rails app. When a user uploads multiple files of size more than 400mb (approx.) they get a timeout error.
Note: I've hosted my app rails app on heroku.
Uploading large files through Heroku is generally not recommended. They limit the request to 30 seconds, which would not be enough time for 400MB.
If you are open to using S3, Heroku provides a potential Rails solution for that.
First solution is you can use refile gem.
Reason is " Effortless direct uploads, even to S3 "
This gem is from Jonas Nicklas, who is behind Carrierwave gem
refile
Blog on reason for refile gem
Second solution
move upload file into background jobs
Related
I'm using carrierwave_direct gem for uploading files directly to the aws but as it is mentioned on the documentation that only one file can be uploaded at a time, can some one please suggest me a way to upload multiple files at a time.
I uploaded my app to heroku. The app was built with ruby on rails and had the activeadmin and paperclip gem installed. The files are configured properly and i was able to upload images using he active admin gem. this worked perfectly loaclly and worked fine on heroku until i uploaded more images, now all the images have disappeared. Any ideas on how this can be sorted?
Heroku has an ephemeral filesystem.
That is because each running dyno is an independent container (much like docker) which shared nothing with other dynos, and is destroyed when the app is restarted/deployed.
So any file written on disk will be lost when the dyno is restarted, and is not recoverable.
You need to configure paperclip to upload images to a dedicated file storage system like Amazon S3.
You can't store images on heroku, or anything for that matter. Your uploaded files are available for temporary use only, long enough to process them away to cloud storage elsewhere.
http://cloudinary.com/ offers development accounts for free; https://aws.amazon.com/s3/ has become ridiculously cheap and gives new devs a long free trial. There are other resources but ultimately you have to choose storage which isn't transient.
You can use gems like carrierwave, or attachinary to easily access and store with either of those storage locations.
I want to invalidate files in cloudfront cache. The files are saved in amazon S3.
My requirement is
When I delete a post in my application, I want delete the file in S3 and send an invalidation request to cloudfront.
File delete from S3 is done. But I do not know how to send invalidation request to S3. I read about cloudfront-invalidator gem from https://github.com/reidiculous/cloudfront-invalidator/network/members. But I am not getting any specific example using that gem.
I got my solution using cloudfront-invalidator.
I changed as per my requirement and using it successfully..
I get problem in my rails 2.3.8 and ruby 1.8.7. So customize it and now it is successfully running
Here is the link
https://github.com/krishnasahoo/cloudfront-invalidator
I'm having issues with CKEditor. I can upload and insert pictures without issues, but when I try to do the same with files, the link to my file is set to something like javascript:void(0)/*130*/, with the number changing. This is happening on FF/Safari/Chrome.
My app runs on rails 3.1.3, using MongoDB/Mongoid as database/ODM, with paperclip for handling attachments and using S3 for hosting assets. When I explore my bucket I can see that the files are uploaded correctly, so the problem (probably) come from somewhere else. I'm using this gem, and both the rc2 & the master branch doesn't fix that.
Thanks for your time.
Well it's been a while. I solved it by forking the gem (cf ksol/ckeditor), but the diff is too obfuscated to remember what was wrong. Hopefully the original gem is working now.
One client project will be online two months later. One of the requirements changed is to support large files (10 to 15MB per RAW camera file, expected 1000 to 5000 files download per day) download worldwide for their customers. The process will be:
there is upload screen via paperclip to the rails local public folder
a hourly task to upload to web storage (S3?)
update the download url from paperclip url to the web url
Questions:
is there a gem/plug-in for this
purpose?
if no, any gem/plug-in
for S3 to recommend?
Questions about the storage provider:
is S3 recommended?
or other service to recommend?
The baseline is: the client's web server does not and will not have the bandwidth to handle the downloads.
Thanks
I don't think there is anything that will do all of this out of the box for you. Paperclip will push files sychronousy to S3 on upload, so you will need to make this ansychronous yourself.
S3 is rock-solid, I have used it in production on a number of projects. Totally recommended.
You can upload files directly to S3 which may help by reducing the double handling of the file (no longer need to upload to your app before pushing to Amazon):
http://developer.amazonwebservices.com/connect/entry.jspa?categoryID=139&externalID=1434
The aws-s3 and delayed_job gems are probably what you want.
gem install aws-s3
S3 is popular and widely used as far as I am aware.
If you end up going the route of uploading directly to S3 which offloads the work from your Rails server and makes it asynchronous, please check out my sample projects:
Sample project using Rails 3, Flash and MooTools-based FancyUploader to upload directly to S3: https://github.com/iwasrobbed/Rails3-S3-Uploader-FancyUploader
Sample project using Rails 3, Flash/Silverlight/GoogleGears/BrowserPlus and jQuery-based Plupload to upload directly to S3: https://github.com/iwasrobbed/Rails3-S3-Uploader-Plupload