I'm using paperclip to upload some text/csv files to an S3 bucket.
I need to edit those files occasionally. How can I edit and re-save those?
Before using S3, I was just using File.open for saving the files, but that throws a "No such file or directory" error now.
File.open opens a file system object. To access S3 you need to connect to the S3 server and issue commands, such as GET.
There are some gems available from the Amazon developer's site which may help.
http://aws.amazon.com/ruby/
Related
I want to allow my user to download a bundle of files that are stored on s3 using the zipline gem. The files are already hosted on an s3 server but they aren't there as part of a paperclip or carrierwave attachment in my app. Will I need to create some records in my database to sort of trick zipline into thinking they are paperclip attachments, or is there a way I can send the zip file without bothering with an attachment gem? At the moment, trying to download the files with zipline doesn't throw an error message at all. It just seems to skip right over and nothing downloads.
See the part of the zipline README where an enumerator gets used to include remote files into the ZIP. It uses absolute URLs, to generate those from your S3 objects you will need to use presigned URLs (which Zipline is going to pass on to Curb):
Aws::S3::Bucket.new(your_bucket_name).object(your_key).presigned_url(:get)
In rails 4.0.2, I am using s3_direct_upload gem to upload files directly to s3, after that I want to store the files & convert it to html, also I want to store some dependency files like converted .html / .txt files back to s3. But I am not able to get the files from s3 & also it is not saving to the database.
I have referred this from http://blog.littleblimp.com/post/53942611764/direct-uploads-to-s3-with-rails-paperclip-and.
How can I overcome this problem? awaiting for the solution.
I already have files in my s3, which were uploaded via FTP, and I'd just like to attach them to my models, which allow upload via CarrierWave, if I'm using the same bucket to store the files I upload via CarrierWave and the ones I upload via FTP, is there a way to just assign the s3 key for the file to the (new) associated record (via a model) where the file itself is handled via CarrierWave's attachment strategy?
Assuming you have the thumbnails already created, and can store the files in the correct directory on S3, you could simply:
#user.update_column(:image, "your-image-name.png")
This will not execute any callbacks, and carrierwave will assume all processing has already been completed.
That's a lot of assumptions though, so this would likely not work for you in reality.
What would be the best solution to upload files to file storage, upload to S3 (via Resque), and then modify the storage type for carrierwave on the record? Is this possible?
Thanks!
You could try the carrierwave_backgrounder gem which has Resque support for uploading files to S3 in the background after the file has hit your server.
Alternatively you could try uploading directly to S3 from the user's browser using the new CORS support. Eg. s3_direct_upload
I'd like to be able to upload a zip file to my Rails application that contains a number of images. Then I'd like Rails to unzip that file and attach the images inside to my Photo's model via Paperclip, so that they are ultimately stored on my Amazon S3 account (configured through Paperclip).
I'd like do do this all on my Rails site hosted on Heroku, which unfortunately doesn't allow local storage of any kind (so far as I'm aware) to temporarily do the unzipping before the Paperclip parsing.
How would I do this??
I would recommend uploading directly to S3 which bypasses Heroku entirely so you're not restricted to the 30 second request timeout they enforce (which drops your uploads after that time is hit) or the 1gb /tmp directory limit. After the file is uploaded, you can make a POST to your Rails app with the file's name and location and then do your unzipping operation. If you'd like to use Paperclip for post-processing, I have attached a link below. If you end up going the route of uploading directly to S3 which offloads the work from your Rails server, please check out my sample projects:
Sample project using Rails 3, Flash and MooTools-based FancyUploader to upload directly to S3: https://github.com/iwasrobbed/Rails3-S3-Uploader-FancyUploader
Sample project using Rails 3, Flash/Silverlight/GoogleGears/BrowserPlus and jQuery-based Plupload to upload directly to S3: https://github.com/iwasrobbed/Rails3-S3-Uploader-Plupload
Here is the link for the Paperclip post processing for an example like images:
http://www.railstoolkit.com/posts/fancyupload-amazon-s3-uploader-with-paperclip
dmagkic is correct about the rails_root/tmp. I recommend something like the following:
Upload files through heroku to S3
Setup a background job to zip the files (store the file names that you need to group)
run the BJ that downloads the files from S3, zips them, sends the zip to S3, removes the unzipped files.
That way your application will still be responsive'ish during the upload process.
If you try to upload multiple files, you COULD write to /tmp, but just make sure that all the files come across in the same post request.
Heroku does allow writing to #{RAILS_ROOT}/tmp.
But you need to take in mind that file will be there only as long as request lasts. Probably longer, but that is not guaranteed. You could try to block request while you unzip and send to S3, but you should take care of the time it takes.
It sounds to me like you need some flash uploader that can unzip and send to S3, without Heroku.