Rails direct upload to Amazon S3 - ruby-on-rails

Question

I'm looking to add functionality to my Rails app to upload files directly to Amazon S3. From my research the general consensus seems to be to use the s3-swf-upload-plugin. I've setup a sample app using that gem but I can't get it to play nice with only allowing the selection of a single file. I'd also like to create a record post upload and use paperclip to create a thumbnail for which I can find little guidance.
So my questions are:
(1) am I on the right track using that gem or should I be taking another appraoch?
(2) are there any samples out there that I could use for reference?
Any assistance would be much appreciated.
Chris

Answer 1

Try a new Gem called CarrierWaveDirect it allows you to upload files direct to S3 using a html form and easily move the image processing into a background process

Answer 2

Not sure about whether you can modify it easily to only upload one file at a time, but this gem works very well for me. It is based on one of Ryan Bates' Railscast:
https://github.com/waynehoover/s3_direct_upload

Answer 3

Try looking into carrierwave https://github.com/jnicklas/carrierwave (supports s3)
Multi file uploads with carrierwave and uploadify http://blog.assimov.net/post/4306595758/multi-file-upload-with-uploadify-and-carrierwave-on

Answer 4

If you are using Rails 3, please check out my sample projects:
Sample project using Rails 3, Flash and MooTools-based FancyUploader to upload directly to S3: https://github.com/iwasrobbed/Rails3-S3-Uploader-FancyUploader
Sample project using Rails 3, Flash/Silverlight/GoogleGears/BrowserPlus and jQuery-based Plupload to upload directly to S3: https://github.com/iwasrobbed/Rails3-S3-Uploader-Plupload
By the way, you can do post-processing with Paperclip using something like this blog post describes:
http://www.railstoolkit.com/posts/fancyupload-amazon-s3-uploader-with-paperclip

Answer 5

I have adapted Heroku's direct to S3 upload solution in Rails (which uses jQuery-File-Upload and the aws-sdk gem) so uploads to S3 can be made remotely using ajax. I hope this is useful:
posts_controller.rb
before_action :set_s3_direct_post, only: [:index, :create]
before_action :delete_picture_from_s3, only: [:destroy]
class PostsController < ApplicationController
def index
.
.
end
def create
#post = #user.posts.build(post_params)
if #post.save
format.html
format.js
end
end
def destroy
Post.find(params[:id]).destroy
end
private
def set_s3_direct_post
return S3_BUCKET.presigned_post(key: "uploads/#{SecureRandom.uuid}/${filename}", success_action_status: '201', acl: 'public-read')
end
def delete_picture_from_s3
key = params[:picture_url].split('amazonaws.com/')[1]
S3_BUCKET.object(key).delete
return true
rescue => e
# If anyone knows a good way to deal with a defunct file sitting in the bucket, please speak up.
return true
end
def post_params
params.require(:post).permit(:content, :picture_url)
end
end
posts.html.erb
<div class="info" data-url="<%= #s3_direct_post.url %>"
data-formdata="<%= (#s3_direct_post.fields.to_json) %>"
data-host="<%= URI.parse(#s3_direct_post.url).host %>">
</div>
The form
<%= form_for(:post, url: :posts, method: :post,
html: { class: "post_form", id: "post_form-#{post.id}" }
) do |f| %>
<%= f.text_area :content, id: "postfield-#{post.id}", class: "postText" %>
<%= f.button( :submit, name: "Post", title: "Post" ) do %>
<span class="glyphicon glyphicon-ok" aria-hidden="true"></span>
<% end %>
<span class="postuploadbutton" id="postUp-<%= post.id %>" title="Add file" >
<span class="glyphicon glyphicon-upload" aria-hidden="true"></span>
</span>
<span title="Cancel file" class="noticecancelupload" id="postCancel-<%= post.id %>" >
<span class="glyphicon glyphicon-remove-circle" aria-hidden="true"></span>
</span>
<%= f.file_field :picture_url, accept: 'image/jpeg,image/gif,image/png',
class: "notice_file_field", id: "postFile-#{post.id}" %>
<% end %>
_post.html.erb
<%= button_to post_path(
params: {
id: post.id,
picture_url: post.picture_url
}
),
class: 'btn btn-default btn-xs blurme',
data: { confirm: "Delete post: are you sure?" },
method: :delete do %>
<span class="glyphicon glyphicon-remove" aria-hidden="true"></span>
<% end %>
Javascript in each _post.html.erb
$(document).off('click',"#postUp-<%= post.id %>");
$(document).on('click', '#postUp-<%= post.id %>', function(e) {
prepareUpload("#post_form-<%= post.id %>");
$('#postFile-<%= post.id %>').trigger("click");
});
$(document).off('click',"#postCancel-<%= post.id %>");
$(document).on('click', '#postCancel-<%= post.id %>', function(e) {
$(".appendedInput").remove(); // $('#postFile-<% post.id %>').val(""); doesn't work for me
$('.progBar').css('background','white').text("");
});
$(document).off('submit',"#post_form-<%= post.id %>"); // without this the form submitted multiple times in production
$(document).on('submit', '#post_form-<%= post.id %>', function(e) { // don't use $('#post_form-<%= post.id %>').submit(function() { so it doesn't bind to the #post_form (so it still works after ajax loading)
e.preventDefault(); // prevent normal form submission
if ( validatePostForm('<%= post.id %>') ) {
$.ajax({
type: 'POST',
url: $(this).attr('action'),
data: $(this).serialize(),
dataType: 'script'
});
$('#postCancel-<%= post.id %>').trigger("click");
}
});
function validatePostForm(postid) {
if ( jQuery.isBlank($('#postfield-' + postid).val()) && jQuery.isBlank($('#postFile-' + postid).val()) ) {
alert("Write something fascinating or add a picture.");
return false;
} else {
return true;
}
}
Javascript in application.js
function prepareUpload(feckid) {
$(feckid).find("input:file").each(function(i, elem) {
var fileInput = $(elem);
var progressBar = $("<div class='progBar'></div>");
var barContainer = $("<div class='progress'></div>").append(progressBar);
fileInput.after(barContainer);
var maxFS = 10 * 1024 * 1024;
var info = $(".info");
var urlnumbnuts = info.attr("data-url");
var formdatanumbnuts = jQuery.parseJSON(info.attr("data-formdata"));
var hostnumbnuts = info.attr("data-host");
var form = $(fileInput.parents('form:first'));
fileInput.fileupload({
fileInput: fileInput,
maxFileSize: maxFS,
url: urlnumbnuts,
type: 'POST',
autoUpload: true,
formData: formdatanumbnuts,
paramName: 'file',
dataType: 'XML',
replaceFileInput: false,
add: function (e, data) {
$.each(data.files, function (index, file) {
if (file.size > maxFS) {
alert('Alas, the file exceeds the maximum file size of 10MB.');
form[0].reset();
return false;
} else {
data.submit();
return true;
}
});
},
progressall: function (e, data) {
var progress = parseInt(data.loaded / data.total * 100, 10);
progressBar.css('width', progress + '%')
},
start: function (e) {
progressBar.
css('background', 'orange').
css('display', 'block').
css('width', '0%').
text("Preparing...");
},
done: function(e, data) {
var key = $(data.jqXHR.responseXML).find("Key").text();
var url = '//' + hostnumbnuts + '/' + key;
var input = $('<input />', { type:'hidden', class:'appendedInput',
name: fileInput.attr('name'), value: url });
form.append(input);
progressBar.
css('background', 'green').
text("Ready");
},
fail: function(e, data) {
progressBar.
css("background", "red").
css("color", "black").
text("Failed");
}
});
});
} // function prepareUpload()
create.js.erb
$(".info").attr("data-formdata", '<%=raw #s3_direct_post.fields.to_json %>'); // don't use .data() to set attributes
$(".info").attr("data-url", "<%= #s3_direct_post.url %>");
$(".info").attr("data-host", "<%= URI.parse(#s3_direct_post.url).host %>");
$('.post_form')[0].reset();
$('.postText').val('');
application.js
//= require jquery-fileupload/basic
config/initializers/aws.rb
Aws.config.update({
region: 'us-east-1',
credentials: Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY']),
})
S3_BUCKET = Aws::S3::Resource.new.bucket(ENV['S3_BUCKET'])
Notes:
This solution is designed for multiple post forms on the index.html.erb page. This is why the #s3_direct_post information is placed inside a div of class info inside index.html.erb, rather than in each post form. This means there is only one #s3_direct_post presented on the page at any one time, irrespective of the number of forms on the page. The data inside the #s3_direct_post is only grabbed (with a call to prepareUpload()) upon clicking the file upload button. Upon submission a fresh #s3_direct_post is generated in the posts controller, and the information inside .info is updated by create.js.erb. Storing the #s3_direct_post data inside the form means many different instances of #s3_direct_post can exist at once, leading to errors with the file name generation.
You need to :set_s3_direct_post in both the posts controller index action (ready for the first upload) and the create action (ready for the second and subsequent uploads).
Normal form submission is prevented by e.preventDefault(); so it can be done 'manually' with $.ajax({. Why not just use remote: true in the form? Because in Rails, file upload is done with an HTML request and page refresh even when you try to do it remotely.
Use info.attr() rather than info.data() to set and retrieve the #s3_direct_post attributes because info.data doesn't get updated
(for example see this question). This means you also have to manually parse the attribute into an object using jQuery.parseJSON() (which .data() actually does automatically).
Don't use //= require jquery-fileupload in application.js. This bug was a real ballache to identify (see here). The original Heroku solution didn't work until I changed this.

Answer 6

You can use Paperclip to upload to S3 (see documentation) and to create thumbnails, although it uploads to temporary folder first, after that image processing can be applied before uploading file to S3.
As for the examples of such configuration, there are plenty of them throughout the blogosphere and on StackOverflow, e.g. this.