What's the more straightforward way to download a file from given URL and uploading it immediately to Amazon S3 (+ save into database some information about the file, like name, size etc)?
Right now, I am not using Paperclip neither Carrierwave.
Thank you
Straightforward:
require 'open-uri'
require 's3'
amazon = S3::Service.new(access_key_id: 'KEY', secret_access_key: 'KEY')
bucket = amazon.buckets.find('image_storage')
url = 'http://www.example.com/url'
download = open(url)
file = bucket.objects.build('image.png')
file.content = (File.read download)
if file.save
# Make a new ActiveRecord::Base class for this
LogFile.create(size: download.size, type: download.type, name: url)
end
https://github.com/qoobaa/s3
Related
I am using aws-skd-s3 gem in my Rails project.
Create S3 resoure
s3 = Aws::S3::Resource.new(access_key_id: #####,
secret_access_key: #####,
region: 'us-east-1')
Create an S3 object
path = 'sample'
key = test.csv
obj = s3.bucket(#{bucket_name}).object("#{path}" + key)
Store CSV in S3
obj.put(body: csv_response, content_type: 'text/csv')
How to verify that put method stored the csv in S3 without any issues?
Is there any status code available for put method in S3 to verify?
Two ways to go about it:
Store the result. It should be a PutObjectOutput type object. You can check out the official method documentation of the put request method.
The second way to go about it is to make a exists? call right after your put request is completed. Something like this:
s3 = Aws::S3::Resource.new(region: 'ap-southeast-1') # change to the region you use
obj = s3.bucket('bucket-name').object("path/to/object/in/bucket")
if obj.exists?
# Object was uploaded successfully!
else
# No it wasn't!
end
Hope that helps!
One way I've seen or read other people doing it is calculating a md5 hash of the original file before upload and then match that with the etag value from the response of obj.put
In my Rails app I save customer RMA shipping labels to an S3 bucket on creation. I just updated to V2 of the aws-sdk gem, and now my code for setting the ACL doesn't work.
Code that worked in V1.X:
# Saves label to S3 bucket
s3 = AWS::S3.new
obj = s3.buckets[ENV['S3_BUCKET_NAME']].objects["#{shippinglabel_filename}"]
obj.write(open(label.label('pdf').postage_label.label_pdf_url, 'rb'), :acl => :public_read)
.write seems to have been deprecated, so I'm using .put now. Everything is working, except when I try to set the ACL.
New code for V2.0:
# Saves label to S3 bucket
s3 = Aws::S3::Resource.new
obj = s3.bucket(ENV['S3_BUCKET_NAME']).object("#{shippinglabel_filename}")
obj.put(Base64.decode64(label_base64), { :acl => :public_read })
I get an Aws::S3::Errors::InvalidArgument error, pointed at the ACL.
This code works for me:
photo_obj = bucket.object object_name
photo_obj.upload_file path, {acl: 'public-read'}
so you need to use the string 'public-read' for the acl. I found this by seeing an example in object.rb
I have the following problem,
In my rails 4 app I am hosting images / videos on s3. Currently I made all the files public and for example an image I can access by storing the public link in the database.
However, I want some of the images videos to be private.
I looked at the presigned url options using the following
s3 = Aws::S3::Client.new(
region: AWS_REGION,
access_key_id: S3_CONFIG['access_key_id'],
secret_access_key: S3_CONFIG['secret_access_key']
)
resource = Aws::S3::Resource.new(client: s3)
bucket = resource.bucket(BUCKET_NAME)
utilities = bucket.objects(prefix: '/folder').each do |obj|
obj.presigned_url(:get, expires_in: 3600).to_s
end
This works fine, but how would I use the presigned url since I can obviously not store them in the db like the public links.
I am using aws-sdk version 2
I am also wondering if this in general is a good solution?
Thanks for any hints,
Jean
Here is the Presigner Doc
Example:
signer = Aws::S3::Presigner.new
url = signer.presigned_url(:put_object, bucket: "bucket", key: "path")
I am developing a website on ruby on rails where users can upload pictures thanks to paperclip, it is stored in amazon S3. After, they can modify pictures thanks to aviary. But when i want to save the new pictures, aviary just gave me an temporary URL where i can get my modified picture.
Does paperclip can do it ? I don't think it can save an picture from an URL and store it to S3 ?
I've searched for a week now, and i don't know the best way to do it. I've read about filepicker, but the account to store data in S3 files isn't free ...
Finally i've heard about this s3 https://github.com/qoobaa/s3, but i don't understand how to use it. I have installed gem s3, but when i set require 's3' , it is not recognize.
What is the best to do?
Why don't you pass the URL that Aviary generates to your server and upload the new photo from there? The code below does that in Python/Django:
#login_required
#csrf_exempt
def upload_from_url(request):
origin_url = request.POST.get("origin_url")
name = request.POST.get("name")
try:
conn = boto.connect_s3(settings.AWS_ACCESS_KEY_ID, settings.AWS_SECRET_ACCESS_KEY)
bucket_name = settings.AWS_UGC_STORAGE_BUCKET_NAME
bucket = conn.get_bucket(bucket_name)
k = Key(bucket)
k.key = name
file_object = urllib2.urlopen(origin_url)
fp = StringIO.StringIO(file_object.read())
k.set_contents_from_file(fp)
return HttpResponse("Success")
except Exception, e:
return HttpResponse(e, mimetype='application/javascript')
Hope this helps.
Paperclip has matured a lot since this question was answered. If you want to save files by passing a URL, as of Paperclip v3.1.4, you can just assign the URL to your Paperclip attachment attribute.
Let's say I have a class User and my attachment is called avatar. We'll have the following in our User model:
has_attached_file :avatar
# Validate the attached image is image/jpg, image/png, etc
# This is required by later releases of Paperclip
validates_attachment_content_type :avatar, :content_type => /\Aimage\/.*\Z/
In our view, we can define a hidden field that will accept the temporary URL received from Aviary:
= f.hidden_field :avatar, id: 'avatar'
We can set the value of this hidden field with the Aviary onSave callback:
var featherEditor = new Aviary.Feather({
apiKey: '#{ENV['AVIARY_KEY']}',
onSave: function(imageID, newURL) {
var img = document.getElementById(imageID);
img.src = newURL;
var avatar = document.getElementById('avatar');
avatar.value = newURL;
featherEditor.close();
}
});
Within onSave, you can use AJAX to update the User object, use jQuery's .submit() to submit the form, or let the user submit it when they want.
I have a problem with S3 and CarrierWave:
I have a pseudo-form that uploads data and files, I wrote "pseudo" because it's an ajax form so data is sent with jquery to rails with a POST request. Files cannot be uploaded in this way...so I have a popup windows that upload files to rails, I save in the session the reference to the uploaded files and when the ajax request uploads the rest of the form, I link the files uploaded to the rest of the data.
With storage :file it works without any problems, when i receive the file I do:
uploader = ImgObjUploader.new
uploader.store!(params[:image_form][:image])
session["image"] = uploader.url
and then when I get the rest of the data:
if (session[:image] != nil) then
obj.image = File.open(session[:image])
end
And my model is:
mount_uploader :image, ImgObjUploader
This code work without any problems, for amazon s3 I switched to:
uploader = ImgObjUploader.new
uploader.retrieve_from_store!(session[:image])
puts uploader
#obj.image = uploader
obj.image = uploader.url
but it doesn't work...I didn't receive an error but I don't have the image saved inside obj object. Puts uploader prints the url of amazon S3.
Anyone can help me?
Thank You.