I am trying to upload the file to s3 server, using s3_direct_upload gem. I have followed the instruction in document. But I am unable to upload the file. I am getting following error:
<Code>InvalidRequest</Code>
<Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message>
My bucket region is us-east-2.
And following is my initializer content:
S3DirectUpload.config do |c|
c.access_key_id = ENV["AWS_ACCESS_KEY_ID"]
c.secret_access_key = ENV["AWS_SECRET_ACCESS_KEY"]
c.bucket = ENV["AWS_S3_BUCKET"]
c.region = ENV["AWS_REGION"]
c.url = "https://#{ENV['AWS_S3_BUCKET']}.s3.amazonaws.com"
end
Related
I am using Ruby on Rails AWS SDK to copy files from one bucket to another and store the url in carrierwave. The app uses carrierwave and I need to store the new copied url to the database field that carrierwave uses. The problem is the url is private so I tried to just generate the url and store it in remote_file_url but it can't access the file. The files are very large so I can't use carrierwave to upload the file to sdk. I tried and had no success.
obj.public_url did not work for me. When I copied the file it worked but not when moved it from one bucket to another.
Here is what I have:
picture_name = "my_picture.jpg"
#picture = Picture.find(id)
upload_dir = "uploads/picture/file/#{#picture.id}/my_picture.jpg"
s3 = Fog::Storage.new(provider: 'AWS', :aws_access_key_id => access_key, :aws_secret_access_key => secret_access_key ,:region => region)
obj = s3.copy_object('my-temp-bucket',
picture_name,
"my-target-bucket",
upload_dir, acl: 'public-read')
#picture.remote_image_url = obj.url_for(:read, :expires => 10*60)
#picture.save
I also tried with no luck.
#picture.remote_image_url = obj.public_url
#picture.save
I get an error undefined method `bucket' for #
Thank you for your help!!!
I'm trying to upload a file to amazon s3 but I'm getting the following error:
Aws::S3::Errors::PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
Here is how I'm doing it:
s3 = Aws::S3::Resource.new(
credentials: Aws::Credentials.new('xxxx', 'xxx'),
region: 'us-west-2'
)
obj = s3.bucket('dev.media.xxx.com.br').object('key')
obj.upload_file('/Users/andrefurquin/Desktop/teste.pdf', acl:'public-read')
obj.public_url
I'm sure I'm using the right region, credentials etc..
What can I do ?
I have the following problem,
In my rails 4 app I am hosting images / videos on s3. Currently I made all the files public and for example an image I can access by storing the public link in the database.
However, I want some of the images videos to be private.
I looked at the presigned url options using the following
s3 = Aws::S3::Client.new(
region: AWS_REGION,
access_key_id: S3_CONFIG['access_key_id'],
secret_access_key: S3_CONFIG['secret_access_key']
)
resource = Aws::S3::Resource.new(client: s3)
bucket = resource.bucket(BUCKET_NAME)
utilities = bucket.objects(prefix: '/folder').each do |obj|
obj.presigned_url(:get, expires_in: 3600).to_s
end
This works fine, but how would I use the presigned url since I can obviously not store them in the db like the public links.
I am using aws-sdk version 2
I am also wondering if this in general is a good solution?
Thanks for any hints,
Jean
Here is the Presigner Doc
Example:
signer = Aws::S3::Presigner.new
url = signer.presigned_url(:put_object, bucket: "bucket", key: "path")
I have been able to have third party clients upload files directly to AWS s3 and then process those files with paperclip with the following line in the model:
my_object.file_attachment = URI.parse(URI.escape(my_bucket.s3.amazonaws.com/whatever.ext))
That line downloads the file, processes it and then saves it appropriately. The problem is, in order for that line to work, I have to provide anonymous read privileges for the upload location. So my question is: How do avoid that? My thought is to use the aws-sdk to download the file - so I have been trying stuff like:
file = Tempfile.new('temp', :encoding => 'ascii-8bit')
bucket.objects[aws_key].read do |chunk|
file.write chunk
end
my_object.file_attachment = file
and variations on that theme, but nothing is working so far. Any insights would be most helpful.
Solution I am not very happy with
You can generate a temporary privileged URL using the AWS SDK:
s3 = AWS::S3.new
bucket = s3.buckets['bucket_name']
my_object.file_attachment = bucket.objects['relative/path/of/uploaded/file.ext'].url_for(:read)
As #laertiades says in his amended question, one solution is to create a temporary, pre-signed URL using the AWS SDK.
AWS SDK version 1
In AWS SDK version 1, that looks like this:
s3 = AWS::S3.new
bucket = s3.buckets['bucket_name']
my_object.file_attachment = bucket.objects['relative/path/of/uploaded/file.ext'].url_for(:read)
AWS documentation: http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/S3/S3Object.html#url_for-instance_method
AWS SDK version 2
In AWS SDK version 2, it looks like this with the optional expires_in parameter (credit to this answer on another question):
presigner = Aws::S3::Presigner.new
my_object.file_attachment = presigner.presigned_url(:get_object, # get_object method means read-only
bucket: 'bucket-name',
key: "relative/path/of/uploaded/file.ext",
expires_in: 10.minutes.to_i # time should be in seconds
).to_s
AWS documentation: http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Presigner.html
What's the more straightforward way to download a file from given URL and uploading it immediately to Amazon S3 (+ save into database some information about the file, like name, size etc)?
Right now, I am not using Paperclip neither Carrierwave.
Thank you
Straightforward:
require 'open-uri'
require 's3'
amazon = S3::Service.new(access_key_id: 'KEY', secret_access_key: 'KEY')
bucket = amazon.buckets.find('image_storage')
url = 'http://www.example.com/url'
download = open(url)
file = bucket.objects.build('image.png')
file.content = (File.read download)
if file.save
# Make a new ActiveRecord::Base class for this
LogFile.create(size: download.size, type: download.type, name: url)
end
https://github.com/qoobaa/s3