My app uploads an image to S3 and returns the URL of that image. However, it returns a URL like http://my-client-dev.s3.amazonaws.com/gifs/export_gif_3_1438603398.gif?AWSAccessKeyId=AKILLV5BH1BNAT3D3D3Q&Expires=1438607003&Signature=SBQmnULXR5F5cAtyZgqbFl0%2Bn%2Fk%3D
How can I get it to return the simple URL that doesn't require access key and signature in the URL? Also, the links expire pretty quickly.
Here is my code:
bucket.objects[filename].write(result.to_blob, {:acl=>:public_read})
url = bucket.objects[filename].url_for(:read, :secure => false).to_s
Have you looked at the S3 SDK? It has a #public_url method.
Related
I am using aws-skd-s3 gem in my Rails project.
Create S3 resoure
s3 = Aws::S3::Resource.new(access_key_id: #####,
secret_access_key: #####,
region: 'us-east-1')
Create an S3 object
path = 'sample'
key = test.csv
obj = s3.bucket(#{bucket_name}).object("#{path}" + key)
Store CSV in S3
obj.put(body: csv_response, content_type: 'text/csv')
How to verify that put method stored the csv in S3 without any issues?
Is there any status code available for put method in S3 to verify?
Two ways to go about it:
Store the result. It should be a PutObjectOutput type object. You can check out the official method documentation of the put request method.
The second way to go about it is to make a exists? call right after your put request is completed. Something like this:
s3 = Aws::S3::Resource.new(region: 'ap-southeast-1') # change to the region you use
obj = s3.bucket('bucket-name').object("path/to/object/in/bucket")
if obj.exists?
# Object was uploaded successfully!
else
# No it wasn't!
end
Hope that helps!
One way I've seen or read other people doing it is calculating a md5 hash of the original file before upload and then match that with the etag value from the response of obj.put
I am currently using the Ruby aws-sdk, version 2 gem with server-side customer-provided encryption key(SSE-C). I am able to upload the object from a rails form to Amazon S3 with no issues.
def s3
Aws::S3::Object.new(
bucket_name: ENV['S3_BUCKET'],
key: 'hello',
)
end
def upload_object
customer_key = OpenSSL::Cipher::AES.new(256, :CBC).random_key
customer_key_md5 = Digest::MD5.new.digest(customer_key)
object_key = 'hello'
options = {}
options[:key] = object_key
options[:sse_customer_algorithm] = 'AES256'
options[:sse_customer_key] = customer_key
options[:sse_customer_key_md5] = customer_key_md5
options[:body] = 'hello world'
options[:bucket] = ENV['S3_BUCKET']
s3.put(options)
test_params = {
object_key: object_key,
customer_key: Base64.encode64(customer_key),
md5_key: Base64.encode64(customer_key_md5),
}
Test.create(test_params)
end
But I'm having some issues with retrieving the object and generating a signed url link for the user to download it.
def retrieve_object(customer_key, md5)
options = {}
options[:key] = 'hello
options[:sse_customer_algorithm] = 'AES256'
options[:sse_customer_key] = Base64.decode64(customer_key)
options[:sse_customer_key_md5] = Base64.decode64(md5)
options[:bucket] = ENV['S3_BUCKET']
s3.get(options)
url = s3.presigned_url(:get)
end
The link is generated, but when I clicked on it, it directs me to an amazon page saying.
<Error>
<Code>InvalidRequest</Code>
<Message>
The object was stored using a form of Server Side Encryption. The correct parameters must be provided to retrieve the object.
</Message>
<RequestId>93684EEBA062B1C2</RequestId>
<HostId>
OCnn5EG7ydfoKzsmEDMbqK5kOhLFpNXxVRdekfhOfnBc6s+jtPYFsKi8IZsEPcd9ConbYUHgwC8=
</HostId>
</Error>
The error message is not helping as I'm unsure what parameters I need to add. I think I might be missing some permissions parameters.
Get Method
http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Object.html#get-instance_method
Presigned_Url Method
http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Object.html#presigned_url-instance_method
When you generate a pre-signed GET object URL, you need to provide all of the same params that you would pass to Aws::S3::Object#get.
s3.get(sse_customer_algorithm: 'AES256', sse_customer_key: customer_key).body.read
This means you need to pass the same sse_customer_* options to #presigned_url:
url = obj.presigned_url(:get,
sse_customer_algorithm: 'AES256',
sse_customer_key: customer_key)
This will ensure that the SDK correctly signs the headers that Amazon S3 expects when you make the final GET request. The next problem is that you are now responsible for sending those values along with the GET request as headers. Amazon S3 will not accept the algorithm and key in the query string.
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Get.new(uri.request_uri, {
"x-amz-server-side-encryption-customer-algorithm" => 'AES256',
"x-amz-server-side-encryption-customer-key" => Base64.encode64(cpk),
"x-amz-server-side-encryption-customer-key-MD5" => Base64.encode64(OpenSSL::Digest::MD5.digest(cpk))
})
Please note - while testing this, I found a bug in the presigned URL implementation of the current v2.0.33 version of the aws-sdk gem. This has been fixed now and should be part of v2.0.34 once it releases.
See the following gitst for a full example that patches the bug and demonstrates:
Uploads an object using a cpk
Gets the object using the SDK
Generates a presigned GET url
Downloads the object using just Net::HTTP and the presigned URL
You can view the sample script here:
https://gist.github.com/trevorrowe/49bfb9d59f83ad450a9e
Just replace the bucket_name and object_key variables at the top of the script.
I have the following problem,
In my rails 4 app I am hosting images / videos on s3. Currently I made all the files public and for example an image I can access by storing the public link in the database.
However, I want some of the images videos to be private.
I looked at the presigned url options using the following
s3 = Aws::S3::Client.new(
region: AWS_REGION,
access_key_id: S3_CONFIG['access_key_id'],
secret_access_key: S3_CONFIG['secret_access_key']
)
resource = Aws::S3::Resource.new(client: s3)
bucket = resource.bucket(BUCKET_NAME)
utilities = bucket.objects(prefix: '/folder').each do |obj|
obj.presigned_url(:get, expires_in: 3600).to_s
end
This works fine, but how would I use the presigned url since I can obviously not store them in the db like the public links.
I am using aws-sdk version 2
I am also wondering if this in general is a good solution?
Thanks for any hints,
Jean
Here is the Presigner Doc
Example:
signer = Aws::S3::Presigner.new
url = signer.presigned_url(:put_object, bucket: "bucket", key: "path")
I'm currently storing files privately on S3. In my Rails app, in the attachment.rb model I can obtain a public URL for the private file like so:
def cdn_url ( style='original' )
attachment.s3_object(style).url_for( :read, secure: true, response_content_type: self.meta['file_content_type'], expires: 1.hour ).to_s
end
The problem is this is providing a URL to S3 and rewriting the URL to use my Cloudfront origin url is erroring with:
The request signature we calculated does not match the signature you provided. Check your key and signing method.
How can I get a public URL asset like below but serve the asset via Cloudfront?
First Way (Easy)
Just use aws_cf_signer gem. Put it in you bundler.
WIth this you can do something like
def cdn_url (options = {})
style = options[:style] || 'original'
cloudfront_domain = options[:cloudfront_domain] || 'example.cloudfront.net'
cloudfront_pem_key_path = options[:cloudfront_pem_key_path]
cloudfront_key_paid_id = options[:cloundfrount_key_paid_id]
path = attachment.path(style) #path of the file
# you can get this values from your aws a/c , most probably by going int
# https://console.aws.amazon.com/iam/home?#security_credential
signer = AwsCfSigner.new(cloudfront_pem_key_path, cloudfront_key_paid_id)
# this configuration may vary.
# visit https://github.com/dylanvaughn/aws_cf_signer
# and check all available settings/options
url = signer.sign(path, :ending => Time.now + 3600)
cloudfront_domain + url
end
With this you can access the url with something like this
cdn_url(cloudfront_pem_key_path: '/users/downloads/pri.pem' , cloudfront_key_paid_id: '33243424XXX')
Second way
# A simple function to return a signed, expiring url for Amazon Cloudfront.
# This will require openssl, digest/sha1, base64 and maybe other libraries.
module CloudFront
def get_signed_expiring_url(domain,path, expires_in, private_key_filename, key_pair_id)
# AWS works on UTC, so make sure you are not using local time
expires = (Time.now.getutc + expires_in).to_i.to_s
private_key = OpenSSL::PKey::RSA.new(File.read(private_key_filename))
# path should be your S3 path without a leading slash and without a file extension.
# e.g. files/private/52
policy = %Q[{"Statement":[{"Resource":"#{path}","Condition":{"DateLessThan":{"AWS:EpochTime":#{expires}}}}]}]
signature = Base64.strict_encode64(private_key.sign(OpenSSL::Digest::SHA1.new, policy))
# I'm not sure exactly why this is required, but it's in Amazon's perl script and seems necessary
# Different base64 implementations maybe?
signature.tr!("+=/", "-_~")
"#{domain}#{path}?Expires=#{expires}&Signature=#{signature}&Key-Pair-Id=#{key_pair_id}"
end
end
With this you can do something like
def cdn_url ( style='original',cloudfront_pem_key_path,key_pair_id)
path = attachment.path(style) #path of the file
# you can get this values from your aws a/c , most probably by going int
CloudFront.get_signed_expiring_url 'example.cloudfront.net', path, 45.seconds ,'/users/downloads/pri.pem', 'as12XXXXX')
end
Give a try, may be it will work. Be sure to properly set properly bucket access policy. check this out if you are seeing accessDenied error http://www.jppinto.com/2011/12/access-denied-to-file-amazon-s3-bucket/
Use the aws sdk gem.
See the API Documentation
Details about generating a presigned URL for an operation on the object
Provide the access-key-id and secret-access-key:-
S3 = AWS::S3.new(
:access_key_id => 'access_key_id',
:secret_access_key => 'secret_access_key')
In controller put these lines:--
bucket = S3.buckets['bucket_name']
s3_obj = bucket.objects["Path-to-file"]
return s3_obj.url_for(:read, :expires => 60*60).to_s
This link will expires in 1 Hour. After that the link will be not accessible.
I'm uploading the image using aws-s3 gem, it uploads perfectly and give me the url. Image is in PNG format but when I hit the public url in my browser, it starts downloading, but I don't want to download it, I want to see the image in browser.
Please tell me if you know which parameter in the following request should be added for this or what change is required to achieve this?
AWS::S3::S3Object.store(base_name,open(local_file),bucket,:content_type => mime_type,:access => :public_read,:authenticated => false)
s3_url = AWS::S3::S3Object.url_for(params[:name],bucket)[/[^?]+/]
Check :content_type => mime_type if mime_type is correct.