I started to integrate CloudFront into my exciting Rails App, everything with CloudFront is working fine, except that the old uploaded images can't be accessed.
CarrierWave.configure do |config|
config.fog_credentials = {
:provider => 'AWS',
:aws_access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:aws_secret_access_key => ENV['AWS_SECRET_ACCESS_KEY'],
}
config.asset_host = ENV['CLOUDFRONT_ENDPOINT']
config.fog_directory = 'oktob-editor'
config.fog_public = true
config.fog_attributes = {'Cache-Control'=>"max-age=#{365.day.to_i}"}
end
Example of old uploaded image
https://oktob-editor.s3.amazonaws.com/uploads/post/image/127/thumb_Ruby_on_Rails.svg.png
After I integrated CloudFront and set asset_host it becomes
http://ID.cloudfront.net/uploads/post/image/127/thumb_Ruby_on_Rails.svg.png
with
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>D368D2E641BBBB64</RequestId><HostId></HostId></Error>
So is there a way that enable old images to work properly with CloudFront
Seems like changing the Restrict Bucket Access to Yes makes it to work
Related
Hello I have included given code
def store_s3(file)
# We create a connection with amazon S3
AWS.config(access_key_id: ENV['S3_ACCESS_KEY'], secret_access_key: ENV['S3_SECRET'])
s3 = AWS::S3.new
bucket = s3.buckets[ENV['S3_BUCKET_LABELS']]
object = bucket.objects[File.basename(file)]
# the file is not the content of the file is the route
# file_data = File.open(file, 'rb')
object.write(file: file)
# save the file and return an url to download it
object.url_for(:read, response_content_type: 'text/csv')
end
this code is working correctly in my local data is stored in amazon but when I had deployed code in heroku server I had made variables on server too.
is there anything which I am missing here please let me know cause of issue.
I don't see region, in your example is S3_Hostname your region?
for myself, region was just like 'us-west-2'.
If you want to setup your s3 with carrierwave and gem fog you can do it like this on config/initializers/carrierwave.rb
CarrierWave.configure do |config|
config.fog_provider = 'fog/aws'
config.fog_directory = 'name for s3 directory'
config.fog_credentials = {
:provider => 'AWS',
:aws_access_key_id => 'your access key',
:aws_secret_access_key => 'your secret key',
:region => 'your region ex: eu-west-2'
}
end
I have inherited a project that uses Paperclip for image processing, which also uploads to a AWS bucket, normally I use Carrierwave and choose to save files locally when in Test or Development environments
CarrierWave.configure do |config|
if Rails.env.test?
config.storage = :file
config.enable_processing = false
else
config.fog_credentials = {
:provider => 'AWS',
:aws_access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:aws_secret_access_key => ENV['AWS_SECRET_ACCESS_KEY'],
:region => 'eu-west-1'
}
config.fog_directory = ENV['AWS_BUCKET']
config.fog_public = true
config.fog_attributes = {'Cache-Control'=>'max-age=315576000'}
end
end
How can I achieve the same thing with paperclip? I have read that you can Define Defaults in a initializer file.
But I am a bit unsure on what options to pass.
You can create an initializer like this
# config/initializers/paperclip.rb
if Rails.env.development? || Rails.env.test?
Paperclip::Attachment.default_options[:storage] = 'filesystem'
else
Paperclip::Attachment.default_options[:storage] = 's3'
Paperclip::Attachment.default_options[:s3_credentials] = {
bucket: ENV['AWS_BUCKET'],
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
}
# other config...
end
For more options about S3, see also http://www.rubydoc.info/gems/paperclip/Paperclip/Storage/S3
Just add those options to the Paperclip::Attachment.default_options hash :)
Paperclip can have different storage for each field, so look for how s3 storage is selected.
Probably it's has_attached_file :foo, storage: :s3, ..., to save locally storage should be :filesystem
I'm trying to set up the Amazon Simple Storage Service for use with rails. I'm getting this error message:
The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.
The problem is that I chose the Frankfurt S3 region, and there only the V4 scheme is supported.
It's the same error message as in this post, which directs you to the solution
here, with instructions how to "set the :s3_signature_version parameter to :v4 when constructing the client". The command is:
s3 = AWS::S3::Client.new(:s3_signature_version => :v4)
My question is, how do I do this? Where do I put this code?
EDIT:
I tried putting :s3_signature_version => :v4 in carrier_wave.rb as follows, but during the upload to heroku it said [fog][WARNING] Unrecognized arguments: s3_signature_version, and it didn't make any difference, I still get the error.
config/initializers/carrier_wave.rb:
if Rails.env.production?
CarrierWave.configure do |config|
config.fog_credentials = {
# Configuration for Amazon S3
:provider => 'AWS',
:aws_access_key_id => ENV['S3_ACCESS_KEY'],
:aws_secret_access_key => ENV['S3_SECRET_KEY'],
:s3_signature_version => :v4
}
config.fog_directory = ENV['S3_BUCKET']
end
end
EDIT:
I've created a new bucket using the Northern California region, for which this isn't supposed to be a problem, but I'm still getting exactly the same error message.
EDIT:
This doesn't make any difference either:
if Rails.env.production?
CarrierWave.configure do |config|
config.fog_credentials = {
# Configuration for Amazon S3
:provider => 'AWS',
:aws_access_key_id => ENV['S3_ACCESS_KEY'],
:aws_secret_access_key => ENV['S3_SECRET_KEY']
}
config.fog_directory = ENV['S3_BUCKET']
config.fog_attributes = {:s3_signature_version => :v4}
end
end
I had the problem, that Spree v2.3 was fixated to aws-sdk v1.27.0. But the parameter s3_signature_version was introduced in v1.31.0 (and set per default for China).
So in my case the following configuration for Frankfurt has totally been ignored:
AWS.config(
region: 'eu-central-1',
s3_signature_version: :v4
)
I found this old question from the other direction, trying to take the advice in https://github.com/fog/fog/issues/3450 and set signature to version 2 (to test a hypothesis). Delving into the source a bit, it turns out the magic phrase is :aws_signature_version => 4, so like this:
config.fog_credentials = {
# Configuration for Amazon S3
:provider => 'AWS',
:aws_access_key_id => ENV['S3_ACCESS_KEY'],
:aws_secret_access_key => ENV['S3_SECRET_KEY'],
:aws_signature_version => 4
}
I had this same problem and could not find any guidance on where to implement the s3_signature_version: :v4 command.
In the end, I basically deleted the existing bucket in Frankfurt and created on in the Standard US zone and it works (after updating the permissions policy attached to the user accessing the bucket to reflect that the bucket has changed).
I would love to have the bucket in Frankfurt but I don't have another 16 hours to spend going round in circles with this issue so if anybody is able to add a bit more direction on how to incorporate the s3_signature_version: :v4 line, that would be great.
For other users following Michael Hartl's Rails Tutorial:
you (might*) need at least v 1.26 of the 'fog' gem. Modify your Gemfile accordingly, and don't forget to '$ bundle install'.
*the reason is that some S3 buckets require authorization signature version 4. In the future probably all of them will, and at least Frankfurt (zone eu-central-1) requires v4 authorization.
This has been supported since fog v1.26:
https://github.com/fog/fog/blob/v1.26.0/lib/fog/aws/storage.rb
I'm working on a small rails site which allows some users to upload images and others to see them. I started using CarrierWave with S3 as the storage medium and everything worked great but then I wanted to experiment with using CouldFront. I first added a distribution to my S3 bucket and then changed the CarrierWave configuration I was using to this:
CarrierWave.configure do |config|
config.storage = :fog
config.fog_credentials = {
:provider => 'AWS', # required
:aws_access_key_id => ENV['S3_ACCESS_KEY_ID'], # required
:aws_secret_access_key => ENV['S3_SECRET_ACCESS_KEY'], # required
:region => 'eu-west-1',
}
config.asset_host = 'http://static.my-domain.com/some-folder'
config.fog_public = true # optional, defaults to true
config.fog_attributes = {'Cache-Control'=>'max-age=315576000'} # optional, defaults to {}
end
I should mention that http://static.my-domain.com is a CNAME entry pointing to a CloudFront endpoint (some-id.cloudfront.net). The result is that the pictures are shown correctly, URLs look like this: http://static.my-domain.com/some-folder/uploads/gallery_image/attachment/161/large_image.jpg but whenever I try to upload a photo or for that matter get the size of the uploaded attachment I get the following exception:
Excon::Errors::MovedPermanently: Expected(200) <=> Actual(301 Moved Permanently)
response => #<Excon::Response:0x007f61fc3d1548 #data={:body=>"",
:headers=>{"x-amz-request-id"=>"some-id", "x-amz-id-2"=>"some-id",
"Content-Type"=>"application/xml", "Transfer-Encoding"=>"chunked",
"Date"=>"Mon, 31 Mar 2014 21:16:45 GMT", "Connection"=>"close", "Server"=>"AmazonS3"},
:status=>301, :remote_ip=>"some-ip"}, #body="", #headers={"x-amz-request-id"=>"some-id",
"x-amz-id-2"=>"some-id", "Content-Type"=>"application/xml",
"Transfer-Encoding"=>"chunked", "Date"=>"Mon, 31 Mar 2014 21:16:45 GMT",
"Connection"=>"close", "Server"=>"AmazonS3"}, #status=301, #remote_ip="some-ip"
Just to add some more info, I tried the following:
removing the region entry
using the CloudFront URL directly instead of the CNAME
specifying the Amazon endpoint (https://s3-eu-west1.amazonaws.com)
but all of them had no effect.
Is there something I'm missing or is it that CarrierWave does not support this at this time?
The answer to the question is YES. The reason why it didn't work with my configuration is that I was missing the fog_directory entry. When I added my asset_host, I removed fog_directory since the CDN urls being generated where malformed. I later found out that this was due to having fog_public set to false. After getting the proper CDN urls, I forgot to add fog_directory back since I could see my images and thought everything was fine. Anyway the correct configuration is:
CarrierWave.configure do |config|
config.storage = :fog
config.fog_credentials = {
:provider => 'AWS', # required
:aws_access_key_id => ENV['S3_ACCESS_KEY_ID'], # required
:aws_secret_access_key => ENV['S3_SECRET_ACCESS_KEY'], # required
:region => 'eu-west-1'
}
config.fog_directory = '-bucket-name-/-some-folder-'
config.asset_host = 'https://static.my-domain.com/-some-folder-'
config.fog_public = true # optional, defaults to true
config.fog_attributes = {'Cache-Control'=>'max-age=315576000'} # optional, defaults to {}
end
Try setting :asset_host in your Uploader like so:
class ScreenshotUploader < CarrierWave::Uploader::Base
storage :fog
# Configure uploads to be stored in a public Cloud Files container
def fog_directory
'my_public_container'
end
# Configure uploads to be delivered over Rackspace CDN
def asset_host
"c000000.cdn.rackspacecloud.com"
end
end
Inspired from https://github.com/carrierwaveuploader/carrierwave/wiki/How-to%3A-Store-private-public-uploads-in-different-Cloud-Files-Containers-with-Fog
I have installed carrierwave and fog, have successfully uploaded the images and viewed them the first time, but now it does not show the images anymore.
Here is my config file app/config/initializers/carrierwave.rb
CarrierWave.configure do |config|
config.fog_credentials = {
:provider => 'AWS', # required
:aws_access_key_id => 'AKIAJKOHTE4WTXCCXAMA', # required
:aws_secret_access_key => 'some secret key here', # required
:region => 'eu-east-1', # optional, defaults to 'us-east-1'
:host => 'https://s3.amazonaws.com', # optional, defaults to nil
:endpoint => 'https://s3.amazonaws.com:8080' # optional, defaults to nil
}
config.fog_directory = 'createmysite.co.za' # required
config.fog_public = false # optional, defaults to true
#config.fog_attributes = {'Cache-Control'=>'max-age=315576000'} # optional, defaults to {}
end
This is what the url looks like of the image that is supposed to display
<img alt="Normal_selection_003" src="https://createmysite.co.za.s3.amazonaws.com/uploads/portfolio/image/3/normal_Selection_003.png?AWSAccessKeyId=AKIAJKOHTE4WTXCCXAMA&Signature=8PLq8WCkfrkthmfVGfXX9K6s5fc%3D&Expires=1354859553">
when I open the image url this is the output from amazon
https://createmysite.co.za.s3.amazonaws.com/uploads/portfolio/image/3/normal_Selection_003.png?AWSAccessKeyId=AKIAJKOHTE4WTXCCXAMA&Signature=8PLq8WCkfrkthmfVGfXX9K6s5fc%3D&Expires=1354859553
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>3F179B7CE417BC12</RequestId>
<HostId>
zgh46a+G7UDdpIHEEIT0C/rmijShOKAzhPSbLpEeVgUre1iDc9f7TSOwaJdQpR65
</HostId>
</Error>
Update
new config file (added fog url expiry) app/config/initializers/carrierwave.rb
CarrierWave.configure do |config|
config.fog_credentials = {
:provider => 'AWS', # required
:aws_access_key_id => 'AKIAJKOHTE4WTXCCXAMA', # required
:aws_secret_access_key => 'chuck norris', # required
}
config.fog_directory = 'createmysite.co.za' # required
config.fog_public = false # optional, defaults to true
config.fog_authenticated_url_expiration = 600 # (in seconds) => 10 minutes
end
works like a charm!
You've set config.fog_public to false and are using Amazon S3 for storage. URLs for private files through S3 are temporary (they're signed and have an expiry). Specifically, the URL posted in your question has an Expires=1354859553 parameter.
1354859553 is Fri, 07 Dec 2012 05:52:33 GMT, which is in the past from the current time, so the link has effectively expired, which is why you're getting the Access Denied error.
You can adjust the expiry out further (the default is 600 seconds) by setting
config.fog_authenticated_url_expiration = ... # some integer here
If you want non-expiring links either
set config.fog_public to true
have your application act as a middle man, serving the files up through send_file. Here is at least one question on SO covering this