When using fog via paperclip with the following configuration:
config.paperclip_defaults = {
:storage => :fog,
:fog_credentials => {
:provider => 'AWS',
:aws_access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:aws_secret_access_key => ENV['AWS_SECRET_ACCESS_KEY'],
:region => 'eu-central-1'
},
:fog_directory => ENV['FOG_DIRECTORY']
}
Access to S3 fails with the following error:
Excon::Errors::Forbidden: Expected(200) <=> Actual(403 Forbidden)
SignatureDoesNotMatch - The request signature we calculated does not match the signature you provided. Check your key and signing method.
Logging directly with the awscli tools using the same credentials and setting the same region works. I double checked the keys. Also, aws s3api get-bucket-location --bucket mybucket returns eu-central-1.
Update
I got it working with the aws-sdk gem instead of fog, which is what paperclip recommends in their readme. But I think fog should work too so I'm not marking this as resolved.
Related
I'm working on a simple project using Paperclip to upload images. Everything has been working just fine until I attempted to integrate S3 with Paperclip. Upon 'uploading' a user's image I get a NoMethodError (undefined method 'match' for nil:NilClass): error. This only happens when I have my S3 configuration running - if I comment it out the file uploads perfectly.
My configuration:
development.rb:
....
....
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['AWS_BUCKET_ID'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
My Model:
class User < ActiveRecord::Base
has_attached_file :image_file, default_url: "/myapp/images/:style/missing.png"
validates_attachment_file_name :image_file, matches: [/png\Z/, /jpeg\Z/, /tiff\Z/, /bmp\Z/, /jpg\Z/]
entire error output from console:
NoMethodError (undefined method `match' for nil:NilClass):
app/controllers/images_controller.rb:33:in `block in create'
app/controllers/images_controller.rb:32:in `create'
Things I tried:
I added the AWS keys and bucket name directly into the code instead
of as an environmental variable.
As mentioned above, I commented out the AWS configuration in my environment file and it seemed to work perfectly.
It's probably worth mentioning that I installed the fog gem earlier to start configuring for Google Cloud Storage, but decided to stick with S3 instead. I used gem uninstall fog to remove the gem but it appears some dependencies stayed behind.
Add :s3_region to your config map:
E.g.
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['AWS_BUCKET_ID'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
},
:s3_region => ENV['AWS_REGION']
}
Unfortunately this is a v5.0.0 feature (in Beta). And therefore there's nothing about it in the docs. However there is a comment about it in the actual 5.0 code.
Also good to note that the s3_permissions key in papertrail expects a string value now (it used to accept symbols).
:s3_permissions => 'public-read'
You need to specify the region in you s3_credentials, in a way such as region: ENV["AWS_REGION"].
As for cleaning up unused gems you can run bundle clean.
I have a rails 4 app deployed on heroku. I'm using Amazon s3 bucket to host images (using paperclip gem), but when I try to upload an image to my articles controller i get this error:
Permission denied # dir_s_mkdir - /articles
I have gone through the steps listed here. As i understand this could be a folder permission issue, but not really to sure how to tackle the issue.
Try putting the following code in your /config/environments/production.rb file and update ENV secret key names accordingly. The host name is dependent on your region, us-west-1, eu-west-1, etc.
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY'],
:s3_host_name => 's3-us-west-1.amazonaws.com'
}
}
I am using latest aws-sdk for Ruby. My rake file contains following line to connect to S3 bucket:
require 'aws-sdk'
...
AWS.config(
:access_key_id => 'xxxxx',
:secret_access_key => 'xxxxx'
)
s3 = AWS::S3.new
object = s3.buckets['bucket_in_ireland'].objects['data.csv']
When accessing an AWS S3 bucket in region US that works perfectly fine. But getting following error while accessing AWS S3 bucket in region Ireland:
AWS::S3::Errorrs::PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
From Google I could find two solutions:
Put AWS::S3::DEFAULT_HOST.replace "s3-eu-west-1.amazonaws.com" at the bottom of environment.rb
Set :s3_host_name => 's3-eu-west-1.amazonaws.com' for paperclip default settings
When I placed the line mentioned in (1) at the bottom of environment.rb, the error message changed to following:
NameError: uninitialized constant AWS::S3::DEFAULT_HOST
Then I tried (2) by updating following code in production.rb but that didn't help.
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY'],
:s3_host_name => 's3-eu-west-1.amazonaws.com'
}
}
Note: Also tried to apply this fix [https://github.com/aws/aws-sdk-ruby/issues/510] but nothing really changed after modifications in environment.rb, but nothing work.
Any help please...
Setting s3_endpoint to AWS config solved the issue for me.
AWS.config(
:access_key_id => 'xxxx',
:secret_access_key => 'xxxx',
:s3_endpoint => 's3-eu-west-1.amazonaws.com'
)
I'm writing a Rails 3 app that uses Paperclip to transcode a video file attachment into a bunch of other formats, and then to store the resulting files. It all works fine for local storage, but I am trying to make it work using Paperclip's Fog support to store files in a bucket on our own Ceph cluster. However, I can't seem to find the right configuration options to make Fog talk to my Ceph server.
Here is a snippet from my Rails class:
has_attached_file :videofile,
:storage => :fog,
:fog_credentials => { :aws_access_key_id => 'xxx', :aws_secret_access_key => 'xxx', :provider => 'AWS'},
:fog_public => true,
:url => ":id/:filename",
:fog_directory => 'replay',
:fog_host => 'my-hostname',
Writes using this setup fail because Paperclip attempts to save to Amazon S3 rather than the host I've provided. I have a non-Rails / non-Paperclip toy script working just fine:
conn = Fog::Storage.new({
:aws_access_key_id => 'xxx',
:aws_secret_access_key => 'xxx',
:host => 'my-hostname',
:path_style => true,
:provider => "AWS",
})
This correctly connects to my local Ceph server. So I suspect there is something I'm not configuring in Paperclip properly - but what?
Here's the relevant hunk from fog.rb that I think is causing the connection to only go to AWS:
def host_name_for_directory
if #options[:fog_directory].to_s =~ Fog::AWS_BUCKET_SUBDOMAIN_RESTRICTON_REGEX
"#{#options[:fog_directory]}.s3.amazonaws.com"
else
"s3.amazonaws.com/#{#options[:fog_directory]}"
end
end
the error was just from an improperly configured Ceph cluster. For anyone who finds this thread, as long as you:
Have your wildcard DNS set up properly for your Ceph frontend;
Ceph configured to recognize as such
Pass in :host in :fog_credentials, which would be the FQDN of the Ceph frontend
:fog_host, which apparently needs to be the URL for your bucket, e.g. https://bucket.ceph-server.foobar.com.
Paperclip will work out of the box. I don't think that it is documented anywhere that you can use :host but it works.
I'm pretty new to ROR. I've recently deployed an app on heroku and have tried to add an attachment function to the app via paperclip.
I've followed all the steps in adding aws-s3 to my app. Here was my initial code:
user.rb (model)
has_attached_file :avatar,
:styles => {:small => "70x70>"},
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:path => ":attachment/:id/:style/:basename.:extension"
validates_attachment_size :avatar, :less_than => 1.megabytes
validates_attachment_content_type :avatar, :content_type => ['image/jpeg', 'image/png']
s3.yml (file is located in config folder) note: all of these buckets exist on my aws-s3
development:
bucket: my_avatar-dev
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
test:
bucket: myapp_avatar-test
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
production:
bucket: myapp_avatar-pro
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
gemfile
gem 'aws-s3'
When running this configuration, I would get a error page 500 error when loading my app. Running Heroku logs showed the following error: AWS::S3::MissingAccessKey (You did not provide both required access keys.
So I followed some advice and defined the key and secret_key as environment variables to heroku, using the following line of code:
heroku config:add S3_KEY=amazonaccesskey S3_SECRET=amazon_secret_key
I then added an initializer to test environments and launch via key or .yml file depending on environment, code is as follows:
initializers/s3.rb
if Rails.env == "production"
# set credentials from ENV hash
S3_CREDENTIALS = { :access_key_id => ENV['S3_KEY'], :secret_access_key => ENV['S3_SECRET'], :bucket => "myapp_avatar-pro"}
else
# get credentials from YML file
S3_CREDENTIALS = Rails.root.join("config/s3.yml")
end
user.rb model was then update to the following:
has_attached_file :avatar, :storage => :s3, :s3_credentials => S3_CREDENTIALS
I then deployed to heroku and tested the app, but I still keep getting the same error (page 500) and error code: AWS::S3::MissingAccessKey (You did not provide both required access keys.
How is this possible if I have defined the variables in heroku? Is there something I am missing? Is it possible it's something with the gem? Also, I'm using HAML for styling... not sure that matters at all, but just in case it does. I'm quite lost, so any help would be greatly appreciated. Thank you so much!
Having just worked through the same problem and trawling a number of similar posts. I found that any of the possible configurations in the above answer i.e. declaring all of the hashes in the model, using the .yml or using the initializer all work fine from my dev and on heroku as long as the S3 bucket is of US Standard type> The choice is just about how DRY you want to be.
When I originally set S3 up, I used a European bucket. This gave me the spurious error message:
AWS::S3::MissingAccessKey (You did not provide both required access keys.
I note from the AWS site : http://docs.amazonwebservices.com/general/latest/gr/index.html?rande.html
that AWS uses a specific endpoint address for each region to reduce latency and am guessing (because I am a novice coder) that the US standard is either a default or coded into the AWS-S3 plugin. (Maybe someone can edit this up into a more complete answer?)
I solved this problem with this:
:s3_credentials => {
:access_key_id => 'mykey',
:secret_access_key => 'mykey'