aws-s3 error: AWS::S3::MissingAccessKey error, but keys have been defined? - ruby-on-rails

I'm pretty new to ROR. I've recently deployed an app on heroku and have tried to add an attachment function to the app via paperclip.
I've followed all the steps in adding aws-s3 to my app. Here was my initial code:
user.rb (model)
has_attached_file :avatar,
:styles => {:small => "70x70>"},
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:path => ":attachment/:id/:style/:basename.:extension"
validates_attachment_size :avatar, :less_than => 1.megabytes
validates_attachment_content_type :avatar, :content_type => ['image/jpeg', 'image/png']
s3.yml (file is located in config folder) note: all of these buckets exist on my aws-s3
development:
bucket: my_avatar-dev
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
test:
bucket: myapp_avatar-test
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
production:
bucket: myapp_avatar-pro
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
gemfile
gem 'aws-s3'
When running this configuration, I would get a error page 500 error when loading my app. Running Heroku logs showed the following error: AWS::S3::MissingAccessKey (You did not provide both required access keys.
So I followed some advice and defined the key and secret_key as environment variables to heroku, using the following line of code:
heroku config:add S3_KEY=amazonaccesskey S3_SECRET=amazon_secret_key
I then added an initializer to test environments and launch via key or .yml file depending on environment, code is as follows:
initializers/s3.rb
if Rails.env == "production"
# set credentials from ENV hash
S3_CREDENTIALS = { :access_key_id => ENV['S3_KEY'], :secret_access_key => ENV['S3_SECRET'], :bucket => "myapp_avatar-pro"}
else
# get credentials from YML file
S3_CREDENTIALS = Rails.root.join("config/s3.yml")
end
user.rb model was then update to the following:
has_attached_file :avatar, :storage => :s3, :s3_credentials => S3_CREDENTIALS
I then deployed to heroku and tested the app, but I still keep getting the same error (page 500) and error code: AWS::S3::MissingAccessKey (You did not provide both required access keys.
How is this possible if I have defined the variables in heroku? Is there something I am missing? Is it possible it's something with the gem? Also, I'm using HAML for styling... not sure that matters at all, but just in case it does. I'm quite lost, so any help would be greatly appreciated. Thank you so much!

Having just worked through the same problem and trawling a number of similar posts. I found that any of the possible configurations in the above answer i.e. declaring all of the hashes in the model, using the .yml or using the initializer all work fine from my dev and on heroku as long as the S3 bucket is of US Standard type> The choice is just about how DRY you want to be.
When I originally set S3 up, I used a European bucket. This gave me the spurious error message:
AWS::S3::MissingAccessKey (You did not provide both required access keys.
I note from the AWS site : http://docs.amazonwebservices.com/general/latest/gr/index.html?rande.html
that AWS uses a specific endpoint address for each region to reduce latency and am guessing (because I am a novice coder) that the US standard is either a default or coded into the AWS-S3 plugin. (Maybe someone can edit this up into a more complete answer?)

I solved this problem with this:
:s3_credentials => {
:access_key_id => 'mykey',
:secret_access_key => 'mykey'

Related

AWS::S3::Errors::NoSuchKey No Such Key

I have a Rails 4 app with Paperclip running on Heroku. I recently started encountering a problem that is preventing me from updating any models with attached images. For example, any time I make a change through my site's CMS system to my Company model, the changes are not saved. I've tried changing only the image, the image and various other attributes, and only non-image attributes. It's as if the #company.save is never called in my controller's update action.
I found this issue report and tried the various solutions suggested, but nothing worked.
Here is my Company model:
class Company < ActiveRecord::Base
has_attached_file :logo, :styles => { :medium => "300x300>", :thumb => "100x100>" }
validates_attachment_content_type :logo, :content_type => /\Aimage\/.*\Z/
end
In my logs, I get the following error:
[AWS S3 404 0.022711 0 retries] head_object(:bucket_name=>"myApp",:key=>"companies/profile_photos/000/000/118/original/2015-09-05_19.05.34.jpg") AWS::S3::Errors::NoSuchKey No Such Key
I verified that my S3 credentials are set using heroku config and that they are valid.
I'm not certain, but I suspect that this may have something to do with my recent addition of the following buildpacks:
=== myApp Buildpack URLs
1. https://github.com/bobbus/image-optim-buildpack.git
2. https://github.com/heroku/heroku-buildpack-ruby.git
I've not been able to reproduce the problem on localhost.
Update
I've since opened up an issue report on this problem.
Here is my config.paperclip_defaults block in production.rd
config.paperclip_defaults = {
:storage => :s3,
:s3_protocol => :https,
:s3_host_name => 's3.amazonaws.com',
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
Here is my aws.yml:
development:
access_key_id: <%= ENV['AWS_ACCESS_KEY_ID'] %>
secret_access_key: <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
production:
access_key_id: <%= ENV['AWS_ACCESS_KEY_ID'] %>
secret_access_key: <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
Documentation from the GitHub paperclip wiki
Assuming you are using dotenv-rails...
In development.rb and production.rb files:
config.paperclip_defaults = {
storage: :s3,
s3_host_name: ENV['S3_HOST_NAME'],
bucket: ENV['S3_BUCKET_NAME']
}
Make sure to set these variables in your Heroku environment:
$ heroku config:set S3_HOST_NAME=your-s3-hostname-here S3_BUCKET_NAME=your-production-bucket-name-here
Paperclip-with-Amazon-S3
Documentation from rubydoc.info
This states you should have a yaml file with these details specified or write a method on your model.
rubydoc.info Paperclip::Storage::S3
No Such Key is S3's way of saying "404 Not Found."
I don't believe it has any alternate meanings.
The request was authorized and syntactically valid, but there's no file in the bucket at the specified path.
You may want to inspect the contents of your bucket from the AWS console.

Paperclip/Rails not reading ENV variables with AWS information

I am trying to use paperclip with S3 AWS. I have set the secret keys and bucket name in the .bash_profile but Rials/Paperclip cannot seem to read them as i get the following error when i try to upload an image in development...
'missing required :bucket option'
If i replace the ENV['S3_BUCKET_NAME'] with the actual name, i then get the following...
'Missing Credentials. Unable to find AWS credentials'
Here is the set-up in the model...
has_attached_file :image, :styles => { :medium => "300x300>", :thumb => "100x100>" },
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
},
:s3_host_name => 's3-eu-west-1.amazonaws.com'
I have tried putting the credentials in the development config file in Rails but i have the same errors. Do i need to tell rails where to look for the ENV variables or am I doing something else wrong here? It seems others are having similar issues but i cannot find a solution.
Thanks for reading.
Just store the AWS bucket, key and key_id in a YAML file and in 'config/initializers', create a new file to read the YAML file contents and set the corresponding ENV variables. Don't forget to restart your rails server
Example
google_client = YAML.load_file("#{Rails.root.join('config/google_client.yml')}")
ENV['GOOGLE_APP_NAME'] = google_client['APP_NAME']
ENV['GOOGLE_CLIENT_ID'] = google_client['CLIENT_ID']
ENV['GOOGLE_CLIENT_SECRET'] = google_client['CLIENT_SECRET']
ENV['GOOGLE_CLIENT_SCOPE'] = google_client['CLIENT_SCOPE']
Typically, we do not push the aws info in Github commit but just ssh to the remote server and place the YAML file in the config folder in our Rails app. So only people who have access to your server can read your aws credentials and not all people even if they have Github access.
tell me if this solves your problem.

Paperclip uploads on Heroku using S3

I'm sorry to rehash an old gripe but I'm at my wits end and not sure where to go next. I am using Paperclip on Heroku and have S3 uploads configured. I was able to get things working in my local development environment but once it's running on Heroku I run into this error:
AWS::S3::Errors::PermanentRedirect (The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
I've googled this error and read through the Heroku documentation and I believe I have everything set up correctly. I initially thought that my problems stemmed from having my bucket in the s3-us-west-1.amazonaws.com region, but I'm not convinced anymore.
Here are the relevant parts of my Heroku config:
AWS_REGION: us-west-1
S3_BUCKET_NAME: my-super-awesomely-amazing-bucket
From my config/environments/production.rb file:
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
My paperclip.rb initialize file:
if Rails.env.production?
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = '/:class/:attachment/:id_partition/:style/:filename'
Paperclip::Attachment.default_options[:s3_host_name] = 's3-us-west-1.amazonaws.com'
end
And my paperclip config from the relevant model:
has_attached_file :document,
:styles => { },
:default_url => "/image_styles/:style/missing.png"
So...what am I doing wrong here? At this point I'm sure I've missed something obvious but I'm stumped on where to go from here. I feel like I've assiduously configured everything and yet that PermanentRedirect error keeps coming up.
Bucket
This might not be the direct solution, but we've found that you have to include the bucket option outside of your s3_credentials block:
#config/environments/production.rb
config.paperclip_defaults = {
storage: :s3,
s3_host_name: 's3-eu-west-1.amazonaws.com',
s3_credentials: {
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
},
bucket: ENV['S3_BUCKET_NAME']
}
This is working 100% for us on Heroku, but whether it wil work for you (as your bucket is in a different region) is a different matter
If you need more help, ask a comment and I'll gladly give you some ideas
Try this link http://adamthedeveloper.blogspot.in/2014/02/awss3permanentredirect-bucket-you-are.html . Previously this happened to Europe region buckets only and get resolved by setting AWS::S3::DEFAULT_HOST. Hope this solve your issue.

aws s3 paperclip and rails 3.0 error

First off thanks so much for taking the time to read my question. Thank you! It appears that i'm having trouble implementing aws s3 on my webapp. I have a ROR app deployed o heroku and I'd like to allow users to upload a profile picture to their profile.
I've installed imagemagick and paperclip to handle the attachments. Then someone informed me that heroku does not accept uploads and that I'll need to subscribe to aws-s3. That made sense. So I signed up for aws and added the following code to my project:
user.rb (model)
has_attached_file :avatar,
:styles => {:small => "70x70>"},
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:path => ":attachment/:id/:style/:basename.:extension"
validates_attachment_size :avatar, :less_than => 1.megabytes
validates_attachment_content_type :avatar, :content_type => ['image/jpeg', 'image/png']
s3.yml (file is located in config folder) note: all of these buckets exist on my aws-s3
development:
bucket: my_avatar-dev
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
test:
bucket: myapp_avatar-test
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
production:
bucket: myapp_avatar-pro
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
gemfile
gem 'aws-s3'
When I run the application in the development environment (localhost) everything functions properly... I checked AWS and my uploads appear there; however, when I deploy my app to heroku it breaks. To elaborate, the app loads the sign-in screen but as soon as a user signs in, the application breaks and redirects to the error 500 page: "We're sorry, but something went wrong. We've been notified about this issue and we'll take a look at it shortly."
When I hide the following code (and replace :path with some other value) and re-deploy... the app loads, but obviously it lacks the functionality it needs to redirect to aws-s3.
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:path => ":attachment/:id/:style/:basename.:extension"
As I mentioned, I'm pretty new to rails... so I'm not sure what I'm doing wrong. Was I supposed to link s3.yml somewhere else, a route or something? Maybe it's something that I need to do while deploying? I'd like to thank anyone who can help me, and I am thankful for your time!

AWS::S3::MissingAccessKey in Paperclip but I've defined both

I'm on Heroku, and this is a portfolio thing which I'm putting up on github for potential employers to look at, so obviously I don't want to stick my keys in a S3.yml file. I've exported S3_KEY and S3_SECRET to my environment both on Heroku and my machine and ruby can access them. But when I try and upload, it gives me the following error:
AWS::S3::MissingAccessKey in Portfolio itemsController#update
You did not provide both required access keys. Please provide the access_key_id and the secret_access_key.
The trace is irrelevant except for my controller line #, which works fine until I try and upload a file. Here's what I have:
class Asset < ActiveRecord::Base
attr_accessible :image, :image_file_name, :image_content_type, :image_file_size, :portfolio_item_id, :order
has_attached_file :image,
:styles => {
:thumb => "100x100#",
:small => "300x300",
:large => "600x600>"
},
:storage => :s3,
:s3_credentials => {
:access_key_id => ENV["S3_KEY"],
:secret_access_key => ENV["S3_SECRET"]
},
:bucket => "bucketybucket",
:path => "portfolio"
end
Anyone know what's going on here? How am I constructing this hash wrong?
Oh, and I've followed this thread, no dice: Paperclip and Amazon S3 Issue
same problem...
seems like that ENV const doesn't load before loading the module. solve by using file argument
like this
:s3_credentials => Rails.root.join('config/amazon_s3.yml')
and in amazon_s3.yml
access_key_id: 'your_key'
secret_access_key: 'your_sec_key'
bucket: 'somebucket'
furthermore, you can set environment variable by using heroku config:add command, which is describe in Heroku DevCenter
The problem is because the Enviroment variable in heroku is different that the enviroment variable in your system, so it may happen that the application works just in one enviroment

Resources