Rails ArgumentError (missing required :bucket option) fixes not working? - ruby-on-rails

My Rails app is hosted on Heroku but I need to get Paperclip working locally so I can upload photos in testing. I'm really close, but I'm stuck on the "missing required :bucket option" error.
I've tried moving around settings in my environment files as well as well as adding a config/application.yml file with my AWS credentials in it, but no dice.
Here's my config from development.rb (looks the same in production.rb):
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
I've tried moving :bucket attribute outside of :s3_credentials but it didn't change anything. Application.yml looks like this:
AWS_ACCESS_KEY_ID="xxxxxxxxxxxxxxxxx"
AWS_SECRET_ACCESS_KEY="xxxxxxxxxxxxxxxxxxxx"
S3_BUCKET_NAME="xxxxxx"
I also installed the dot-env gem and added an s3.env file to the index of my app which contains the same credentials as above, but even that hasn't worked. Nor did adding my Paperclip config inline with the has_attached_file in my model.
What do I do??

Alternatively, if someone runs into this problem and doesn't want to use a .env file, the following works.
Assuming your bucket is hosted in the US region, put a file called aws.yml under the config dir. The file should be in the following format:
development:
access_key_id: AWS_ACCESS_KEY_ID
secret_access_key: AWS_SECRET_KEY_ID
production:
access_key_id: AWS_ACCESS_KEY_ID
secret_access_key: AWS_SECRET_KEY_ID
In your model:
class Model < ActiveRecord::Base
has_attached_file :avatar, styles: {
thumb: '100x100>',
square: '200x200#',
medium: '300x300>'}, :storage => :s3,
:s3_credentials => "#{Rails.root}/config/aws.yml",
:bucket => "your_bucket_name"
validates_attachment_content_type :avatar, :content_type => /\Aimage\/.*\Z/
Replace your_bucket_name with the name of your actual bucket.
Gemfile:
gem 'aws-sdk', '< 2.0'
Under Environments, production and development:
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
To put it in production on Heroku, you have to set the following using the terminal:
$ heroku config:set S3_BUCKET_NAME=your_bucket_name
$ heroku config:set AWS_ACCESS_KEY_ID=your_access_key_id
$ heroku config:set AWS_SECRET_ACCESS_KEY=your_secret_access_key

Your Application.yml file doesn't look like YAML. It should look like this:
AWS_ACCESS_KEY_ID: "xxxxxxxxxxxxxxxxx"
AWS_SECRET_ACCESS_KEY: "xxxxxxxxxxxxxxxxxxxx"
S3_BUCKET_NAME: "xxxxxx"
If you are loading the yml file like they advise here you'll want your yml file to look like this:
development:
AWS_ACCESS_KEY_ID: "xxxxxxxxxxxxxxxxx"
AWS_SECRET_ACCESS_KEY: "xxxxxxxxxxxxxxxxxxxx"
S3_BUCKET_NAME: "xxxxxx"
You need a way to set the environment variables for your app in your development environment. Using Application.yml or .env file should work, but you probably don't need both.

Related

`fetch': key not found: "S3_BUCKET_NAME" (KeyError) with paperclip/aws s3/rails

I am getting a 'fetch': key not found: "S3_BUCKET_NAME" (KeyError) error in rails (4.2.3) using 'aws-sdk', '~> 2.3' and "paperclip", "~> 5.0.0"
I have set the Keys in my environment via terminal and running heroku config shows them listed.
In both my config/environments/development.rb as well as in my config production.rb I have included:
config.paperclip_defaults = {
storage: :s3,
s3_credentials: {
bucket: ENV.fetch('S3_BUCKET_NAME'),
access_key_id: ENV.fetch('AWS_ACCESS_KEY_ID'),
secret_access_key: ENV.fetch('AWS_SECRET_ACCESS_KEY'),
s3_region: ENV.fetch('AWS_REGION'),
}
}
I have also included the above code in my user.rb model, but for the sake of reference it looks like this in the model:
has_attached_file :avatar,
styles: { medium: "300x300#", thumb: "100x100#" },
:convert_options => {
:thumb => "-quality 75 -strip" },
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY'],
:region => ENV['AWS_REGION']
},
:path => ":filename.:extension",
# :path => ":rails_root/public/system/:attachment/:id/:style/:filename",
:default_url => "default_img.png"
validates_attachment_content_type :avatar, content_type: /\Aimage\/.*\Z/
I have also included the env vars in my secrets.yml:
development:
secret_key_base: 817c07d41b8524495628fbe91fb1f0535ade65aa96a3fee379a8d16c29cc1f7b167f537442e547422ab17ee9700028a95896eb1c0717de06dfe7895d15ddb5ce
secret_key: sk_test_xxx
publishable_key: pk_test_xxx
access_key_id: xxx
secret_access_key: xxx
s3_bucket_name: 'bucket-name'
test:
secret_key_base: a38e71848a4d9bc63fa8dce4522add10a4931b10e6786f0cab6a9eb1643e271b992f52fa6eff672b0d03687003099c0632477dd26b246ac4e637c52c69ec4ab0
# Do not keep production secrets in the repository,
# instead read values from the environment.
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
secret_key: <%= ENV["SECRET_KEY"] %>
publishable_key: <%= ENV["PUBLISHABLE_KEY"] %>
access_key_id: <%= ENV['AWS_ACCESS_KEY_ID'] %>
secret_access_key: <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
s3_bucket_name: <%= ENV["S3_BUCKET_NAME"]%>
although that may not have been required. Googling around and going through the few other posts on SO related to this error have given me little to go on - does anybody have any ideas on what the issue may be?
'fetch': key not found: "S3_BUCKET_NAME" means that the environment variables S3_BUCKET_NAME does not have a value.
In your case you are using Heroku. Follow the instructions in the link below.
For Heroku: https://devcenter.heroku.com/articles/config-vars.
If you are using ENV['variables'] you need to have them set in every environment. test, production and development. If you are developing on a PC, MAC, or Linux you need to make sure the environment variables are set.
For linux:
https://www.digitalocean.com/community/tutorials/how-to-read-and-set-environmental-and-shell-variables-on-a-linux-vps
For OSX:
I do it the same was as linux, I just set them in my ~/.zshrc by adding a line like this. If you aren't using zshell then add a line to your ~/.bashrc or ~/.bash_profile.
export ENV_VARIABLE_NAME="value"
For Windows:
I don't know how, but I am sure google does.

AWS::S3::Errors::NoSuchKey No Such Key

I have a Rails 4 app with Paperclip running on Heroku. I recently started encountering a problem that is preventing me from updating any models with attached images. For example, any time I make a change through my site's CMS system to my Company model, the changes are not saved. I've tried changing only the image, the image and various other attributes, and only non-image attributes. It's as if the #company.save is never called in my controller's update action.
I found this issue report and tried the various solutions suggested, but nothing worked.
Here is my Company model:
class Company < ActiveRecord::Base
has_attached_file :logo, :styles => { :medium => "300x300>", :thumb => "100x100>" }
validates_attachment_content_type :logo, :content_type => /\Aimage\/.*\Z/
end
In my logs, I get the following error:
[AWS S3 404 0.022711 0 retries] head_object(:bucket_name=>"myApp",:key=>"companies/profile_photos/000/000/118/original/2015-09-05_19.05.34.jpg") AWS::S3::Errors::NoSuchKey No Such Key
I verified that my S3 credentials are set using heroku config and that they are valid.
I'm not certain, but I suspect that this may have something to do with my recent addition of the following buildpacks:
=== myApp Buildpack URLs
1. https://github.com/bobbus/image-optim-buildpack.git
2. https://github.com/heroku/heroku-buildpack-ruby.git
I've not been able to reproduce the problem on localhost.
Update
I've since opened up an issue report on this problem.
Here is my config.paperclip_defaults block in production.rd
config.paperclip_defaults = {
:storage => :s3,
:s3_protocol => :https,
:s3_host_name => 's3.amazonaws.com',
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
Here is my aws.yml:
development:
access_key_id: <%= ENV['AWS_ACCESS_KEY_ID'] %>
secret_access_key: <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
production:
access_key_id: <%= ENV['AWS_ACCESS_KEY_ID'] %>
secret_access_key: <%= ENV['AWS_SECRET_ACCESS_KEY'] %>
Documentation from the GitHub paperclip wiki
Assuming you are using dotenv-rails...
In development.rb and production.rb files:
config.paperclip_defaults = {
storage: :s3,
s3_host_name: ENV['S3_HOST_NAME'],
bucket: ENV['S3_BUCKET_NAME']
}
Make sure to set these variables in your Heroku environment:
$ heroku config:set S3_HOST_NAME=your-s3-hostname-here S3_BUCKET_NAME=your-production-bucket-name-here
Paperclip-with-Amazon-S3
Documentation from rubydoc.info
This states you should have a yaml file with these details specified or write a method on your model.
rubydoc.info Paperclip::Storage::S3
No Such Key is S3's way of saying "404 Not Found."
I don't believe it has any alternate meanings.
The request was authorized and syntactically valid, but there's no file in the bucket at the specified path.
You may want to inspect the contents of your bucket from the AWS console.

How to set up Amazon S3, paperclip, and ENV variables

I have tried many different ways to set up S3 using ENV variables for image uploads and cannot get it to work. I know my keys and bucket name work, because when I put them straight into the code, my images upload correctly. However, when I try to switch to ENV variables, things do not work.
I used the figaro gem, which created application.yml. In that file, I have:
S3_BUCKET_NAME "xxxxx"
AWS_ACCESS_KEY_ID: "AAAAAAAAA"
AWS_SECRET_ACCESS_KEY: "BBBbbbBBBB"
Not sure if there should be any quotation marks there or not, but right now, I have them in. I have tried without, also.
In my model (listing.rb), I have:
has_attached_file :image,
:styles => { :medium => "200x" , :thumb => "100x100" },
:default_url => "default.png",
:storage => :s3,
:s3_credentials => Proc.new{|a| a.instance.s3_credentials }
def s3_credentials
{:bucket => ENV["S3_BUCKET_NAME"], :access_key_id => ENV["AWS_ACCESS_KEY_ID"],
:secret_access_key => ENV["AWS_SECRET_ACCESS_KEY"]
end
Like I said, when I hard code the values into def s3_credentials, everything works fine. It's just when I try to swap out ENV variables that things fall apart.
In paperclip.rb, I have:
Paperclip::Attachment.default_options[:s3_host_name] = 's3-us-west-2.amazonaws.com'
I also have this code in both production.rb and development.rb:
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
Here is the error message I get when uploading a new image: "The request signature we calculated does not match the signature you provided. Check your key and signing method." On line: "if #listing.save". The ones that were uploaded w/ the credentials hard-coded are still able to be seen in my app.
I'm fairly new to rails and have looked here and other places, including the S3 and paperclip docs, and cannot find a solution that will work. Please let me know if you need to see any other code. I plan on deploying to heroku, if that matters, and saw that figaro is supposed to play nicely with heroku. THANK YOU.
EDIT/UPDATE: For anyone else reading this in the future, Sachin's answer below worked. However, there was a '+' in one of my key IDs. When trying to added the ENV variable via command line, all the characters after the '+' (and including it) were cut off. Simply wrap them in "", and you should be good to go.
Also, I dropped use of the figaro gem, and set up an aws.rb initializer file (per Amazon's instructions). Here are the contents of that file:
AWS.config(
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
)
S3_BUCKET = AWS::S3.new.buckets[ENV['S3_BUCKET']]
And I don't know if this made any difference, but my development.rb and production.rb files now have the following as paperclip defaults:
config.paperclip_defaults = {
:storage => :s3,
:bucket => "your_real_bucket_name_here_in_quotes",
:s3_credentials => {
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
Also note the switch to referring to the ENV bucket name as S3_BUCKET vs. S3_BUCKET_NAME.
And the code in my model (listing.rb) is now this:
has_attached_file :image, :styles => { :medium => "200x", :thumb "100x100"}, :default_url => "default.png", :storage => :s3, :bucket => "your_real_bucket_name_here_in_quotes"
validates_attachment_content_type :image, :content_type => ["image/jpg", "image/jpeg", "image/png", "image/gif"]
You can do one thing:
You can set this configuration in your development.rb or production.rb
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
And If you want to set this environment variables into local then use this:
sudo nano ~/.profile
Then add your variables over here
export S3_BUCKET_NAME="your bucket name"
export AWS_ACCESS_KEY_ID="your access key id"
export AWS_SECRET_ACCESS_KEY="your secret access key"
And then reload your ~/.profile with . ~/.profile
Check added variable with echo $S3_BUCKET_NAME
And for Heroku
You can set your variable like:
heroku config:set S3_BUCKET_NAME="your bucket name"
heroku config:set AWS_ACCESS_KEY_ID="your access key id"
heroku config:set AWS_SECRET_ACCESS_KEY="your secret access key"
Check that variables added or not in heroku with heroku config
For more detail you can refer form here.
Let me know if you need me more..

Paperclip/heroku/S3 Configuration is setup as best as I can tell but getting error on upload

Here's the error I get when I try to upload an image through the live site:
ArgumentError (missing required :bucket option):
app/controllers/editions_controller.rb:53:in `block in create'
app/controllers/editions_controller.rb:52:in `create'
I have the latest versions of aws-sdk and paperclip
I've got the following in production.rb
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['bucket_name'],
:access_key_id => ENV['key'],
:secret_access_key => ENV['key2']
}
I've got my AWS_BUCKET set on my heroku app:
AWS_BUCKET: bucket_name
S3_BUCKET_NAME: bucket_name
AWS_ACCESS_KEY_ID: key
AWS_SECRET_ACCESS_KEY: key2
Can anybody suggest why this isn't working? Thanks.
You've got:
:s3_credentials => {
:bucket => ENV['bucket_name']
}
I've always been told that you should capitalize the constants like so:
:s3_credentials => {
:bucket => ENV['BUCKET_NAME']
}
Additionally, you have not actually set an ENV variable called BUCKET_NAME on heroku, you have the following:
AWS_BUCKET: bucket_name
But if your env var is called BUCKET_NAME, then you need the following instead:
BUCKET_NAME: bucket_name
So from the commane line you'd want to run something like heroku config:set BUCKET_NAME=bucket_name
And finally, you've changed the names of your env vars from those that are suggested in the paperclip readme. You've gone with ENV['BUCKET_NAME'] when everyone else uses ENV['AWS_BUCKET']. I dunno if that will effect anything, but try using the process outlined here by heroku.
Try those changes and let me know how they work out. Good luck.

aws-s3 error: AWS::S3::MissingAccessKey error, but keys have been defined?

I'm pretty new to ROR. I've recently deployed an app on heroku and have tried to add an attachment function to the app via paperclip.
I've followed all the steps in adding aws-s3 to my app. Here was my initial code:
user.rb (model)
has_attached_file :avatar,
:styles => {:small => "70x70>"},
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:path => ":attachment/:id/:style/:basename.:extension"
validates_attachment_size :avatar, :less_than => 1.megabytes
validates_attachment_content_type :avatar, :content_type => ['image/jpeg', 'image/png']
s3.yml (file is located in config folder) note: all of these buckets exist on my aws-s3
development:
bucket: my_avatar-dev
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
test:
bucket: myapp_avatar-test
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
production:
bucket: myapp_avatar-pro
access_key_id: amazonaccesskey
secret_access_key: amazon_secret_access_key
gemfile
gem 'aws-s3'
When running this configuration, I would get a error page 500 error when loading my app. Running Heroku logs showed the following error: AWS::S3::MissingAccessKey (You did not provide both required access keys.
So I followed some advice and defined the key and secret_key as environment variables to heroku, using the following line of code:
heroku config:add S3_KEY=amazonaccesskey S3_SECRET=amazon_secret_key
I then added an initializer to test environments and launch via key or .yml file depending on environment, code is as follows:
initializers/s3.rb
if Rails.env == "production"
# set credentials from ENV hash
S3_CREDENTIALS = { :access_key_id => ENV['S3_KEY'], :secret_access_key => ENV['S3_SECRET'], :bucket => "myapp_avatar-pro"}
else
# get credentials from YML file
S3_CREDENTIALS = Rails.root.join("config/s3.yml")
end
user.rb model was then update to the following:
has_attached_file :avatar, :storage => :s3, :s3_credentials => S3_CREDENTIALS
I then deployed to heroku and tested the app, but I still keep getting the same error (page 500) and error code: AWS::S3::MissingAccessKey (You did not provide both required access keys.
How is this possible if I have defined the variables in heroku? Is there something I am missing? Is it possible it's something with the gem? Also, I'm using HAML for styling... not sure that matters at all, but just in case it does. I'm quite lost, so any help would be greatly appreciated. Thank you so much!
Having just worked through the same problem and trawling a number of similar posts. I found that any of the possible configurations in the above answer i.e. declaring all of the hashes in the model, using the .yml or using the initializer all work fine from my dev and on heroku as long as the S3 bucket is of US Standard type> The choice is just about how DRY you want to be.
When I originally set S3 up, I used a European bucket. This gave me the spurious error message:
AWS::S3::MissingAccessKey (You did not provide both required access keys.
I note from the AWS site : http://docs.amazonwebservices.com/general/latest/gr/index.html?rande.html
that AWS uses a specific endpoint address for each region to reduce latency and am guessing (because I am a novice coder) that the US standard is either a default or coded into the AWS-S3 plugin. (Maybe someone can edit this up into a more complete answer?)
I solved this problem with this:
:s3_credentials => {
:access_key_id => 'mykey',
:secret_access_key => 'mykey'

Resources