When running my rails app in development mode from Nitrous.io, I cannot access my development bucket which I set up on AWS S3. The upload button opens my personal computer, from where I don't want to load files. (even when I try to load files from my computer, I get a long error message stating "The request signature we calculated does not match the signature you provided. Check your key and signing method"
I think I don't have AWS S3 configured properly.
Currently, I have one IAM user, which I've assigned to AdministratorAccess Also, I am using the proper AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in my application.yml file. In fog.rb I have it read from the enviroment.
I should add too that I currently enrolled in a web development apprenticeship program.
Sorry for not showing my files
Here's my application.yml with the sensitive data taken out:
SENDGRID_PASSWORD: alphanumberic
SENDGRID_USERNAME: -------#heroku.com
AWS_ACCESS_KEY_ID: alphanumeric
AWS_SECRET_ACCESS_KEY: alphanumeric
development:
AWS_BUCKET: vmanamino-bloccit-development
production:
AWS_BUCKET: vmanamino-bloccit-production
development:
secret_key_base: alphanumeric
test:
secret_key_base: alphanumeric
Here's my fog.rb file which reads the values from the environment
CarrierWave.configure do |config|
config.fog_credentials = {
provider: 'AWS',
aws_access_key_id: ENV['AWS_ACCESS_KEY_ID'],
aws_secret_access_key: ENV['AWS_ACCESS_KEY_ID'],
}
config.fog_directory = ENV['AWS_BUCKET']
config.fog_public = true
end
You're using the AWS_ACCESS_KEY_ID environment variable for both the access key and the secret access key whereas the latter should of course be using ENV['AWS_SECRET_ACCESS_KEY']
I didn't realize I needed to enclose the KEY ID and the SECRET KEY in quotes. Once I did that, I got it to work. I can upload images from my conputer to S3. Also, I didn't understand the assignment perfectly. I thought my app would upload images from S3. Now, the error raised earlier makes sense. I upload from my computer sending the image to S3.
Related
I have an app running on a ubuntu server. I have a production mode and a staging mode.
Problem is that actions being done on the production site relative to uploading and retrieving images from an S3 bucket are being done to the same bucket as my staging. When I have my configurations set up differently.
production.rb
config.s3_bucket = 'bucket-production'
config.s3_path = 'https://bucket-production.s3.us-east-2.amazonaws.com/'
staging.rb && development.rb
config.s3_bucket = 'bucket-staging'
config.s3_path = 'https://bucket-staging.s3.us-east-2.amazonaws.com/'
storage.yml
amazon:
service: S3
access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
region: us-east-2
bucket: bucket-staging
endpoint: http://bucket-staging.us-east-2.amazonaws.com
I'm thinking it could be something with storage.yml but I deleted this entire file and restarted the localhost server and it didn't change anything. Is storage.yml production only?
Also, my logs are logging to staging from production.
I would like to ask is prod server/staging server(Ubuntu) running in AWS.if yes, you must have IAM ROLE attached to the server which must be fine-grained to which env application should access which bucket of S3. Also storing access_key_id and secret access key id should not be used as best practice. IAM role can take care of that. Also, I would like to add if the server is in the private subnet you need to explore either NAT gateway or use VPC S3 Endpoint to access the bucket.
Also try printing logs of S3 connection in prod mode how it is acquiring cred to access bucket. You might be granting access using some ENV variables or IAM role. Best way to see is use
printenv
before and after of S3 connection to see the variables and which bucket access is provided.
Thanks
Ashish
I am attempting to create a Users controller in my ruby on rails project, which I have also configured with heroku and an aws-s3 bucket. I set my .env and my heroku local with the S3_BUCKET, AWS_ACCESS_KEY_ID, and AWS_SECRET_ACCESS_KEY. I also set my initializer/aws.rb file to look like this:
Aws.config.update({
region: 'us-east-1',
credentials: Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY']),
})
S3_BUCKET = Aws::S3::Resource.new.bucket(ENV['S3_BUCKET'])
I have bundle installed the aws gem like this:
gem 'aws-sdk', '~> 3'
However when I run the command
rails g controller Users new
I get the following error in my terminal:
aws-sdk-s3/bucket.rb:658:in `extract_name': missing required option :name (ArgumentError)
I looked at that file and it is trying to find the S3 bucket name, but I have set that already in .env and heroku local. Is there some other place where this needs to be set? None of the guides I have read mention this error.
Hi please check whether you have specified the right credentials and bucket name. Also, make sure you have provided the right region .Try the below code
s3 = Aws::S3::Resource.new(
credentials: Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY']),
region: 'us-west-1'
)
obj = s3.bucket(ENV['S3_BUCKET']).object('key')
If you want to upload file or something
obj.upload_file(file, acl:'public-read')
This would help you, I have used like this in my project.
1. Create the file aws.rb in your /config/initializers folder.
2. Then copy the below code,
S3Client = Aws::S3::Client.new(
access_key_id: 'ACCESS_KEY_ID',
secret_access_key: 'SECRET_ACCESS_KEY',
region: 'REGION'
)
Thats all, this works.
Happy coding :)
I deploy my rails app onto Heroku, and I've set aws access keys on the server as environment variables. However, to test my application in development environment, I need to initialize them somewhere on my local machine. So I decided to the following.
/config/initailizers/init_aws_locally.rb
ENV['AWS_ACCESS_KEY_ID'] = 'my key'
ENV['AWS_SECRET_ACCESS_KEY'] = 'my secret key'
This file is added in .gitignore
However, when I upload in development environment, I get this error message:
Missing required arguments: aws_access_key_id, aws_secret_access_key
I think somehow I overlooked a simple step to include my aws keys in my development environment. But I'm not sure why the reason for the error when I already initialized the keys.
For your reference, I'm using carrierwave, S3, and Fog.
config/initializers/fog.rb
CarrierWave.configure do |config|
config.fog_credentials = {
:provider => 'AWS', # required
:aws_access_key_id => ENV['AWS_ACCESS_KEY_ID'], # required
:aws_secret_access_key => ENV['AWS_SECRET_ACCESS_KEY'], # required
:region => 'us-east-1', # optional, defaults to 'us-east-1'
}
config.fog_directory = 'd' # required
config.fog_public = true # optional, defaults to true
end
Thank you. I appreciate your help!
Your initializers will be run in alphabetical order. See the docs:
If you have any ordering dependency in your initializers, you can
control the load order by naming. For example, 01_critical.rb will be
loaded before 02_normal.rb.
The problem you're experiencing is that your fog.rb initializer is running before your init_aws_locally.rb one (because f comes before i). So ENV['AWS_ACCESS_KEY_ID'] has not been defined (yet) when you set fog_credentials.
I'd avoid putting any credentials in code. It's such a terrible idea and Heroku has the right idea. So what I do is use RVM and put a file .rvmrc in my project folder. I put .rvmrc in .gitignore as well.
Then you edit .rvmrc to have
export AWS_ACCESS_KEY_ID="BLAH"
so on and so forth. Anytime I "cd" into this directory my env is setup for me for that project by RVM. If you're not using RVM there is other alternatives out there.
rails s
and it will have all the environment variables setup that you put in your .rvmrc script. No need for initializer or development only yaml config files that you keep out of source control. From my experience this is the simplest solution.
I went to my shell and typed:
$ echo $AWS_SECRET_ACCESS_KEY
and it came back blank. It turns out I recently moved to a new virtual machine and forgot to add this to the .bashrc file. It's worth checking the shell environment just in case. Once I added those two lines to my .bashrc, everything was happy again.
I am doing some development involving the Stripe API. In my development environment I am using my stripe test key. In production I am using the real api key so that I can process real transactions of course.
I am currently just changing out the test api key immediately before I deploy to my production environment; this does not feel very good. A strategy that I am pretty sure would work is to just make a development branch with a gitignore (ignoring my initializer that loads the api key) and then just merging it with the master branch before I deploy; this way the api keys would always be correct in their respective environments. I don't really like this approach though. Is there some sort of way to configure these api keys somewhere so that the app just knows which one to use when in dev/prod?
In rails 4.1 we have the config/secrets.yml file so you can set the api keys there like so:
development:
secret_key_base: 'xxx'
publishable_key: xxx
secret_key: xxx
production:
secret_key_base: 'xxx'
publishable_key: xxx
secret_key: xxx
And in your stripe.rb file you can do this:
Rails.configuration.stripe = {
:publishable_key => Rails.application.secrets.publishable_key,
:secret_key => Rails.application.secrets.secret_key
}
Stripe.api_key = Rails.configuration.stripe[:secret_key]
I prefer using yetting.
Though adding API keys in to the environments files gets the job done, adding them to a seperate file seems a whole lot of a cleaner and abstracted way to me.
Once the gem is installed. you can make a yetting.yml file in the config directory.
development:
facebook_app_id: xxxxxx
staging:
facebook_app_id: xxxxxx
production:
facebook_app_id: xxxxxx
This is how I do it :
In config/initializers/stripe.rb
if(Rails.env == 'development' || Rails.env == 'staging')
Stripe.api_key = "XXXXXXXXXXXXXXXXXXXXXXXXXXX"
STRIPE_PUBLIC_KEY = "XXXXXXXXXXXXXXXXXXXXXXXXXXX"
elsif(Rails.env == 'production')
Stripe.api_key = "XXXXXXXXXXXXXXXXXXXXXXXXXXX"
STRIPE_PUBLIC_KEY = "XXXXXXXXXXXXXXXXXXXXXXXXXXX"
end
There are many ways to skin this cat.
Quick and simple, look at Rails.env.
It will return the environment the server is running in.
You just need to set the key accordingly.
The way I would really recommend is to create application config variables with a yaml file or using Rails.application.config
config/environments/development.rb
config.api_key = "my dev api key"
config/environments/production.rb
config.api_key = "my prod api key"
To access your key
MyApp::Application.config.api_key
Check out this question for other examples.
Where should I store keys specific to my development, test, production servers in my ruby project? For example where should store my development-specific amazon s3 secret and key? my config/development.rb file? One issue I see with that is if the file was a part of a public github project it would show for everyone.
Thanks!
You store separate environment variables in config/development.rb, config/testing.rb and config/production.rb respectively.
However, if your files will be stored in a public git repo, you do not want to hardcode any sensitive information into them. The best method is to use either yaml files that are part of your .gitignore or to use Environment variables in your shell. I prefer the latter, like this:
PAPERCLIP_OPTIONS = { storage: :s3,
bucket: ENV['S3_BUCKET'],
s3_credentials: { access_key_id: ENV['S3_KEY'],
secret_access_key: ENV['S3_SECRET'] }}
You then just set the environment variables on the system that is running the app.
If you use the yaml config file method, you must add the sensitive config files to your .gitignore file. Otherwise they will still be uploaded to your public repo.
If you look at config directory, you will see a YAML file, containing database credential information. You could do the same for your cloud environments.
development:
server: xiy-234
username: hello
password: 1325abc
production:
...
You can put those information inside a .yml file in the config directory.
For instance:
production:
access_key_id: xxx
secret_access_key: xx
bucket: xxx
development:
access_key_id: xxx
secret_access_key: xxx
bucket: xxx
staging:
access_key_id: xxx
secret_access_key: xxx
bucket: xxx
Once this is done, you have to store those keys in a Hash by doing the following:
APIS_CONFIG = {'amazons3' => YAML.load_file("#{RAILS_ROOT}/config/amazons3.yml")[Rails.env]}
(You can put the previous line of code inside a .rb file situated in the config/initializers directory)
Note that you might find this Railscast interesting.