I just installed the gem asset_sync and I am trying to get set up with my AWS account. When I run bundle exec rake assets:precompile I get the following errror:
AssetSync::Config::Invalid: Fog provider can't be blank, Fog directory can't be blank
I understand the simply reason that I am getting this error, namely that I havent pushed the Fog provider or directory to heroku. What I am stumped about is where to put the Following code (Taken from the Fog README). In config/initializers/fog.rb? Is this all I need to do to start using fog, other than installing the gem?
require 'rubygems'
require 'fog'
# create a connection
connection = Fog::Storage.new({
:provider => 'AWS',
:aws_access_key_id => YOUR_AWS_ACCESS_KEY_ID,
:aws_secret_access_key => YOUR_AWS_SECRET_ACCESS_KEY
})
# First, a place to contain the glorious details
directory = connection.directories.create(
:key => "fog-demo-#{Time.now.to_i}", # globally unique name
:public => true
)
not a problem, getting started tends to be the hardest part.
The answer is, it depends. I'd actually venture to say it would be best to put this in your environment based initializers, ie config/init/development or config/init/production, etc. Relatedly, you probably will not want to generate a new directory every time you start your app (there is an account level limit of 100 total I believe). So you might want to either set a key for each environment for that create or simply create the directory somewhere outside the initializers (and within the initializer you can assume it exists).
If you want to use that directory directly, you'll still need to create a reference, but you can create a local reference without making any api calls with #new like this:
directory = connection.directories.new(:key => ...)
As for asset_sync, it needs those keys and a reference to the directory key, which you will probably want to provide via ENV vars (to avoid checking your credentials into version control). You can find details on which keys and how to set them here: https://github.com/rumblelabs/asset_sync#built-in-initializer-environment-variables (the readme also describes how to do it via initializers, but that probably isn't the best plan).
Hope that helps!
Related
I have a problem I am following step by step this tutorial to include the gem stripe to my rails application : https://stripe.com/docs/checkout/rails
But I have an issue when I launch my payment:
you did not set a valid publishable key. Call Stripe.setPublishableKey() with your publishable key.
The tutorial I am following doesn't use this method Stripe.setPublishableKey() and it seems working as well ...
I am using a .env file to get the API keys and I use the test keys from the tutorial.
Any idea for the solution ?!
Thank you very much :) If you need more information please tell me
My actual code is a copy/paste from the tutorial I use.
Edit 1
I don't understand why but it working when I push the project on Heroku ... but I still don't understand why it's failing on local
From the guide you linked:
The application makes use of your publishable and secret API keys
to interact with Stripe. An initializer is a good place to set these
values, which will be provided when the application is started.
Add the following to config/initializers/stripe.rb:
Rails.configuration.stripe = {
:publishable_key => ENV['PUBLISHABLE_KEY'],
:secret_key => ENV['SECRET_KEY']
}
Stripe.api_key = Rails.configuration.stripe[:secret_key]
These keys values are pulled out of environmental variables so as not
to hardcode them. It’s best practice not to write API keys into your
code, where they could easily wind up in source control repositories
and other non-private destinations.
Have you added that data into config/initializers/stripe.rb ?
If so, you may need to restart your server as the files in config/initializers are only loaded on server startup
I am using a .env file to get the API keys and I use the test keys from the tutorial.
Rails don't autoload environment variables from .env file. So, in the tutorial author set required variables from the console before server start, but you use .env file.
To fix that use gem dotenv-rails: just add in your Gemfile(not need add the same lines as in gem description)
I was able to overcome the same type of error in another tutorial by changing the last line of the stripe.rb file to:
Stripe.api_key = Rails.application.credentials.stripe[:secret_key]
with the key being saved in the credentials.yml.enc file in rails 5.2
Running heroku run rake db:migrate, I get this error: Missing required arguments: aws_access_key_id, aws_secret_access_key.
I made the correction recommended in SO question 25596504, specifically changing the file carrier_wave.rb to carrierwave.rb with no luck. I am following the Hartl tutorial page 688 which specifies adding them as $heroku config:set S3_ACCESS_KEY=<access key> I replaced the '' with quoted and unquoted versions of the actual key. The keys show when I run heroku config, e.g., S3_ACCESS_KEY:
The application was working before I began working on this section (11.4) of the tutorial re uploading images. I, incidentally, know about the Figaro gem; however, I'd like to try following the tutorial's approach. What am I missing? Any thoughts would be appreciated. Thanks!
Go on Heroku, on your application, go to settings, hit Reveal Config Vars.
Click on on Edit on the right side and enter your secrets there:
S3_BUCKET: name of your bucket goes here
S3_ACCESS_KEY: xxxxx
S3_SECRET_KEY: xxxx
On config/initializers/carrierwave.rb or wherever you're entering your secrets should have:
CarrierWave.configure do |config|
config.root = Rails.root.join('tmp') # adding these...
config.cache_dir = 'carrierwave' # ...two lines
config.fog_credentials = {
:provider => 'AWS', # required
:s3_access_key_id => ENV['S3_ACCESS_KEY'], # required
:s3_secret_access_key => ENV['S3_SECRET_KEY'], # required
:region => 'eu-west-1', # optional, defaults to 'us-east-1'
:host => 's3.example.com', # optional, defaults to nil
:endpoint => 'https://s3.example.com:8080' # optional, defaults to nil
}
config.fog_directory = ENV['S3_Bucket'] # required
config.fog_public = false # optional, defaults to true
config.fog_attributes = {'Cache-Control'=>'max-age=315576000'} # optional, defaults to {}
end
Here is a tutorial I made after much work getting AWS to work with Heroku as described in Chapter 11 of the Ruby on Rails Tutorial by Michael Hartl. I hope it helps:
Getting the Rails Tutorial Sample App to work between Heroku and AWS was a huge pain in the ass. But I did it. If you found this tutorial, that means you're probably encountering an error you can't get past. That's fine. I had a few of them.
The first thing you need to do is go back over the code that Hartl provided. Make sure you typed it (or copy/pasted it) in exactly as shown. Out of all the code in this section, there is only one small addition you might need to make. The "region" environment variable. This is needed if you create a bucket that is not in the default US area. More on this later. Here is the code for /config/initializers/carrier_wave.rb:
if Rails.env.production?
CarrierWave.configure do |config|
config.fog_credentials = {
# Configuration for Amazon S3
:provider => 'AWS',
:aws_access_key_id => ENV['S3_ACCESS_KEY'],
:aws_secret_access_key => ENV['S3_SECRET_KEY'],
:region => ENV['S3_REGION']
}
config.fog_directory = ENV['S3_BUCKET']
end
end
That line :region => ENV['S3_REGION'] is a problem for a lot of people. As you continue this tutorial you will learn what it's for.
You should be using that block of code exactly as shown. Do NOT put your actual keys in there. We'll send them to Heroku separately.
Now let's move on to your AWS account and security.
First of all, create your AWS account. For the most part, it is like signing up for any web site. Make a nice long password and store it someplace secure, like an encrypted password manager. When you make your account, you will be given your first set of AWS keys. You will not be using those in this tutorial, but you might need them at some point in the future so save those somewhere safe as well.
Go to the S3 section and make a bucket. It has to have a unique
name, so I usually just put the date on the end and that does it. For example, you might name it "my-sample-app-bucket-20160126". Once you
have created your bucket, click on the name, then click on Properties.
It's important for you to know what "Region" your bucket is in. Find it,
and make a note of it. You'll use it later.
Your main account probably has full permissions to everything, so let's not use that for transmitting random data between two web services. This could cost you a lot of money if it got out. We'll make a limited user instead. Make a new User in the IAM section. I named it "fog", because that's the cloud service software that handles the sending and receiving. When you create it, you will have the option of displaying and/or downoading the keys associated with the new user. It's important you keep this in a safe
and secure place. It does NOT go into your code, because that will probably
end up in a repository where other people can see it. Also, don't give this
new user a password, since it will not be logging into the AWS dashboard.
Make a new Group. I called mine "s3railsbucket". This is where the
permissions will be assigned. Add "fog" to this group.
Go to the Policies section. Click "Create Policy" then select "Create Your
Own Policy". Give it a name that starts with "Allow" so it will show up near
the top of the list of policies. It's a huge list. Here's what I did:
Policy Name: AllowFullAccessToMySampleAppBucket20160126
Description: Allows remote write/delete access to S3 bucket named
my-sample-app-bucket-20160126.
Policy Document:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::my-sample-app-bucket-20160126",
"arn:aws:s3:::my-sample-app-bucket-20160126/*"
]
}
]
}
Go back to the Group section, select the group you made, then add
your new policy to the group.
That's it for AWS configuration. I didn't need to make a policy to allow
"fog" to list the contents of the bucket, even though most tutorials I tried
said that was necessary. I think it's only necessary when you want a user
that can log in through the dashboard.
Now for the Heroku configuration. This stuff gets entered in at your
command prompt, just like 'heroku run rake db:migrate' and such. This is
where you enter the actual Access Key and Secret Key you got from the "fog" user you created earlier.
$ heroku config:set S3_ACCESS_KEY=THERANDOMKEYYOUGOT
$ heroku config:set S3_SECRET_KEY=an0tHeRstRing0frAnDomjUnK
$ heroku config:set S3_REGION=us-west-2
$ heroku config:set S3_BUCKET=my-sample-app-bucket-20160126
Look again at that last one. Remember when you looked at the Properties of
your S3 bucket? This is where you enter the code associated with your
region. If your bucket is not in Oregon, you will have to change us-west-2 to your actual region code. This link worked when this tutorial was written:
http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region
If that doesn't work, Google "AWS S3 region codes".
After doing all this and double-checking for mistakes in the code, I got
Heroku to work with AWS for storage of pictures!
I think this error occurred because the var name does not match.
In carrierwave.rb, you should replace :s3_access_key_id and :s3_secret_access_key with the prefix "aws".
if Rails.env.production?
CarrierWave.configure do |config|
config.root = Rails.root.join('tmp')
config.cache_dir = 'carrierwave'
config.fog_credentials = {
# Configuration for Amazon S3
:provider => 'AWS', # change var's name
:aws_access_key_id => ENV['S3_ACCESS_KEY'], # change var's name
:aws_secret_access_key => ENV['S3_SECRETE_KEY']
}
config.fog_directory = ENV['S3_BUCKET']
end
end
And it can be deployed.
For whatever reason, running rake assets:precompile RAILS_ENV=development fixed this for me.
[and env vars don't need to be named S3_ACCESS_KEY etc, I used aws.access_key_id]
Running rake assets:precompile without any RAILS_ENV was the only way I could get it to precompile.
My application.rb:
S3_CREDENTIALS = YAML.load(File.read(File.expand_path(Rails.root.join("config","s3_credentials.yml"))))["production"]
# AWS::S3::Base.establish_connection! S3_CREDENTIALS['connection']
AWS::S3::Base.establish_connection!(
:access_key_id => S3_CREDENTIALS['access_key_id'],
:secret_access_key => S3_CREDENTIALS['secret_access_key'],
:persistent => true, # from http://www.ruby-forum.com/topic/110842
s3_credentials.yml:
production: &defaults
access_key_id: <%=ENV['AWS_ACCESS_KEY_ID']%>
secret_access_key: <%=ENV['AWS_SECRET_ACCESS_KEY']%>
persistent: true
I launch my web server and in my customers_controller I check for the connection:
Rails.logger.info("S3 service connected? " + AWS::S3::Base.connected?.to_s)
And the log says it's connected:
S3 service connected? true
So I know the following:
My env variables are correct.
My initializer and yml files are correct.
Gem is fine
So then I do something simple like this in the controller:
#documents = Service.buckets()
I reload the page and here we go:
AWS::S3::InvalidAccessKeyId in CustomersController#edit
The AWS Access Key Id you provided does not exist in our records.
THE KICKER -
When I print my secret key and access key in plain text into the application.rb file (get rid of the s3_credentials.yml file, just put the keys into the appropriate lines in the application.rb file - I won't get the error.
Why is it that using environment variables with the aws-s3 gem fails spectacularly using the API methods but loads just fine? The keys can't be good and bad at the same time. It wouldn't connect at all if the keys were wrong, correct?
What I found after hours is that there is something inherently borked in the version of aws-s3 gem that I'm using. Period. There is no solution other than to fix the gem, and I don't have time to go that far. I checked the source and when to the connection.rb module to find where it's calling up the keys. It uses his own extract_keys! method, and after poking at that for a while I gave up.
I ended up using the aws-sdk gem. Config is a bit different than with aws-s3, but same ENV vars and it is all working now.
I submitted a request to the owner of the aws-s3 gem to take a look. But I don't know how active that gem is any longer.
aws-sdk is overkill when all you need is some simple access to S3 stuff, but whatever. It works.
Here is some of the config just in case someone finds it useful in the future:
aws.yml:
development: &defaults
access_key_id: <%=ENV['AWS_ACCESS_KEY_ID']%>
secret_access_key: <%=ENV['AWS_SECRET_ACCESS_KEY']%>
persistent: true
# bucket: YOUR BUCKET
max_file_size: 10485760
acl: public-read
NOTE : The <%= %> syntax above doesn't have spaces. Normally I format it with spaces. This might sound crazy to some, but if you put spaces between the tag and the value inside you will get an error from Amazon saying some crazy stuff about only requiring a single space. This drove me nuts. The leading or trailing space shouldn't matter! But it did. I took the spaces out - started working. Call me crazy. I spent hours battling this so you wouldn't have to!
application.rb (portion):
# Establish a base connection to Amazon S3
S3_CREDENTIALS = YAML.load(File.read(File.expand_path(Rails.root.join("config","aws.yml"))))["development"]
AWS.config(
:access_key_id => S3_CREDENTIALS['access_key_id'],
:secret_access_key => S3_CREDENTIALS['secret_access_key'],
)
And of course export AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to your environment.
On Mac just 'export AWS_ACCESS_KEY_ID=YOUR_KEY_GOES_HERE AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY_GOES_HERE' and hit return.
Heroku is:
heroku config:set AWS_ACCESS_KEY_ID=YOUR_KEY_GOES_HERE AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY_GOES_HERE
Can't do anything for you if you're using Windows.
You're right that it would never work if the key was just wrong. Have you tried to make sure that S3_CREDENTIALS['access_key_id'] and ENV['AWS_ACCESS_KEY_ID'] contain what you think they contain? My first guess is that S3_CREDENTIALS['access_key_id'] ends up containing the literal string of <%=ENV['AWS_ACCESS_KEY_ID']%> (not interpreted) since you're not processing the yaml file as erb before loading the file.
I have many credentials that I have to handle in order to hook my app up to amazon s3 and other services.
I got my heroku app up and running with s3, and it works great. I defined my s3 access credentials following this example: http://devcenter.heroku.com/articles/config-vars
However, I want now to be able to have access to s3 from my local development environment. Obviously, the config vars that I defined on heroku aren't available on my localhost. How can I define these keys locally? Also, I'm looking in particular for a solution that is secure (for example if I define my keys in plain text in an intializer or something, I don't want that file to be pushed on heroku).
For background, here is what I add to my model to get paperclip running with s3
has_attached_file :photo,
:storage => :s3,
:bucket => 'bucket_name',
:s3_credentials => {
:access_key_id => ENV['S3_KEY'],
:secret_access_key => ENV['S3_SECRET']
}
The best place to define stuff like this, if you don't want it shared, is probably an initializer.
# config/initializers/s3_constants.rb
if Rails.env.development?
S3_KEY = "mys3key"
S3_SECRET = "mys3secret"
end
Ensure this file is added to .gitignore so it won't be pushed along with the rest of your repository.
Realistically speaking, constants that differ on a per-environment basis should really be located in the file for that environment (say development.rb here)... but those files should also really be added to your version control system, and if you definitely, definitely want this data excluded from git, then a separate file that you do not commit is probably your best bet.
Just define the environment variables in your .bash_profile file like any other environment variable. Maybe leave a comment to demarcate the section as Rails-specific environment variables.
#~/.bash_profile
# Rails constants
S3_KEY="blady"
S3_SECRET="bladybloo123"
Also, maybe you want to change the name to something more specific so that you can have more than one s3 connection defined...
heroku provides heroku config:add and you provide KEY=value. see config vars documentation
I'm learning Ruby On Rails. I'm trying to use the aws-s3 gem to access Amazon S3. One line of my controller code reads:
AWS::S3::Base.establish_connection!(
:access_key_id => 'myrealaccesskeyishere',
:secret_access_key => 'myrealsecretkeyishere'
)
I've noticed that if I make an error, sometimes rails will come back and show a few lines of code where it thinks the error might be. Should I not be writing these out in the .rb controller files like this? Am I potentially risking my secret key? If so, how should I be doing this instead?
You should put this in an initializer. Place it in config/intializers/amazon_s3.rb
Is there a reason you are putting this code directly in the controller?