I'm having an issue deploying my rails 6 app to Digital Ocean using spaces. After the deploy, the webserver fails. In the Puma logs I find:
Unable to load application: Aws::Sigv4::Errors::MissingCredentialsError: missing credentials, provide credentials with one of the following options:
Feb 05 03:33:25: - :access_key_id and :secret_access_key
Feb 05 03:33:25: - :credentials
Feb 05 03:33:25: - :credentials_provider
Here is my relevant rails setup:
gem file
gem 'aws-sdk-s3'
config/storage.yml
digitalocean_spaces:
service: S3
access_key_id: <%= Rails.application.credentials.dig(:digitalocean_spaces, :access_key_id) %>
secret_access_key: <%= Rails.application.credentials.dig(:digitalocean_spaces, :secret_access_key) %>
region: nyc3
endpoint: 'https://nyc3.digitaloceanspaces.com'
bucket: foobar-bucket
environments/production.rb
config.active_storage.service = :digitalocean_spaces
credentials.yml.enc
digitalocean_spaces:
access_key_id: foobar
secret_access_key: secretfoobar
When you generated the credentials file by executing EDITOR="nano -w" rails credentials:edit --environment production, after saving something like this was printed to the console:
Adding config/credentials/production.key to store the encryption key: c1463bd04a5521e81cbe2a496a91af52
Save this in a password manager your team can access.
If you lose the key, no one, including you, can access anything encrypted with it.
create config/credentials/production.key
Ignoring config/credentials/production.key so it won't end up in Git history:
append .gitignore
The server must know the key in order to decrypt the credentials.
I was using DO Apps and had to go to my app -> Settings -> App-Level Environment Variables -> Edit and add RAILS_MASTER_KEY which contains the generated key
Related
I created a key/secret pair and Pre-Authenticated Request and updated storage.yml to use it with rails active storage + provided an endpoint.
This is the error that I'm getting when trying to upload a file.
Seahorse::Client::NetworkingError (SSL_connect returned=1 errno=0 state=error: certificate verify failed (unspecified certificate verification error)):
The endpoint looks like this https://my-namespace.compat.objectstorage.eu-frankfurt-1.oraclecloud.com
It's been awhile since you posted this, but since I just recently ran into this, I'll share what worked for me in case it's helpful to you (or others).
The force_path_style: true parameter in the Active Storage settings (config/storage.yml) is likely missing. The OCI S3-compatibility interface looks for the bucket name in the path, not the hostname (which this option sets).
Here's an example:
# config/storage.yml
oci:
service: S3
access_key_id: <%= ENV['OCI_KEY'] %>
secret_access_key: <%= ENV['OCI_SECRET'] %>
bucket: <BUCKET NAME>-<%= Rails.env %>
region: <%= ENV['OCI_REGION'] %>
endpoint: https://<%= ENV['OCI_NAMESPACE'] %>.compat.objectstorage.<%= ENV['OCI_REGION'] %>.oraclecloud.com
force_path_style: true
I've been spending hours on this but cannot come up with any solutions. I've scoured the docs to see what I'm missing but can't find anything.
I'm using JWT and ActiveStorage with Amazon S3. I need to hide this keys before I can commit to Github. Following many examples, here's how my credentials.yml.enc file looks:
amazon:
key: <my access key for AWS S3>
secret: <my secret id for AWS S3>
jwt: <my jwt token encode/decode password>
I save this then in my storage.yml file I have:
test:
service: Disk
root: <%= Rails.root.join("tmp/storage") %>
local:
service: Disk
root: <%= Rails.root.join("storage") %>
# Use rails credentials:edit to set the AWS secrets (as aws:access_key_id|secret_access_key)
amazon:
service: S3
access_key_id: <%= Rails.application.credentials[:amazon][:key] %>
secret_access_key: <%= Rails.application.credentials[:amazon][:secret] %>
region: us-east-1
bucket: your_own_bucket
I've also tried with
<%= Rails.application.credentials[Rails.env.to_sym][:amazon][:key] %>
If I goto console and try Rails.application.credentials[:amazon][:key] or Rails.application.credentials.amazon[:key] or any other variants, it's always nil.
Rails.application.credentials
gives me:
=> #<ActiveSupport::EncryptedConfiguration:0x00007faf1284aa80
#key_path=#<Pathname:/Users/demiansims/Development/Beastly/beastly-
backend/config/master.key>, #content_path=#
<Pathname:/Users/demiansims/Development/Beastly/beastly-
backend/config/credentials.yml.enc>, #env_key="RAILS_MASTER_KEY",
#raise_if_missing_key=false, #encryptor=#
<ActiveSupport::MessageEncryptor:0x00007faf1287bb80
#secret=">^\x04\x9Bh\xFEb\x00\x8B\xB3O5\xDC\x8E\xA6b",
#sign_secret=nil, #cipher="aes-128-gcm", #aead_mode=true,
#verifier=ActiveSupport::MessageEncryptor::NullVerifier,
#serializer=Marshal, #options={:cipher=>"aes-128-gcm"}, #rotations=[]>,
#config={}
You credential file might not be saving your changes. I had this same problem. I was using sublime and EDITOR="subl --wait" bin/rails credentials:edit would bring up the editor but it would not recognize when I had saved the file. I never solved this, so I switched to vi. EDITOR="vi" bin/rails credentials:edit and problem solved. Now it all works as expected. If you are not familiar with vi commands like me, these are helpful to know:
i -> "insert" will allow to move around with your arrow keys, delete, and add code
esc -> when you are finished editing hit escape
:wq -> this will save and exit the file. after you hit escape just type in :wq
hope this helps someone.
I think there's a problem in the format of your credentials.yml.enc it should be like:
amazon:
key: <my access key for AWS S3>
secret: <my secret id for AWS S3>
jwt: <my jwt token encode/decode password>
So you can access your key like Rails.application.credentials.amazon[:key]
I want to set SECRET_KEY_BASE which is used in secrets.yml:
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
I have tried to add code as follow in .profile:
export SECRET_KEY_BASE=cfbc3b45d65db30b853cdc0557e0be85609cf75974ebb706f46a00abe09eee9454b3d311e48ee4157e1e5d5e3de5b8d2a329dff13871837cbaeae6af2bc2e42f
it works well, but this is still not that better, I know that dotenv can add this in a .env file in root path of app, so I add
gem 'dotenv-rails'
gem 'dotenv-deployment'
then I add code as follow into .env.production in root path of rails app:
SECRET_KEY_BASE=cfbc3b45d65db30b853cdc0557e0be85609cf75974ebb706f46a00abe09eee9454b3d311e48ee4157e1e5d5e3de5b8d2a329dff13871837cbaeae6af2bc2e42f
But why this doesn't work?
In Rails 4.1, config/secrets.yml is the new default location for secret_key_base of your application. It can, however, be used also for storing other secret variables, making it a good place for environment-specific tokens, API keys etc.
Fill the file with the secrets you want to store, for example:
development:
secret_key_base: your_development_secret
api_key: some_key
production:
secret_key_base: your_production_secret
twitter_consumer_key: production_twitter_key
twitter_consumer_secret: production_twitter_secret
twitter_oauth_token: production_oauth_token
twitter_oauth_token_secret: production_oauth_secret
In your code, you can access these secrets with Rails.application.secrets:
Twitter.configure do |config|
config.consumer_key = Rails.application.secrets.twitter_consumer_key
config.consumer_secret = Rails.application.secrets.twitter_consumer_secret
config.oauth_token = Rails.application.secrets.twitter_oauth_token
config.oauth_token_secret = Rails.application.secrets.twitter_oauth_token_secret
end
The secrets.yml will be checked into git by default, add it to your .gitignore file.
To avoid passing in access keys and secret aws access on a yml file I use the following :
development:
bucket: development
access_key_id: <%= ENV["S3_KEY"] %>
secret_access_key: <%= ENV["S3_SECRET"] %>
and then when running i get the error
Could not log "sql.active_record" event. ArgumentError: invalid byte sequence in UTF-8
PG::Error: ERROR: invalid byte sequence for encoding "UTF8": 0xe7 0xe3 0x6f
If I write my access key and secret directly on yml, like:
development:
bucket: development
access_key_id: MYACCESSKEY
secret_access_key: MYSECRETKEY
it goes smoothly.
Why does this error happen? How can i fix it without loading my key and secret into the yml file?
Edit
To load the environment variables onto development, I'm using the solution explained here
# Load the app's custom environment variables here, so that they are loaded before environments/*.rb
app_environment_variables = File.join(Rails.root, 'config', 'app_environment_variables.rb')
load(app_environment_variables) if File.exists?(app_environment_variables)
Might this be a problem with the loading process?
Edit 2
In the meanwhile, I tried to log what seems to be on my S3_CONFIG variable, loaded with:
config/initializers/load_config.rb
S3_CONFIG = YAML.load_file("#{::Rails.root}/config/s3.yml")[Rails.env]
I get
S3 Config: {"bucket"=>"mybucket", "access_key_id"=>"<%= ENV[\"S3_KEY\"] %>", "secret_access_key"=>"<%= ENV[\"S3_SECRET\"] %>"}
Wasn't it supposed to load the environment key already? May this be my problem?
This problem was happening when I was downloading the file from S3 with :
s3=AWS::S3.new(
access_key_id: S3_CONFIG["access_key_id"],
secret_access_key: S3_CONFIG["secret_access_key"])
and S3_CONFIG["access_key_id"] is just a string <%= ENV[\"S3_KEY\"] %>.
My solution for this was using just
s3=AWS::S3.new(
access_key_id: ENV['S3_KEY'],
secret_access_key: ENV['S3_SECRET'])
Guess sometimes one just needs to understand what he is doing, before pasting in lines of code...
I'm trying to get my refinery cms image storage to Amazon s3 and I'm following this guide:
http://refinerycms.com/guides/how-to-use-amazon-s3-for-storage
But I'm blocked here:
There are a number of ways to set
these with your credentials, including
unix variables or settings them
manually through Ruby using ENV.
How do I define these credentials. Do I put something like :S3_KEY =>
"my_key" in my environments.rb file? I tried this and it didn't work.
I also tried this:
AWS::S3::Base.establish_connection!(
:access_key_id => ENV['S3_KEY'] || 'key_goes_here',
:secret_access_key => ENV['S3_SECRET'] || 's3_secret_key_here',
)
Can't figure out how to do this. Any ideas are greatly appreciated.
The safest way is to specify them as environment variables, so they aren't included in your source code. If you're the only one with access to the source, then specifying them as you describe should work.
You can specify them in your ~/.bashrc
export S3_KEY=mykey
export S3_SECRET=mysecret
Or if you're just testing locally you can prepend them to your rails command.
$ S3_KEY=mykey S3_SECRET=mysecret rails server
If you don't want to/can't use environment variables, another method is to use an initializer to load credentials from a yml file: config/initializers/s3_credentials.rb
# Load AWS::S3 configuration values
#
S3_CREDENTIALS = \
YAML.load_file(File.join(Rails.root, 'config/s3_credentials.yml'))[Rails.env]
# Set the AWS::S3 configuration
#
AWS::S3::Base.establish_connection! S3_CREDENTIALS['connection']
config/s3_credentials.yml
development: &defaults
connection:
:access_key_id: AAAAAA_your-key-here
:secret_access_key: 4rpsi235js_your-secret-here
:use_ssl: true
bucket: project-development
acl: public-read
production:
<<: *defaults
bucket: project