To avoid passing in access keys and secret aws access on a yml file I use the following :
development:
bucket: development
access_key_id: <%= ENV["S3_KEY"] %>
secret_access_key: <%= ENV["S3_SECRET"] %>
and then when running i get the error
Could not log "sql.active_record" event. ArgumentError: invalid byte sequence in UTF-8
PG::Error: ERROR: invalid byte sequence for encoding "UTF8": 0xe7 0xe3 0x6f
If I write my access key and secret directly on yml, like:
development:
bucket: development
access_key_id: MYACCESSKEY
secret_access_key: MYSECRETKEY
it goes smoothly.
Why does this error happen? How can i fix it without loading my key and secret into the yml file?
Edit
To load the environment variables onto development, I'm using the solution explained here
# Load the app's custom environment variables here, so that they are loaded before environments/*.rb
app_environment_variables = File.join(Rails.root, 'config', 'app_environment_variables.rb')
load(app_environment_variables) if File.exists?(app_environment_variables)
Might this be a problem with the loading process?
Edit 2
In the meanwhile, I tried to log what seems to be on my S3_CONFIG variable, loaded with:
config/initializers/load_config.rb
S3_CONFIG = YAML.load_file("#{::Rails.root}/config/s3.yml")[Rails.env]
I get
S3 Config: {"bucket"=>"mybucket", "access_key_id"=>"<%= ENV[\"S3_KEY\"] %>", "secret_access_key"=>"<%= ENV[\"S3_SECRET\"] %>"}
Wasn't it supposed to load the environment key already? May this be my problem?
This problem was happening when I was downloading the file from S3 with :
s3=AWS::S3.new(
access_key_id: S3_CONFIG["access_key_id"],
secret_access_key: S3_CONFIG["secret_access_key"])
and S3_CONFIG["access_key_id"] is just a string <%= ENV[\"S3_KEY\"] %>.
My solution for this was using just
s3=AWS::S3.new(
access_key_id: ENV['S3_KEY'],
secret_access_key: ENV['S3_SECRET'])
Guess sometimes one just needs to understand what he is doing, before pasting in lines of code...
Related
While creating the presigned url for my private image.png file in my s3 bucket, i used the below template
require 'aws-sdk-s3'
s3 = Aws::S3::Client.new(
region: 'us-east-1',
access_key_id: Access_key_id,
secret_access_key: Secret_access_key
)
signer = Aws::S3::Presigner.new(client: s3)
url = signer.presigned_url(
:get_object,
bucket: 'mybuck1',
key: "${image.png}-#{SecureRandom.uuid}"
)
while running the code, i get the follwing error as
AuthorizationQueryParametersError
Query-string authentication version 4 requires the X-Amz-Algorithm, X-Amz-Credential, X-Amz-Signature, X-Amz-Date, X-Amz-SignedHeaders, and X-Amz-Expires parameters.
so what might be the reason of this error, and how to fix this error
thanks in advance
I'm having an issue deploying my rails 6 app to Digital Ocean using spaces. After the deploy, the webserver fails. In the Puma logs I find:
Unable to load application: Aws::Sigv4::Errors::MissingCredentialsError: missing credentials, provide credentials with one of the following options:
Feb 05 03:33:25: - :access_key_id and :secret_access_key
Feb 05 03:33:25: - :credentials
Feb 05 03:33:25: - :credentials_provider
Here is my relevant rails setup:
gem file
gem 'aws-sdk-s3'
config/storage.yml
digitalocean_spaces:
service: S3
access_key_id: <%= Rails.application.credentials.dig(:digitalocean_spaces, :access_key_id) %>
secret_access_key: <%= Rails.application.credentials.dig(:digitalocean_spaces, :secret_access_key) %>
region: nyc3
endpoint: 'https://nyc3.digitaloceanspaces.com'
bucket: foobar-bucket
environments/production.rb
config.active_storage.service = :digitalocean_spaces
credentials.yml.enc
digitalocean_spaces:
access_key_id: foobar
secret_access_key: secretfoobar
When you generated the credentials file by executing EDITOR="nano -w" rails credentials:edit --environment production, after saving something like this was printed to the console:
Adding config/credentials/production.key to store the encryption key: c1463bd04a5521e81cbe2a496a91af52
Save this in a password manager your team can access.
If you lose the key, no one, including you, can access anything encrypted with it.
create config/credentials/production.key
Ignoring config/credentials/production.key so it won't end up in Git history:
append .gitignore
The server must know the key in order to decrypt the credentials.
I was using DO Apps and had to go to my app -> Settings -> App-Level Environment Variables -> Edit and add RAILS_MASTER_KEY which contains the generated key
I've been spending hours on this but cannot come up with any solutions. I've scoured the docs to see what I'm missing but can't find anything.
I'm using JWT and ActiveStorage with Amazon S3. I need to hide this keys before I can commit to Github. Following many examples, here's how my credentials.yml.enc file looks:
amazon:
key: <my access key for AWS S3>
secret: <my secret id for AWS S3>
jwt: <my jwt token encode/decode password>
I save this then in my storage.yml file I have:
test:
service: Disk
root: <%= Rails.root.join("tmp/storage") %>
local:
service: Disk
root: <%= Rails.root.join("storage") %>
# Use rails credentials:edit to set the AWS secrets (as aws:access_key_id|secret_access_key)
amazon:
service: S3
access_key_id: <%= Rails.application.credentials[:amazon][:key] %>
secret_access_key: <%= Rails.application.credentials[:amazon][:secret] %>
region: us-east-1
bucket: your_own_bucket
I've also tried with
<%= Rails.application.credentials[Rails.env.to_sym][:amazon][:key] %>
If I goto console and try Rails.application.credentials[:amazon][:key] or Rails.application.credentials.amazon[:key] or any other variants, it's always nil.
Rails.application.credentials
gives me:
=> #<ActiveSupport::EncryptedConfiguration:0x00007faf1284aa80
#key_path=#<Pathname:/Users/demiansims/Development/Beastly/beastly-
backend/config/master.key>, #content_path=#
<Pathname:/Users/demiansims/Development/Beastly/beastly-
backend/config/credentials.yml.enc>, #env_key="RAILS_MASTER_KEY",
#raise_if_missing_key=false, #encryptor=#
<ActiveSupport::MessageEncryptor:0x00007faf1287bb80
#secret=">^\x04\x9Bh\xFEb\x00\x8B\xB3O5\xDC\x8E\xA6b",
#sign_secret=nil, #cipher="aes-128-gcm", #aead_mode=true,
#verifier=ActiveSupport::MessageEncryptor::NullVerifier,
#serializer=Marshal, #options={:cipher=>"aes-128-gcm"}, #rotations=[]>,
#config={}
You credential file might not be saving your changes. I had this same problem. I was using sublime and EDITOR="subl --wait" bin/rails credentials:edit would bring up the editor but it would not recognize when I had saved the file. I never solved this, so I switched to vi. EDITOR="vi" bin/rails credentials:edit and problem solved. Now it all works as expected. If you are not familiar with vi commands like me, these are helpful to know:
i -> "insert" will allow to move around with your arrow keys, delete, and add code
esc -> when you are finished editing hit escape
:wq -> this will save and exit the file. after you hit escape just type in :wq
hope this helps someone.
I think there's a problem in the format of your credentials.yml.enc it should be like:
amazon:
key: <my access key for AWS S3>
secret: <my secret id for AWS S3>
jwt: <my jwt token encode/decode password>
So you can access your key like Rails.application.credentials.amazon[:key]
According to this blog post, the new version of the Aws gem switches the namespace from AWS to Aws. But what am I supposed to use instead of
Aws.config({
access_key_id: "something",
secret_access_key: "something"
})
It's explained here but doesn't say what the alternative is:
http://ruby.awsblog.com/post/TxFKSK2QJE6RPZ/Upcoming-Stable-Release-of-AWS-SDK-for-Ruby-Version-2
Instead, I get an error:
Uncaught exception: wrong number of arguments (1 for 0)
AWS.config is no longer a method in v2. You now call Aws.config.update with a simple hash:
# v1
AWS.config({
access_key_id: "something",
secret_access_key: "something"
})
# v2
Aws.config.update({
access_key_id: "something",
secret_access_key: "something"
})
Here you have the link to the configuration options for more info related to #v2.
Looking at this section in the doc: http://docs.aws.amazon.com/sdkforruby/api/index.html#Configuration
it seems that the way you configure the credentials has changed.
I can't find the .config method in the docs anymore, it is now an attribute of Aws.
I'm trying to get my refinery cms image storage to Amazon s3 and I'm following this guide:
http://refinerycms.com/guides/how-to-use-amazon-s3-for-storage
But I'm blocked here:
There are a number of ways to set
these with your credentials, including
unix variables or settings them
manually through Ruby using ENV.
How do I define these credentials. Do I put something like :S3_KEY =>
"my_key" in my environments.rb file? I tried this and it didn't work.
I also tried this:
AWS::S3::Base.establish_connection!(
:access_key_id => ENV['S3_KEY'] || 'key_goes_here',
:secret_access_key => ENV['S3_SECRET'] || 's3_secret_key_here',
)
Can't figure out how to do this. Any ideas are greatly appreciated.
The safest way is to specify them as environment variables, so they aren't included in your source code. If you're the only one with access to the source, then specifying them as you describe should work.
You can specify them in your ~/.bashrc
export S3_KEY=mykey
export S3_SECRET=mysecret
Or if you're just testing locally you can prepend them to your rails command.
$ S3_KEY=mykey S3_SECRET=mysecret rails server
If you don't want to/can't use environment variables, another method is to use an initializer to load credentials from a yml file: config/initializers/s3_credentials.rb
# Load AWS::S3 configuration values
#
S3_CREDENTIALS = \
YAML.load_file(File.join(Rails.root, 'config/s3_credentials.yml'))[Rails.env]
# Set the AWS::S3 configuration
#
AWS::S3::Base.establish_connection! S3_CREDENTIALS['connection']
config/s3_credentials.yml
development: &defaults
connection:
:access_key_id: AAAAAA_your-key-here
:secret_access_key: 4rpsi235js_your-secret-here
:use_ssl: true
bucket: project-development
acl: public-read
production:
<<: *defaults
bucket: project