I want to use Rails ActiveStorage, but I am using non-AWS S3 API object storage.
amazon:
service: S3
access_key_id: ""
secret_access_key: ""
region: ""
bucket: ""
On the documentation, it says that we need the region, my S3 API has no region. Is the any way to use custom S3 API?
I solved it by using the endpoint key on the configuration file. It look like this.
amazon:
service: S3
access_key_id: "123"
secret_access_key: "asd"
endpoint: "http://192.168.1.201:30103"
bucket: "test"
Do not need to use any region since there is not any. I found it at the bottom of the S3 Ruby gem AWS documentation https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html
Related
While creating the presigned url for my private image.png file in my s3 bucket, i used the below template
require 'aws-sdk-s3'
s3 = Aws::S3::Client.new(
region: 'us-east-1',
access_key_id: Access_key_id,
secret_access_key: Secret_access_key
)
signer = Aws::S3::Presigner.new(client: s3)
url = signer.presigned_url(
:get_object,
bucket: 'mybuck1',
key: "${image.png}-#{SecureRandom.uuid}"
)
while running the code, i get the follwing error as
AuthorizationQueryParametersError
Query-string authentication version 4 requires the X-Amz-Algorithm, X-Amz-Credential, X-Amz-Signature, X-Amz-Date, X-Amz-SignedHeaders, and X-Amz-Expires parameters.
so what might be the reason of this error, and how to fix this error
thanks in advance
I created a key/secret pair and Pre-Authenticated Request and updated storage.yml to use it with rails active storage + provided an endpoint.
This is the error that I'm getting when trying to upload a file.
Seahorse::Client::NetworkingError (SSL_connect returned=1 errno=0 state=error: certificate verify failed (unspecified certificate verification error)):
The endpoint looks like this https://my-namespace.compat.objectstorage.eu-frankfurt-1.oraclecloud.com
It's been awhile since you posted this, but since I just recently ran into this, I'll share what worked for me in case it's helpful to you (or others).
The force_path_style: true parameter in the Active Storage settings (config/storage.yml) is likely missing. The OCI S3-compatibility interface looks for the bucket name in the path, not the hostname (which this option sets).
Here's an example:
# config/storage.yml
oci:
service: S3
access_key_id: <%= ENV['OCI_KEY'] %>
secret_access_key: <%= ENV['OCI_SECRET'] %>
bucket: <BUCKET NAME>-<%= Rails.env %>
region: <%= ENV['OCI_REGION'] %>
endpoint: https://<%= ENV['OCI_NAMESPACE'] %>.compat.objectstorage.<%= ENV['OCI_REGION'] %>.oraclecloud.com
force_path_style: true
I've been spending hours on this but cannot come up with any solutions. I've scoured the docs to see what I'm missing but can't find anything.
I'm using JWT and ActiveStorage with Amazon S3. I need to hide this keys before I can commit to Github. Following many examples, here's how my credentials.yml.enc file looks:
amazon:
key: <my access key for AWS S3>
secret: <my secret id for AWS S3>
jwt: <my jwt token encode/decode password>
I save this then in my storage.yml file I have:
test:
service: Disk
root: <%= Rails.root.join("tmp/storage") %>
local:
service: Disk
root: <%= Rails.root.join("storage") %>
# Use rails credentials:edit to set the AWS secrets (as aws:access_key_id|secret_access_key)
amazon:
service: S3
access_key_id: <%= Rails.application.credentials[:amazon][:key] %>
secret_access_key: <%= Rails.application.credentials[:amazon][:secret] %>
region: us-east-1
bucket: your_own_bucket
I've also tried with
<%= Rails.application.credentials[Rails.env.to_sym][:amazon][:key] %>
If I goto console and try Rails.application.credentials[:amazon][:key] or Rails.application.credentials.amazon[:key] or any other variants, it's always nil.
Rails.application.credentials
gives me:
=> #<ActiveSupport::EncryptedConfiguration:0x00007faf1284aa80
#key_path=#<Pathname:/Users/demiansims/Development/Beastly/beastly-
backend/config/master.key>, #content_path=#
<Pathname:/Users/demiansims/Development/Beastly/beastly-
backend/config/credentials.yml.enc>, #env_key="RAILS_MASTER_KEY",
#raise_if_missing_key=false, #encryptor=#
<ActiveSupport::MessageEncryptor:0x00007faf1287bb80
#secret=">^\x04\x9Bh\xFEb\x00\x8B\xB3O5\xDC\x8E\xA6b",
#sign_secret=nil, #cipher="aes-128-gcm", #aead_mode=true,
#verifier=ActiveSupport::MessageEncryptor::NullVerifier,
#serializer=Marshal, #options={:cipher=>"aes-128-gcm"}, #rotations=[]>,
#config={}
You credential file might not be saving your changes. I had this same problem. I was using sublime and EDITOR="subl --wait" bin/rails credentials:edit would bring up the editor but it would not recognize when I had saved the file. I never solved this, so I switched to vi. EDITOR="vi" bin/rails credentials:edit and problem solved. Now it all works as expected. If you are not familiar with vi commands like me, these are helpful to know:
i -> "insert" will allow to move around with your arrow keys, delete, and add code
esc -> when you are finished editing hit escape
:wq -> this will save and exit the file. after you hit escape just type in :wq
hope this helps someone.
I think there's a problem in the format of your credentials.yml.enc it should be like:
amazon:
key: <my access key for AWS S3>
secret: <my secret id for AWS S3>
jwt: <my jwt token encode/decode password>
So you can access your key like Rails.application.credentials.amazon[:key]
According to this blog post, the new version of the Aws gem switches the namespace from AWS to Aws. But what am I supposed to use instead of
Aws.config({
access_key_id: "something",
secret_access_key: "something"
})
It's explained here but doesn't say what the alternative is:
http://ruby.awsblog.com/post/TxFKSK2QJE6RPZ/Upcoming-Stable-Release-of-AWS-SDK-for-Ruby-Version-2
Instead, I get an error:
Uncaught exception: wrong number of arguments (1 for 0)
AWS.config is no longer a method in v2. You now call Aws.config.update with a simple hash:
# v1
AWS.config({
access_key_id: "something",
secret_access_key: "something"
})
# v2
Aws.config.update({
access_key_id: "something",
secret_access_key: "something"
})
Here you have the link to the configuration options for more info related to #v2.
Looking at this section in the doc: http://docs.aws.amazon.com/sdkforruby/api/index.html#Configuration
it seems that the way you configure the credentials has changed.
I can't find the .config method in the docs anymore, it is now an attribute of Aws.
To avoid passing in access keys and secret aws access on a yml file I use the following :
development:
bucket: development
access_key_id: <%= ENV["S3_KEY"] %>
secret_access_key: <%= ENV["S3_SECRET"] %>
and then when running i get the error
Could not log "sql.active_record" event. ArgumentError: invalid byte sequence in UTF-8
PG::Error: ERROR: invalid byte sequence for encoding "UTF8": 0xe7 0xe3 0x6f
If I write my access key and secret directly on yml, like:
development:
bucket: development
access_key_id: MYACCESSKEY
secret_access_key: MYSECRETKEY
it goes smoothly.
Why does this error happen? How can i fix it without loading my key and secret into the yml file?
Edit
To load the environment variables onto development, I'm using the solution explained here
# Load the app's custom environment variables here, so that they are loaded before environments/*.rb
app_environment_variables = File.join(Rails.root, 'config', 'app_environment_variables.rb')
load(app_environment_variables) if File.exists?(app_environment_variables)
Might this be a problem with the loading process?
Edit 2
In the meanwhile, I tried to log what seems to be on my S3_CONFIG variable, loaded with:
config/initializers/load_config.rb
S3_CONFIG = YAML.load_file("#{::Rails.root}/config/s3.yml")[Rails.env]
I get
S3 Config: {"bucket"=>"mybucket", "access_key_id"=>"<%= ENV[\"S3_KEY\"] %>", "secret_access_key"=>"<%= ENV[\"S3_SECRET\"] %>"}
Wasn't it supposed to load the environment key already? May this be my problem?
This problem was happening when I was downloading the file from S3 with :
s3=AWS::S3.new(
access_key_id: S3_CONFIG["access_key_id"],
secret_access_key: S3_CONFIG["secret_access_key"])
and S3_CONFIG["access_key_id"] is just a string <%= ENV[\"S3_KEY\"] %>.
My solution for this was using just
s3=AWS::S3.new(
access_key_id: ENV['S3_KEY'],
secret_access_key: ENV['S3_SECRET'])
Guess sometimes one just needs to understand what he is doing, before pasting in lines of code...