I'm working in a Rails 5 application which is deployed by the moment on Heroku. I'm using Postgrsql for data storage, Paperclip to manage image uploads and AWS S3 to store all the uploaded images.
To accomplish this I used this very detailed and useful tutorial from Heroku dev which really help me a lot.
I use the same configuration for development env to be able to test it. Actually it works like a charm in dev.
When I deploy to Heroku and after running migrations, setting up ENV variables I create a new Brochure which accepts a cover image; and everything goes good. The images are stored correctly in AWS S3.
But when I render de image in a view I wit just don't work. I got the following errors in browsers console:
Safari browser:
Failed to load resource: The certificate for this server is invalid. You might be connecting to a server that is pretending to be “sponsors.matchxperience.s3.amazonaws.com” which could put your confidential information at risk.
Chrom canary:
Failed to load resource: net::ERR_INSECURE_RESPONSE
I don't know what's the matter because in development environment everything works.
Can anybody help me out with this or any idea what's going on?
production.rb (the same in development.rb)
Rails.application.configure do
# We’ll also need to specify the AWS configuration variables for the production Environment.
config.paperclip_defaults = {
storage: :s3,
# s3_protocol: 'http',
s3_credentials: {
bucket: ENV.fetch('AWS_S3_BUCKET'),
access_key_id: ENV.fetch('AWS_ACCESS_KEY_ID'),
secret_access_key: ENV.fetch('AWS_SECRET_ACCESS_KEY'),
s3_region: ENV.fetch('AWS_REGION')
}
}
end
brochure.rb
class Brochure < ApplicationRecord
# This method associates the attribute ":cover" with a file attachment
has_attached_file :cover, styles: {
card: '500x330#',
}
# Validate the attached image is image/jpg, image/png, etc
validates_attachment_content_type :cover, :content_type => /\Aimage\/.*\Z/
end
paperclip.rb at config/initializers/
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = '/:class/:attachment/:id_partition/:style/:filename'
After searching and reading different sources for this error I found many different solutions for similar errors but no one seemed to be related with Rails 5 directly and sincerely I don't even though it has something to see with Ruby or Rails.
I was convinced it was something with the AWS S3 server and I was right, I fixed it. Finally reading the official documentation for creating a new bucket I realized that it was something ridiculously simple.
In the documentation it says we can use periods . and hyphens - in our bucket's name:
Can contain lowercase letters, numbers, periods (.), and hyphens (-).
Must start with a number or letter.
Must be between 3 and 63 characters long.
...
So, I named my bucket as:
sponsors.matchxperience
Which was right written, BUT talking about server's URLs it may confuse the browser's requests to point to a different server and that was happening in my case. That's why I got that error.
The solution
Just create another bucket (or renaming the actual one may function) and copied all the content named as:
sponsors-matchxperience
And magically it just works fine in production on Heroku. I don't know what's going on with the AWS documentation but for me, that whats the error, periods . in my bucket's name.
I hope it could be useful for someone else.
Related
I'm using AmazonS3 to store Paperclip attachments on all non-test environments.
For test specifically I use a local path/url setup to avoid interacting with S3 remotely
Paperclip::Attachment.default_options[:path] =
":rails_root/public/system/:rails_env/:class/:attachment/:id_partition/:filename"
Paperclip::Attachment.default_options[:url] =
"/system/:rails_env/:class/:attachment/:id_partition/:filename"
I define my attachment as follows in the model
has_attached_file :source_file, use_timestamp: false
In my Production code I need to access the file using Model.source_file.url because .url returns the remote fully qualified Amazon S3 path to the file. This generally works fine for non-test environments.
However on my test environment I can't use .url because Paperclip creates and stores the file under the path defined by :path above. So I need to use .path. If I use .url I get the error -
Errno::ENOENT:
No such file or directory # rb_sysopen - /system/test/imports/source_files/000/000/030/sample.txt
which makes sense because paperclip didn't store the file there...
How do I get paperclip on my test environment to store/create my file under the :url path so I can use .url correctly?
Edit: If it helps, in test I create the attachment from a locally stored fixture file
factory :import do
source_file { File.new(Rails.root + "spec/fixtures/files/sample.tsv") }
end
Edit2: Setting :path and :url to be the same path in the initializer might seem like a quick fix, but I'm working on a larger app with several contributors, so I don't the have the luxury to do that or break any one else's specs. Plus it looks like Thoughtbot themselves recommend this setup, so there should be a "proper" way to get it working as is.
Thanks!
Have you tried using s3proxy in your test environment to simulate S3 instead of directly have paperclip write to local files?
Im busy testing Paperclip in an application that sends documents to another service. However, I noticed that during testing my spec/documents folder begins filling up with attachments from unit tests. I know this has something to do with my config, but I cant seem to figure out why it puts the files in this folder or how to prevent it from happening.
if Rails.env.production?
Paperclip::Attachment.default_options.merge!({
...
})
elsif Rails.env.test?
Paperclip::Attachment.default_options.merge!({
storage: :filesystem,
url: "/test_attachments",
path: ":rails_root/spec:url"
})
else
Paperclip::Attachment.default_options.merge!({
storage: :filesystem,
url: "/uploads/:class/:attachment/:id_partition/:style/:filename",
path: ":rails_root/public:url"
})
end
Any thoughts? I was hoping there was some config option that would allow me to disable storage in the test environment but I haven't been able to find any.
From what I can tell from the documentation, it suggests to remove the directory. It doesn't look like you can specify Paperclip avoid storage altogether.
See Documentation on Testing here https://github.com/thoughtbot/paperclip#testing
I'm working on a Rails website that uses Paperclip to upload files to Amazon S3 and then serve them through Cloudfront. I've got the uploading working fine, and Cloudfront is serving the files properly, but for some reason ModelObj.attachment.url isn't working properly on my production server. It works fine on my development server (WEBrick under Windows 8) but is returning the wrong URL on the live site (LAMP with Phusion Passenger). It returns almost the right url, but instead of
http://[stuff].cloudfront.net/kidbooks/snds/5072_original.mp3?1393858446
I'm getting
http:///kidbooks/snds/5072_original.mp3?1393858446
It's configured as follows: In environment.rb:
Paperclip::Attachment.default_options.merge!({
:storage => :s3,
:bucket => APP_CONFIG['s3_bucket'],
:path => "/#{APP_CONFIG['s3_path']}/:attachment/:id_:style.:extension",
:s3_credentials => {
...
}
})
And in the model:
has_attached_file :snd,
:url => ':s3_alias_url',
:s3_host_alias => APP_CONFIG['cloudfront_domain']
APP_CONFIG is being set properly on both servers; I've checked. (Or at least, it's being set properly in the console on the live server; I guess I could check more directly. There doesn't appear to be anything relevant in environments/development.rb or production.rb. What's going wrong here, and how can I fix it?
(I found this question, which is close to what I need, but only involves S3.)
Update: Found something weird. Model.snd.options returns a hash that includes, among other things, :s3_host_alias=>nil. Maybe my model is somehow being loaded before my configuration files?
Aha, found it. For some reason my model is either being loaded before my config files, or the loader doesn't have access to APP_CONFIG. This meant that :s3_host_alias was being set to nil, which of course broke things. I moved :s3_host_alias => APP_CONFIG['cloudfront_domain'] from the model to environment.rb, and now it's working fine.
I have no idea why my model would be loading before my config files, though.
My application.rb:
S3_CREDENTIALS = YAML.load(File.read(File.expand_path(Rails.root.join("config","s3_credentials.yml"))))["production"]
# AWS::S3::Base.establish_connection! S3_CREDENTIALS['connection']
AWS::S3::Base.establish_connection!(
:access_key_id => S3_CREDENTIALS['access_key_id'],
:secret_access_key => S3_CREDENTIALS['secret_access_key'],
:persistent => true, # from http://www.ruby-forum.com/topic/110842
s3_credentials.yml:
production: &defaults
access_key_id: <%=ENV['AWS_ACCESS_KEY_ID']%>
secret_access_key: <%=ENV['AWS_SECRET_ACCESS_KEY']%>
persistent: true
I launch my web server and in my customers_controller I check for the connection:
Rails.logger.info("S3 service connected? " + AWS::S3::Base.connected?.to_s)
And the log says it's connected:
S3 service connected? true
So I know the following:
My env variables are correct.
My initializer and yml files are correct.
Gem is fine
So then I do something simple like this in the controller:
#documents = Service.buckets()
I reload the page and here we go:
AWS::S3::InvalidAccessKeyId in CustomersController#edit
The AWS Access Key Id you provided does not exist in our records.
THE KICKER -
When I print my secret key and access key in plain text into the application.rb file (get rid of the s3_credentials.yml file, just put the keys into the appropriate lines in the application.rb file - I won't get the error.
Why is it that using environment variables with the aws-s3 gem fails spectacularly using the API methods but loads just fine? The keys can't be good and bad at the same time. It wouldn't connect at all if the keys were wrong, correct?
What I found after hours is that there is something inherently borked in the version of aws-s3 gem that I'm using. Period. There is no solution other than to fix the gem, and I don't have time to go that far. I checked the source and when to the connection.rb module to find where it's calling up the keys. It uses his own extract_keys! method, and after poking at that for a while I gave up.
I ended up using the aws-sdk gem. Config is a bit different than with aws-s3, but same ENV vars and it is all working now.
I submitted a request to the owner of the aws-s3 gem to take a look. But I don't know how active that gem is any longer.
aws-sdk is overkill when all you need is some simple access to S3 stuff, but whatever. It works.
Here is some of the config just in case someone finds it useful in the future:
aws.yml:
development: &defaults
access_key_id: <%=ENV['AWS_ACCESS_KEY_ID']%>
secret_access_key: <%=ENV['AWS_SECRET_ACCESS_KEY']%>
persistent: true
# bucket: YOUR BUCKET
max_file_size: 10485760
acl: public-read
NOTE : The <%= %> syntax above doesn't have spaces. Normally I format it with spaces. This might sound crazy to some, but if you put spaces between the tag and the value inside you will get an error from Amazon saying some crazy stuff about only requiring a single space. This drove me nuts. The leading or trailing space shouldn't matter! But it did. I took the spaces out - started working. Call me crazy. I spent hours battling this so you wouldn't have to!
application.rb (portion):
# Establish a base connection to Amazon S3
S3_CREDENTIALS = YAML.load(File.read(File.expand_path(Rails.root.join("config","aws.yml"))))["development"]
AWS.config(
:access_key_id => S3_CREDENTIALS['access_key_id'],
:secret_access_key => S3_CREDENTIALS['secret_access_key'],
)
And of course export AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to your environment.
On Mac just 'export AWS_ACCESS_KEY_ID=YOUR_KEY_GOES_HERE AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY_GOES_HERE' and hit return.
Heroku is:
heroku config:set AWS_ACCESS_KEY_ID=YOUR_KEY_GOES_HERE AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY_GOES_HERE
Can't do anything for you if you're using Windows.
You're right that it would never work if the key was just wrong. Have you tried to make sure that S3_CREDENTIALS['access_key_id'] and ENV['AWS_ACCESS_KEY_ID'] contain what you think they contain? My first guess is that S3_CREDENTIALS['access_key_id'] ends up containing the literal string of <%=ENV['AWS_ACCESS_KEY_ID']%> (not interpreted) since you're not processing the yaml file as erb before loading the file.
I have many credentials that I have to handle in order to hook my app up to amazon s3 and other services.
I got my heroku app up and running with s3, and it works great. I defined my s3 access credentials following this example: http://devcenter.heroku.com/articles/config-vars
However, I want now to be able to have access to s3 from my local development environment. Obviously, the config vars that I defined on heroku aren't available on my localhost. How can I define these keys locally? Also, I'm looking in particular for a solution that is secure (for example if I define my keys in plain text in an intializer or something, I don't want that file to be pushed on heroku).
For background, here is what I add to my model to get paperclip running with s3
has_attached_file :photo,
:storage => :s3,
:bucket => 'bucket_name',
:s3_credentials => {
:access_key_id => ENV['S3_KEY'],
:secret_access_key => ENV['S3_SECRET']
}
The best place to define stuff like this, if you don't want it shared, is probably an initializer.
# config/initializers/s3_constants.rb
if Rails.env.development?
S3_KEY = "mys3key"
S3_SECRET = "mys3secret"
end
Ensure this file is added to .gitignore so it won't be pushed along with the rest of your repository.
Realistically speaking, constants that differ on a per-environment basis should really be located in the file for that environment (say development.rb here)... but those files should also really be added to your version control system, and if you definitely, definitely want this data excluded from git, then a separate file that you do not commit is probably your best bet.
Just define the environment variables in your .bash_profile file like any other environment variable. Maybe leave a comment to demarcate the section as Rails-specific environment variables.
#~/.bash_profile
# Rails constants
S3_KEY="blady"
S3_SECRET="bladybloo123"
Also, maybe you want to change the name to something more specific so that you can have more than one s3 connection defined...
heroku provides heroku config:add and you provide KEY=value. see config vars documentation