Can't get Paperclip to set my S3 URLs properly - ruby-on-rails

I'm using paperclip and aws-sdk gems in a Rails 4 app.
I define the :path option in my paperclip.rb config, with no :url option:
Paperclip::Attachment.default_options[:path] = ":class/:attachment/:id_partition/:style/:filename"
It saves my uploaded images like such:
http://s3.amazonaws.com/mybucket-development/profiles/avatars/000/000/026/original/image_file_name.png?1420575189
All fine, it gets saved to S3. However it refuses to let me read the images for display, e.g. =profile.avatar.url(:medium). When I go to that URL in the browser it tells me to re-format it with the bucket name as a domain. Like:
http://mybucket-development.s3.amazonaws.com/profiles/avatars/000/000/026/original/image_file_name.png?1420575189
OK, not a problem either. I go to that URL, I can view my image. So now I need to figure out how to get Paperclip to format the URLs like this automatically. I read in the Paperclip docs that you just have to set
Paperclip::Attachment.default_options[:url] = ":s3_domain_url"
And that I also have to set the :path parameter or I will just get a Paperclip::Errors::InfiniteInterpolationError.
So I set my config file with both combined:
Paperclip::Attachment.default_options[:path] = ":class/:attachment/:id_partition/:style/:filename"
Paperclip::Attachment.default_options[:url] = ":s3_domain_url"
Not working... I try scrapping the paperclip.rb and putting it in config/environments/* But no matter what I do, it still saves the URLs without the domain with the bucket name in the path.
So two questions:
1) How can I get Paperclip to automatically format the saved URLs in domain style?
2) Or even better, how can I get S3 to accept the non-domain style URLs, the one that Paperclip is currently generating?
EDIT
So, if I add in the s3_host_name option then it saves the URLs domain style. So I have to have all 3 of:
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = ":class/:attachment/:id_partition/:style/:filename"
Paperclip::Attachment.default_options[:s3_host_name] = 's3-us-west-2.amazonaws.com'
And it will save my URLs on the model like so:
http://mybucket-development.s3-us-west-2.amazonaws.com/profiles/avatars/000/000/026/original/image_file_name.png%3F1420580224
But now I see that I have a %3F encoding ("?") in the URL which messes it up.

Alright, so as mentioned in the above update, to get the domain-style URLs to be saved by Paperclip I have to include all 3 of the following in my paperclip.rb:
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = ":class/:attachment/:id_partition/:style/:filename"
Paperclip::Attachment.default_options[:s3_host_name] = 's3-us-west-2.amazonaws.com'
I believe there is a related issue from recent gem upgrades, this produces URLs with encodings that won't work on their own.
So in my views I have had to add URI.unescape, such as
=image_tag URI.unescape(profile.avatar.url(:medium))
I could also set a callback on the model to replace the %3F with the "?" before save.
Strange issue with Paperclip... not sure what was going on. First app I've worked on where I encountered that issue.

In paperclip.rb
Paperclip::Attachment.default_options[:s3_host_name] = 's3-ap-south-1.amazonaws.com'
In production.rb
config.paperclip_defaults = {
storage: :s3,
s3_protocol: :https,
s3_credentials: {
bucket: ENV.fetch('S3_BUCKET_NAME'),
access_key_id: ENV.fetch('AWS_ACCESS_KEY_ID'),
secret_access_key: ENV.fetch('AWS_SECRET_ACCESS_KEY'),
s3_region: ENV.fetch('AWS_REGION')
}
This pretty much worked well for me with image_tag. It should work for you too.

Related

Certificate for this server is invalid with S3 and Paperclip

I'm working in a Rails 5 application which is deployed by the moment on Heroku. I'm using Postgrsql for data storage, Paperclip to manage image uploads and AWS S3 to store all the uploaded images.
To accomplish this I used this very detailed and useful tutorial from Heroku dev which really help me a lot.
I use the same configuration for development env to be able to test it. Actually it works like a charm in dev.
When I deploy to Heroku and after running migrations, setting up ENV variables I create a new Brochure which accepts a cover image; and everything goes good. The images are stored correctly in AWS S3.
But when I render de image in a view I wit just don't work. I got the following errors in browsers console:
Safari browser:
Failed to load resource: The certificate for this server is invalid. You might be connecting to a server that is pretending to be “sponsors.matchxperience.s3.amazonaws.com” which could put your confidential information at risk.
Chrom canary:
Failed to load resource: net::ERR_INSECURE_RESPONSE
I don't know what's the matter because in development environment everything works.
Can anybody help me out with this or any idea what's going on?
production.rb (the same in development.rb)
Rails.application.configure do
# We’ll also need to specify the AWS configuration variables for the production Environment.
config.paperclip_defaults = {
storage: :s3,
# s3_protocol: 'http',
s3_credentials: {
bucket: ENV.fetch('AWS_S3_BUCKET'),
access_key_id: ENV.fetch('AWS_ACCESS_KEY_ID'),
secret_access_key: ENV.fetch('AWS_SECRET_ACCESS_KEY'),
s3_region: ENV.fetch('AWS_REGION')
}
}
end
brochure.rb
class Brochure < ApplicationRecord
# This method associates the attribute ":cover" with a file attachment
has_attached_file :cover, styles: {
card: '500x330#',
}
# Validate the attached image is image/jpg, image/png, etc
validates_attachment_content_type :cover, :content_type => /\Aimage\/.*\Z/
end
paperclip.rb at config/initializers/
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = '/:class/:attachment/:id_partition/:style/:filename'
After searching and reading different sources for this error I found many different solutions for similar errors but no one seemed to be related with Rails 5 directly and sincerely I don't even though it has something to see with Ruby or Rails.
I was convinced it was something with the AWS S3 server and I was right, I fixed it. Finally reading the official documentation for creating a new bucket I realized that it was something ridiculously simple.
In the documentation it says we can use periods . and hyphens - in our bucket's name:
Can contain lowercase letters, numbers, periods (.), and hyphens (-).
Must start with a number or letter.
Must be between 3 and 63 characters long.
...
So, I named my bucket as:
sponsors.matchxperience
Which was right written, BUT talking about server's URLs it may confuse the browser's requests to point to a different server and that was happening in my case. That's why I got that error.
The solution
Just create another bucket (or renaming the actual one may function) and copied all the content named as:
sponsors-matchxperience
And magically it just works fine in production on Heroku. I don't know what's going on with the AWS documentation but for me, that whats the error, periods . in my bucket's name.
I hope it could be useful for someone else.

How to use Amazon S3 on Spree-2-2-stable

I need to configure my images on Amazon S3, but when I tried to do so under version 2-2-stable from Spree, I realized that this configuration were moved away from the admin panel.
I read somewhere that this configuration was creating some problems, and thus it was removed on 2-2. But I assume that the functionality is still working somehow.
When I try to add these configs to my config/initialize/spree.rb, I´m getting an error because these preferences are no longer existing.
preference :s3_access_key, :string
preference :s3_bucket, :string
preference :s3_secret, :string
These preferences are found on 2-1-stable but not on 2-2-stable
https://github.com/spree/spree/blob/2-1-stable/core/app/models/spree/app_configuration.rb
https://github.com/spree/spree/blob/2-2-stable/core/app/models/spree/app_configuration.rb
Is there any way how to get it working in order to use it together with Heroku?
Here is Spree's commit with the changes and some instruction as to how to make configuration changes.
https://github.com/spree/spree/commit/b1d6c5e4b9801d888cc76c05116b814945122207
My understanding is that you can still use paperclip to manage uploading to S3, which I have successfully done using their instructions. I have however had problems getting the save path on S3 to configure correctly. This might get you started ... in an environment config file put the following:
# Paperclip configs
config.paperclip_defaults = {
:storage => :s3,
:bucket => ENV['S3_BUCKET_NAME'],
:s3_credentials => {
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
I use environment variables for the S3 credentials, so yours will most likely be different. This chunk of code has made uploading files to S3 work, like I said I just can't force a specific file path on upload. Hope that helps.
EDIT - ADDITIONAL INFO:
I added the following to the spree.rb initializer to define a custom upload path and a custom url path.
# S3 upload path and url path configuration
Spree::Image.attachment_definitions[:attachment][:path] = 'products/:id/:style/:basename.:extension'
Spree::Image.attachment_definitions[:attachment][:url] = 'products/:id/:style/:basename.:extension'
To change default upload sizes you can override the Spree image decorator model. So under app/models add a spree directory and add a file named image_decorator.rb. You can then control the sizes with the following:
Spree::Image.class_eval do
attachment_definitions[:attachment][:styles] = {
:mini => '48x48>', # thumbs under image
:small => '350x700>', # images on category view
:product => '1024x768>', # full product image
:large => '600x600>' # light box image
}
end
Check out this page for specifics --> http://guides.spreecommerce.com/developer/logic.html
So to sum up, you can do all your general image/S3 configuration by updating your environment initializer, your Spree initializer and overriding the spree image_decorator model.

How and where paperclip via aws-sdk store URLs?

Sorry for maybe dumb question, but I have some misunderstanding about how Paperclip works. If I use it with aws-sdk gem, where image urls are stored?
My model has only fields like image_name, image_size, but not something like image_url. I also tried to list all tables from my db in sql console, but there wasn't any special image-related tables. I know what I can access url by calling my_model.image.url, but I need understanding, where this url is located.
Thanks.
The urls are auto generated based on the ruleset you have in your paperclip config file. They usually include the name and id of your model unless you have changed the defaults. Set the :default_url option when you call has_attached_file if you want to change it:
:default_url => '/images/:attachment/missing_:style.png',
:path => "avatars/:id/:style/:filename"
More background: rails paperclip default image with S3

Paperclip 2.3.5 to 3.3.1 causes url to differ from path

I am in the process of upgrading my rails app from 2.3 to 3.2, and I am having a problem with paperclip. My app was previously using paperclip v2.3.5 and now I am using rails 3.0.20 with paperclip v3.3.1
The problem happens when I try to upload an avatar to S3. It looks like paperclip is escaping the path before sending it to S3, but when I ask for the url of a specific style, the url uses an unescaped version of the path, and this leads to a "NoSuchKey" error from S3 (more like a 404 not found)
In my model I have
Paperclip.interpolates :last_modified do |attachment, style|
attachment.instance.updated_at.to_i
end
:path => "folder/:id/:style.:extension?:last_modified",
:url => ":s3_domain_url",
So the old version of my app was using urls from S3 like:
http://my-bucket.s3.amazonaws.com/folder/123/thumbnail.png?123456789
But now, everytime I upload an avatar, S3 will store the url like this:
http://my-bucket.s3.amazonaws.com/folder/123/thumbnail.png%3F123456789
and asking my model for the url of thumbnail style, will return:
http://my-bucket.s3.amazonaws.com/folder/123/thumbnail.png?123456789
which S3 can't find.
I know it seems to be an easy to fix issue, but my main concern is that the production app has a lot of users with many pictures and updating all of their images is a delicate task, and manually escaping the question mark is not a solution.
I just figured out how to fix a problem that was similar to this. There is a new escape_url option. Maybe try turning that to false?
:path => "folder/:id/:style.:extension?:last_modified",
:url => ":s3_domain_url",
:escape_url => false

Paperclip S3 - Can upload images but cannot view them

I'm created a Rails app running on Heroku, with Paperclip and S3. I've managed to upload images to my S3 bucket through the site (I can see them show up in my bucket on the Amazon control panel).
But when I add an Image tag i.e. <%= image_tag x.photo.url %>, I get the following html (tags omitted here), with no image displayed!
img alt="Test_tree" src="http://s3.amazonaws.com/hiphotos/ads/photos/000/000/015/original/test_tree.jpg?1344661020"
Why can't I see the images even though they're in the bucket?
Create a file a called paperclip initializer:
# config/initializers/paperclip.rb
# We are actually setting this to 's3_domain_url',
# so it's not a placeholder for something else.
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = '/:class/:attachment/:id_partition/:style/:filename'
Or you could also place this inside production.rb:
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
},
:url =>':s3_domain_url',
:path => '/:class/:attachment/:id_partition/:style/:filename',
}
Firstly, the url you are trying to use up there in your code is this:
http://s3.amazonaws.com/hiphotos/ads/photos/000/000/015/original/test_tree.jpg
When you visit that link in the browser, you see the following:
<message>
The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
</Message>
<RequestId>810A6AE1D141304C</RequestId>
<Bucket>hiphotos</Bucket>
<HostId>
XXZ+s+slgZLsRWy5NiU/G0yAKBLftw0oT2dDKpas532qXJEPSrISVPqfZsEgpb2J
</HostId>
<Endpoint>hiphotos.s3.amazonaws.com</Endpoint>
So if we modify the url using the correct endpoint we get this:
http://hiphotos.s3.amazonaws.com/ads/photos/000/000/015/original/test_tree.jpg
Which does return the correct image.
If you are using European buckets, this can happen, and it might be the fault of the gem you are using to push things to s3.
There are loads of articles on how to get Paperclip, S3 and European buckets to play nicely together.
I have found though, that since I started using the asset_sync gem, which uses Fog instead of aws-s3 gem, I don't have any more trouble with paperclip and S3.
So I suspect Fog has something to do with making this problem go away for me. I'd recommend switching to it, if you're using something else.

Resources