Upload image from URL with Paperclip to AWS S3 - ruby-on-rails

I have a list of products. For each product, I have one image_url that I want to upload to my AWS S3 account and associate to it.
I am able to do it manually via the form, but with ~1500 products, I can't do it one by one.
My Paperclip config looks like this :
config.paperclip_defaults = {
:url => 'xxxx',
:path => '/:class/:attachment/:id_partition/:style/:filename',
:command_path => "/usr/bin/convert",
:storage => :s3,
:s3_credentials => {
:bucket => "xxx",
:access_key_id => "XXX",
:secret_access_key => "XXXX"
}
}
Here is the code I use to upload manually my image
class Product < ActiveRecord::Base
# This method associates the attribute ":image" with a file attachment
has_attached_file :image, styles: {
thumb: '100x100>',
square: '200x200#',
medium: '300x300>'
}
# Validate the attached image is image/jpg, image/png, etc
validates_attachment_content_type :image, :content_type => /\Aimage\/.*\Z/
end
Everything goes well, I can see at the end of the log :
[paperclip] saving /products/images/000/001/361/original/xxx.jpg
[AWS S3 200 1.107584 0 retries] put_object(:acl=>:public_read,:bucket_name=>"xxx",:content_length=>47766,:content_type=>"image/jpeg",:data=>Paperclip::UploadedFileAdapter: xxx.jpg,:key=>"products/images/000/001/361/original/xxx.jpg")
[paperclip] saving /products/images/000/001/361/thumb/xxx.jpg
[AWS S3 200 0.444224 0 retries] put_object(:acl=>:public_read,:bucket_name=>"xxx",:content_length=>26502,:content_type=>"image/jpeg",:data=>Paperclip::FileAdapter: 3ef480ec0ebbef3921b4c074897fecc320150625-13674-roduas,:key=>"products/images/000/001/361/thumb/xxx.jpg")
[paperclip] saving /products/images/000/001/361/square/xxx.jpg
[AWS S3 200 0.492781 0 retries] put_object(:acl=>:public_read,:bucket_name=>"xxx",:content_length=>36071,:content_type=>"image/jpeg",:data=>Paperclip::FileAdapter: 3ef480ec0ebbef3921b4c074897fecc320150625-13674-1xrcndz,:key=>"products/images/000/001/361/square/xxx.jpg")
[paperclip] saving /products/images/000/001/361/medium/xxx.jpg
[AWS S3 200 0.553079 0 retries] put_object(:acl=>:public_read,:bucket_name=>"xxx",:content_length=>43927,:content_type=>"image/jpeg",:data=>Paperclip::FileAdapter: 3ef480ec0ebbef3921b4c074897fecc320150625-13674-8dq1h4,:key=>"products/images/000/001/361/medium/xxx.jpg")
But now I want to automate this task (when seeding the database). I added open-uri to retrieve image from their URL.
require 'open-uri'
class Product < ActiveRecord::Base
# This method associates the attribute ":image" with a file attachment
has_attached_file :image, styles: {
thumb: '100x100>',
square: '200x200#',
medium: '300x300>'
}
# Validate the attached image is image/jpg, image/png, etc
validates_attachment_content_type :image, :content_type => /\Aimage\/.*\Z/
def image_from_url(url)
self.image = URI.escape(url)
end
end
What I basically do is to create a new Product with something like :
Product.create! :name => "Test!!!"
Then try to add image to it :
Product.last.image_from_url "MY_URL"
I have no error with that but it does not work. I can see on the log the conversion for each format (thumb, square, medium) but it is not push to AWS like before.

I found the solution to my question just after finishing to write it.
Maybe it can help someone so I posted it anyway.
The error was that I didn't update the attribute image. So instead of doing :
Product.last.image_from_url "MY_URL"
I did :
Product.last.update_attribute :image, Product.last.image_from_url("MY_URL")
Now the image is generated AND uploaded to AWS

Related

Getting URL for public assets in Rails model

I have a Rails application that uses Paperclip and saves images to S3. When the user uploads an asset without an image, it gets the default image set in the Paperclip setup.
My API serves those assets and has the links to the images in the JSON response (using jbuilder), however I can't seem to return the default image URL, it only returns "missing.png" and I wanted it to return the entire URL to the server with the missing image path attached to it.
I'm setting the default url in the model, and I've tried using ActionView::Helpers::AssetUrlHelper to get the image_url but it never works even though it is working inside the rails console. Any idea on what can I do to solve it?
The JBuilder file:
json.profile_picture_smallest asset.profile_picture.url(:smallest)
json.profile_picture_small asset.profile_picture.url(:small)
json.profile_picture_medium asset.profile_picture.url(:medium)
json.profile_picture_large asset.profile_picture.url(:large)
json.profile_picture_original asset.profile_picture.url(:original)
The part of paperclip that is included in the Model
module Picturable
extend ActiveSupport::Concern
included do
has_attached_file :profile_picture, path: '/images/' + name.downcase.pluralize + '/:style/:basename', default_url: "missing.png",
styles: {
smallest: '50x50>',
small: '100x100>',
medium: '200x200>',
large: '400x400>',
png: ['400x400>',:png]
}, :convert_options => {
smallest: '-trim',
small: '-trim',
medium: '-trim',
large: '-trim',
png: '-trim'
}
# Validate the attached image is image/jpg, image/png, etc
validates_attachment_content_type :profile_picture, :content_type => /\Aimage\/.*\Z/
end
def set_uuid_name
begin
self.profile_picture_file_name = SecureRandom.uuid
end while self.class.find_by(:profile_picture_file_name => self.profile_picture_file_name)
end
end
Paperclip config:
Paperclip::Attachment.default_options[:s3_host_name] = 's3hostname'
Development config:
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => 'paperclipdev',
:access_key_id => 'accesskey',
:secret_access_key => 'secretaccesskey'
}
}
I think the way to do this is use the asset helpers in your jbuilder file:
json.profile_picture_smallest asset_url(asset.profile_picture.url(:smallest))
It's worth a mention here that you can also pass a symbol method name to paperclip for the default_url parameter if you want the default url to be dynamic based on the model.

Paperclip, S3, Heroku: Missing Image

Long time viewer, first time asker. I've searched for this topic but don't believe I've found the answer.
I have a Post model that has an image. I'm using the Paperclip gem, saving to Amazon S3, and hosting on Heroku.
The file upload form works fine, because I can see that images are sent to my S3 bucket.
The issue is that, the images don't actually show in production.
Here's my model:
class Post < ActiveRecord::Base
attr_accessor :image_file_name, :image_content_type, :image_file_size, :image_updated_at
belongs_to :user
has_many :reviews
has_attached_file :image, styles: { medium: "700x500#", small: "350x250>" }
validates_attachment_content_type :image, content_type: /\Aimage\/.*\Z/
And here's my config/production.rb:
# Required for Paperclip / AWS
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
And here's my show.html.haml file:
.clearfix
.post_image_description
= image_tag #post.image.url(:medium)
.description= simple_format(#post.address)
.description= simple_format(#post.description)
Shouldn't the #post.image.url be enough? What may I be missing to properly route to the image?
This is what I see when I pull Heroku Logs:
2015-06-23T15:38:26.181383+00:00 app[web.1]: ActionController::RoutingError (No route matches [GET] "/images/medium/missing.png"):
For reference, here's my repository for the project: https://github.com/lucasvocos/pitstop
Please let me know if there is anything else to provide in the question, too. As this is my first time asking. Thanks everyone.
I had a similar problem that the upload was working but the display was showing a broken link. I checked the source for the link and it was not pointing to the url shown for the image on S3. I had to add the host name to my paperclip config
config.paperclip_defaults = {
:storage => :s3,
:s3_host_name => 's3-us-west-2.amazonaws.com',
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
You need to set heroku environment variable for s3 bucket variable:
heroku config:set S3_BUCKET_NAME='Your Bucket Name'
heroku config:set AWS_ACCESS_KEY_ID='Your AWS ID'
heroku config:set AWS_SECRET_ACCESS_KEY='Your AWS Secrete Key'
You have to commit the missing.png in \public\images\medium.(make sure \public\images\medium\missing.png exist, then commit). This images is the default until you upload some valid image.
However, also is recommended to define the url in model, something like this:
has_attached_file :image, styles: {
medium: "700x500#",
small: "350x250>" }, :url => '/:class/:attachment/:id/:style_:basename.:extension'

Paperclip post processing conversion does not update file_name, content_type attributes

I would like to restrict File uploads to images only, and convert them automatically to .png. To do so, I use this class:
class ImageAttachment < ActiveRecord::Base
attr_accessible :file, :file_file_name, :file_content_type, :file_file_size
validates_attachment :file,
:content_type => { :content_type => ["image/jpg", "image/tiff", "image/png"] }
has_attached_file :file,
:styles => { :original => ["100%", :png],
:large => ["500x500", :png],
:medium => ["150x150", :png],
:thumb => ["75x100", :png]
},
:default_url => "/system/missing_thumb.png"
end
As I understand, the :styles => { :original => ["100%", :png], ...} should convert all uploaded files that pass validation to .png files. Therefore, I expect the following things to happen when uploading a file example.tiff:
convert the file to .png
change the file name accordingly to example.png
change the content type accordingly to "image/png"
Here's a spec I use:
it "should convert all image types to .png" do
test_file = File.new(Rails.root + "spec/fixtures/images/test.tiff")
attachment = ImageAttachment.create :file => test_file
attachment.file.url.should == "some/paperclip/path/.../test.png"
attachment.file_file_name.should == "test.png"
attachment.file_content_type.should == "image/png"
end
The first assertion is true, and I can also see ImageMagick output in the terminal,
but attachment.file_file_name still returns example.tiff, and attachment.file_content_type returns "image/tiff".
Is my assumption that paperclip automatically updates the file_file_name and the file_content_type attributes wrong?
If so, how would I best do this on my own?

Paperclip tries to upload image with content length 0

I'm trying to upload an image using paperclip and the AWS S3 storage
I've set the default configuration for paperclip
config.paperclip_defaults = {
:storage => :s3,
:bucket => '<bucket_name>',
:s3_credentials => Rails.root.join('config', 'aws.yml').to_s
}
In my model i have this
as_attached_file :img, :styles => { :medium => "300x300>", :thumbnail => "100x100>" }, :path => "assets/products/:attachment/:id/:style.:extension"
In the controller i have a simple property update because my model is first created and after the images are uploaded :
def upload_thumbnail
#product = Product.find(params[:id])
if #product.update_attribute(:img, params[:image])
render :json => { :done => true, :img_path => #product.img.url }
else
render :json => #product.errors }
end
end
When i upload an image i get in the console this error
[paperclip] Saving attachments.
[paperclip] saving assets/products/imgs/10/original.jpg
[AWS S3 200 4.134236 3 retries] put_object(:acl=>:public_read,:bucket_name=>"<bucket_name>",:content_length=>0,:content_type=>"image/jpeg",:data=>Paperclip::UploadedFilAdapter: image.jpg,:key=>"assets/products/imgs/10/original.jpg") AWS::Core::Client::NetworkError AWS::Core::Client::NetworkError
Looking in S3 the image is created at the path specify by has the size set to 0 bytes.
Also I've notice that i get this when the plugin is trying to resize the image
[paperclip] Error while determining content type: Cocaine::CommandNotFoundError
but the original image should be uploaded correct.
What i'm doing wrong ?

Rails, PaperClip, S3, Heroku: Model icon fields not being saved

I am using Rails 3.2 + Heroku + S3 + Paperclip to store an icon on my User model. The model is not saving the 4 icon fields though. The images are getting processed and saved on S3 correctly and no errors are occurring. I also have another model that has a document being stored via Paperclip and S3. That model works perfectly in all cases. The User icon works locally but not on Heroku.
production.rb relevant configuration
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['AWS_BUCKET'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
User model code:
class User < ActiveRecord::Base
attr_accessible :icon
has_attached_file :icon, :url => "/system/:rails_env/:attachment/:style/:hash.:extension",
:hash_data => ":class/:attachment/:id",
:hash_secret => "superSecretThing",
:styles => { :medium => "300x300>", :thumb => "100x100>" },
:default_url => "/blank.png"
...
Controller code: (This code is kind of crazy because I am AJAXing files Base64 encoded.)
params[:user][:icon_data]
decoded_file = Base64.decode64(data)
begin
split_name = params[:user][:icon_file_name].split(".")
file = Tempfile.new([split_name[0..-2].join("."), ".#{split_name[-1]}"])
file.binmode
file.write(decoded_file)
file.close
#user.icon = open(file)
#user.icon_file_name = params[:user][:icon_file_name]
ensure
file.unlink
end
#user.save
I do an almost identical process on another model with a Paperclip attachment and it works flawlessly. In both cases the attachment is being saved correctly to S3 and no errors are being raised. This gist has example output for a controller action from the Heroku logs.
I am pretty baffled because the other model works fine. The only real difference is that the User attachment does image processing but that part appears to be working fine.
The problem is the same as this one, but the solution there does not apply.
Thoughts?
So the problem is that not including the :path argument makes it try to use the :url parameter for both the url and the path. The real fix is to include the :path parameter in addition to the url.
So for example a fixed configuration that works both locally and on Heroku:
has_attached_file :icon,
:url => "/system/:rails_env/:attachment/:style/:hash.:extension",
:path => "public/system/:rails_env/:attachment/:style/:hash.:extension",
:hash_data => ":class/:attachment/:id",
:hash_secret => "superDuperSecret",
:styles => { :medium => "300x300>", :thumb => "100x100>" },
:default_url => "/blank.png"

Resources