I'm trying to upload an image using paperclip and the AWS S3 storage
I've set the default configuration for paperclip
config.paperclip_defaults = {
:storage => :s3,
:bucket => '<bucket_name>',
:s3_credentials => Rails.root.join('config', 'aws.yml').to_s
}
In my model i have this
as_attached_file :img, :styles => { :medium => "300x300>", :thumbnail => "100x100>" }, :path => "assets/products/:attachment/:id/:style.:extension"
In the controller i have a simple property update because my model is first created and after the images are uploaded :
def upload_thumbnail
#product = Product.find(params[:id])
if #product.update_attribute(:img, params[:image])
render :json => { :done => true, :img_path => #product.img.url }
else
render :json => #product.errors }
end
end
When i upload an image i get in the console this error
[paperclip] Saving attachments.
[paperclip] saving assets/products/imgs/10/original.jpg
[AWS S3 200 4.134236 3 retries] put_object(:acl=>:public_read,:bucket_name=>"<bucket_name>",:content_length=>0,:content_type=>"image/jpeg",:data=>Paperclip::UploadedFilAdapter: image.jpg,:key=>"assets/products/imgs/10/original.jpg") AWS::Core::Client::NetworkError AWS::Core::Client::NetworkError
Looking in S3 the image is created at the path specify by has the size set to 0 bytes.
Also I've notice that i get this when the plugin is trying to resize the image
[paperclip] Error while determining content type: Cocaine::CommandNotFoundError
but the original image should be uploaded correct.
What i'm doing wrong ?
Related
I am using Paperclip to upload videos and keep getting a Security Validation error about the content type
The error when saving an mp4 to my model class is "content type discovered from file command: video/mp4. See documentation to allow this combination."
The save looks like this
AssignmentEventVideo.create(video: "https://s3-ap-southeast-2.amazonaws.com/dev/upload/0c857445-09ad-44b6-bbfa-810a9974a501/ScreenCaptureProject4.mp4")
The model class
class AssignmentEventVideo < ActiveRecord::Base
has_attached_file :video, :styles => {
:medium => { :geometry => "640x480", :format => 'mp4' },
:android => { :geometry => "640x480", :format => 'webm'},
:mobile => { :geometry => "300x300", :format => 'png', :time => 2 },
:thumb => { :geometry => "100x100#", :format => 'png', :time => 2 }
}
validates_attachment_content_type :video, content_type: ['video/mp4']
end
If have tried disabling validation all together with the code below but it still throws the error
do_not_validate_attachment_file_type :video
I have confirmed that the file command is return the correct type with
file -b --mime ScreenCaptureProject3.mp4
which returns
video/mp4; charset=binary
The save is working fine for another model class that accepts images and checks content using
validates_attachment_content_type :photo, content_type: /\Aimage\/.*\Z/
I'm not sure where to turn next - except to recreate the class and change the column name to something that doesn't clash with video?
Hope someone can help!
Thanks katafrakt - you got me on the right path.
I was using a presigned_post and uploading to S3 using JQuery FileUploader. This was not setting the Content-Type and I was getting back a binary/octet type that Paperclip didn't know how to deal with.
I set content_type on the presigned post, which stores the right meta data in S3 and all is well.
I have a list of products. For each product, I have one image_url that I want to upload to my AWS S3 account and associate to it.
I am able to do it manually via the form, but with ~1500 products, I can't do it one by one.
My Paperclip config looks like this :
config.paperclip_defaults = {
:url => 'xxxx',
:path => '/:class/:attachment/:id_partition/:style/:filename',
:command_path => "/usr/bin/convert",
:storage => :s3,
:s3_credentials => {
:bucket => "xxx",
:access_key_id => "XXX",
:secret_access_key => "XXXX"
}
}
Here is the code I use to upload manually my image
class Product < ActiveRecord::Base
# This method associates the attribute ":image" with a file attachment
has_attached_file :image, styles: {
thumb: '100x100>',
square: '200x200#',
medium: '300x300>'
}
# Validate the attached image is image/jpg, image/png, etc
validates_attachment_content_type :image, :content_type => /\Aimage\/.*\Z/
end
Everything goes well, I can see at the end of the log :
[paperclip] saving /products/images/000/001/361/original/xxx.jpg
[AWS S3 200 1.107584 0 retries] put_object(:acl=>:public_read,:bucket_name=>"xxx",:content_length=>47766,:content_type=>"image/jpeg",:data=>Paperclip::UploadedFileAdapter: xxx.jpg,:key=>"products/images/000/001/361/original/xxx.jpg")
[paperclip] saving /products/images/000/001/361/thumb/xxx.jpg
[AWS S3 200 0.444224 0 retries] put_object(:acl=>:public_read,:bucket_name=>"xxx",:content_length=>26502,:content_type=>"image/jpeg",:data=>Paperclip::FileAdapter: 3ef480ec0ebbef3921b4c074897fecc320150625-13674-roduas,:key=>"products/images/000/001/361/thumb/xxx.jpg")
[paperclip] saving /products/images/000/001/361/square/xxx.jpg
[AWS S3 200 0.492781 0 retries] put_object(:acl=>:public_read,:bucket_name=>"xxx",:content_length=>36071,:content_type=>"image/jpeg",:data=>Paperclip::FileAdapter: 3ef480ec0ebbef3921b4c074897fecc320150625-13674-1xrcndz,:key=>"products/images/000/001/361/square/xxx.jpg")
[paperclip] saving /products/images/000/001/361/medium/xxx.jpg
[AWS S3 200 0.553079 0 retries] put_object(:acl=>:public_read,:bucket_name=>"xxx",:content_length=>43927,:content_type=>"image/jpeg",:data=>Paperclip::FileAdapter: 3ef480ec0ebbef3921b4c074897fecc320150625-13674-8dq1h4,:key=>"products/images/000/001/361/medium/xxx.jpg")
But now I want to automate this task (when seeding the database). I added open-uri to retrieve image from their URL.
require 'open-uri'
class Product < ActiveRecord::Base
# This method associates the attribute ":image" with a file attachment
has_attached_file :image, styles: {
thumb: '100x100>',
square: '200x200#',
medium: '300x300>'
}
# Validate the attached image is image/jpg, image/png, etc
validates_attachment_content_type :image, :content_type => /\Aimage\/.*\Z/
def image_from_url(url)
self.image = URI.escape(url)
end
end
What I basically do is to create a new Product with something like :
Product.create! :name => "Test!!!"
Then try to add image to it :
Product.last.image_from_url "MY_URL"
I have no error with that but it does not work. I can see on the log the conversion for each format (thumb, square, medium) but it is not push to AWS like before.
I found the solution to my question just after finishing to write it.
Maybe it can help someone so I posted it anyway.
The error was that I didn't update the attribute image. So instead of doing :
Product.last.image_from_url "MY_URL"
I did :
Product.last.update_attribute :image, Product.last.image_from_url("MY_URL")
Now the image is generated AND uploaded to AWS
I am using Rails 3.2 + Heroku + S3 + Paperclip to store an icon on my User model. The model is not saving the 4 icon fields though. The images are getting processed and saved on S3 correctly and no errors are occurring. I also have another model that has a document being stored via Paperclip and S3. That model works perfectly in all cases. The User icon works locally but not on Heroku.
production.rb relevant configuration
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['AWS_BUCKET'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
User model code:
class User < ActiveRecord::Base
attr_accessible :icon
has_attached_file :icon, :url => "/system/:rails_env/:attachment/:style/:hash.:extension",
:hash_data => ":class/:attachment/:id",
:hash_secret => "superSecretThing",
:styles => { :medium => "300x300>", :thumb => "100x100>" },
:default_url => "/blank.png"
...
Controller code: (This code is kind of crazy because I am AJAXing files Base64 encoded.)
params[:user][:icon_data]
decoded_file = Base64.decode64(data)
begin
split_name = params[:user][:icon_file_name].split(".")
file = Tempfile.new([split_name[0..-2].join("."), ".#{split_name[-1]}"])
file.binmode
file.write(decoded_file)
file.close
#user.icon = open(file)
#user.icon_file_name = params[:user][:icon_file_name]
ensure
file.unlink
end
#user.save
I do an almost identical process on another model with a Paperclip attachment and it works flawlessly. In both cases the attachment is being saved correctly to S3 and no errors are being raised. This gist has example output for a controller action from the Heroku logs.
I am pretty baffled because the other model works fine. The only real difference is that the User attachment does image processing but that part appears to be working fine.
The problem is the same as this one, but the solution there does not apply.
Thoughts?
So the problem is that not including the :path argument makes it try to use the :url parameter for both the url and the path. The real fix is to include the :path parameter in addition to the url.
So for example a fixed configuration that works both locally and on Heroku:
has_attached_file :icon,
:url => "/system/:rails_env/:attachment/:style/:hash.:extension",
:path => "public/system/:rails_env/:attachment/:style/:hash.:extension",
:hash_data => ":class/:attachment/:id",
:hash_secret => "superDuperSecret",
:styles => { :medium => "300x300>", :thumb => "100x100>" },
:default_url => "/blank.png"
I'm using Paperclip 2.3.5 within a Rails app to store PDF documents on Amazon S3. For every PDF a JPG thumbnail is generated by ImageMagick. Im' using this configuration in the model:
has_attached_file :file,
:styles => { :thumb => { :geometry => "200x200>",
:format => :jpg
} },
:whiny => false,
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:s3_permissions => 'authenticated-read',
:s3_headers => { 'Expires' => 1.year.from_now.httpdate },
:url => "s3.amazonaws.com",
:path => "documents/:id/:style/:basename.:extension",
:bucket => 'mybucket'
But there is problem: The generated thumbnail is uploaded to S3 with the content_type "application/pdf", which is WRONG, because it's a JPG (you can see the content_type of a file on S3 with a S3 exploring tool like Cyberduck). For the original PDF file this content_type is correct, but not for the thumbnail. This causes trouble in some browsers (e.g. Chrome or Safari) which don't show the thumbnail inline.
Beware: The content_type stored in my database (field "file_content_type") is "application/pdf", which is still correct, because it's the content_type for the original file.
How can I override the content_type for a thumbnail if it should be different from the original file?
This is how we fixed it on brighterplanet.com/research, which has pdf documents and png previews:
has_attached :pdf_document,
:storage => :s3,
# [... other settings ...]
# PDFs work better in Windows 7 / IE if you give them content-type: attachment
:s3_headers => { 'Content-Disposition' => 'attachment' },
:styles => { :preview => { :geometry => '135', :format => :png } }
after_save :fix_thumbnail
def fix_thumbnail(force = false)
# application/pdf and application/x-pdf have both been seen...
return unless force or pdf_document_content_type.include?('pdf')
# set content type and disposition
s3 = AWS::S3.new(YAML.load(File.read("#{RAILS_ROOT}/config/aws_s3.yml")))
t = s3.buckets[PAPERCLIP_BUCKET].objects[pdf_document.path(:thumbnail)]
content = t.read
t.write(:data => content, :content_type => 'image/png', :content_disposition => 'inline', :acl => :public_read)
nil
end
I had to overcome this, not the most elegant solution but I forked Paperclip and hold the patch in my own git repo - https://github.com/svetzal/paperclip
It is a direct replacement for Paperclip, just put in your environment.rb
gem 'twm_paperclip', :lib => 'paperclip'
This is fixed in paperclip >= 2.7, as you can see here:
https://github.com/thoughtbot/paperclip/blob/v2.7/lib/paperclip/storage/s3.rb#L290
the mime-type of the file that is written to S3 is determined specifically before uploading.
class User < ActiveRecord::Base
has_attached_file :photo, :styles => { :square => "100%", :large => "100%" },
:convert_options => {
:square => "-auto-orient -geometry 70X70#",
:large => "-auto-orient -geometry X300" },
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:path => ":attachment/:id/:style.:extension",
:bucket => 'mybucket'
validates_attachment_size :photo,
:less_than => 5.megabyte
end
Works great on local machine, but gives me an error on Heroku: There was an error processing the thumbnail for stream.20143
The thing is I want to auto-orient photos before resizing, so they resized properly.
The only working variant now(thanks to jonnii) is resizing without auto-orient:
...
as_attached_file :photo, :styles => { :square => "70X70#", :large => "X300" },
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:path => ":attachment/:id/:style.:extension",
:bucket => 'mybucket'
...
How to pass additional convert options to paperclip on Heroku?
UPD
I discover, the trouble in "-auto-orient" option. It seems like this option is broken in version of ImageMagick used by Heroku. I created custom paperclip image processor inherited from paperclip's standard thumbnail:
module Paperclip
class Ao < Thumbnail
def transformation_command
super + " -auto-orient"
end
end
end
It works perfect on local machine, but fails on Heroku.
These are the sizes I use. They all work fine on heroku:
SIZES = {
:original => "640x480>",
:thumb => "150x150#",
:mini => "60x60#",
:micro => "30x30#"
}
Make sure your gem version of paperclip is the same as heroku's. You can specify the specific gem version in your .gems file and in your environment.rb to make sure they line up.
I'm not sure exactly why your convert_options are causing problems, but if I remember correctly paperclip uses ImageScience directly and your chosen options might be incompatible with the read only heroku file system.
If this is critical and you need an answer right now I'd raise a support ticket on heroku. If you get a response make sure you post it back here!