Paperclip on S3 is changing file extensions from a url - ruby-on-rails

I wrote a rake task that downloads an image from wikipedia given a celebrity name, but for some reason when storing on S3 the file extension is either being dropped or changed to .txt
The file otherwise is correct.
Any ideas?
From my celeb model:
has_attached_file :pic,
:styles => { :medium => "300x300>", :thumb => "100x100>" },
:default_style => :medium,
:storage => :s3,
:s3_credentials => "#{Rails.root}/config/s3.yml",
:path => "/:style/:img_name.:extension"
From my rake task:
desc "Update celeb pics from wiki"
task :update_celeb_pics => [:environment] do
include ApplicationHelper
Celeb.all.each do |celeb|
if !celeb.name.start_with?("(")
puts celeb.name
celeb.pic = open(getImage(celeb.name))
celeb.save
end
end
end
the getImage method is a helper that returns a string
require 'open-uri'
require 'uri'
module ApplicationHelper
def getInfo(name)
Nokogiri::XML(open(URI.escape("http://en.wikipedia.org/w/api.php?action=opensearch&search=#{name}&limit=1&namespace=0&format=xml")))
end
def nokoPage(name)
Nokogiri::XML(open(getURL(name)))
end
def getImage(name)
"http:" + nokoPage(name).css("table img")[0].attribute("src").value if !nokoPage(name).css("table img").empty?
end
def getDescription(name)
getInfo(name).css("Description").text
end
def getURL(name)
getInfo(name).css("Url").text
end
def getBday(name)
bday = nokoPage(name).css("span.bday")
return Date.parse(bday[0].text) if !bday.empty?
return Date.today
end
def getDday(name)
dday = nokoPage(name).css("span.dday")
return Date.parse(dday[0].text) if !dday.empty?
end
end

This is because
self.pic = open("http://something.com/bla/image.png")
is not the best solution here. Just yesterday, i got a Pull Request merged into Paperclip that lets you do this
self.pic = URI.parse(getImage(name))
This will ensure that your pic's content type matches the downloaded file, pic's filename is set to the name of the file downloaded.
The reason you get txt extension is because open returns a StringIO object which infact names the filename as "stringio.txt". Your filename is probably changed by the s3 code but the extension remains as '.txt'
I suggest you link your Gemfile to paperclip's github repo, run bundle and try again.
Cheers,
Aditya

Related

Why doesn't work Paperclip interpolations?

I am trying to make work Paperclip interpolations the whole afternoon, but still not success.
Here is how I have set up the Image model:
has_attached_file :image,
:styles => { :thumb => '300x300#',
:medium => "300x300>",
:original => "900x900>" },
:path => ":rails_root/public/images/:user_id/:style/:basename.:extension",
:url => "/images/:user_id/:style/:basename.:extension"
In /config/initializers/paperclip.rb is following:
Paperclip.options[:command_path] = "/usr/local/bin"
module Paperclip
module Interpolations
def user_id attachment, style_name
attachment.instance.user_id.to_s
end
end
end
But every time I save a file, the files is saved as
/images//original/file-name.jpg
The user's ID is missing.
What is wrong in this sample? I still cannot find the right config of Paperclip setup. I would be very grateful for every help.
Thank you
Try to add this in your Image model
Paperclip.interpolates :user_id do |attachment, style|
attachment.instance.user_id.to_s
end

Paperclip save remote images without extension

Here I have Controller
require 'open-uri'
user = User.new
url = "some_remote_image.jpg" #remote image WITH extension
user.image = open(url)
user.save
Model
has_attached_file :image,
:styles => { :thumb => "25x25>", :large => "1000x1000>" },
:path => ":rails_root/images/users/:id/:style/:hash.:extension",
:url => "/images/users/:id/:style/:hash.:extension",
:hash_secret => "hash string"
This work, but images stores without extension, for ex. "some_remote_image."
If uploading images by post form everything uploading WITH extension.
I'm confused.
I solved it by updating Paperclip to last github version and set image like this instead of user.image = open(url)
user.image = URI.parse(url)
In case if someone wants reverse thing - add extension to no extension files
def besfore_save
tempfile = data.queued_for_write[:original]
unless tempfile.nil?
extension = File.extname(tempfile.original_filename)
if !extension || extension == ''
mime = tempfile.content_type
ext = Rack::Mime::MIME_TYPES.invert[mime]
self.data.instance_write :file_name, "#{tempfile.original_filename}#{ext}"
end
end
true
end

Rails 3 - Tempfile Path?

I have the following:
attachments.each do |a|
Rails.logger.info a.filename
tempfile = Tempfile.new("#{a.filename}", "#{Rails.root.to_s}/tmp/")
Rails.logger.info tempfile.path
end
Where attachments is from paperclip.
Here's the output:
billgates.jpg
/Users/bhellman/Sites/cline/tmp/billgates.jpg20101204-17402-of0u9o-0
Why is the file name getting 20101204-17402-of0u9o-0 appended to at the end? That's breaking everything with paperclip etc. Anyone seen this before? For the life of I have no idea what's doing it?
Thanks
UPDATE
Paperclip: Paperclip on github
a is the attachment file
tempfile = Tempfile.new("#{a.filename}", "#{Rails.root.to_s}/tmp/")
tempfile << a.body
tempfile.puts
attachments.build(
:attachment => File.open(tempfile.path)
)
best make sure your tempfile has the correct extension, saving you to trying and change it after:
file = Tempfile.new(['hello', '.jpg'])
file.path # => something like: "/tmp/hello2843-8392-92849382--0.jpg"
more here: http://apidock.com/ruby/v1_9_3_125/Tempfile/new/class
The first argument for Tempfile.new is just a basename. To make sure each Tempfile is unique the characters are appended to the end of the file.
You should use Paperclip's API for this:
tempfiles = []
attachments.each do |a|
# use Attachment#to_file to get a :filesystem => file, :s3 => tempfile
tempfiles << a.to_file
end
tempfiles.each do |tf|
Rails.logger.debug tf.filename
end
attachment = attachments.build(
:attachment => File.open(tempfile.path)
)
# change the displayed file name stored in the db record here
attachment.attachment_file_name = a.filename # or whatever else you like
attachment.save!
The best way I found to deal with this was to specify the file extension in the Paperclip attribute. For example:
has_attached_file :picture,
:url => "/system/:hash.jpg",
:hash_secret => "long_secret_string",
:storage => :s3,
:s3_credentials => "#{Rails.root}/config/s3.yml"
Note that the :url is declared as '.jpg' rather than the traditional .:extension.
Good luck!

How can I set paperclip's storage mechanism based on the current Rails environment?

I have a rails application that has multiple models with paperclip attachments that are all uploaded to S3. This app also has a large test suite that is run quite often. The downside with this is that a ton of files are uploaded to our S3 account on every test run, making the test suite run slowly. It also slows down development a bit, and requires you to have an internet connection in order to work on the code.
Is there a reasonable way to set the paperclip storage mechanism based on the Rails environment? Ideally, our test and development environments would use the local filesystem storage, and the production environment would use S3 storage.
I'd also like to extract this logic into a shared module of some kind, since we have several models that will need this behavior. I'd like to avoid a solution like this inside of every model:
### We don't want to do this in our models...
if Rails.env.production?
has_attached_file :image, :styles => {...},
:path => "images/:uuid_partition/:uuid/:style.:extension",
:storage => :s3,
:url => ':s3_authenticated_url', # generates an expiring url
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:s3_permissions => 'private',
:s3_protocol => 'https'
else
has_attached_file :image, :styles => {...},
:storage => :filesystem
# Default :path and :url should be used for dev/test envs.
end
Update: The sticky part is that the attachment's :path and :url options need to differ depending on which storage system is being used.
Any advice or suggestions would be greatly appreciated! :-)
I like Barry's suggestion better and there's nothing keeping you from setting the variable to a hash, that can then be merged with the paperclip options.
In config/environments/development.rb and test.rb set something like
PAPERCLIP_STORAGE_OPTIONS = {}
And in config/environments/production.rb
PAPERCLIP_STORAGE_OPTIONS = {:storage => :s3,
:s3_credentials => "#{Rails.root}/config/s3.yml",
:path => "/:style/:filename"}
Finally in your paperclip model:
has_attached_file :image, {
:styles => {:thumb => '50x50#', :original => '800x800>'}
}.merge(PAPERCLIP_STORAGE_OPTIONS)
Update: A similar approach was recently implemented in Paperclip for Rails 3.x apps. Environment specific settings can now be set with config.paperclip_defaults = {:storage => :s3, ...}.
You can set global default configuration data in the environment-specific configuration files. For example, in config/environments/production.rb:
Paperclip::Attachment.default_options.merge!({
:storage => :s3,
:bucket => 'wheresmahbucket',
:s3_credentials => {
:access_key_id => ENV['S3_ACCESS_KEY_ID'],
:secret_access_key => ENV['S3_SECRET_ACCESS_KEY']
}
})
After playing around with it for a while, I came up with a module that does what I want.
Inside app/models/shared/attachment_helper.rb:
module Shared
module AttachmentHelper
def self.included(base)
base.extend ClassMethods
end
module ClassMethods
def has_attachment(name, options = {})
# generates a string containing the singular model name and the pluralized attachment name.
# Examples: "user_avatars" or "asset_uploads" or "message_previews"
attachment_owner = self.table_name.singularize
attachment_folder = "#{attachment_owner}_#{name.to_s.pluralize}"
# we want to create a path for the upload that looks like:
# message_previews/00/11/22/001122deadbeef/thumbnail.png
attachment_path = "#{attachment_folder}/:uuid_partition/:uuid/:style.:extension"
if Rails.env.production?
options[:path] ||= attachment_path
options[:storage] ||= :s3
options[:url] ||= ':s3_authenticated_url'
options[:s3_credentials] ||= File.join(Rails.root, 'config', 's3.yml')
options[:s3_permissions] ||= 'private'
options[:s3_protocol] ||= 'https'
else
# For local Dev/Test envs, use the default filesystem, but separate the environments
# into different folders, so you can delete test files without breaking dev files.
options[:path] ||= ":rails_root/public/system/attachments/#{Rails.env}/#{attachment_path}"
options[:url] ||= "/system/attachments/#{Rails.env}/#{attachment_path}"
end
# pass things off to paperclip.
has_attached_file name, options
end
end
end
end
(Note: I'm using some custom paperclip interpolations above, like :uuid_partition, :uuid and :s3_authenticated_url. You'll need to modify things as needed for your particular application)
Now, for every model that has paperclip attachments, you just have to include this shared module, and call the has_attachment method (instead of paperclip's has_attached_file)
An example model file: app/models/user.rb:
class User < ActiveRecord::Base
include Shared::AttachmentHelper
has_attachment :avatar, :styles => { :thumbnail => "100x100>" }
end
With this in place, you'll have files saved to the following locations, depending on your environment:
Development:
RAILS_ROOT + public/attachments/development/user_avatars/aa/bb/cc/aabbccddeeff/thumbnail.jpg
Test:
RAILS_ROOT + public/attachments/test/user_avatars/aa/bb/cc/aabbccddeeff/thumbnail.jpg
Production:
https://s3.amazonaws.com/your-bucket-name/user_avatars/aa/bb/cc/aabbccddeeff/thumbnail.jpg
This does exactly what I'm looking for, hopefully it'll prove useful to someone else too. :)
-John
How about this:
Defaults are established in application.rb. The default storage of :filesystem is used, but the configuration for s3 is initialized
Production.rb enables :s3 storage and changes the default path
Application.rb
config.paperclip_defaults =
{
:hash_secret => "LongSecretString",
:s3_protocol => "https",
:s3_credentials => "#{Rails.root}/config/aws_config.yml",
:styles => {
:original => "1024x1024>",
:large => "600x600>",
:medium => "300x300>",
:thumb => "100x100>"
}
}
Development.rb (uncomment this to try with s3 in development mode)
# config.paperclip_defaults.merge!({
# :storage => :s3,
# :bucket => "mydevelopmentbucket",
# :path => ":hash.:extension"
# })
Production.rb:
config.paperclip_defaults.merge!({
:storage => :s3,
:bucket => "myproductionbucket",
:path => ":hash.:extension"
})
In your model:
has_attached_file :avatar
Couldn't you just set an environment variable in production/test/development.rb?
PAPERCLIP_STORAGE_MECHANISM = :s3
Then:
has_attached_file :image, :styles => {...},
:storage => PAPERCLIP_STORAGE_MECHANISM,
# ...etc...
My solution is same with #runesoerensen answer:
I create a module PaperclipStorageOption in config/initializers/paperclip_storage_option.rb
The code is very simple:
module PaperclipStorageOption
module ClassMethods
def options
Rails.env.production? ? production_options : default_options
end
private
def production_options
{
storage: :dropbox,
dropbox_credentials: Rails.root.join("config/dropbox.yml")
}
end
def default_options
{}
end
end
extend ClassMethods
end
and use it in our model
has_attached_file :avatar, { :styles => { :medium => "1200x800>" } }.merge(PaperclipStorageOption.options)
Just it, hope this help
Use the :rails_env interpolation when you define the attachment path:
has_attached_file :attachment, :path => ":rails_root/storage/:rails_env/attachments/:id/:style/:basename.:extension"

Paperclip S3 download remote images

How I can download a remote image (http protocol, the url is in the image_remote_url attribute) and save it as an attachment to S3 via Paperclip ?
class Product < ActiveRecord::Base
require 'open-uri'
attr_accessor :image_remote_url
has_attached_file :photo,
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:path => ":class/:id/:style.:extension",
:bucket => "my_bucket",
:styles => {
:icon => "32x32#",
}
def fetch_image
# how should this method look ?
end
end
How should the method "fetch_image" look ?
Here's a link to a page that explains exactly what you need.
http://trevorturk.wordpress.com/2008/12/11/easy-upload-via-url-with-paperclip/
I've implemented it successfully on my own site.
I'm not sure this is still useful for you or not, but in a pull request to paperclip just a few hours ago, I've managed to make this super easy.
def set_photo
self.photo = URI.parse(self.image_remote_url)
end
This should do the job now on paperclip (version > 3.1.3) (not 3.1.3 but whatever comes after).

Resources