Paperclip Upload to S3 with Delayed Job - ruby-on-rails

I am trying to upload a file to s3 in background using Rails (4.2) with delayed_job gem.
My code is basically what is shown in this post:http://airbladesoftware.com/notes/asynchronous-s3/ (with small changes).
The Photo Modal
class Photo < ActiveRecord::Base
belongs_to :gallery
has_attached_file :local_photo,
path: ":rails_root/public/system/:attachment/:id/:style/:basename.:extension",
url: "/system/:attachment/:id/:style/:basename.:extension"
has_attached_file :image,
styles: {large: '500x500#', medium: '180x180#'},
storage: :s3,
s3_credentials: lambda { |attachment| attachment.instance.s3_keys },
s3_permissions: :private,
s3_host_name: 's3-sa-east-1.amazonaws.com',
s3_headers: {'Expires' => 1.year.from_now.httpdate,
'Content-Disposition' => 'attachment'},
path: "images/:id/:style/:filename"
validates_attachment_content_type :image,
content_type: [
"image/jpg",
"image/jpeg",
"image/png"]
def s3_keys
{
access_key_id: SECRET["key_id"],
secret_access_key: SECRET["access_key"],
bucket: SECRET["bucket"]
}
end
after_save :queue_upload_to_s3
def queue_upload_to_s3
Delayed::Job.enqueue ImageJob.new(id) if local_photo? && local_photo_updated_at_changed?
end
def upload_to_s3
self.image = Paperclip.io_adapters.for(local_photo)
save!
end
end
class ImageJob < Struct.new(:image_id)
def perform
image = Photo.find(image_id)
image.upload_to_s3
image.local_photo.destroy
end
end
With this code, the job (ImageJob) run in background (I can see it on delayed_job_web), no error. But the file is not uploaded.
If I "disable background" and use only:
ImageJob.new(id).perform if local_photo? && local_photo_updated_at_changed?
The file is uploaded to the amazon and local file is deleted too.
Any suggestions ?
Thanks in advance
UPDATE #1 Now I can see the error:
Job failed to load: undefined class/module ImageJob. Handler: "--- !ruby/struct:ImageJob\nimage_id: 412\n"
UPDATE #2 I make this working using delay method, something like this:
def perform
if local_photo? && local_photo_updated_at_changed?
self.delay.move_to_s3
end
end

Related

Rails 4 - Paperclip is not uploading files

I am testing an Rails 4 application in localhost which uses Paperclip, and even though submitting images creates entries in the Database, the images folder is always empty.
Model
class Tile < ActiveRecord::Base
belongs_to :game
# Paperclip
has_attached_file :image,
styles: { medium: "300x300>", thumb: "100x100>" },
url: "/images/:style/:filename",
default_url: "/images/:style/missing.png"
validates_attachment_content_type :image, content_type: /\Aimage\/.*\Z/
end
Controller
class TilesController < ApplicationController
before_action :set_game
def create
tile = #game.tiles.create tile_params
redirect_to #game, notice: tile
end
private
# Use callbacks to share common setup or constraints between actions.
def set_game
#game = Game.find(params[:game_id])
end
# Never trust parameters from the scary internet, only allow the white list through.
def tile_params
params.require(:tile).permit(:image)
end
end
View form
<%= form_for([#game, Tile.new], multipart: true) do |form| %>
<%= form.file_field :image %>
<% end %>
Everytime I try to print <p><%= tile.image.url %></p> I get /images/original/missing.png.
Why is this?
Update
I am using Ubuntu 14.04.
Imagemagick is installed.
Be sure to also update path long with url, ie
has_attached_file :image,
styles: { medium: "300x300>", thumb: "100x100>" },
path: "/images/:style/:filename",
url: "/images/:style/:filename",
default_url: "/images/:style/missing.png"
As explained in this Railscasts ~5:20, url sets where images will be retrieved from, and path sets where they will be stored.
Assuming you're using window, to show image on the screen: image_tag(tile.image.url(:small))
Also, did you install ImageMagick and file for Windown?'
and if yes, did you set your environment to: Paperclip.options[:command_path] = 'C:\Program Files (x86)\GnuWin32\bin' in the config/environments/development.rb? then restart your server.

Paperclip multiple storage

I want to move my assets folder to Amazon S3 and since it has a big size, during the transaction i need to upload files both in my local storage and amazon s3 through paperclip.
Is there a way to configure paperclip to store uploaded files both on filesystem and amazon s3?
Maybe you'd benefit from this:
http://airbladesoftware.com/notes/asynchronous-s3/
What you'll have to do is firstly upload to your local storage, and then "asynchronously" upload to S3
This is typically done through the likes of Resque or DelayedJob (as the tutorial demonstrates), and will require you to run some sort of third-party processing engine on your server (typically Redis or similar)
From the tutorial:
### Models ###
class Person < ActiveRecord::Base
has_attached_file :local_image,
path: ":rails_root/public/system/:attachment/:id/:style/:basename.:extension",
url: "/system/:attachment/:id/:style/:basename.:extension"
has_attached_file :image,
styles: {large: '500x500#', medium: '200x200#', small: '70x70#'},
convert_options: {all: '-strip'},
storage: :s3,
s3_credentials: "#{Rails.root}/config/s3.yml",
s3_permissions: :private,
s3_host_name: 's3-eu-west-1.amazonaws.com',
s3_headers: {'Expires' => 1.year.from_now.httpdate,
'Content-Disposition' => 'attachment'},
path: "images/:id/:style/:filename"
after_save :queue_upload_to_s3
def queue_upload_to_s3
Delayed::Job.enqueue ImageJob.new(id) if local_image? && local_image_updated_at_changed?
end
def upload_to_s3
self.image = local_image.to_file
save!
end
end
class ImageJob < Struct.new(:image_id)
def perform
image = Image.find image_id
image.upload_to_s3
image.local_image.destroy
end
end
### Views ###
# app/views/people/edit.html.haml
# ...
= f.file_field :local_image
# app/views/people/show.html.haml
- if #person.image?
= image_tag #person.image.expiring_url(20, :small)
- else
= image_tag #person.local_image.url, size: '70x70'

no validates_attachment_file_name when upgrading to Paperclip 4.1 from 3.5

We have code that looks like run of the mill paper clip:
has_merchants_attached_file :pdf,
storage: :s3,
s3_credentials: Mbc::DataStore.s3_credentials,
s3_permissions: :private,
path: ":identifier_template.pdf",
bucket: Mbc::DataStore.forms_and_templates_bucket_name
validates_attachment_file_name :pdf, :matches => [/pdf\Z/]
Which generates an error:
undefined method `validates_attachment_file_name' for #<Class:0x007fba67d25fe0>
Interestingly enough, when we down grade back to 3.5, we encounter the same issue.
The controller that is generating this is:
def index
#fidelity_templates = FidelityTemplate.order("identifier asc").all
end
Additionally:
def has_merchants_attached_file(attribute, options={})
if Rails.env.test? || Rails.env.development?
has_attached_file attribute,
path: "paperclip_attachments/#{options[:path]}"
else
has_attached_file attribute, options
end
end
Any thoughts on what could be causing this?
You can read about the provided validators here:
https://github.com/thoughtbot/paperclip#validations
The included validators are:
AttachmentContentTypeValidator
AttachmentPresenceValidator
AttachmentSizeValidator
They can be used in either of these ways:
# New style:
validates_with AttachmentPresenceValidator, :attributes => :avatar
# Old style:
validates_attachment_presence :avatar
UPDATE ...
If you read further down the link I've given above you'll get to a section on Security Validations (Thanks Kirti Thorat):
https://github.com/thoughtbot/paperclip#security-validations
They give an example on how to validate the filename format:
# Validate filename
validates_attachment_file_name :avatar, :matches => [/png\Z/, /jpe?g\Z/]
From your code snippet it looks like your validation should work as-is.
However, I've never seen paperclip used with this syntax:
has_merchants_attached_file ...
Perhaps that's the source of your issues? You would usually use the following to attach files to your model:
has_attached_file :pdf ...

Paperclip on S3 is changing file extensions from a url

I wrote a rake task that downloads an image from wikipedia given a celebrity name, but for some reason when storing on S3 the file extension is either being dropped or changed to .txt
The file otherwise is correct.
Any ideas?
From my celeb model:
has_attached_file :pic,
:styles => { :medium => "300x300>", :thumb => "100x100>" },
:default_style => :medium,
:storage => :s3,
:s3_credentials => "#{Rails.root}/config/s3.yml",
:path => "/:style/:img_name.:extension"
From my rake task:
desc "Update celeb pics from wiki"
task :update_celeb_pics => [:environment] do
include ApplicationHelper
Celeb.all.each do |celeb|
if !celeb.name.start_with?("(")
puts celeb.name
celeb.pic = open(getImage(celeb.name))
celeb.save
end
end
end
the getImage method is a helper that returns a string
require 'open-uri'
require 'uri'
module ApplicationHelper
def getInfo(name)
Nokogiri::XML(open(URI.escape("http://en.wikipedia.org/w/api.php?action=opensearch&search=#{name}&limit=1&namespace=0&format=xml")))
end
def nokoPage(name)
Nokogiri::XML(open(getURL(name)))
end
def getImage(name)
"http:" + nokoPage(name).css("table img")[0].attribute("src").value if !nokoPage(name).css("table img").empty?
end
def getDescription(name)
getInfo(name).css("Description").text
end
def getURL(name)
getInfo(name).css("Url").text
end
def getBday(name)
bday = nokoPage(name).css("span.bday")
return Date.parse(bday[0].text) if !bday.empty?
return Date.today
end
def getDday(name)
dday = nokoPage(name).css("span.dday")
return Date.parse(dday[0].text) if !dday.empty?
end
end
This is because
self.pic = open("http://something.com/bla/image.png")
is not the best solution here. Just yesterday, i got a Pull Request merged into Paperclip that lets you do this
self.pic = URI.parse(getImage(name))
This will ensure that your pic's content type matches the downloaded file, pic's filename is set to the name of the file downloaded.
The reason you get txt extension is because open returns a StringIO object which infact names the filename as "stringio.txt". Your filename is probably changed by the s3 code but the extension remains as '.txt'
I suggest you link your Gemfile to paperclip's github repo, run bundle and try again.
Cheers,
Aditya

Paperclip S3 download remote images

How I can download a remote image (http protocol, the url is in the image_remote_url attribute) and save it as an attachment to S3 via Paperclip ?
class Product < ActiveRecord::Base
require 'open-uri'
attr_accessor :image_remote_url
has_attached_file :photo,
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:path => ":class/:id/:style.:extension",
:bucket => "my_bucket",
:styles => {
:icon => "32x32#",
}
def fetch_image
# how should this method look ?
end
end
How should the method "fetch_image" look ?
Here's a link to a page that explains exactly what you need.
http://trevorturk.wordpress.com/2008/12/11/easy-upload-via-url-with-paperclip/
I've implemented it successfully on my own site.
I'm not sure this is still useful for you or not, but in a pull request to paperclip just a few hours ago, I've managed to make this super easy.
def set_photo
self.photo = URI.parse(self.image_remote_url)
end
This should do the job now on paperclip (version > 3.1.3) (not 3.1.3 but whatever comes after).

Resources