I have a client who wants to bulk-upload photo files to her existing Ruby on Rails application that I developed. I've scratched together code and scripts to unzip a zip file and rename all included files sequentially:
photo0001.png, photo0002.png etc...
However, I'm stuck on how to write a model or method to go through and assign each photo to a new instance, as paperclip defines the following:
class Picture
has_attached_file :photo
So I'm looking for a way to do something like this:
for i in 1..5
Picture.create(:caption => "Test", :photo => "photo/photo000#{i}.png")
`rm photo/photo000#{i}.png`
end
I'm sure there's more to it than that, but that's the basic idea...
I have a rake task for uploading photos, the meat of it looks like this:
Dir.glob(photo_path).entries.each do |e|
puts "Uploading #{e}"
Picture.create!(:photo => File.open(e))
end
I don't know if this helps, but it should hopefully point you in the right direction.
Related
First of all, these are my environment versions:
Rails: 4.1.0
Ruby: 2.1.1p76
Paperclip: 4.1
I created a scaffold (rails g scaffold Entry description:text) and further added Paperclip to the then existing model (rails g paperclip entry image).
Afterwards I migrated and everything worked fine so far.
Now when I upload an image it just doesn't display, instead "/images/original/missing.png" get's shown and there's no record of the image I just uploaded at all.
This is my model (models/entry.rb):
class Entry < ActiveRecord::Base
has_attached_file :image,
:path => ":rails_root/public/images/:class/:attachement/:id/:basename.:extension",
:url => "/images/:class/:attachement/:id/:basename.:extension"
end
My view (show.html.slim):
p#notice = notice
p
strong Description:
= #entry.description
= image_tag #entry.image.url
= link_to 'Edit', edit_entry_path(#entry)
'|
= link_to 'Back', entries_path
I have ImageMagick installed and even set the Paperclip.options within my development.rb.
I have no idea what I am missing here, it just doesn't seem to upload any images whatsoever, nor throw out any error messages.
After a little more research and a few cold drinks I found the solution!
It's necessary to either explicitly allow certain formats to get uploaded OR remove the verification check (I recommend this for development).
Doing this is as simple as adding the following line to your model (for me, entry.rb)
(SOURCE: https://stackoverflow.com/a/21898204/3686898)
do_not_validate_attachment_file_type :image
Also, I added another check in my controller (same as attr_accessible in earlier Rails versions):
private
# Use callbacks to share common setup or constraints between actions.
def set_entry
#entry = Entry.find(params[:id])
end
# Never trust parameters from the scary internet, only allow the white list through.
def entry_params
params.require(:entry).permit(:description, :image)
end
Alrighty, I hope this helps someone out :)
(Always remember to have a look at your server-logs. It provides golden information!)
Check your
:path => ":rails_root/public/images/:class/:attachement/:id/:basename.:extension"
make sure that is tracing back to where the image is stored
I have a model with a method to return a url to a person's avatar that looks like this:
def avatar_url
if self.avatar?
self.avatar.url # This uses paperclip
else
"/images/avatars/none.png"
end
end
I'm in the midst of upgrading to 3.1, so now the hard-coded none image needs be referenced through the asset pipeline. In a controller or view, I would just wrap it in image_path(), but I don't have that option in the model. How can I generate the correct url to the image?
I struggled with getting this right for a while so I thought I'd post the answer here. Whilst the above works for a standard default image (i.e. same one for each paperclip style), if you need multiple default styles you need a different approach.
If you want to have the default url play nice with the asset pipeline and asset sync and want different default images per style then you need to generate the asset path without fingerprints otherwise you'll get lots of AssetNotPrecompiled errors.
Like so:
:default_url => ActionController::Base.helpers.asset_path("/missing/:style.png", :digest => false)
or in your paperclip options:
:default_url => lambda { |a| "#{a.instance.create_default_url}" }
and then an instance method in the model that has the paperclip attachment:
def create_default_url
ActionController::Base.helpers.asset_path("/missing/:style.png", :digest => false)
end
In this case you can still use the interpolation (:style) but will have to turn off the asset fingerprinting/digest.
This all seems to work fine as long as you are syncing assets without the digest as well as those with the digest.
Personally, I don't think you should really be putting this default in a model, since it's a view detail. In your (haml) view:
= image_tag(#image.avatar_url || 'none.png')
Or, create your own helper and use it like so:
= avatar_or_default(#image)
When things like this are hard in rails, it's often a sign that it's not exactly right.
We solved this problem using draper: https://github.com/jcasimir/draper. Draper let us add a wrapper around our models (for use in views) that have access to helpers.
Paperclip has an option to specify default url
has_attached_file :avatar, :default_url => '/images/.../missing_:style.png'
You can use this to serve default image' in case user has not uploaded avatar.
Using rails active storage I solved this problem by doing this:
# Post.rb
def Post < ApplicationRecord
has_one_attached :image
def thumbnail
self.image.attached? ? self.image.variant(resize: "150x150").processed.service_url : 'placeholder.png';
end
def medium
self.image.attached? ? self.image.variant(resize: "300x300").processed.service_url : 'placeholder.png';
end
def large
self.image.attached? ? self.image.variant(resize: "600x600").processed.service_url : 'placeholder.png';
end
end
Then in your views simply call:
<%= image_tag #post.thumbnail %>,
I'm using the Polymorphic fork of Paperclip in Rails, but have been having some massive problems with regards to the overwriting of unique filenames. No matter whether I put a time-stamp (more on that in a second) or the id of the asset in the URL, if a file with the same name is uploaded subsequently, then the previous one is overwritten.
Also, it was working before, but the Time interpolation is now outputting just "0" instead of the timestamp.
module Paperclip
module Interpolations
def stamp(attachment, style)
attachment.instance_read(:created_at).to_i
end
end
end
Now just outputs;
0
This is what my URL field is;
:url => "/assets/images/:stamp/:id_:style.:extension"
Thanks.
Try adding this to config/initializers/paperclip.rb
Paperclip.interpolates :stamp do |attachment, style|
attachment.created_at.to_i
end
Total Rspec noob here. Writing my first tests tonight.
I've got a model called Image. Using paperclip I attach a file called photo. Standard stuff. I've run the paperclip generator and everything works fine in production and test modes.
Now I have a spec file called image.rb and it looks like this (it was created by ryanb's nifty_scaffold generator):
require File.dirname(__FILE__) + '/../spec_helper'
describe Image do
it "should be valid" do
Image.new.should be_valid
end
end
This test fails and I realise that it's because of my model validations (i.e. validates_attachment_presence)
The error that I get is:
Errors: Photo file name must be set., Photo file size file size must be between 0 and 1048576 bytes., Photo content type is not included in the list
So how do I tell rspec to upload a photo when it runs my test?
I'm guessing that it's got somethign to do with fixtures.... maybe not though. I've tried playing around with them but not having any luck. For the record, I've created a folder called images inside my fixtures folder and the two files I want to use in my tests are called rails.png and grid.png)
I've tried doing the following:
it "should be valid" do
image = Image.new :photo => fixture_file_upload('images/rails.png', 'image/png').should be_valid
# I've also tried adding stuff like this
#image.stub!(:has_attached_file).with(:photo).and_return( true )
#image.stub!(:save_attached_files).and_return true
#image.save.should be_true
end
But rspec complains about "fixture_file_upload" not being recognised... I am planning to get that Rspec book. And I've trawled around the net for an answer but can't seem to find anything. My test database DOES get populated with some data when I remove the validations from my model so I know that some of it works ok.
Thanks in advance,
EDIT:
images.yml looks like this:
one:
name: MyString
description: MyString
two:
name: MyString
description: MyString
This should work with Rails 2.X:
Image.new :photo => File.new(RAILS_ROOT + '/spec/fixtures/images/rails.png')
As of Rails 3, RAILS_ROOT is no longer used, instead you should use Rails.root.
This should work with Rails 3:
Image.new :photo => File.new(Rails.root + 'spec/fixtures/images/rails.png')
Definitely get the RSpec book, it's fantastic.
Rails.root is a pathname object so you can use it like this:
Image.new :photo => Rails.root.join("spec/fixtures/images/rails.png").open
Edit - probably does not work in Rails 3...
see answer by #Paul Rosania
In case anyone else finds this via Google, RAILS_ROOT is no longer valid in Rails 3.0. That line should read:
Image.new :photo => File.new(Rails.root + 'spec/fixtures/images/rails.png')
(Note the lack of leading slash!)
I use the multipart_body gem in my integration tests. Its a bit truer to BDD than testing.
http://steve.dynedge.co.uk/2010/09/19/multipart-body-a-gem-for-working-with-multipart-data/
With respect to rspec and paperclip, the has_attached_file :photo directive creates a virtual attribute of sorts i.e. :photo ... when you assign a file or a path to photo, paperclip takes over, stores the file, optionally does processing on it e.g. auto-create thumbnails, import a spreadsheet, etc. You aren't telling rspec to test paperclip. You are invoking code and telling rspec what the results of that code -should- be.
In $GEM_HOME/gems/paperclip-2.3.8/README.rdoc, about 76% of the way through the file under ==Post Processing (specifically lines 147 and 148):
---[ BEGIN QUOTE ]---
NOTE: Because processors operate by turning the original attachment into the styles, no processors will be run if there are no styles defined.
---[ END QUOTE ]---
Reading the code, you'll see support :original ... does your has_attached_file define a style?
I use a generic ":styles => { :original => { :this_key_and => :this_value_do_not_do_anything_unless_a_lib_paperclip_processors__foo_dot_rb__does_something_with_them } }" ... just to get paperclip to move the file from some temp directory into my has_attached_file :path
One would think that would be default or more obvious in the docs.
I'm using paperclip for uploading images in S3.
But I've noted that this upload is very slow. I think because before complete the submit the file has to pass by my server, be processed and be sent to the S3 server.
Is there a method for accelerate this?
thanks
You did not post any code so I'm going to make a few assumptions here:
in your project you have an Album and Image model
An Album has_many :images
You already have
paperclip and
aws-sdk
set up correctly with buckets and all else
You are uploading many images at once
In order to upload many images, your form will look something like this:
<%= form_for #album, html: { multipart: true } do |f| %>
<%= f.file_field :files, accept: 'image/png,image/jpeg,image/gif', multiple: true %>
<%= f.submit %>
<% end %>
Your controller will look something like this
class AlbumsController < ApplicationController
def update
#album = Album.find params[:id]
#album.update album_params
redirect_to #album, notice: 'Images saved'
end
def album_params
params.require(:album).permit files: []
end
end
In order to manipulate images using an album you'll need
class Album < ApplicationRecord
has_many :images, dependent: :destroy
accepts_nested_attributes_for :images, allow_destroy: true
def files=(array = [])
array.each do |f|
images.create file: f
end
end
end
Your Image file will look like this
class Image < ApplicationRecord
belongs_to :album
has_attached_file :file, styles: { thumbnail: '500x500#' }, default_url: '/default.jpg'
validates_attachment_content_type :file, content_type: /\Aimage\/.*\Z/
end
This is just the important stuff. With this setup, an upload of 22 images with a total of 12MB takes the :files= method 41.1806895 seconds to execute on average on my local server. To check how long a method takes to run, use:
def files=(array = [])
start = Time.now
array.each do |f|
images.create file: f
end
p "ELAPSED TIME: #{Time.now - start}"
end
You ask for a faster upload of many images. There are a few ways to do this. Using
jobs
won't work because you can't pass complex data like images to a job.
Use delayed_paperclip instead. It moves image styles creation (like thumbnail: '500x500#') into background jobs.
Gemfile
source 'https://rubygems.org'
ruby '2.3.0'
...
gem 'delayed_paperclip'
...
Image file
class Image < ApplicationRecord
...
process_in_background :file
end
It speeds up the :files= method. The same upload as before (22 images, 12MB) with this setup took 23.13998 seconds on my machine. That's 1.77963 times faster than before.
Another way of speeding things up is by using Threads. Remove delayed_paperclip from the Gemfile and the process_in_background :file line. Update your :files= method:
def files=(array = [])
threads = []
array.each do |f|
threads << Thread.new do
images.create file: f
end
end
threads.each(&:join)
end
You might try this, but get some weird error and only see that 4 images saved. You must also use Mutex. Also, you must not use :join on the threads because if you join, the method will wait until the threads are done running.
def files=(array = [])
semaphore = Mutex.new
array.each do |f|
Thread.new do
semaphore.synchronize do
images.create file: f
end
end
end
end
With this simple change to the method and no added gems, the same upload as before runs in 0.017628 seconds. That is 1,313 times faster than delayed_paperclip. It's also 2,336 times faster than the regular setup.
What happens if you use delayed_paperclip AND Threads?
Don't change the :files= method. Just turn delayed_paperclip back on in your Gemfile and add back the process_in_background :file line.
With this setup on my machine, the method runs in 0.001277 seconds on average. That's
13.8 times faster than Threads
18,120.6 times faster than delayed_paperclip
32,248.0 times faster than regular setup
Remember, this is on my machine and I have not tested this in production. I am also on wifi, not ethernet. All these things can change the results but I think the numbers speak for themselves.
Upload images faster. Done.
UPDATE: Don't use delayed_paperclip. It can cause a busy database, and some images might not get saved. I've tested it. I think just using threads is fast enough. Remove the process_in_background line from the Image file. Also, here's what my files= method looks like:
def files=(array = [])
Thread.new do
begin
array.each { |f| images.create file: f }
ensure
ActiveRecord::Base.connection_pool.release_connection
end
end
end
Note: Since we push the image saving to a background task and then redirect. The page that loads will not have images on them yet. The user has to
refresh
to update the page. One way around this is to use
polling.
Polling is when JavaScript checks for any changes every 5 seconds or so and makes changes if any to the page.
Another option is to use
Web Sockets.
Now that we have Rails 5, we can use ActionCable. Every time an image gets created, we broadcast an update for the album. If the user is on that page for that album, they will see updates happen as soon as they happen on the database without having the user refresh or the browser make a request every 5 seconds on an infinite loop.
Cool stuff.
Do you want to improve the appearance of the upload being faster or actually make the upload faster?
If it's the former you can put your image handling logic into a background task using something like delayed_job. This way when a user clicks the button they'll immediately go to their next page while you process the image (you can show a "processing in progress" image placeholder until the task is finished).
If it's the latter then it's entirely down to your server and internet connection. Where are you hosting?
How about uploading direct to S3?
Not sure if paperclip does this out of the box, but you could make it.
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/index.html?UsingHTTPPOST.html
Use delayed jobs, this is a good example here
Or you can use flash upload.
If you end up going the route of uploading directly to S3 which offloads the work from your Rails server, please check out my sample projects:
Sample project using Rails 3, Flash and MooTools-based FancyUploader to upload directly to S3: https://github.com/iwasrobbed/Rails3-S3-Uploader-FancyUploader
Sample project using Rails 3, Flash/Silverlight/GoogleGears/BrowserPlus and jQuery-based Plupload to upload directly to S3: https://github.com/iwasrobbed/Rails3-S3-Uploader-Plupload
By the way, you can do post-processing with Paperclip using something like this blog post describes:
http://www.railstoolkit.com/posts/fancyupload-amazon-s3-uploader-with-paperclip
As cwninja recommends, we upload direct to s3 so as to get rid of this extra upload. We use a modified version of the plugin described in this blog post:
http://elctech.wpengine.com/2009/02/updates-on-rails-s3-flash-upload-plugin/
Ours is modified to handle multiple file uploads (rewrote the the flex object
Not sure how well this plays with paperclip, we use attachment_fu, but it wasn't so bad to get it to work with that.