I'm using Carrierwave to upload some images and generate different versions of them (size, color...). Every image is given a unique name when uploaded and I'm now trying to process the images in the background in order to avoid blocking web processes. I use carrierwave_backgrounder for that.
The problem is that when my image is processed, a new name is given to it. So the model doesn't know this new name and the original image is now twice on the server.
I end up with this kind of files:
f9f97657-eaab-40ce-b965-31bb128066ee.jpg // First uploaded images
e4244551-7f43-4c03-8747-e8f2f2e57156.jpg // Copy of the original image created while processed in the background
thumb_e4244551-7f43-4c03-8747-e8f2f2e57156.jpg
And event.image returns f9f97657-eaab-40ce-b965-31bb128066ee.jpg.
How can I avoid to generate a new filename if we are just processing the different versions in the background and not uploading it?
My model:
class Event < ActiveRecord::Base
mount_uploader :cover_image, CoverImageUploader
process_in_background :cover_image
end
My uploader:
class CoverImageUploader < CarrierWave::Uploader::Base
include CarrierWave::RMagick
include CarrierWave::MimeTypes
include Sprockets::Helpers::RailsHelper
include Sprockets::Helpers::IsolatedHelper
include ::CarrierWave::Backgrounder::Delay
def filename
"#{secure_token}.#{file.extension}" if original_filename
end
process :set_content_type
version :thumb do
process resize_to_fill: [200, 200]
end
# ...
# Some other versions processing...
# ...
protected
def secure_token
var = :"##{mounted_as}_secure_token"
model.instance_variable_get(var) || model.instance_variable_set(var, SecureRandom.uuid)
end
end
can try this:
def filename
if original_filename.present?
if model.respond_to?("#{mounted_as}_processing") && model.send("#{mounted_as}_processing")
#name ||= model.send("#{mounted_as}_identifier")
else
#name ||= "#{secure_token}.#{file.extension}"
end
end
end
only works if you have a "processing" column in your table
Related
I'm using Carrierwave and Fog gems to store a file to my Amazon S3 bucket (to /files/file_id.txt). I need to store a slightly different version of the file to a different location in the bucket (/files/file_id_processed.txt) at the same time (right after the original is stored). I don't want to create a separate uploader attribute for it on the model - is there any other way?
This my current method that stores the file:
def store_file(document)
file_name = "tmp/#{document.id}.txt"
File.open(file_name, 'w') do |f|
document_content = document.content
f.puts document_content
document.raw_export.store!(f)
document.save
end
# I need to store the document.processed_content
File.delete(file_name) if File.exist?(file_name)
end
This is the Document model:
class Document < ActiveRecord::Base
mount_uploader :raw_export, DocumentUploader
# here I want to avoid adding something like:
# mount_uploader :processed_export, DocumentUploader
end
This is my Uploader class:
class DocumentUploader < CarrierWave::Uploader::Base
storage :fog
def store_dir
"files/"
end
def extension_white_list
%w(txt)
end
end
This is how my final solution looks like (kinda) - based on Nitin Verma's answer:
I had to add a custom processor method for the version to the Uploader class:
# in document_uploader.rb
...
version :processed do
process :do_the_replacements
end
def do_the_replacements
original_content = #file.read
File.open(current_path, 'w') do |f|
f.puts original_content.gsub('Apples','Pears')
end
end
considering that you need similar file but with different name.
for this you need to create a version for file in uploader.
version :processed do
process
end
and now second file name will be processed_{origional_file}.extension. if you want to change file name of second file you can use this link https://github.com/carrierwaveuploader/carrierwave/wiki/How-to:-Customize-your-version-file-names
so I have a site already in production using the carrierwave gem with images stored on amazon s3. my uploader uses the store_dir method to specify a particular structure to put in processed images.
well, now in my dev environment I've added the carrierwave_direct gem to start uploading directly to S3. The problem is this gem completely overrides the store_dir and filename defaults in my uploader. I can't push my fully working uploader live because all my old image links will be broken.
it's my understanding that the CWdirect gem would upload a raw image file to a "temp" directory on S3, then S3 responds and gives you a key variable so you can grab this file and process it as you see fit. so, should i be using a completely separate image uploader class in carrierwave to process the images and place them in the correct folders? meaning I'll have one uploader dedicated to carrierwave_direct that uploads wherever this gem seems to want to upload to; and I'll use another uploader.rb class linked to my real model that keeps my current store_dir and filename structure?
In any case, my basic question is, how can I use CarrierWave_Direct gem if I already have CW running in production with images in a specific folder structure?
Ok, so I did figure out how to do this and I'll explain below. My hunch was correct to use two different CarrierWave uploader classes--one class dedicated to uploading to S3 (using CarrierWave_Direct gem), and a second class used only for image processing (the class I was already using in production). I'll try and post relevant code below but if anyone has questions let me know. I'm unsure why I haven't seen others using separate classes like this but it seems to work for me.
My image Uploader class app\uploaders\image_uploader.rb utilizing carrierwave_direct gem:
class ImageUploader < CarrierWave::Uploader::Base
include CarrierWaveDirect::Uploader
include ActiveModel::Conversion
extend ActiveModel::Naming
include CarrierWave::MimeTypes
process :set_content_type
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
def extension_white_list
%w(jpg jpeg gif png)
end
# Include the Sprockets helpers for Rails 3.1+ asset pipeline compatibility:
include Sprockets::Helpers::RailsHelper
include Sprockets::Helpers::IsolatedHelper
# Override the directory where uploaded files will be stored.
# CarrierWaveDirect::Uploader puts raw uploaded files in this directory on S3 as a first step
def store_dir
"unprocessed_uploads"
end
end
**notice there's no processing being done in this class
My image PROCESSING class app\uploaders\image_processor.rb (what was already in place in production):
class ImageProcessor < CarrierWave::Uploader::Base
# Include RMagick or MiniMagick support:
# include CarrierWave::RMagick
include CarrierWave::MiniMagick
include CarrierWave::MimeTypes
process :set_content_type
# Include the Sprockets helpers for Rails 3.1+ asset pipeline compatibility:
include Sprockets::Helpers::RailsHelper
include Sprockets::Helpers::IsolatedHelper
# Choose what kind of storage to use for this uploader:
storage :fog
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
# "uploads/#{model.class.to_s.underscore}/path/#{model.id}"
end
# Provide a default URL as a default if there hasn't been a file uploaded:
def default_url
"logos/" + [version_name, "default.png"].compact.join('_')
end
# Process files fetched from S3 after they are uploaded:
def make_thumbnail(width, height)
# uses MiniMagick classes to get a square, centered thumbnail image
manipulate! do |img|
if img[:width] < img[:height]
remove = ((img[:height] - img[:width])/2).round
img.shave("0x#{remove}")
elsif img[:width] > img[:height]
remove = ((img[:width] - img[:height])/2).round
img.shave("#{remove}x0")
end
img.resize("#{width}x#{height}")
img
end
end
# Create different versions of your uploaded files:
# the process statement below isn't defined within a version block on purpose--this means the ORIGINAL uploaded photo is constrained to 1050 pics
process :resize_to_limit => [1050, 1050]
process :quality => 85 # this reduces filesize greatly and saves space
version :thumb do
process :make_thumbnail => [100, 100]
process :quality => 85 # this reduces filesize greatly and saves space
end
version :big_thumb do
process :make_thumbnail => [350, 350]
process :quality => 85 # this reduces filesize greatly and saves space
end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
def extension_white_list
%w(jpg jpeg gif png)
end
# Override the filename of the uploaded files:
# Avoid using model.id or version_name here, see uploader/store.rb for details.
def filename
if original_filename
if model && model.read_attribute(:image).present?
model.read_attribute(:image)
else
"#{secure_token}.#{file.extension}"
end
end
end
protected
def secure_token
var = :"##{mounted_as}_secure_token"
model.instance_variable_get(var) or model.instance_variable_set(var, SecureRandom.uuid)
end
end
My Photo model (summarized):
class Photo < ActiveRecord::Base
mount_uploader :image, ImageProcessor
def save_and_process_image(options = {})
s3_unprocessed_image_url = self.image.asset_host + '/' + self.key
# this next line downloads the image from S3
# and this save line below will process the image and reupload to S3 according to ImageProcessor settings
self.remote_image_url = s3_unprocessed_image_url
save
end
end
I also have Photo controller and view code available, if ya'll want it let me know. Basically I use the ImageUploader class to do the initial upload to S3 to a folder called unprocessed_uploads. Then S3 responds with a key field in the URL which I pass on to ImageProcessor class--this is attached to a Photo and processes the thumbnail and other images then re-uploads them to my uploads folder on S3.
This separation meant i didn't need to change my current folder structure on S3 when adding the carrierwave_direct gem. Hope this helps others. Let me know if you need more code I'm kind of tired of typing :)
UPDATE--Adding more code:
Photos Controller:
class PhotosController < ApplicationController
def index
#photos = #photos.sort_by(&:created_at)
#uploader = ImageUploader.new
#uploader.success_action_redirect = new_tank_photo_url(#tank)
respond_to do |format|
format.html # index.html.erb
format.json { render json: #photos }
end
end
def create
respond_to do |format|
if #photo.save_and_process_image
format.html { redirect_to tank_photos_path(#tank), notice: 'Photo uploaded successfully and is being processed...' }
format.json { render json: #photo, status: :created, location: #photo }
else
format.html { render :new }
format.json { render json: #photo.errors, status: :unprocessable_entity }
end
end
end
Photos Index view, the form with the Upload button:
<%= direct_upload_form_for #uploader, :html => {:class => "form-inline"} do |f| %>
<%= f.file_field :image %>
<%= f.submit "Upload", :class => "btn btn-primary btn-medium" %>
<% end %>
So with the above view/controller code added, here is a summary of the steps taken. Please note the difference between the ImageUploader class and the ImageProcessor class:
In my Photos Controller I create the #uploader variable which is of class ImageUploader--this means that images submitted in my index/view form which uses #uploader will upload the image to a "temp" folder--Since this uses the ImageUploader class then no processing is done when this image is uploaded (yet).
When the user clicks "Upload" button on the form, the Photo>>create action is called. This action calls #photo.save_and_process_image -- look at the model to see that this method actually grabs the recently uploaded image from the S3 'temp' folder, processes it, then re-uploads the processed images to their final destination. This is all possible because my Photo model is linked to the ImageProcessor class, which does the processing/uploading. It's not linked to the separate ImageUploader class.
Hope this helps explain what i have going on
I'm try to set up carrierwave_direct for some image uploading in my rails 3.2 application but seem unable to render the form by using the direct_upload_form_for tags.
I'm getting the error "undefined method `direct_fog_url' for #< PostmarkerImage:0x007fdbe07b39f0 >" when loading the page containing the form.
PostmarkerImage is the model in which I've mounted the uploader like so:
class PostmarkerImage < ActiveRecord::Base
attr_accessible :image, :image_cache
belongs_to :postmarker
validates :image, :presence => true
mount_uploader :image, PostmarkerImageUploader
end
The image column is a string and I've also ensured that the migration has been run to create the column. The uploader now looks like this:
class PostmarkerImageUploader < CarrierWave::Uploader::Base
include CarrierWaveDirect::Uploader
# Include RMagick or MiniMagick support:
# include CarrierWave::RMagick
include CarrierWave::MiniMagick
# Include the Sprockets helpers for Rails 3.1+ asset pipeline compatibility:
include Sprockets::Helpers::RailsHelper
include Sprockets::Helpers::IsolatedHelper
# Choose what kind of storage to use for this uploader:
# storage :file
# storage :fog
include CarrierWave::MimeTypes
process :set_content_type
# Provide a default URL as a default if there hasn't been a file uploaded:
# def default_url
# # For Rails 3.1+ asset pipeline compatibility:
# # asset_path("fallback/" + [version_name, "default.png"].compact.join('_'))
#
# "/images/fallback/" + [version_name, "default.png"].compact.join('_')
# end
# Process files as they are uploaded:
# process :scale => [200, 300]
#
# def scale(width, height)
# # do something
# end
# Create different versions of your uploaded files:
# version :thumb do
# process :scale => [50, 50]
# end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
def extension_white_list
%w(jpg jpeg gif png)
end
# Override the filename of the uploaded files:
# Avoid using model.id or version_name here, see uploader/store.rb for details.
# def filename
# "something.jpg" if original_filename
# end
end
I've also ensured that carrierwave_direct, fog, mini_magick and Imagemagick have already been installed. May I know what else I have missed out? It almost seems like the mounting of the uploader has not been done properly.
Managed to figure it out :) Just in case anyone has this problem:
I am implementing a separate model to hold my images instead of adding a column to an existing table. So the direct_upload_form_for expects the column (e.g. Image.new.image) instead of the Object (e.g. Image.new) which I was returning. I guess I was thrown off by the form_for part which usually expects an Object!
For anyone else who runs into this error, you are likely not passing the correct object to direct_upload_form_for. Let's say we had a User model with an avatar column. It should look like.
Model:
class User < ActiveRecord::Base
mount_uploader :avatar, AvatarUploader
end
Controller:
def edit # Could be any action, but I always direct upload to existing objects
#uploader = #user.avatar
#uploader.success_action_redirect = edit_registration_url(resource)
end
View (erb):
<%= direct_upload_form_for #uploader do |f| %>
<%= f.file_field :image %>
<% end %>
Notice how the #uploader object is actually the column #user.avatar, not the object #user. That's what the OP is talking about in his answer. It's just more confusing in his case because the object is not a User, it's a PostmarkerImage, which I assume is a model that only exists to hold images. This is common when you want one object (ie. a Postmarker) to have multiple images. So you create a model only for images, and set up a has_many relationship between them.
Pitfall: I was also getting this error a lot in production because I was forgetting to copy the code in the controller from the edit action to the update action. If the upload fails due to validation (like file type), you need to remember to set #uploader again! Similarly, if you have your uploader in new, remember to assign #uploader in create, as well.
I am using Rails and ActiveRecord.
I have carrierwave mounted on one of the columns(:logo) of a model(Listing). My default filename is "disp_logo". Let's say I just do Listing.create! In this case, I haven't really uploaded any file. I did not do Listing.logo=<some file> or Listing.remote_logo_url=<some url>. But, carrierwave still inserts the string "disp_logo" in the :logo column. Why does it do that? How can I prevent carrierwave from doing so?
My uploader class has the following methods:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
def filename
"disp_logo"
end
That's what I mean by default filename.
I want the column :logo to have NULL if image is not uploaded. Instead it has "disp_logo".
It would seem you created your migration with "disp_logo" as the default value for your column. You should set the default image in your uploader instead, like this:
# Provide a default URL as a default if there hasn't been a file uploaded:
def default_url
"/" + [version_name, "disp_logo.jpg"].compact.join('_')
end
Or if you use rails 3.1 and the assets pipeline:
# Include the Sprokets helpers for Rails 3.1+ asset pipeline compatibility:
include Sprockets::Helpers::RailsHelper
include Sprockets::Helpers::IsolatedHelper
# Provide a default URL as a default if there hasn't been a file uploaded:
def default_url
asset_path [version_name, "disp_logo.jpg"].compact.join('_')
end
Edit:
Modify your filename method as follows:
def filename
"something.jpg" unless original_filename.nil?
end
I came up with a solution that fits my use-case better.
class LogoUploader < CarrierWave::Uploader::Base
attr_accessor :upload_failed
def filename
upload_failed ? "failed" : "disp_logo"
end
end
class SomeModel < ActiveRecord::Base
mount_uploader :logo, LogoUploader
def some_method_that_assigns_a_file_and_uploads_it
self.remote_logo_url = <some_url>
OR
self.logo = <some file>
<some_other_logic>
save!
rescue
logo.upload_failed = true
save!
end
end
This way, I know which records were processed:logo #=> "disp_logo", which records failed:logo #=> "failed" and which are yet to be processed:logo #=> nil
This is another question in pursuit of an answer to this question.
In my watermark processor if I set the watermark image's path to a static watermark image, everything works just fine.
I have three models: Watermark, Gallery, and Photo. Watermark has_many :galleries. Gallery belongs_to :watermark and has_many :photos. Photo belongs_to :gallery and mount_uploader :image, PhotoUploader.
Here's what I'd like to do in photo_uploader.rb:
version :large do
process :watermark
process :resize_to_limit => [600, 600]
end
def watermark
manipulate! do |img|
watermark = Magick::Image.read(model.gallery.watermark.image_url).first
img = img.composite(watermark, Magick::CenterGravity, Magick::OverCompositeOp)
end
end
While using model methods/attributes works fine in the store_dir method, it doesn't work in the watermark processor. How can I pass the model.gallery.watermark.image_url argument to the watermark processor?
the problem is the that by the time the image is mounted on the Model, the variables inside the watemark def, inside the uploader class, are not avialiabled. Thats what did. I'm using mongoid to deal with mongoDB. I set a param inside the Model, that will hold the value that i want eg. the value the_current_model.gallery.watermark.image_url. This param will be user_image.
class Asset
include Mongoid::Document
mount_uploader :image, AssetUploader
field :user_image
attr_accessible :user_image
after_save :mark_it
private
def mark_it
image.recreate_versions! if user_image.present?
end
end
The key to solve this problem is recreate the image (as you see on the model calling the mark_it def) after the data being saved on the database and the param user_image allowed to be used for you uploader class.
class AssetUploader < CarrierWave::Uploader::Base
include CarrierWave::RMagick
storage :file
# storage :fog
after :cache, :unlink_original_cache
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
# Provide a default URL as a default if there hasn't been a file uploaded:
# def default_url
# "/images/fallback/" + [version_name, "default.png"].compact.join('_')
# end
process :resize_to_fit => [800, 800]
process :quality => 80
version :thumb do
process :resize_to_fit => [200, 200]
end
version :large do
process :watermark
end
def watermark
if model.user_name.present?
manipulate! do |img|
text = Magick::Draw.new
text.gravity = Magick::CenterGravity
text.fill = 'white'
text.pointsize = 40
text.stroke = 'none'
text.annotate(img, 0, 0, 0, 0, "#{model.user_name}")
img
end
end
end
def unlink_original(file)
File.delete path if version_name.blank?
end
def unlink_original_cache(file)
File.delete if version_name.blank?
end
def extension_white_list
%w(jpg jpeg gif png)
end
# Override the filename of the uploaded files:
# Avoid using model.id or version_name here, see uploader/store.rb for details.
# def filename
# "something.jpg" if original_filename
# end
end
Now the key is here:
def watermark
if model.user_name.present?
manipulate! do |img|
text = Magick::Draw.new
text.gravity = Magick::CenterGravity
text.fill = 'white'
text.pointsize = 40
text.stroke = 'none'
text.annotate(img, 0, 0, 0, 0, "#{model.user_name}")
img
end
end
end
The method watermark only creates the image with the watermark on it, if the user_name param on the model is set (created on the db). So when the data is saved on the db, this param is avaliabled for you uploader class to use as want.
Thats what I did, and it worked very well!
I hope it could help!