CarrierWave Backgrounder not uploading versions images to AWS S3 - ruby-on-rails

I was using carrierwave 0.10.0 gem with RMagic to upload images on AWS S3. Everything was working fine except it was taking too much time to upload on AWS S3. So thought using carrierwave backgrounder to upload images in background. I set up carrierwave backgrounder (0.4.2) but In this one My original file is always get upload to S3 but versions of that image is never gets uploaded on S3.
Here is my carrierwave_backgrounder.rb
CarrierWave::Backgrounder.configure do |c|
c.backend :sidekiq, queue: :carrierwave
end
I have defined my queue in sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { :url => "redis://#{ENV['REDIS_ENDPOINT']}:6379", :namespace=> "#{ENV['REDIS_NAMESPACE']}" }
config.options =
queues: %w{
critical
carrierwave
}
})
end
Here is my photo_uploader.rb
class PhotoUploader < CarrierWave::Uploader::Base
include ::CarrierWave::Backgrounder::Delay
include CarrierWave::RMagick
storage :fog
def store_dir
"uploads/images/"
end
def filename
"#{secure_token}.#{file.extension}" if original_filename.present?
end
def orient_image
manipulate! do |img|
img.auto_orient
img
end
end
# Create different versions of your uploaded files:
version :thumb_small do
process :resize_to_fill => [100,100]
process :strip
end
def strip
manipulate! do |img|
img.strip!
img = yield(img) if block_given?
img
end
end
def extension_white_list
%w(jpg jpeg gif png)
end
def get_version_dimensions
model.width, model.height = `identify -format "%wx%h " #{file.path}`.split(/x/)
end
protected
def secure_token
var = :"##{mounted_as}_secure_token"
model.instance_variable_get(var) || model.instance_variable_set(var, SecureRandom.hex(5))
end
end
Here is my profile.rb file
mount_uploader :image_url, PhotoUploader
process_in_background :image_url
I have started the sidekiq worker using this command
bundle exec sidekiq -d -L log/sidekiq.log -C config/sidekiq.yml -e development
When I upload image_url only the original image is uploaded. This is sidekiq log after uploading original file. But I don't see any version file uploaded. I checked the S3 bucket also(No version file only the original file)
2016-01-11T08:52:20.772Z 3983 TID-ownpyrrxk CarrierWave::Workers::ProcessAsset JID-91e3803d50defb2d1419cef1 INFO: start
2016-01-11T08:52:31.119Z 3983 TID-ownpyrrxk CarrierWave::Workers::ProcessAsset JID-91e3803d50defb2d1419cef1 INFO: done: 10.347 sec
Is There something I am missing. Please Help
Thanks in Advance

After investigating with few documentations, here is my suggestion:
From careerwave_backgrounder readme: https://github.com/lardawge/carrierwave_backgrounder#background-options
Its clearly shows,
# This stores the original file with no processing/versioning.
# It will upload the original file to s3.
From this #113 , the author said
I found a bug related to Rmagick but no issue with versions
You can try with MiniMagick/ImageMagick instead of RMagick.
Documentation to look for the similar issue:
https://github.com/lardawge/carrierwave_backgrounder/issues/113
https://github.com/lardawge/carrierwave_backgrounder/issues/130
Rails CarrierWave versions are not created for some reason
Thanks!

Related

heroku with carrierwave dropbox storage and rails showing always same image

so I have a web application that runs fine in development, and with carrierwave and imagemagick I make some changes on the photos that I need to upload. The problem is that when on heroku when i ask for the main version of the photo it still gives me the thumb version
class BannerUploader < CarrierWave::Uploader::Base
include CarrierWave::MiniMagick
if Rails.env.development?
storage :file
else
storage :dropbox
end
def store_dir
if version_name.nil?
"uploads/#{model.class.to_s.underscore}/#{mounted_as}"
else
"uploads/#{model.class.to_s.underscore}/#{mounted_as}_#{version_name}"
end
end
process convert: :jpg
process :crop
process resize_to_limit: [1200, 260]
def crop
if model.crop_x.present?
manipulate! do |img|
x = model.crop_x.to_i
y = model.crop_y.to_i
w = model.crop_w.to_i
h = model.crop_h.to_i
r = model.crop_r.to_i
img.rotate r
img.crop([[w, h].join('x'), [x, y].join('+')].join('+'))
end
end
end
version :thumb do
process resize_to_fill: [640, 320]
end
def extension_whitelist
%w(jpg jpeg png)
end
def filename
if original_filename
"#{secure_token}.#{file.extension}"
elsif file
file.filename
end
end
protected
def secure_token
var = :"##{mounted_as}_secure_token"
model.instance_variable_get(var) or model.instance_variable_set(var, SecureRandom.uuid)
end
end
this is my uploader
then in my views i have image_tag(model.banner_url(:thumb) to get the thumb version and image_tag #model.banner_url to get the large version. The problem is that the second one on my local machine runs just fine, but when on heroku it gives me the same image of the first one. It does create the right folders and files, and it does crop them right, but it doesn't retrieve the correct one. I am using the
gem 'dropbox-sdk-v2', '~> 0.0.3'
gem 'carrierwave-dropbox', '~> 2.0'
as heroku storage, with obviously a dropbox account
Often you'll want to add different versions of the same file. The
classic example is image thumbnails. There is built in support for
this*
I think the problem is the way you define the store_dir method.
Define it this way:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
And also CarrierWave documentation recommends to use the uploader this way:
uploader = AvatarUploader.new
uploader.store!(my_file) # size: 1024x768
uploader.url # => '/url/to/my_file.png' # size: 800x800
uploader.thumb.url # => '/url/to/thumb_my_file.png' # size: 200x200
Source: https://github.com/carrierwaveuploader/carrierwave

Carrierwave upload to Amazon S3 from a rake task: can't get upload to work

I am trying to generate a csv file in a rake task and...
Email it
Upload it to Amazon s3.
Here is the task.
desc "This task is called by the Heroku scheduler add-on"
require 'csv'
task :send_report => :environment do
file = Baseline.to_csv
ReportMailer.database_report(file).deliver_now
Report.create!(:data => file)
end
The generation of the csv file and attachment to the email works fine (not shown). Its the carrierwave upload that isn't working. Please note that I have other uploaders for other models and they work fine so my bucket settings are correct.
Here are the other files.
class Report < ActiveRecord::Base
mount_uploader :data, ReportUploader
end
and
class ReportUploader < CarrierWave::Uploader::Base
storage :fog
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
def extension_white_list
%w(jpg jpeg gif png csv xls)
end
end
I have tried various permutations such as store! with not luck. I should add that if I look at the database, the new report is being created (and the data attribute is "nil", with no upload in sight)
Thanks

Cannot access the pictures uploaded by Carrierwave versions in production environment

This is my Uploader:
class PhotoUploader < CarrierWave::Uploader::Base
include CarrierWave::MiniMagick
process :resize_to_fit => [nil, 600]
version :thumb do
process :resize_to_fill => [150,150]
end
# Choose what kind of storage to use for this uploader:
storage :file
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
def cache_dir
"#{Rails.root}/tmp/uploads"
end
def extension_white_list
%w(jpg jpeg gif png)
end
def filename
if original_filename
#name ||= Digest::MD5.hexdigest(current_path)
"#{#name}.#{file.extension}"
end
end
end
In production.rb, I set config.serve_static_assets = false.
And I then deployed this project on a production server(Nginx + Passenger) using Capistrano. When I upload a picture, it will generate 2 copies under the /home/deploy/Gallary/current/public/uploads/picture/photo/ dir, just as following shows:
And I can access the first one through browser(because this one is the default file which Carrierwave generated), while the second one(generated by version :thumb) threw a exception just like this:
ActionController::RoutingError (No route matches [GET] "/uploads/picture/photo/49/thumb_6d9596c7449d3714eadb74b9c71beec2.jpg")
Actually this file thumb_6d9596c7449d3714eadb74b9c71beec2.jpg does exist in the dic right there.
So, what's wrong? And what should I do?
I had a similar problem. Found the answer in another thread.
So try this:
Server unable to find public folder in rails 3 production environment
Worked for me

CarrierWave: Uploads and saves original file...I don't want it to

I am using carrierwave_backgrounder to upload images to S3 in a background process with Sidekiq.
Here's my background_uploader.rb class...
class BackgroundUploader < CarrierWave::Uploader::Base
include ::CarrierWave::Backgrounder::Delay
include CarrierWave::RMagick
include CarrierWave::MimeTypes
process :set_content_type
# Include the Sprockets helpers for Rails 3.1+ asset pipeline compatibility:
include Sprockets::Helpers::RailsHelper
include Sprockets::Helpers::IsolatedHelper
storage :fog
def store_dir
"uploads/backgrounds/#{model.id}"
end
def default_url
"/assets/default.jpg"
end
process :resize_to_fit => [1024, 1024]
process :convert => 'jpg'
process :fix_exif_rotation
def extension_white_list
%w(jpg jpeg png)
end
def filename
#name ||= Digest::MD5.hexdigest(File.dirname(current_path.to_s))
"#{#name}.#{file.extension}" if original_filename
end
# Rotates the image based on the EXIF Orientation & applies gaussian blur
def fix_exif_rotation
manipulate! do |img|
img.auto_orient!
img = yield(img) if block_given?
img = img.gaussian_blur(0.0, 20.0)
img
end
end
end
carrierwave_backgrounder.rb:
CarrierWave::Backgrounder.configure do |c|
c.backend :sidekiq, queue: :carrierwave
end
background.rb contains:
mount_uploader :image, BackgroundUploader
process_in_background :image
I then run sidekiq -q carrierwave to fire up the background workers. All is working good! Upload a file, I see the queue accept it and start working...
If I immediately open up my AWS S3 Console I see the original file in there. Non-resized and un-blurred. Once the job is complete...I refresh S3 and there's the resized/blurred version. Now both images are in there, but I only want the blurred image to be there. In my view I use...
<%= image_tag(#background.image.to_s) %>
It displays the original file. If I check the checkbox to remove the file, it does so as it should (deletes the original from S3), but the blurred version stays there.
What's getting uploaded to S3...
original.jpg (immediate...I don't want this uploaded at all)
modified.jpg (after job completes)
Long story short: I don't want the original file to upload to S3.
I think your problem is the filename method, which carrierwave may rely on to find (and delete) the original file. Does the problem go away when you use a filename that doesn't change between when it's initially stored and when you process it?

exif image rotation issue using carrierwave and rmagick to upload to s3

I've got a photo upload feature in my rails app. The app uploads direct to s3 through carrierwave via rmagick and fog. The issue I am having is when a photo is uploaded via mobile through the "take a photo option" in portrait (note this is with iphone but I believe android has the same issue). Once uploaded the image appears fine on mobile, however when viewed on desktop the image appears rotated 90 degrees.
Through my research it looks to be an issue with exif. This stackoverflow responder outlines 2 potential solutions. This gist also looks promising as well.
So far I have found a few solutions posted but none have worked. Ideally I would like the photo to be saved to s3 as a portrait, then just display the image as is.
Any suggestions are well appreciated.
Below is my code
app/uploaders/image_uploader.rb
class ImageUploader < CarrierWave::Uploader::Base
include CarrierWaveDirect::Uploader
include CarrierWave::RMagick
# Include the Sprockets helpers for Rails 3.1+ asset pipeline compatibility:
include Sprockets::Helpers::RailsHelper
include Sprockets::Helpers::IsolatedHelper
include CarrierWave::MimeTypes
process :fix_exif_rotation
process :set_content_type
version :thumb do
process resize_to_fill: [200, 200]
end
def extension_white_list
%w(jpg jpeg png)
end
def fix_exif_rotation #this is my attempted solution
manipulate! do |img|
img = img.auto_orient!
end
end
end
app/models/s3_image.rb
class S3Image < ActiveRecord::Base
attr_accessible :image, :name, :user_id
mount_uploader :image, ImageUploader
belongs_to :user
def image_name
File.basename(image.path || image.filename) if image
end
class ImageWorker
include Sidekiq::Worker
def perform(id, key)
s3_image = S3Image.find(id)
s3_image.key = key
s3_image.remote_image_url = s3_image.image.direct_fog_url(with_path: true)
s3_image.save!
s3_image.update_column(:image_processed, true)
end
end
end
config/initializers/carrierwave.rb
CarrierWave.configure do |config|
config.fog_credentials = {
provider: "AWS",
aws_access_key_id: " ... ",
aws_secret_access_key: " ... "
}
config.fog_directory = " ... "
end
btw I used this Railscast as a guide for setting up my s3 upload.
Well I got this working using fog instead or carrierwave_direct.
Below is the code that ended up working for me:
app/uploaders/image_uploader.rb
class ImageUploader < CarrierWave::Uploader::Base
include CarrierWave::MiniMagick
include Sprockets::Helpers::RailsHelper
include Sprockets::Helpers::IsolatedHelper
storage :fog
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
def fix_exif_rotation #this is my attempted solution
manipulate! do |img|
img.tap(&:auto_orient)
end
end
process :fix_exif_rotation
end
app/models/s3_image.rb
class S3Image < ActiveRecord::Base
attr_accessible :image, :name, :user_id, :image_cache
mount_uploader :image, ImageUploader
belongs_to :user
end
initializers/carrierwave.rb
CarrierWave.configure do |config|
config.fog_credentials = {
provider: "AWS",
aws_access_key_id: " ... ",
aws_secret_access_key: " ... ",
region: 'us-west-2'
}
config.fog_directory = " ... "
end
I had a similar problem and fixed it with an approach nearly identical to yours.
# In the uploader:
def auto_orient
manipulate! do |img|
img = img.auto_orient
end
end
(Note that I am not calling auto_orient! - just auto_orient, without the bang.)
Then I have process :auto_orient as the first line of any version I create. For example:
version :square do
process :auto_orient
process :resize_to_fill => [600, 600]
end
My solution (quite similar to Sumeet) :
# painting_uploader.rb
process :right_orientation
def right_orientation
manipulate! do |img|
img.auto_orient
img
end
end
It's really important to return an image. Otherwise, you'll get an
NoMethodError (undefined method `write' for "":String):
Lando2319's answer was not working for me.
I am using RMagick.
I managed to make ImageMagick apply the correct orientation (and to reset the EXIF rotation data in order to avoid a double rotation by the viewer) by using :
def fix_exif_rotation # put this before any other process in the Carrierwave uploader
manipulate! do |img|
img.tap(&:auto_orient!)
end
The difference between my solution & Lando's is the bang (!). In my case it was absolutely necessary.

Resources