Carrierwave precalculated md5 checksum of a file as a filename - ruby-on-rails

Question

Using carrierwave uploader for images, trying to provide uniqueness of uploaded images using md5 checksum as filename
looks like I'm doing something wrong
model is defined like:
class Image < ActiveRecord::Base
attr_accessible :description, :img
mount_uploader :img, ImageUploader
My uploader code is as following:
class ImageUploader < CarrierWave::Uploader::Base
include CarrierWave::MiniMagick
storage :file
def store_dir
"images/#{filename[0,2]}"
end
def md5
#md5 ||= ::Digest::MD5.file(current_path).hexdigest
end
def filename
#name ||= "#{md5}#{::File.extname(current_path)}" if super
end
first of all, I suspect this approach inflicts calculation of checksum each time image entry is queried to display
secondly, after image entry is saved, every other of img.original_filename img.filename img.path img.current_path seem to be undefined with following error:
You have a nil object when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.[]
app/uploaders/image_uploader.rb:17:in `store_dir'
carrierwave (0.5.7) lib/carrierwave/uploader/store.rb:43:in `store_path'
carrierwave (0.5.7) lib/carrierwave/storage/file.rb:41:in `retrieve!'
carrierwave (0.5.7) lib/carrierwave/uploader/store.rb:95:in `block in retrieve_from_store!'
carrierwave (0.5.7) lib/carrierwave/uploader/callbacks.rb:17:in `with_callbacks'
carrierwave (0.5.7) lib/carrierwave/uploader/store.rb:94:in `retrieve_from_store!'
carrierwave (0.5.7) lib/carrierwave/mount.rb:311:in `uploader'
any kind of help or tip is appreciated
UPD:
changed uploader this way:
def store_dir
"images/#{model.img_identifier[0,2]}"
end
def filename
#name ||= "#{md5}#{::File.extname(current_path)}"
end
protected
def md5
var = :"##{mounted_as}_md5"
model.instance_variable_get(var) or model.instance_variable_set(var, ::Digest::MD5.file(current_path).hexdigest)
end
current_path seems to refer to full path of form-submitted tempfile, thus being valid for extension extraction and digest calculation
img_identifier stands for persisting resulting filename and thus goes valid for prefix extraction for our store_dir
still not sure if any caveat is induced with this approach
also still not convinced about the way file uniqueness validation should be performed
UPD:
I've added this before_validation callback in my model class:
validates_uniqueness_of :checksum
before_validation :assign_checksum
def assign_checksum
self.checksum = img.md5 if img.present? and img_changed?
end
where checksum is a separate string field in my model's db table
it is quite redundant as it duplicates the img field in general, but I still can't figure out the way to validate uniqueness of img itself.
UPD:
Moved away from db redundancy this way. In my model:
validate :img_uniqueness
def img_uniqueness
errors.add :img, "Image already exists in database" if Image.where(:img => self.img.filename).first
end
now there's no need in checksum field

Answer 1

This could help:
How to: Use file`s MD5 as filename
https://github.com/carrierwaveuploader/carrierwave/wiki/How-to:-Use-file%60s-MD5-as-filename
And maybe
How-to: Use file's digest (e.g. MD5, SHA-1) as file path
https://github.com/carrierwaveuploader/carrierwave/wiki/How-to:-Use-file's-digest-(e.g.-MD5,-SHA-1)-as-file-path

Answer 2

Add a md5hash field to your model, then add the following code to your Model:
before_validation :compute_hash
validates_uniqueness_of :md5hash, :on => :create
def compute_hash
self.md5hash = Digest::MD5.hexdigest(self.file.read)
end
That should do the trick.

Answer 3

1. When you define store_dir .. filename seems to be NIL!!
That seems to be the immediate error -- try a puts statement to print out what filename is set to..
If the filename is NIL, you'll see the error you're seeing:
You have a nil object when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.[]
app/uploaders/image_uploader.rb:17:in `store_dir'
NOTE:
you're overriding both filename and store_dir ... and using filename inside the definition of store_dir...
There could be a "chicken and the egg" type of problem here.. better check on that
store_dir should just be a directory, e.g. /somewhere/on/your/disk/images
filename should just be a filename without path, e.g. 24371592d9ea16625854ed68ac4b5846 , or 24371592d9ea16625854ed68ac4b5846.jpg
e.g. check how those two are used in the code in store.rb (at end below)
Question:
using filename[0,2] - you're using directories with 2-letter prefix of the MD5-sum to store the images?
2. Side-note: What is current_path?? Seems a misnomer. Should be a path+filename, not just a path
3. Check the second code snippet (below) from store.rb
Seems like you set store_dir to a relative directory -- that's very fragile and error prone.. it might be a better idea to set it to an absolute path (starting with '/')
4. Try setting filename and store_dir to constants for debugging
Just as a sanity check, does it work when you do this:
def store_dir
'/tmp/' # from the source code, it looks like this needs to start with '/' !!
end
def filename
'my_uploaded_file'
end
that should work before setting filename to MD5 sum..
From the source code: (0.5.7)
lib/carrierwave/storage/file.rb
lib/carrierwave/uploader/store.rb
You can override CarrierWave::Uploader::Store#filename to point to a filename of your choice (see source code):
Overriding it in CarrierWave::Uploader::Base should work, as 'Store' is included in 'Base' ; this overrides the default filename.
Fromstore.rb
# Override this in your Uploader to change the filename.
#
# Be careful using record ids as filenames. If the filename is stored in the database
# the record id will be nil when the filename is set. Don't use record ids unless you
# understand this limitation.
#
# Do not use the version_name in the filename, as it will prevent versions from being
# loaded correctly.
#
# === Returns
#
# [String] a filename
#
def filename
#filename
end
You could also check for the cached filename, compute the MD5-sum (or better SHA1 sum) on it, and then use the result to name the file.
Fromstore.rb
# Calculates the path where the file should be stored. If +for_file+ is given, it will be
# used as the filename, otherwise +CarrierWave::Uploader#filename+ is assumed.
#
# === Parameters
#
# [for_file (String)] name of the file <optional>
#
# === Returns
#
# [String] the store path
#
def store_path(for_file=filename) # DEFAULT argument is filename - you don't need to override both
File.join([store_dir, full_filename(for_file)].compact)
end
File Equality:
files_equal?( filename1, filename2 )
return true if File.size(filename1) == File.size(filename2)
# return MD5(filename1) == MD5(filename2)
# as we assume that the filename == MD5 + some suffix :
return File.basename(filename1) == File.basename(filename2) # if names == MD5 are the same
end

Answer 4

I would consider the wiki way for file name uniquess,
https://github.com/jnicklas/carrierwave/wiki/How-to%3A-Create-random-and-unique-filenames-for-all-versioned-files