I have an upload system that performs several conversion operations on uploaded files (needs the real file paths to do them) and then stores paths of the resulting files.
Those files will be accessed later to create a zip file (again real path needed)
The problem with Capistrano is obviously when I store the file path using Rails.root.join('public', 'uploads', 'designer_user_uploaded') and then try to create my zip file like 10 days later and the release has changed, the stored file path no longer points to the correct location.
My directory is in the linked dirs for Capistrano. So the files are not lost but they just can't be retrieved anymore.
I'm not sure how I should handle this. I think it would be easy to solve by just doing something like File.join(shared_path, 'public', 'uploads', 'designer_user_uploaded') but I can't find how to get the shared path from controllers. Also this might not work in development.
I'm on Rails 4.2.6
output_folder = Rails.root.join('public', 'uploads', 'designer_user_uploaded')
FileUtils.mkdir_p(output_folder) unless File.directory?(output_folder)
file_name = sanitize_filename(Time.now.to_s + "_" + SecureRandom.uuid)
original_file_path = output_folder.join(change_file_extension(file_name, ext))
designer_file_path = ext == '.ai' || ext == '.eps' ?
output_folder.join(change_file_extension(file_name, '.svg')) : output_folder.join(change_file_extension(file_name, ext))
if ext == '.ai' || ext == '.eps'
copy_file(file.path, original_file_path)
copy_file(svg_path, designer_file_path)
#response[:original_file_path] = original_file_path
#response[:designer_file_path] = designer_file_path
#response[:file_name] = file_name + '.svg'
#response[:file_type] = "svg"
#response[:colors] = get_image_colors(original_file_path)
# Store created file paths for future access
#up = UserUpload.create(designer_path: designer_file_path, original_path: original_file_path)
puts #up.id
#response[:file_id] = #up.id
else
copy_file(file.path, designer_file_path)
#response[:original_file_path] = nil
#response[:designer_file_path] = designer_file_path
#response[:file_name] = file_name + ext
#response[:file_type] = ext == '.svg' ? 'svg' : "image"
#response[:colors] = get_image_colors(designer_file_path)
# Store created file path for future access
#up = UserUpload.create(designer_path: designer_file_path)
puts #up.id
#response[:file_id] = #up.id
end
# Embroidery conversion
is_embroidery = params[:is_embroidery].to_s.downcase == 'true'
if is_embroidery && (ext == '.svg' || ext == '.ai' || ext == '.eps')
embroidery_file_name = "embroidery_#{file_name}.svg"
embroidery_path = output_folder.join(embroidery_file_name)
Embroidery::EmbroideryConversionJob.perform_async(original_file_path, embroidery_path, #up)
#response[:designer_file_path] = embroidery_path
#response[:file_name] = embroidery_file_name
#response[:file_type] = "embroidery"
end
#response[:output_url] = '/uploads/designer_user_uploaded/'
file.close(true)
Related
Would it be possible to access the ActiveStorageBlob or ActiveStorageAttachment like it would be a native model ?
E.g.
I want to do ActiveStorageBlob.first to access the first record of this model/table.
or. ActiveStorageAttachment.all.as_json to generate json formated print.
The background idea is to find a way how to dump the content of these ActiveStorage related tables as json formated files. Then change simething on these files, and load it back.
----Extending this text after got correct answer-----
Thank you very much Sarah Marie.
And I hope you know how to load the JSON data back into these tables ?
I have tried this :
dump_file_path = File.join(Rails.root, "backup", active_storage_blobs_file)
load_json = JSON.parse(File.read(dump_file_path))
load_json.each do |j|
ActiveStorage::Blob.create(j)
end
But thats not working.
ActiveModel::UnknownAttributeError (unknown attribute
'attachable_sgid' for ActiveStorage::Blob.)
ActiveStorage::Blob.first
ActiveStorage::Attachment.all.as_json
---- For second extended question ----
ActiveStorage::Blob.create_before_direct_upload!(
filename: j[:filename],
content_type: j[:content_type],
byte_size: j[:byte_size],
checksum: j[:checksum]
)
# or
ActiveStorage::Blob.create_before_direct_upload!(**j.symbolize_keys)
Reference: https://github.com/rails/rails/blob/5f3ff60084ab5d5921ca3499814e4697f8350ee7/activestorage/app/controllers/active_storage/direct_uploads_controller.rb#L8-L9
https://github.com/rails/rails/blob/098fd7f9b3d5c6f540911bc0c17207d6b48d5bb3/activestorage/app/models/active_storage/blob.rb#L113-L120
Now I have a complete solution, how to dump and load the ActiveStorage tables as JSON files.
...dump it
active_storage_blobs_file = "active_storage_blob.json"
active_storage_attachments_file = "active_storage_attachment.json"
puts("...dump active_storage_blob")
dump_file_path = File.join(Rails.root, "backup",active_storage_blobs_file)
dump_file = File.open(dump_file_path, "w")
dump_file.write(JSON.pretty_generate(ActiveStorage::Blob.all.as_json))
dump_file.close()
puts("...dump active_storage_attachment")
dump_file_path = File.join(Rails.root, "backup",
active_storage_attachments_file)
dump_file = File.open(dump_file_path, "w")
dump_file.write(JSON.pretty_generate(ActiveStorage::Attachment.all.as_json))
dump_file.close()
...load it back
puts("...load active_storage_blob")
dump_file_path = File.join(Rails.root, "backup", active_storage_blobs_file)
abort("File does not exist (" + dump_file_path + ") > abort <") unless File.exist?(dump_file_path)
load_json = JSON.parse(File.read(dump_file_path))
load_json.each do |j|
j = j.except("attachable_sgid")
result = ActiveStorage::Blob.create(j)
if (not result.errors.empty?)
puts(result.errors.full_messages.to_s)
puts(j.inspect)
exit(1)
end
end
puts("...load active_storage_attachment")
dump_file_path = File.join(Rails.root, "backup", active_storage_attachments_file)
abort("File does not exist (" + dump_file_path + ") > abort <") unless File.exist?(dump_file_path)
load_json = JSON.parse(File.read(dump_file_path))
load_json.each do |j|
result = ActiveStorage::Attachment.create(j)
if (not result.errors.empty?)
puts(result.errors.full_messages.to_s)
puts(j.inspect)
exit(1)
end
end
What I want to solve
I want to change the name of the file after downloading the zip. I was trying to figure out how to remove the directory name in the middle of this.
In this case, the intermediate directory name is "output".
I would like to know if there is a better way.
2_J000001719_1202テスト/output/WNS_UP用データ.txt
2_J000001719_1202テスト/output/Images/1051687701.jpg
↓
2_J000001719_1202テスト/WNS_UP用データ.txt
2_J000001719_1202テスト/Images/1051687701.jpg
Original Source Code
add_dir_name = "#{order.priority}_#{order.orderno}_#{company_name[0, 15]}"
Zip::File.open_buffer(obj) do |zip|
zip.each do |entry|
ext = File.extname(entry.name)
file_name = File.basename(entry.name)
dir_name = File.dirname(entry.name)
next if ext.blank? || file_name.count(".") > 1
dir = File.join(add_dir_name, dir_name)
FileUtils.mkpath(dir.to_s)
zip.extract(entry, dir + ext) {true}
file_name.force_encoding("UTF-8")
new_file_name = "#{dir}/#{file_name}"
new_file_name.force_encoding("UTF-8")
File.rename(dir + ext, new_file_name)
#input_dir << new_file_name
end
end
What I tried
I added this method to the source code, but it did not work.
new_dir = Dir.glob(dir_name+"/*").last
demdem = new_dir.split('/').last
new_file_name = dir.sub("/" + demdem, "")
Here's a generic function to move all files (recursively) from one dir to another:
orig_dir = "2_J000001719_1202テスト/output"
new_dir = "2_J000001719_1202テスト"
def move_all_files(orig_dir, new_dir)
Dir.glob("#{orig_dir}/**/*").each do |path|
File.move(path, path.gsub(orig_dir, new_dir))
end
end
I am making pdf file from DataUrl Image in ruby on rails.
I have selected RBPDF to produce pdf file in server side.
But in this code I have following error
#pdf.Image(object["src"] , object["left"], object["top"], object["width"], object["height"])
Here object["src"] is DataUrl Image.
RuntimeError (RBPDF error: Missing image file:
data:image/jpeg;base64,/9j/4REORXhp...
Is it impossible to add RBPDF image from DataUrl image?
Adding files dynamically is not effective I think.
You may monkey patch the origin method.
I use the data_uri gem to parse the image data.
require 'data_uri'
require 'rmagick'
module Rbpdf
alias_method :old_getimagesize, :getimagesize
# #param [String] date_url
def getimagesize(date_url)
if date_url.start_with? 'data:'
uri = URI::Data.new date_url
image_from_blob = Magick::Image.from_blob(uri.data)
origin_process_image(image_from_blob[0])
else
old_getimagesize date_url
end
end
# this method is extracted without comments from the origin implementation of getimagesize
def origin_process_image(image)
out = Hash.new
out[0] = image.columns
out[1] = image.rows
case image.mime_type
when "image/gif"
out[2] = "GIF"
when "image/jpeg"
out[2] = "JPEG"
when "image/png"
out[2] = "PNG"
when " image/vnd.wap.wbmp"
out[2] = "WBMP"
when "image/x-xpixmap"
out[2] = "XPM"
end
out[3] = "height=\"#{image.rows}\" width=\"#{image.columns}\""
out['mime'] = image.mime_type
case image.colorspace.to_s.downcase
when 'cmykcolorspace'
out['channels'] = 4
when 'rgbcolorspace', 'srgbcolorspace' # Mac OS X : sRGBColorspace
if image.image_type.to_s == 'GrayscaleType' and image.class_type.to_s == 'PseudoClass'
out['channels'] = 0
else
out['channels'] = 3
end
when 'graycolorspace'
out['channels'] = 0
end
out['bits'] = image.channel_depth
out
end
end
I have a model that has a method that looks through the filesystem starting at a particular location for files that match a particular regex. This is executed in an after_save callback. I'm not sure how to test this using Rspec and FactoryGirl. I'm not sure how to use something like FakeFS with this because the method is in the model, not the test or the controller. I specify the location to start in my FactoryGirl factory, so I could change that to a fake directory created by the test in a set up clause? I could mock the directory? I think there are probably several different ways I could do this, but which makes the most sense?
Thanks!
def ensure_files_up_to_date
files = find_assembly_files
add_files = check_add_assembly_files(files)
errors = add_assembly_files(add_files)
if errors.size > 0 then
return errors
end
update_files = check_update_assembly_files(files)
errors = update_assembly_files(update_files)
if errors.size > 0 then
return errors
else
return []
end
end
def find_assembly_files
start_dir = self.location
files = Hash.new
if ! File.directory? start_dir then
errors.add(:location, "Directory #{start_dir} does not exist on the system.")
abort("Directory #{start_dir} does not exist on the system for #{self.inspect}")
end
Find.find(start_dir) do |path|
filename = File.basename(path).split("/").last
FILE_TYPES.each { |filepart, filehash|
type = filehash["type"]
vendor = filehash["vendor"]
if filename.match(filepart) then
files[type] = Hash.new
files[type]["path"] = path
files[type]["vendor"] = vendor
end
}
end
return files
end
def check_add_assembly_files(files=self.find_assembly_files)
add = Hash.new
files.each do |file_type, file_hash|
# returns an array
file_path = file_hash["path"]
file_vendor = file_hash["vendor"]
filename = File.basename(file_path)
af = AssemblyFile.where(:name => filename)
if af.size == 0 then
add[file_path] = Hash.new
add[file_path]["type"] = file_type
add[file_path]["vendor"] = file_vendor
end
end
if add.size == 0 then
logger.error("check_add_assembly_files did not find any files to add")
return []
end
return add
end
def check_update_assembly_files(files=self.find_assembly_files)
update = Hash.new
files.each do |file_type, file_hash|
file_path = file_hash["path"]
file_vendor = file_hash["vendor"]
# returns an array
filename = File.basename(file_path)
af = AssemblyFile.find_by_name(filename)
if !af.nil? then
if af.location != file_path or af.file_type != file_type then
update[af.id] = Hash.new
update[af.id]['path'] = file_path
update[af.id]['type'] = file_type
update[af.id]['vendor'] = file_vendor
end
end
end
return update
end
def add_assembly_files(files=self.check_add_assembly_files)
if files.size == 0 then
logger.error("add_assembly_files didn't get any results from check_add_assembly_files")
return []
end
asm_file_errors = Array.new
files.each do |file_path, file_hash|
file_type = file_hash["type"]
file_vendor = file_hash["vendor"]
logger.debug "file type is #{file_type} and path is #{file_path}"
logger.debug FileType.find_by_type_name(file_type)
file_type_id = FileType.find_by_type_name(file_type).id
header = file_header(file_path, file_vendor)
if file_vendor == "TBA" then
check = check_tba_header(header, file_type, file_path)
software = header[TBA_SOFTWARE_PROGRAM]
software_version = header[TBA_SOFTWARE_VERSION]
elsif file_vendor == "TBB" then
check = check_tbb_header(header, file_type, file_path)
if file_type == "TBB-ANNOTATION" then
software = header[TBB_SOURCE]
else
software = "Unified"
end
software_version = "UNKNOWN"
end
if check == 0 then
logger.error("skipping file #{file_path} because it contains incorrect values for this filetype")
asm_file_errors.push("#{file_path} cannot be added to assembly because it contains incorrect values for this filetype")
next
end
if file_vendor == "TBA" then
xml = header.to_xml(:root => "assembly-file")
elsif file_vendor == "TBB" then
xml = header.to_xml
else
xml = ''
end
filename = File.basename(file_path)
if filename.match(/~$/) then
logger.error("Skipping a file with a tilda when adding assembly files. filename #{filename}")
next
end
assembly_file = AssemblyFile.new(
:assembly_id => self.id,
:file_type_id => file_type_id,
:name => filename,
:location => file_path,
:file_date => creation_time(file_path),
:software => software,
:software_version => software_version,
:current => 1,
:metadata => xml
)
assembly_file.save! # exclamation point forces it to raise an error if the save fails
end # end files.each
return asm_file_errors
end
Quick answer: you can stub out model methods like any others. Either stub a specific instance of a model, and then stub find or whatever to return that, or stub out any_instance to if you don't want to worry about which model is involved. Something like:
it "does something" do
foo = Foo.create! some_attributes
foo.should_receive(:some_method).and_return(whatever)
Foo.stub(:find).and_return(foo)
end
The real answer is that your code is too complicated to test effectively. Your models should not even know that a filesystem exists. That behavior should be encapsulated in other classes, which you can test independently. Your model's after_save can then just call a single method on that class, and testing whether or not that single method gets called will be a lot easier.
Your methods are also very difficult to test, because they are trying to do too much. All that conditional logic and external dependencies means you'll have to do a whole lot of mocking to get to the various bits you might want to test.
This is a big topic and a good answer is well beyond the scope of this answer. Start with the Wikipedia article on SOLID and read from there for some of the reasoning behind separating concerns into individual classes and using tiny, composed methods. To give you a ballpark idea, a method with more than one branch or more than 10 lines of code is too big; a class that is more than about 100 lines of code is too big.
This is my method for uploading files:
name = file.original_filename
directory = "images/"
path = File.join(directory, name)
File.open(path, "wb") { |f| f.write(file.read) }
I need to rename the uploaded file - I want to give it a unique name. But how can I obtain the file name and the extension?
One way on how to rename the file is from the filename remove the extension (.jpg - remove last 4 chars), rename the file and then merge the name+extension.
But this is a bit dirty way. Is there something cleaner and more elegant?
A 'little' late but I put this answer for those who are still searching and get here.
photo = params[:photo]
name = photo.original_filename
directory = "public/uploads/photos"
path = File.join(directory, name)
uniq_name = (0...10).map { (65 + rand(26)).chr }.join
time_footprint = Time.now.to_formatted_s(:number)
File.open(path, "wb") do |file|
file.write(photo.read)
#uniq_path = File.join(directory, uniq_name + time_footprint + File.extname(file))
File.rename(file, #uniq_path)
end
I take the random string generation from How to generate a random string in Ruby. And set #uniq_path to use it on a create function after.
What about doing this?
File.rename(file, folder_path + "/" + new_name + File.extname(file))
For example, calling this script on the same folder of the file:
new_name = "TESTING"
File.open("test.txt") do |file|
File.rename(file, new_name + File.extname(file))
end
Will rename the file to: TESTING.txt