The following code is meant to aggregate a handful of PDF docs and send a zip file with all the PDF's contained. The code works, but only 50% of the time.
def donor_reports
#starting_date = params['date'].to_date
#ending_date = #starting_date.to_date.end_of_month
#categories = Category.all
zipfile_name = Tempfile.new(['records', '.zip'])
Zip::File.open(zipfile_name.path, Zip::File::CREATE) do |zipfile|
#categories.each do |category|
temp_pdf = Tempfile.new(['record', '.pdf'])
temp_pdf.binmode
temp_prawn_pdf = CategoryReport.new
temp_prawn_pdf.generate_report(category, #starting_date, #ending_date)
temp_pdf.write temp_prawn_pdf.render
temp_pdf.rewind
zipfile.add("#{category.name} - Donation Report.pdf", "#{temp_pdf.path}")
end
end
send_file zipfile_name
end
The other 50% of the time it throws the following error -
No such file or directory # rb_sysopen - /var/folders/jy/w7tv9n8n7tqgtgmv49qz7zqh0000gn/T/record20160114-10768-1guyoar.pdf
Any advice/guidance here would be much appreciated! First time using Tempfile and Rubyzip.
This is probably because the inner Tempfile gets garbage-collected and deleted from disk (sometimes) before the zip gem actually references when the zipfile block ends. I just push these into an array to get around it:
```
temp_files = []
Zip::File.open(temp_file.path, Zip::File::CREATE) do |zip|
#categories.each do |category|
temp_pdf = Tempfile.new(['record', '.pdf'])
#...
temp_files << temp_pdf
end
end
```
Related
For a variety of reasons I am migrating my uploads from ActiveStorage (AS) to CarrierWave (CW).
I am making rake task and have the logic sorted out - I am stumped at how to feed the AS blob into the CW file.
I am trying something like ths:
#files.each.with_index(1) do | a, index |
if a.attachment.attached?
a.attachment.download do |file|
a.file = file
end
a.save!
end
end
This is based on these two links:
https://edgeguides.rubyonrails.org/active_storage_overview.html#downloading-files
message.video.open do |file|
system '/path/to/virus/scanner', file.path
# ...
end
and
https://github.com/carrierwaveuploader/carrierwave#activerecord
# like this
File.open('somewhere') do |f|
u.avatar = f
end
I tested this locally and the files are not mounted via the uploader. My question(s) would be:
am I missing something obvious here?
is my approach wrong and needs a new one?
Bonus Karma Question:
I can't seem to see a clear path to set the CW filename when I do this?
Here is my final rack task (based on the accepted answer) - open to tweaks. Does the job for me:
namespace :carrierwave do
desc "Import the old AS files into CW"
task import: :environment do
#files = Attachment.all
puts "#{#files.count} files to be processed"
puts "+" * 50
#files.each.with_index(1) do | a, index |
if a.attachment.attached?
puts "Attachment #{index}: Key: #{a.attachment.blob.key} ID: #{a.id} Filename: #{a.attachment.blob.filename}"
class FileIO < StringIO
def initialize(stream, filename)
super(stream)
#original_filename = filename
end
attr_reader :original_filename
end
a.attachment.download do |file|
a.file = FileIO.new(file, a.attachment.blob.filename.to_s)
end
a.save!
puts "-" * 50
end
end
end
desc "Purge the old AS files"
task purge: :environment do
#files = Attachment.all
puts "#{#files.count} files to be processed"
puts "+" * 50
#files.each.with_index(1) do | a, index |
if a.attachment.attached?
puts "Attachment #{index}: Key: #{a.attachment.blob.key} ID: #{a.id} Filename: #{a.attachment.blob.filename}"
a.attachment.purge
puts "-" * 50
#count = index
end
end
puts "#{#count} files purged"
end
end
Now in my case I am doing this in steps - I have branched my master with this rake task and the associated MCV updates. If my site was in true production would probably run the import rake task first then confirm all went well THEN purge the old AS files.
The file object you get from the attachment.download block is a string. More precisely, the response from .download is the file, "streamed and yielded in chunks" (see documentation). I validated this by calling file.class to make sure the class is what I expected.
So, to solve your issue, you need to provide an object on which .read can be called. Commonly that is done using the Ruby StringIO class.
However, considering Carrierwave also expects a filename, you can solve it using a helper model that inherits StringIO (from blogpost linked above):
class FileIO < StringIO
def initialize(stream, filename)
super(stream)
#original_filename = filename
end
attr_reader :original_filename
end
And then you can replace a.file = file with a.file = FileIO.new(file, 'new_filename')
I am using Rails 5.2 and ActiveStorage to let my users upload files in my app. I have all the uploaded files displayed in a table and I can choose to select which ones I want to download. After selecting, I will be able to download all of them in a zip file. To zip the files, I am using Rubyzip and I can't get to work properly.
I have tried two ways :
1 - I tried this way and faced this error No such file or directory # rb_sysopen - /rails/active_storage/blobs/..../2234.pyThis is my controller :
def batch_download
if params["record"].present?
ids = params["record"].to_unsafe_h.map(&:first)
if ids.present?
folder_path = "#{Rails.root}/public/downloads/"
zipfile_name = "#{Rails.root}/public/archive.zip"
FileUtils.remove_dir(folder_path) if Dir.exist?(folder_path)
FileUtils.remove_entry(zipfile_name) if File.exist?(zipfile_name)
Dir.mkdir("#{Rails.root}/public/downloads")
Record.where(id: ids).each do |attachment|
open(folder_path + "#{attachment.file.filename}", 'wb') do |file|
file << open("#{rails_blob_path(attachment.file)}").read
end
end
input_filenames = Dir.entries(folder_path).select {|f| !File.directory? f}
Zip::File.open(zipfile_name, Zip::File::CREATE) do |zipfile|
input_filenames.each do |attachment|
zipfile.add(attachment,File.join(folder_path,attachment))
end
end
send_file(File.join("#{Rails.root}/public/", 'archive.zip'), :type => 'application/zip', :filename => "#{Time.now.to_date}.zip")
end
else
redirect_back fallback_location: root_path
end
end
2 - Secondly I tried to follow the rubyzip documentation and the error was a bit different.
No such file or directory # rb_file_s_lstat - /rails/active_storage/blobs/..../2234.py
if ids.present?
folder = []
input_filenames = []
Record.where(id: ids).each do |attachment|
input_filenames.push("#{attachment.file.filename}")
pre_path = "/rails/active_storage/blobs/"
path_find = "#{rails_blob_path(attachment.file)}"
folder.push(pre_path + path_find.split('/')[4])
end
container = Hash[folder.zip(input_filenames)]
zipfile_name = "/Users/fahimabdullah/Documents/archive.zip"
Zip::File.open(zipfile_name, Zip::File::CREATE) do |zipfile|
# input_filenames.each do |filename|
container.map do |path, filename|
zipfile.add(filename, File.join(path, filename))
end
zipfile.get_output_stream("myFile") { |f| f.write "myFile contains just this" }
end
I expect it to download a zip file containing all the files inside. And this is my first question so please excuse me if the question was too long. Thank you.
I just had a similar problem but saw this hasn't been answered yet. Even though it's a little older and you probably already solved this, here is my approach. The trick here was to create a tempfile in between
def whatever
zip_file = Tempfile.new('invoices.zip')
Zip::File.open(zip_file.path, Zip::File::CREATE) do |zipfile|
invoices.each do |invoice|
next unless invoice.attachment.attached?
overlay = Tempfile.new(['overlay', '.pdf'])
overlay.binmode
overlay.write(invoice.attachment.download)
overlay.close
overlay.path
zipfile.add(invoice.filename, File.join(overlay.path))
end
end
invoices_zip = File.read(zip_file.path)
UserMailer.with(user: user).invoice_export(invoices_zip, 'invoices.zip').deliver_now
ensure
zip_file.close
zip_file.unlink
end
I am downloading multiple files from S3 and zipping them. How do I give a custom name to each file being zipped?
def download_all_files
folder_path = "#{Rails.root}/public/downloads/"
zipfile_name = "#{Rails.root}/public/archive.zip"
FileUtils.remove_dir(folder_path) if Dir.exist?(folder_path)
FileUtils.remove_entry(zipfile_name) if File.exist?(zipfile_name)
Dir.mkdir("#{Rails.root}/public/downloads")
#model_object.each do |attachment|
open(folder_path + "#{attachment.avatar.file.filename}", 'wb') do |file|
file << open("#{attachment.avatar.url}").read
end
end
input_filenames = Dir.entries(folder_path).select {|f| !File.directory? f}
Zip::File.open(zipfile_name, Zip::File::CREATE) do |zipfile|
input_filenames.each do |attachment|
zipfile.add(attachment,File.join(folder_path,attachment))
end
end
send_file(File.join("#{Rails.root}/public/", 'archive.zip'), :type => 'application/zip', :filename => "#{Time.now.to_date}.zip")
end
Looks like you have 2 ways you can accomplish this. To the end user, both of these should yield the same result, just depends how you want them on your server.
1. Change it while zipping the files together
This will leave the files as they are currently named on your system and only change them in the output archive.zip
Looking at this gem it looks like
Zip::File.open(zipfile_name, Zip::File::CREATE) do |zipfile|
input_filenames.each do |filename|
# Two arguments:
# - The name of the file as it will appear in the archive
# - The original file, including the path to find it
zipfile.add(filename, folder + '/' + filename)
end
zipfile.get_output_stream("myFile") { |os| os.write "myFile contains just this" }
end
So, in your code just change the first occurrence of 'attachment' on
zipfile.add(attachment, File.join(folder_path, attachment))
to whatever you want the name to be.
2. Change it while writing the file to your server
Since you are writing the file just a few lines above there
open(folder_path + "#{attachment.avatar.file.filename}", 'wb') do |file|
you could change the file name on this line as well. This would change the filename on your server and in the archive.zip
I'm trying to create a function that would download a zip that contains files. I followed this tutorial, that says that my code should look like the following if you want to download a CSV (I translated the comments) :
def exporter_zip
# CSV's header
csv = "Title;Content;Publishing date;File name;\n"
# Loop over articles saved in database
Article.all.each do |article|
# Creating the CSV with datas
csv += "#{article.title};#{article.content};#{article.publishing_date.strftime("%Y-%m-%d") if article.publishing_date };#{article.file_name}\n"
end
# Creating the zip file inside the folder
zip_tmp = File.new("#{Rails.root}/db/mon_fichier.zip", "w+")
# opening the file in writing mode
Zip::File.open(zip_tmp.path, Zip::File::CREATE) {
|zipfile|
# Inserting the csv variable's content inside the article.csv file, which is inserted into the zip file
zipfile.get_output_stream("articles.csv") { |f| f.puts csv }
}
# Sending the created file
send_file "#{Rails.root}/db/mon_fichier.zip"
end
Here is how i adapted the code:
class DownloadController < ApplicationController
require 'zip'
def zip
# Creating zip file
zip_tmp = File.new("#{Rails.root}/public/zip-#{Time.now.strftime('%d-%m-%Y')}.zip", 'w+')
FileDetail.all.each do |fichier|
Zip::File.open(zip_tmp.path, Zip::File::CREATE) { |zipfile|
# fichier.filename => file.mp3
# fichier.path => path/to/file.mp3
zipfile.get_output_stream(fichier.filename, fichier.path)
}
end
send_file "#{Rails.root}/public/zip-#{Date.today.to_time}.zip"
end
end
However, while I'm not even sure I'm doing this correctly, I get the following error: cannot open entry for reading while its open for writing, targetting the following line: Zip::File.open(zip_tmp.path, Zip::File::CREATE) { |zipfile|
Can anyone can tell me what's going on? I never did this before so I don't know what went wrong..
Thank you in advance
FileDetail.all.each do |fichier|
Zip::File.open(zip_tmp.path, Zip::File::CREATE) { |zipfile|
# fichier.filename => file.mp3
# fichier.path => path/to/file.mp3
zipfile.get_output_stream(fichier.filename, fichier.path)
}
end
Doing it this way will attempt to create the zip file for each member of your FileDetail. You should code it to only open and create once:
Zip::File.open(zip_tmp.path, Zip::File::CREATE) do |zipfile|
FileDetail.all.each do |fichier|
# fichier.filename => file.mp3
# fichier.path => path/to/file.mp3
zipfile.get_output_stream(fichier.filename, fichier.path)
end
end
I have a small Rails 3.2.1 app that uses CarrierWave 0.5.8 for file uploads to S3 (using Fog)
I want users to be able to select some images that they'd like to download, then zip them up and send them a zip. Here is what I've come up with:
def generate_zip
#A collection of Photo objects. The Photo object has a PhotoUploader mounted.
photos = Photo.all
tmp_filename = "#{Rails.root}/tmp/" << Time.now.strftime('%Y-%m-%d-%H%M%S-%N').to_s << ".zip"
zip = Zip::ZipFile.open(tmp_filename, Zip::ZipFile::CREATE)
zip.close
photos.each do |photo|
file_to_add = photo.photo.file
zip = Zip::ZipFile.open(tmp_filename)
zip.add("tmp/", file_to_add.path)
zip.close
end
#do the rest.. like send zip or upload file and e-mail link
end
This doesn't work because photo.photo.file returns an instance of CarrierWave::Storage::Fog::File instead of a regular file.
EDIT: The error this leads to:
Errno::ENOENT: No such file or directory - uploads/photos/name.jpg
I also tried the following:
tmp_filename = "#{Rails.root}/tmp/" << Time.now.strftime('%Y-%m-%d-%H%M%S-%N').to_s << ".zip"
zip = Zip::ZipFile.open(tmp_filename, Zip::ZipFile::CREATE)
zip.close
photos.each do |photo|
processed_uri = URI.parse(URI.escape(URI.unescape(photo.photo.file.authenticated_url)).gsub("[", "%5B").gsub("]", "%5D"))
file_to_add = CarrierWave::Uploader::Download::RemoteFile.new(processed_uri)
zip = Zip::ZipFile.open(tmp_filename)
zip.add("tmp/", file_to_add.path)
zip.close
end
But this gives me a 403. Some help would be greatly appreciated.. It probably is not that hard I'm just Doing it Wrong™
I've managed to solve the problem with help from #ffoeg
The solution offered by #ffoeg didn't work quite so well for me since I was dealing with zip files > 500 MB which caused me problems on Heroku. I've therefor moved the zipping to a background process using resque:
app/workers/photo_zipper.rb:
require 'zip/zip'
require 'zip/zipfilesystem'
require 'open-uri'
class PhotoZipper
#queue = :photozip_queue
#I pass
def self.perform(id_of_object_with_images, id_of_user_to_be_notified)
user_mail = User.where(:id => id_of_user_to_be_notified).pluck(:email)
export = PhotoZipper.generate_zip(id_of_object_with_images, id_of_user_to_be_notified)
Notifications.zip_ready(export.archive_url, user_mail).deliver
end
# Zipfile generator
def self.generate_zip(id_of_object_with_images, id_of_user_to_be_notified)
object = ObjectWithImages.find(id_of_object_with_images)
photos = object.images
# base temp dir
temp_dir = Dir.mktmpdir
# path for zip we are about to create, I find that ruby zip needs to write to a real file
# This assumes the ObjectWithImages object has an attribute title which is a string.
zip_path = File.join(temp_dir, "#{object.title}_#{Date.today.to_s}.zip")
Zip::ZipOutputStream.open(zip_path) do |zos|
photos.each do |photo|
path = photo.photo.path
zos.put_next_entry(path)
zos.write photo.photo.file.read
end
end
#Find the user that made the request
user = User.find(id_of_user_to_be_notified)
#Create an export object associated to the user
export = user.exports.build
#Associate the created zip to the export
export.archive = File.open(zip_path)
#Upload the archive
export.save!
#return the export object
export
ensure
# clean up the tempdir now!
FileUtils.rm_rf temp_dir if temp_dir
end
end
app/controllers/photos_controller.rb:
format.zip do
#pick the last ObjectWithImages.. ofcourse you should include your own logic here
id_of_object_with_images = ObjectWithImages.last.id
#enqueue the Photozipper task
Resque.enqueue(PhotoZipper, id_of_object_with_images, current_user.id)
#don't keep the user waiting and flash a message with information about what's happening behind the scenes
redirect_to some_path, :notice => "Your zip is being created, you will receive an e-mail once this process is complete"
end
Many thanks to #ffoeg for helping me out. If your zips are smaller you could try #ffoeg's solution.
Here is my take. There could be typos but I think this is the gist of it :)
# action method, stream the zip
def download_photos_as_zip # silly name but you get the idea
generate_zip do |zipname, zip_path|
File.open(zip_path, 'rb') do |zf|
# you may need to set these to get the file to stream (if you care about that)
# self.last_modified
# self.etag
# self.response.headers['Content-Length']
self.response.headers['Content-Type'] = "application/zip"
self.response.headers['Content-Disposition'] = "attachment; filename=#{zipname}"
self.response.body = Enumerator.new do |out| # Enumerator is ruby 1.9
while !zf.eof? do
out << zf.read(4096)
end
end
end
end
end
# Zipfile generator
def generate_zip(&block)
photos = Photo.all
# base temp dir
temp_dir = Dir.mktempdir
# path for zip we are about to create, I find that ruby zip needs to write to a real file
zip_path = File.join(temp_dir, 'export.zip')
Zip::ZipFile::open(zip_path, true) do |zipfile|
photos.each do |photo|
zipfile.get_output_stream(photo.photo.identifier) do |io|
io.write photo.photo.file.read
end
end
end
# yield the zipfile to the action
block.call 'export.zip', zip_path
ensure
# clean up the tempdir now!
FileUtils.rm_rf temp_dir if temp_dir
end