Placing folders, inserting files, zipping all and providing download - ruby-on-rails

I have a project running.
And I have a few fundamental Ruby-On-Rails-questions on:
where to place a folder in my rails installation, so i have access to it from within my controllers
how to place files inside that folder when models are saved
how to zip it
providing the zip as a download
now, let me explain:
I want to save to a "pages" folder that has a subfolder "blog"
to that blog-folder I want to add subfolders each representing a post
the name of the post-folder is created by the user, as he provides a title
each post-folder has a MarkDown file in it called "post.md"
if the user clicks a download-button the whole "pages"-folder should be zipped and sent to the client as a download

Technically you can put your folder anywhere, and access your .md file like following (as long as you have permission to read that file):
def show
markdown=Redcarpet::Markdown.new(Redcarpet::Render::HTML)
md_file = IO.read("#{Rails.root}\static\pages\blog\post.md")
#post = markdown.render(md_file)
end
and at view
<%= #post.html_safe %>
According to documentation of 'rubyzip' gem you will need something like this to make your directory zipable:
class ZipFileGenerator
# Initialize with the directory to zip and the location of the output archive.
def initialize(inputDir, outputFile)
#inputDir = inputDir
#outputFile = outputFile
end
# Zip the input directory.
def write()
entries = Dir.entries(#inputDir); entries.delete("."); entries.delete("..")
io = Zip::File.open(#outputFile, Zip::File::CREATE);
writeEntries(entries, "", io)
io.close();
end
# A helper method to make the recursion work.
private
def writeEntries(entries, path, io)
entries.each { |e|
zipFilePath = path == "" ? e : File.join(path, e)
diskFilePath = File.join(#inputDir, zipFilePath)
puts "Deflating " + diskFilePath
if File.directory?(diskFilePath)
io.mkdir(zipFilePath)
subdir =Dir.entries(diskFilePath); subdir.delete("."); subdir.delete("..")
writeEntries(subdir, zipFilePath, io)
else
io.get_output_stream(zipFilePath) { |f| f.print(File.open(diskFilePath, "rb").read())}
end
}
end
end
and at your zip action at controller:
directoryToZip = "/pages"
outputFile = "/public/out#{rand(6**length).to_s(6)}.zip"
zf = ZipFileGenerator.new(directoryToZip, outputFile)
zf.write()
and then you can provide the link of outputFile to your user and you may also need to delete that zip file.

Related

Reading text from a PDF works in Rails console but not in Rails application

I have a simple one-page searchable PDF that is uploaded to a Rails 6 application model (Car) using Active Storage. I can extract the text from the PDF using the 'tempfile' and 'pdf-reader' gems in the Rails console:
> #car.creport.attached?
=> true
> f = Tempfile.new(['file', '.pdf'])
> f.binmode
> f.write(#car.creport.blob.download)
> r = PDF::Reader.new(f.path.to_s)
> r.pages[1].text
=> "Welcome to the ABC Car Report for January 16, 20...
But, if I try the same thing in the create method of my cars_controller.rb, it doesn't work:
# cars_controller.rb
...
def create
#car = Car.new(car_params)
#car.filetext = ""
f = Tempfile.new(['file', '.pdf'])
f.binmode
f.write(#car.creport.blob.download)
r = PDF::Reader.new(f.path.to_s)
#car.filetext = r.pages[1].text
...
end
When I run the Rails application I can create a new Car and select a PDF file to attach. But when I click 'Submit' I get a FileNotFoundError in cars_controller.rb at the f.write() line.
My gut instinct is that the controller is trying to read the blob in order to write it to the temp file too soon (i.e., before the blob has even been written). I tried inserting a sleep(2) to give it time, but I get the same FileNotFoundError.
Any ideas?
Thank you!
I don't get why you're jumping through so many hoops. And using .download without a block loads the entire file into memory (yikes). If #car.creport is an ActiveStorage attachment you can just use the open method instead:
#car.creport.blob.open do |file|
file.binmode
r = PDF::Reader.new(file) # just pass the IO object
#car.filetext = r.pages[1].text
end if #car.creport
This steams the file to disk instead (as a tempfile).
If you're just taking file input via a plain old file input you will get a ActionDispatch::Http::UploadedFile in the parameters that also is extemely easy to open:
params[:file].open do |file|
file.binmode
r = PDF::Reader.new(file) # just pass the IO object
#car.filetext = r.pages[1].text
end if params[:file].respond_to?(:open)
The difference looks like it's with your #car variable.
In the console you have a blob attached (#car.creport.attached? => true). In your controller, you're initializing a new instance of the Car class, so unless you have some initialization going on that attaches something in the background, that will be nil.
Why that would return a 'file not found' error I'm not sure, but from what I can see that's the only difference between code samples. You're trying to write #car.creport.blob.download, which is present on #car in console, but nil in your controller.

Rails 5.2 Import XLSX with ActiveStorage and Creek

I have a model called ImportTemp which is used for store imported XLSX file to database. I'm using ActiveStorage for storing the files.
this is the model code:
class ImportTemp < ApplicationRecord
belongs_to :user
has_one_attached :file
has_one_attached :log_result
end
this is my import controller code:
def import
# Check filetype
case File.extname(params[:file].original_filename)
when ".xlsx"
# Add File to ImportFile model
import = ImportTemp.new(import_type: 'UnitsUpload', user: current_user)
import.file.attach(params[:file])
import.save
# Import unit via sidekiq with background jobs
ImportUnitWorker.perform_async(import.id)
# Notice
flash.now[:notice] = "We are processing your xlsx, we will inform you after it's done via notifications."
# Unit.import_file(xlsx)
else flash.now[:error] = t('shared.info.unknown')+": #{params[:file].original_filename}"
end
end
after upload the xlsx file, then the import will be processed in sidekiq. This is the worker code (still doesn't do the import actually) :
class ImportUnitWorker
include Sidekiq::Worker
sidekiq_options retry: false
def perform(file_id)
import_unit = ImportTemp.find(file_id)
# Open the uploaded xlsx to Creek
creek = Creek::Book.new(Rails.application.routes.url_helpers.rails_blob_path(import_unit.file, only_path: true))
sheet = creek.sheets[0]
puts "Opening Sheet #{sheet.name}"
sheet.rows.each do |row|
puts row
end
units = []
# Unit.import(units)
end
but after i tried it, it gives me error:
Zip::Error (File /rails/active_storage/blobs/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBCZz09IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--3960b6ba5b55f7004e09967d16dfabe63f09f0a9/2018-08-10_10_39_audit_gt.xlsx not found)
but if i tried to open it with my browser, which is the link looks like this:
http://localhost:3000/rails/active_storage/blobs/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBCZz09IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--3960b6ba5b55f7004e09967d16dfabe63f09f0a9/2018-08-10_10_39_audit_gt.xlsx
it's working and the xlsx is downloaded. My question is what's wrong with it? why the file is not found in the sidekiq?
I ended up using Tempfile as suggested by George Claghorn. I don't know if this is the best solution or best practices, but it works for me now. I'm going to use this solution while waiting Rails 6 stable to come out with ActiveStorage::Blob#open feature.
def perform(file_id)
import = ImportTemp.find(file_id)
temp_unit = Tempfile.new([ 'unit_import_temp', '.xlsx' ], :encoding => 'ascii-8bit')
units = []
begin
# Write xlsx from ImportTemp to Tempfile
temp_unit.write(import.file.download)
# Open the temp xlsx to Creek
book = Creek::Book.new(temp_unit.path)
sheet = book.sheets[0]
sheet.rows.each do |row|
# Skip the header
next if row.values[0] == 'Name' || row.values[1] == 'Abbreviation'
cells = row.values
# Add cells to new Unit
unit = Unit.new(name: cells[0], abbrev: cells[1], desc: cells[2])
units << unit
end
# Import the unit
Unit.import(units)
ensure
temp_unit.close
temp_unit.unlink # deletes the temp file
end
end
Rails.application.routes.url_helpers.rails_blob_path doesn’t return the path to the file on disk. Rather, it returns a path that can be combined with a hostname to produce an URL for downloading the file, for use in links.
You have two options:
If you’d prefer to keep ImportUnitWorker indifferent to the storage service in use, “download” the file to a tempfile on disk. Switch to Rails master and use ActiveStorage::Blob#open:
def perform(import_id)
import = ImportTemp.find(import_id)
units = []
import.file.open do |file|
book = Creek::Book.new(file.path)
sheet = creek.sheets[0]
# ...
end
Unit.import(units)
end
If you don’t mind ImportWorker knowing that you use the disk service, ask the service for the path to the file on disk. ActiveStorage::Service::DiskService#path_for(key) is private in Rails 5.2, so either forcibly call it with send or upgrade to Rails master, where it’s public:
def perform(import_id)
import = ImportTemp.find(import_id)
units = []
path = ActiveStorage::Blob.service.send(:path_for, import.file.key)
book = Creek::Book.new(path)
sheet = creek.sheets[0]
# ...
Unit.import(units)
end
The answer now seems to be (unless I am missing something):
Creek::Book.new file.service_url, check_file_extension: false, remote: true

Rails - compressing CSV files

I'm trying to attach a zipped CSV file to an e-mail without any joy. I've tried following http://api.rubyonrails.org/classes/ActiveSupport/Gzip.html:
class UserExportProcessor
#queue = :user_export_queue
def self.perform(person_id, collection_ids)
person = Person.unscoped.find(person_id)
collection = Person.unscoped.where(id: [49522, 70789])
file = ActiveSupport::Gzip.compress(collection.to_csv)
PersonMailer.people_export(person, file).deliver
end
end
This sends an attachment - still as a CSV file - filled with symbols (no letters or numbers).
When I try and remove the compression:
class UserExportProcessor
#queue = :user_export_queue
def self.perform(person_id, collection_ids)
person = Person.unscoped.find(person_id)
collection = Person.unscoped.where(id: [49522, 70789])
PersonMailer.people_export(person, collection.to_csv).deliver
end
end
The system e-mails the CSV file as it should and the CSV is properly formed. What am I doing wrong? Do I need to make a new file out of the compressed data? I've tried various approaches with no joy..
Thanks in advance
EDIT: I'm trying
class UserExportProcessor
require 'zip'
#queue = :user_export_queue
def self.perform(person_id, collection_ids)
person = Person.unscoped.find(person_id)
collection = Person.unscoped.where(id: [49522, 70789])
file = Zip::ZipFile.open("files.zip", Zip::ZipFile::CREATE) { |zipfile|
puts zipfile.get_output_stream(collection.to_csv)
zipfile.mkdir("a_dir")
end
PersonMailer.people_export(person, file).deliver
end
end
However this fails with:
Errno::ENAMETOOLONG: File name too long - /Users/mark/projects/bla/Role,Title,First Name,Last Name,Address 1,Address 2,Address 3,City,Postcode,Country,Email,Telephone,Mobile,Job Title,Company,Area of work,Department,Regions,Account Manager,Sales Coordinator,Production Studios,Production Partners,Genres,Last login,Created date
Is there any way for me to set the file name with the above approach?
The mistake probably happens in the code of PersonMailer.people_export (which you didn't include). So this is my best guess: you probably add the attachment and do not properly define the mime-type.
Make sure you set the correct file extension when you add the attachment:
http://edgeguides.rubyonrails.org/action_mailer_basics.html#adding-attachments
something along this line should work:
attachments['archive.zip'] = ActiveSupport::Gzip.compress(collection.to_csv)

How in Ruby on Rails 4.2.6 make a statement which has to check for file extension and upload to right public folder

I'm working on an application which has an upload functionality for documents. I can download various kind of documents like pdf, docx and etc. However, all is uploaded in one folder like ../uploads/documents.
What I have to reach is when the upload began, a statement will check the file extension and upload it to the right folder named as the extension of the file. As an example, I can have a PDF in upload and the app check if the PDF directory exists and if not create one, then upload to that directory. So far I have done what below but I'm new in RoR so I would like to have some suggestions how to make what mentioned above:
This comes from my CTRL:
module UploaderWidget
class Engine < ::Rails::Engine
end
def initialize(params = {})
#file = params.delete(:file)
super
if #file
self.filename = sanitize_filename(#file.original_filename)
self.content_type = #file.content_type
self.file_contents = #file.read
end
end
def upload_local
path = "#{Rails.root}/public/uploads/document"
FileUtils.mkdir_p(path) unless File.exists?(path)
FileUtils.copy(#file.tempfile, path)
end
private
def sanitize_filename(filename)
return File.basename(filename)
end
def document_file_format
unless ["application/pdf","application/vnd.ms-excel",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"application/msword",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"text/plain", "text/csv", "application/octet-stream"].include? self.content_type
errors.add(:file, 'Invalid file format.')
end
end
NUM_BYTES_IN_MEGABYTE = 1048576
def file_size_under_one_mb
if (#file.size.to_f / NUM_BYTES_IN_MEGABYTE) > 1
errors.add(:file, 'File size cannot be over one megabyte.')
end
end
end
You can use the File.extname() method:
File.extname("test.rb") #=> ".rb"
File.extname("a/b/d/test.rb") #=> ".rb"
File.extname("foo.") #=> ""
File.extname("test") #=> ""
File.extname(".profile") #=> ""
File.extname(".profile.sh") #=> ".sh"
In your particular case, you could do something like this:
sanitize the file name and save it to an instance variable
extract the extension name and save it to an instance variable
eventually manipulate your extension name, e.g.
remove the '.' char
create an hash having every extension as a key and the sub-folder name as a value
define a method to manipulate the extension and return the folder name
build your path like path = Rails.root.join('public', 'uploads', 'document' sub_folder_name)
The rest of your code should work as it is

Error with Paperclip / FasterCSV Processing for optional csv upload

I have a page where a user can import data to the site. either in the form of copy and pasting into a text area from excel, or by uploading a .csv file.
The controller checks if a csv has been uploaded - if so it processes this, else it will process the pasted content. (working on the assumption the user will only choose one option for now).
The copy and paste part works perfectly, however, the problem arises when I try to process the uploaded csv file:
I get the error:
can't convert
ActionController::UploadedTempfile
into String
#events_controller
def invite_save
#event = Event.find(params[:id])
if params[:guest_list_csv]
lines = parse_csv_file(params[:guest_list_csv])
else
#csv file uploaded
lines = params[:guest_list_paste]
end
if lines.size > 0
lines.each do |line|
new_user(line.split)
end
flash[:notice] = "List processing was successful."
else
flash[:error] = "List data processing failed."
end
end
private
def parse_csv_file(path_to_csv)
lines = []
require 'fastercsv'
FasterCSV.foreach(path_to_csv) do |row|
lines << row
end
lines
end
def new_user(line)
#code to create new user would go here
end
I'm essentially trying to upload and process the csv in one smooth action, rather than have to get the user to press a "process" button.
On the line #6 above
if params[:guest_list_csv]
lines = parse_csv_file(params[:guest_list_csv])
else
#csv file uploaded
lines = params[:guest_list_paste]
end
The problem is params[:guest_list_csv] is not the actual string, neither is the path, since it's a file object. What you need is explicitly call #path on it.
# line 6
lines = parse_csv_file(params[:guest_list_csv].path)
Please try it and see if it fixes your problem.

Resources