After the file is uploaded, I want to analyze and immediately process.
I'm currently attaching then processing each:
current_account.archives.attach(archive_params)
current_account.archives.each do |archive|
Job.enqueue(AccountArchiveImportJob.new(current_account.id, archive.id))
end
In the job i'm opening the CSV and parsing junk
attachment = Account.find(account_id).archives.where(id: archive_id).first
CSV.parse(attachment.download) do |row|
do_stuff_with_the_row(row)
end
I would like to do something like:
CSV.foreach(attachment.open) do |row|
do_stuff_with_the_row(row)
end
I cannot find documentation that allows converting the attachment back into a FILE
At least from Rails 6.0 rc1:
model.attachment_changes['attachment_name'].attachable
will give you IO of the original TmpFile BEFORE it is uploaded.
Rails-6 we will get a download method that will yield a file but you can get this very easily!
Add this downloader.rb file as an initializer
Then given this model
class Business < ApplicationRecord
has_one_attached :csvfile
end
you can do
ActiveStorage::Downloader.new(csvfile).download_blob_to_tempfile do |file|
CSV.foreach(file.path, {headers: true}) do |row|
do_something_with_each_row(row.to_h)
end
end
EDIT: not sure why this took me to so long to find service_url. Way more simple, but has been noted that service_url should not be shown to users
open(csvfile.service_url)
From Rails 5.2 official guide
class VirusScanner
include ActiveStorage::Downloading
attr_reader :blob
def initialize(blob)
#blob = blob
end
def scan
download_blob_to_tempfile do |file|
system 'scan_virus', file.path
end
end
end
So you can do
include ActiveStorage::Downloading
attr_reader :blob
def initialize(blob)
#blob = blob
end
def perform
download_blob_to_tempfile do |file|
CSV.foreach(file.path, {headers: true}) do |row|
do_something_with_each_row(row.to_h)
end
end
end
You can get the file path from the attachment, and then open the file.
path = ActiveStorage::Blob.service.send(:path_for, attachment.key)
File.open(path) do |file|
#...
end
Related
In my app I want to upload .csv files to cloud using carrierwave, but before I want to test uploader local, but I don't know how to upload generated in app csv string. In my app it looks like that:
/uploaders/file_uploader.rb
class FileUploader < CarrierWave::Uploader::Base
# Choose what kind of storage to use for this uploader:
storage :file
# define some uploader specific configurations in the initializer
# to override the global configuration
def store_dir
"uploads/#{model.id}"
end
end
/models/rooms.rb
class Rooms < ApplicationRecord
mount_uploader :file, FileUploader
end
/services/generate_csv
require 'csv'
class GenerateCsv
def initialize(room)
#room = room
end
def build
csv_string = gen_csv
#room.file = csv_string
#room.save!
end
def gen_csv
CSV.generate do |csv|
csv << headers
users.each do |user|
csv << GenerateRow.gen(room: #room)
end
end
end
end
And it doesn't work. After room.save! column file is nil. Is there any option to upload generated csv and associate to DB column (after all I only need URL to download)? Maybe I should create csv file instead of csv string? Can you help me with tihs?
#EDIT
Solution
/services/generate_csv
...
def build
csv_string = gen_csv
#room.file = StringIO.new(csv_string)
#room.save!
end
...
In our rails 6 project I want to export active record data to excel file and save xls file into S3 bucket without storing xls data on local db and send file link to email and provide download xls feature from email. Please help me.
1. Create a model which holds a exported file
# app/models/csv_export.rb
class CsvExport < ApplicationRecord
has_one_attached :file
# ...
end
Configure ActiveStorage that it uses S3 as a provider. See https://medium.com/alturasoluciones/setting-up-rails-5-active-storage-with-amazon-s3-3d158cf021ff
2. Create a export job which creates a new CsvExport with data
# app/jobs/csv_export_job.rb
require 'csv'
class CsvExportJob < ApplicationJob
queue_as :default
def perform(csv_export_id)
csv_export = CsvExport.find_by(id: csv_export_id)
csv_export.file.attach \
io: StringIO.new(csv_string), # add csv_string call here
filename: filename
# ...
end
private
# ...
CSV_COLUMNS = %w[id name email].freeze
def csv_string
CSV.generate(headers: true) do |csv|
csv << CSV_COLUMNS
# or whatever data you want to export
User.all.each do |contact|
csv << CSV_COLUMNS.map { |col| contact.send(col) }
end
end
end
end
3. Trigger job
e.g. from a controller action
csv_export = CsvExport.create(status: :started)
CsvExportJob.perform_later csv_export.id
I hope these examples will bring you to the right direction. If you need detailed help you can look at this article https://railsbyexample.com/export-records-to-csv-files-using-activestorage/
Rails 4.2 ap and taglib-ruby gem
I have tried a few approaches to tag my audio files using taglib-ruby.
I have tried as a carrierwave process, and now, I am trying to tag after_save.
My question, if I am doing a callback after_save:
def tag_id3v1(tags)
TagLib::MPEG::File.open(file.path) do |file|
tag = file.id3v1_tag(true)
tag.title = :title
file.save
end
end
what should my file path be? I have tried :file, :file_name, and the uploader version url, #{track.mp3.url}
I am trying to reopen the saved files and write the tags to the files. Does anyone have any hints on the best approach to do this?
Finally
TagLib::MPEG::File.open(file.file) do |file|
Always something like, "file.file". No matter the approach, that is what did the trick for me.
I ended up doing this in a carrierwave callback inside of a :version. Final code
version :mp3 do
process :convert => [:mp3]
def full_filename(for_file)
"#{super.chomp(File.extname(super))}.mp3"
end
after :store, :tag_id3v2
end
def tag_id3v2(for_file)
TagLib::MPEG::File.open(file.file) do |file|
tag = file.id3v2_tag(true)
tag.title = "#{model.title}"
file.save
end
end
(...)
I'm actually looking for advices more than pure coding answers on how to uncompress RAR/ZIP file after upload while keeping a maximum rate of data integrity.
Here is my problem : my application's users are uploading files generated by Adobe Edge (we are using it for animated ads) which are in RAR format. To upload the file, it was really trivial. Here is my uploader :
class MediaUploader < CarrierWave::Uploader::Base
storage :file
def store_dir
"uploads/#{ model.class.to_s.underscore }/#{ mounted_as }/#{ ScatterSwap.hash(model.id) }"
end
def extension_white_list
%w(jpg jpeg gif png rar zip)
end
def filename
"#{ secure_token }.#{ file.extension }" if original_filename.present?
end
protected
def secure_token
var = :"##{ mounted_as }_secure_token"
model.instance_variable_get(var) or model.instance_variable_set(var, SecureRandom.uuid)
end
end
Now, in my case, the RAR file is not actually the one I'll be using. What I need are the files contain inside the archive. Those files generally looks like that :
- edge_includes
|
- images
|
- js
|
| ADS_1234_988x160_edge.js
| ADS_1234_988x160_edgeActions.js
| ADS_1234_988x160.an
| ADS_1234_988x160.html
From the above example, I need to store the reference to ADS_1234_988x160.html file within the database.
For this purpose, I was going to use Carrierwave callbacks in order to :
after :store, :uncompress_and_update_reference
def uncompress_and_update_reference(file)
# uncompress and update reference
end
Uncompress the archive (probably using rubyzip)
Get the path to ADS_1234_988x160.html
Update the reference inside the database
Is there any better way to handle it? How to handle failure or network errors? Any ideas are welcome.
I had similar problem: zip is uploaded but it should not be stored, only unzipped content. I solved it by implementing dedicated storage. Something like:
class MyStorage
class App < CarrierWave::Storage::Abstract
def store!(file)
# invoked when file is moved from cache into store
# unzip and update db here
end
def retrieve!(identifier)
MyZipFile.new(identifier)
end
end
end
class MyZipFile
def initialize(identifier)
#identifier = identifier
end
def path
file.path
end
def delete
# delete unzipped files
end
def file
#file ||= zip_file
end
def zip_file
# create zip file somewhere in tmp dir
end
end
# in uploader file
storage MyStorage
storage :my_storage
If symbol is used than it need to be defined in carrierwave config:
CarrierWave.configure do |config|
config.storage_engines.merge!(my_storage: 'MyStorage')
end
I'm using Carrierwave and Fog gems to store a file to my Amazon S3 bucket (to /files/file_id.txt). I need to store a slightly different version of the file to a different location in the bucket (/files/file_id_processed.txt) at the same time (right after the original is stored). I don't want to create a separate uploader attribute for it on the model - is there any other way?
This my current method that stores the file:
def store_file(document)
file_name = "tmp/#{document.id}.txt"
File.open(file_name, 'w') do |f|
document_content = document.content
f.puts document_content
document.raw_export.store!(f)
document.save
end
# I need to store the document.processed_content
File.delete(file_name) if File.exist?(file_name)
end
This is the Document model:
class Document < ActiveRecord::Base
mount_uploader :raw_export, DocumentUploader
# here I want to avoid adding something like:
# mount_uploader :processed_export, DocumentUploader
end
This is my Uploader class:
class DocumentUploader < CarrierWave::Uploader::Base
storage :fog
def store_dir
"files/"
end
def extension_white_list
%w(txt)
end
end
This is how my final solution looks like (kinda) - based on Nitin Verma's answer:
I had to add a custom processor method for the version to the Uploader class:
# in document_uploader.rb
...
version :processed do
process :do_the_replacements
end
def do_the_replacements
original_content = #file.read
File.open(current_path, 'w') do |f|
f.puts original_content.gsub('Apples','Pears')
end
end
considering that you need similar file but with different name.
for this you need to create a version for file in uploader.
version :processed do
process
end
and now second file name will be processed_{origional_file}.extension. if you want to change file name of second file you can use this link https://github.com/carrierwaveuploader/carrierwave/wiki/How-to:-Customize-your-version-file-names