In my app I want to upload .csv files to cloud using carrierwave, but before I want to test uploader local, but I don't know how to upload generated in app csv string. In my app it looks like that:
/uploaders/file_uploader.rb
class FileUploader < CarrierWave::Uploader::Base
# Choose what kind of storage to use for this uploader:
storage :file
# define some uploader specific configurations in the initializer
# to override the global configuration
def store_dir
"uploads/#{model.id}"
end
end
/models/rooms.rb
class Rooms < ApplicationRecord
mount_uploader :file, FileUploader
end
/services/generate_csv
require 'csv'
class GenerateCsv
def initialize(room)
#room = room
end
def build
csv_string = gen_csv
#room.file = csv_string
#room.save!
end
def gen_csv
CSV.generate do |csv|
csv << headers
users.each do |user|
csv << GenerateRow.gen(room: #room)
end
end
end
end
And it doesn't work. After room.save! column file is nil. Is there any option to upload generated csv and associate to DB column (after all I only need URL to download)? Maybe I should create csv file instead of csv string? Can you help me with tihs?
#EDIT
Solution
/services/generate_csv
...
def build
csv_string = gen_csv
#room.file = StringIO.new(csv_string)
#room.save!
end
...
Related
In our rails 6 project I want to export active record data to excel file and save xls file into S3 bucket without storing xls data on local db and send file link to email and provide download xls feature from email. Please help me.
1. Create a model which holds a exported file
# app/models/csv_export.rb
class CsvExport < ApplicationRecord
has_one_attached :file
# ...
end
Configure ActiveStorage that it uses S3 as a provider. See https://medium.com/alturasoluciones/setting-up-rails-5-active-storage-with-amazon-s3-3d158cf021ff
2. Create a export job which creates a new CsvExport with data
# app/jobs/csv_export_job.rb
require 'csv'
class CsvExportJob < ApplicationJob
queue_as :default
def perform(csv_export_id)
csv_export = CsvExport.find_by(id: csv_export_id)
csv_export.file.attach \
io: StringIO.new(csv_string), # add csv_string call here
filename: filename
# ...
end
private
# ...
CSV_COLUMNS = %w[id name email].freeze
def csv_string
CSV.generate(headers: true) do |csv|
csv << CSV_COLUMNS
# or whatever data you want to export
User.all.each do |contact|
csv << CSV_COLUMNS.map { |col| contact.send(col) }
end
end
end
end
3. Trigger job
e.g. from a controller action
csv_export = CsvExport.create(status: :started)
CsvExportJob.perform_later csv_export.id
I hope these examples will bring you to the right direction. If you need detailed help you can look at this article https://railsbyexample.com/export-records-to-csv-files-using-activestorage/
After the file is uploaded, I want to analyze and immediately process.
I'm currently attaching then processing each:
current_account.archives.attach(archive_params)
current_account.archives.each do |archive|
Job.enqueue(AccountArchiveImportJob.new(current_account.id, archive.id))
end
In the job i'm opening the CSV and parsing junk
attachment = Account.find(account_id).archives.where(id: archive_id).first
CSV.parse(attachment.download) do |row|
do_stuff_with_the_row(row)
end
I would like to do something like:
CSV.foreach(attachment.open) do |row|
do_stuff_with_the_row(row)
end
I cannot find documentation that allows converting the attachment back into a FILE
At least from Rails 6.0 rc1:
model.attachment_changes['attachment_name'].attachable
will give you IO of the original TmpFile BEFORE it is uploaded.
Rails-6 we will get a download method that will yield a file but you can get this very easily!
Add this downloader.rb file as an initializer
Then given this model
class Business < ApplicationRecord
has_one_attached :csvfile
end
you can do
ActiveStorage::Downloader.new(csvfile).download_blob_to_tempfile do |file|
CSV.foreach(file.path, {headers: true}) do |row|
do_something_with_each_row(row.to_h)
end
end
EDIT: not sure why this took me to so long to find service_url. Way more simple, but has been noted that service_url should not be shown to users
open(csvfile.service_url)
From Rails 5.2 official guide
class VirusScanner
include ActiveStorage::Downloading
attr_reader :blob
def initialize(blob)
#blob = blob
end
def scan
download_blob_to_tempfile do |file|
system 'scan_virus', file.path
end
end
end
So you can do
include ActiveStorage::Downloading
attr_reader :blob
def initialize(blob)
#blob = blob
end
def perform
download_blob_to_tempfile do |file|
CSV.foreach(file.path, {headers: true}) do |row|
do_something_with_each_row(row.to_h)
end
end
end
You can get the file path from the attachment, and then open the file.
path = ActiveStorage::Blob.service.send(:path_for, attachment.key)
File.open(path) do |file|
#...
end
I'm actually looking for advices more than pure coding answers on how to uncompress RAR/ZIP file after upload while keeping a maximum rate of data integrity.
Here is my problem : my application's users are uploading files generated by Adobe Edge (we are using it for animated ads) which are in RAR format. To upload the file, it was really trivial. Here is my uploader :
class MediaUploader < CarrierWave::Uploader::Base
storage :file
def store_dir
"uploads/#{ model.class.to_s.underscore }/#{ mounted_as }/#{ ScatterSwap.hash(model.id) }"
end
def extension_white_list
%w(jpg jpeg gif png rar zip)
end
def filename
"#{ secure_token }.#{ file.extension }" if original_filename.present?
end
protected
def secure_token
var = :"##{ mounted_as }_secure_token"
model.instance_variable_get(var) or model.instance_variable_set(var, SecureRandom.uuid)
end
end
Now, in my case, the RAR file is not actually the one I'll be using. What I need are the files contain inside the archive. Those files generally looks like that :
- edge_includes
|
- images
|
- js
|
| ADS_1234_988x160_edge.js
| ADS_1234_988x160_edgeActions.js
| ADS_1234_988x160.an
| ADS_1234_988x160.html
From the above example, I need to store the reference to ADS_1234_988x160.html file within the database.
For this purpose, I was going to use Carrierwave callbacks in order to :
after :store, :uncompress_and_update_reference
def uncompress_and_update_reference(file)
# uncompress and update reference
end
Uncompress the archive (probably using rubyzip)
Get the path to ADS_1234_988x160.html
Update the reference inside the database
Is there any better way to handle it? How to handle failure or network errors? Any ideas are welcome.
I had similar problem: zip is uploaded but it should not be stored, only unzipped content. I solved it by implementing dedicated storage. Something like:
class MyStorage
class App < CarrierWave::Storage::Abstract
def store!(file)
# invoked when file is moved from cache into store
# unzip and update db here
end
def retrieve!(identifier)
MyZipFile.new(identifier)
end
end
end
class MyZipFile
def initialize(identifier)
#identifier = identifier
end
def path
file.path
end
def delete
# delete unzipped files
end
def file
#file ||= zip_file
end
def zip_file
# create zip file somewhere in tmp dir
end
end
# in uploader file
storage MyStorage
storage :my_storage
If symbol is used than it need to be defined in carrierwave config:
CarrierWave.configure do |config|
config.storage_engines.merge!(my_storage: 'MyStorage')
end
I'm using Carrierwave and Fog gems to store a file to my Amazon S3 bucket (to /files/file_id.txt). I need to store a slightly different version of the file to a different location in the bucket (/files/file_id_processed.txt) at the same time (right after the original is stored). I don't want to create a separate uploader attribute for it on the model - is there any other way?
This my current method that stores the file:
def store_file(document)
file_name = "tmp/#{document.id}.txt"
File.open(file_name, 'w') do |f|
document_content = document.content
f.puts document_content
document.raw_export.store!(f)
document.save
end
# I need to store the document.processed_content
File.delete(file_name) if File.exist?(file_name)
end
This is the Document model:
class Document < ActiveRecord::Base
mount_uploader :raw_export, DocumentUploader
# here I want to avoid adding something like:
# mount_uploader :processed_export, DocumentUploader
end
This is my Uploader class:
class DocumentUploader < CarrierWave::Uploader::Base
storage :fog
def store_dir
"files/"
end
def extension_white_list
%w(txt)
end
end
This is how my final solution looks like (kinda) - based on Nitin Verma's answer:
I had to add a custom processor method for the version to the Uploader class:
# in document_uploader.rb
...
version :processed do
process :do_the_replacements
end
def do_the_replacements
original_content = #file.read
File.open(current_path, 'w') do |f|
f.puts original_content.gsub('Apples','Pears')
end
end
considering that you need similar file but with different name.
for this you need to create a version for file in uploader.
version :processed do
process
end
and now second file name will be processed_{origional_file}.extension. if you want to change file name of second file you can use this link https://github.com/carrierwaveuploader/carrierwave/wiki/How-to:-Customize-your-version-file-names
i am trying to upload a file with carrierwave in my rails app and currently this is my code:
Controller:
def fileSave
#code.store!(code)
end
View:
= form_for #code = Code.new(params[:code]), :as => :code, :html => {:multipart => true} do |f|
div class="browse"
span
= f.file_field :code
= f.submit 'Upload'
Uploader:
# encoding: utf-8
class CodeUploader < CarrierWave::Uploader::Base
def pre_limit file
#require 'debugger'; debugger
if file && file.size > 100.megabytes
raise Exception.new('too large')
end
true
end
storage :file
def store_dir
"public/uploads"
end
def extension_white_list
%w(txt js ttf html)
end
def filename
"file.txt" if original_filename
end
end
Model:
require 'carrierwave/orm/activerecord'
class Code < ActiveRecord::Base
attr_accessor :code
mount_uploader :code, CodeUploader
end
And my problem is i can not store the uploaded file. x[ I am sure this is like 3 lines of code but i can not figure it out. Also the file to be uploaded is expected to be txt (probably figured that out looking the extension list).
Thanks to all readers and answerers. :}
P.S. I was wondering if i could create some kind of imaginary file, a file which is not really created. The is idea is if a take a text from a textarea and create a file (the imaginary one), store the text inside and then eventually save the whole file (maybe use carrierwave as and manually store it).