Exporting ActiveRecord objects into POROs - ruby-on-rails

I'm developing a "script generator" to automatize some processes at work.
It has a Rails application running on a server that stores all data needed to make the script and generates the script itself at the end of the process.
The problem I am having is how to export the data from the ActiveRecord format to Plain Old Ruby Objects (POROs) so I can deal with them in my script with no database support and a pure-ruby implementation.
I thought about YAML, CSV or something like this to export the data but it would be a painful process to update these structures if the process changes. Is there a simpler way?
Ty!

By "update these structures if the process changes", do you mean changing the code that reads and writes the CSV or YAML data when the fields in the database change?
The following code writes and reads any AR object to/from CSV (requires the FasterCSV gem):
def load_from_csv(csv_filename, poro_class)
headers_read = []
first_record = true
num_headers = 0
transaction do
FCSV.foreach(csv_filename) do |row|
if first_record
headers_read = row
num_headers = headers_read.length
first_record = false
else
hash_values = {}
for col_index in 0...num_headers
hash_values[headers_read[col_index]] = row[col_index]
end
new_poro_obj = poro_class.new(hash_values) # assumes that your PORO has a constructor that accepts a hash. If not, you can do something like new_poro_obj.send(headers_read[col_index], row[col_index]) in the loop above
#work with your new_poro_obj
end
end
end
end
#objects is a list of ActiveRecord objects of the same class
def dump_to_csv(csv_filename, objects)
FCSV.open(csv_filename,'w') do |csv|
#get column names and write them as headers
col_names = objects[0].class.column_names()
csv << col_names
objects.each do |obj|
col_values = []
col_names.each do |col_name|
col_values.push obj[col_name]
end
csv << col_values
end
end
end

Related

Create timestamped labels on csv files (ruby code)

I am running a transaction download script through Ruby. I was wondering if it is possible to label each .csv it creates with the current date/time the script was run. Below is the end of the script.
CSV.open("transaction_report.csv", "w") do |csv|
csv << header_row
search_results.each do |transaction|
transaction_details_row = header_row.map{ |attribute| transaction.send(attribute) }
csv << transaction_details_row
end
end
Like this?
CSV.open("transaction_report-#{Time.now}.csv", "w") do |csv|
csv << header_row
search_results.each do |transaction|
transaction_details_row = header_row.map{ |attribute| transaction.send(attribute) }
csv << transaction_details_row
end
end
This just appends the time of generation to the file name. For example:
"transaction_report-#{Time.now}.csv"
# => "transaction_report-2019-10-10 16:09:07 +0100.csv"
If you want to avoid spaces in the file name, you can sub these out like so:
"transaction_report-#{Time.now.to_s.gsub(/\s/, '-')}.csv"
# => "transaction_report-2019-10-10-16:09:40-+0100.csv"
Is that what you're after? It sounds right based on the question, though happy to update if you're able to correct me :)

using rails how do I import CSV onto mongodb

I am trying to import CSV files into my mongodb Using Ruby on Rails.
I know how to do it from the shell/terminal but not how to do it from Rails
Following this tutorial as an example Using MongoDB to store and retrieve CSV files content in Ruby, you can store all the values as strings, so you only need to read the CSV file and MongoDB dynamically creates all the needed attributes in objects that should represent each file row for the given CSV file:
class StoredCSV
include Mongoid::Document
include Mongoid::Timestamps
def self.import!(file_path)
columns = []
instances = []
CSV.foreach(file_path) do |row|
if columns.empty?
# We dont want attributes with whitespaces
columns = row.collect { |c| c.downcase.gsub(' ', '_') }
next
end
instances << create!(build_attributes(row, columns))
end
instances
end
private
def self.build_attributes(row, columns)
attrs = {}
columns.each_with_index do |column, index|
attrs[column] = row[index]
end
attrs
end
end
Usage
StoredCSV.import!('data.csv')
stored_data = StoredCSV.all

Saving my rails object from an array of values sourced form a csv

i'm currently trying to update attributes via a csv upload.
My method looks like this:
def upload_csv
CSV.parse(params[:file].read, headers: true) do |row|
foo = Foo.find_by_id(row.to_hash["id"])
row.to_hash.each do |v|
if Foo.new.has_attribute?(v[0]) && v[0] != "id"
foo.update_attributes()
end
end
end
end
When it jumps into where I want to update my attributes, i'm getting an array that looks like this:
["bar", "22"]
How can I save that value to my foo object?
Ok, so reading you're code I'm concluding that your problem is really that you have a CSV that may contain some fields that are not in your model:
def upload_csv
excluded = %w/id created_at updated_at/
CSV.new( params[:file], headers: true) do |row|
rh = row.to_hash
foo = Foo.find_by id: rh['id']
foo.update! rh.slice(*foo.attribute_names).except(*excluded)
end
end
Note that I'm assuming params[:file] is an uploaded file from a form, in which case it's an IO object, and so can be passed into CSV.new directly (no need to read it all into memory and pass it to CSV.parse).

Ruby, Tempfile, CSV

I have the below resque job that produces a csv file and sends it to a mailer. I want to validate that the csv file has data so I do not email blank files. For some reason, when I write a method outside of the perform method, it will not work. For example, the below code will print invalid when I know the csv file has data on the first line. If I uncomment the line below ensure it works properly, however I want to extract this checking of the file into a separate method. Is this correct?
class ReportJob
#queue = :report_job
def self.perform(application_id, current_user_id)
user = User.find(current_user_id)
client_application = Application.find(client_application_id)
transactions = application.transactions
file = Tempfile.open(["#{Rails.root}/tmp/", ".csv"]) do |csv|
begin
csv_file = CSV.new(csv)
csv_file << ["Application", "Price", "Tax"]
transactions.each do |transaction|
csv_file << [application.name, transaction.price, transaction.tax]
end
ensure
ReportJob.email_report(user.email, csv_file)
#ReportMailer.send_report(user.email, csv_file).deliver
csv_file.close(unlink=true)
end
end
end
def self.email_report(email, csv)
array = csv.to_a
if array[1].blank?
puts "invalid"
else
ReportMailer.send_report(email, csv).deliver
end
end
end
You should invoke your method as such:
ReportJob.email_report(email, csv)
Otherwise, get rid of the self in:
def self.email_report(email, csv)
# your implementation here.
end
and define your method as follows:
def email_report(email, csv)
# your implementation.
end
This is something that we call Class Methods and Instance Methods.

How to load the data from a .yml file to database?

There is a table questions, and a data file questions.yml. Assume there is no 'Question' model.
'questions.yml' has some recodes dump from the table.
---
questions_001:
title: ttt1
content: ccc1
questions_002:
title: ttt2
content: ccc2
I want to load the data from the yml file, insert them to database. But I can't use rake db:fixtures:load, because it will treat the content as 'erb' template, which is not want I want
So I want to write another rake task, to load the data manually.
I can read the records by:
File.open("#{RAILS_ROOT}/db/fixtures/#{table_name}.yml", 'r') do |file|
YAML::load(file).each do |record|
# how to insert the record??
end
end
But I don't know how to insert them.
Edit:
I have tried:
Class.new(ActiveRecord::Base).create(record)
and
class Dummy < ActiveRecord::Base {}
Dummy.create(rcord)
But nothing inserted to database
Try this after loading the date from the yml file to records:
class Question < ActiveRecord::Base
# Question model just to import the yml file
end
records.each { |record| Question.create(record) }
You can simply create a model just for importing. You don't need to create the app/models/question.rb. Just write the code above in the script responsible for the importing.
UPDATE:
You can use the following function:
def create_class(class_name, superclass, &block)
klass = Class.new superclass, &block
Object.const_set class_name, klass
end
source
File.open("#{RAILS_ROOT}/db/fixtures/#{table_name}.yml", 'r') do |file|
YAML::load(file).each do |record|
model_name = table_name.singularize.camelize
create_class(model_name, ActiveRecod::Base) do
set_table_name table_name.to_sym
end
Kernel.const_get(model_name).create(record)
end
end
To use the connection directly you can use the following:
ActiveRecord::Base.connection.execute("YOUR SQL CODE")
Got it working thanks to #jigfox 's answer. Had to modify a bit for the full implementation now with Rails 4.
table_names = Dir.glob(Rails.root + 'app/models/**.rb').map { |s| Pathname.new(s).basename.to_s.gsub(/\.rb$/,'') }
table_names.each do |table_name|
table_name = table_name.pluralize
path = "#{Rails.root}/db/fixtures/#{table_name}.yml"
if File.exists?(path)
File.open(path, 'r') do |file|
y = YAML::load(file)
if !y.nil? and y
y.each do |record|
model_name = table_name.singularize.camelize
rec = record[1]
rec.tap { |hs| hs.delete("id") }
Kernel.const_get(model_name).create(rec)
end
end
end
end
end
This loads fixtures into the current RAILS_ENV, which, by default, is development.
$ rake db:fixtures:load

Resources