I am trying to import CSV files into my mongodb Using Ruby on Rails.
I know how to do it from the shell/terminal but not how to do it from Rails
Following this tutorial as an example Using MongoDB to store and retrieve CSV files content in Ruby, you can store all the values as strings, so you only need to read the CSV file and MongoDB dynamically creates all the needed attributes in objects that should represent each file row for the given CSV file:
class StoredCSV
include Mongoid::Document
include Mongoid::Timestamps
def self.import!(file_path)
columns = []
instances = []
CSV.foreach(file_path) do |row|
if columns.empty?
# We dont want attributes with whitespaces
columns = row.collect { |c| c.downcase.gsub(' ', '_') }
next
end
instances << create!(build_attributes(row, columns))
end
instances
end
private
def self.build_attributes(row, columns)
attrs = {}
columns.each_with_index do |column, index|
attrs[column] = row[index]
end
attrs
end
end
Usage
StoredCSV.import!('data.csv')
stored_data = StoredCSV.all
Related
Im new on the ruby on rails
How can i read datas from csv files?
I have a csv file, and it has datas like name surname etc.
Here, there is a name surname variables. I need to read from csv files. How can i do it?
my codes now like this:
require 'csv'
table = CSV.parse(File.read("csv files/department.csv"), headers: true)
class Person
#name
#surname
#role
#department
def printPersonFullname(name, surname)
#name = name
#surname = surname
puts ("#{#name} #{#surname}")
end
end
here csv file image:
By the way, when i print, console print this:
D:\Ruby27-x64\bin\ruby.exe: No such file or directory -- csv.rb (LoadError)
Here goes a working example of reading CSV, instantiating Person object and printing.
# following the directory tree:
# ./
# ./tasks/main.rb
# ./csv/department.csv
#
# csv file example
#
# name,surname,department
# John,Doe,IT
# Bill,Doe,IT
#
# running this code:
# ruby tasks/main.rb
require 'csv'
class Person
def initialize(name, surname, department)
#name = name
#surname = surname
#department = department
end
def print_full_name
puts "#{#name} #{#surname}"
end
end
### main
filepath = "#{__dir__}/../csv/department.csv"
CSV.foreach(filepath, headers: true) do |row|
person = Person.new(
row["name"],
row["surname"],
row["department"]
)
person.print_full_name
end
Your code in the question wasn't written in ruby way. I recommend you to read this ruby styleguide to follow some practices on our community
I want to create a CRON task for daily report. I need guidance where to create my class in my project (in which folder). How to instantiate an object from rails console for the same class. Will that class inherit application controller? I would also like to know since i will be querying my database so would my models be directly accessible in this file or somehow i have to include them like we do in django?
I have created a class /lib/tasks/daily_report.rb. But i am unable to understand how will i use that file to create a task.
module Reports
class Report
class << self
def collect_data
row_data = []
headers = ["Mobile", "Buildings", "Owners", "Tenants", "Members", "Total People"]
row_data.push(*headers)
puts "in side collect data"
date = Date.today.to_s
mobile = ["mobiles"]
for i in mobile do
row = []
row << i
build_count = Buildings.where(created_at: date, added_by: i).count
row << build_count
puts "build_count"
owners_count = Residents.where(created_at: date, added_by: i, role: "owner").count
row << owners_count
puts "owners_count"
tenants_count = Residents.where(created_at: date, added_by: i, role: "tenant").count
row << tenants_count
members_count = MemeberRelations.where(created_at: date, added_by: i).count
row << members_count
total_people = owners_count + tenants_count + members_count
row << total_people
row_data << row
end
puts row_data
return row_data
end
def generate_csv()
puts "walk away"
row_data = self.collect_data
CSV.open('/home/rajdeep/police-api/daily_report.csv', 'w') do |csv|
row_data.each { |ar| csv << ar }
end
end
end
end
end
If you wish to manage cron tasks from Rails, try whenever gem.
Add it to your Gemfile,
Gemfile
gem 'whenever', require: false
Run initialize task from root of your app
$ bundle exec wheneverize .
This will create an initial config/schedule.rb file for you (as long
as the config folder is already present in your project)
(from the gem docs).
After that in config/schedule.rb set proper parameters of call time. For example
config/schedule.rb
every :hour do # Many shortcuts available: :hour, :day, :month, :year, :reboot
runner "Report.generate_csv"
end
More syntax options of schedule.rb here
UPDATE AFTER COMMENTS
Hope, you're under Rails context yet. Create file in public folder at application root path.
result_file = "#{Rails.root}/public/cards-excluded.csv"
CSV.open(result_file, 'w') do |csv|
row_data.each { |ar| csv << ar }
end
ANOTHER UPDATE LATER
Okay, although this is not relevant to the original question, let's try to solve your problem.
We'll proceed from what you have Rails application, not custom Ruby library.
First, create module at your_rals_app/lib/reports.rb file
module Reports
class Report
class << self
def collect_data
# your current code and line below, explicit return
return row_data
end
def generate_csv
row_data = collect_data # btw unnecessary assignment
CSV.open('/home/rajdeep/police-api/daily_report.csv', 'w') do |csv|
row_data.each { |ar| csv << ar }
end
end
end
end
end
Second, make sure, that you have lib files at autoload path. Check it in you config/application.rb
config.autoload_paths += %W(#{config.root}/lib/*)
Thirdly, use Reports module such way (> means that you're at rails console, rails c)
> Reports::Report.generate_csv
Is there a way to achieve exporting the entire active records in a rails application to csv where each relation is a sheet of csv or there is another way to export full db data.Any suggestions?
Try like this you can get datas from your tables. after you can split it by sheet
models = ActiveRecord::Base.connection.tables
models.shift
models.shift
models.map do |model_name|
model_name = model_name.split("")
model_name.pop
model_name = model_name.join("")
model_name.camelize.constantize.all.map do |data|
puts data
end
end
def backup
models = ActiveRecord::Base.connection.tables
all_data = Hash.new
models.map do |model_name|
table_data = []
model_name = model_name.split("")
model_name.pop
model_name = model_name.join("")
model_name.camelize.constantize.all.map do |data|
table_data.push(data)
end
all_data[model_name.camelize] = table_data
end
send_data export_csv(all_data), filename: "Backup - #{Date.today}.csv" and return
end
def export_csv(data)
csvfile = CSV.generate(headers: true) do |csv|
data.each do |key, value|
csv << [key]
attributes = key.camelize.constantize.column_names
csv << attributes
value.each do |val|
csv << val.attributes.values_at(*attributes)
end
csv << ['eot']
end
end
return csvfile
end
I found the solution and a way to export all tables inside a single csv file.
There is a table questions, and a data file questions.yml. Assume there is no 'Question' model.
'questions.yml' has some recodes dump from the table.
---
questions_001:
title: ttt1
content: ccc1
questions_002:
title: ttt2
content: ccc2
I want to load the data from the yml file, insert them to database. But I can't use rake db:fixtures:load, because it will treat the content as 'erb' template, which is not want I want
So I want to write another rake task, to load the data manually.
I can read the records by:
File.open("#{RAILS_ROOT}/db/fixtures/#{table_name}.yml", 'r') do |file|
YAML::load(file).each do |record|
# how to insert the record??
end
end
But I don't know how to insert them.
Edit:
I have tried:
Class.new(ActiveRecord::Base).create(record)
and
class Dummy < ActiveRecord::Base {}
Dummy.create(rcord)
But nothing inserted to database
Try this after loading the date from the yml file to records:
class Question < ActiveRecord::Base
# Question model just to import the yml file
end
records.each { |record| Question.create(record) }
You can simply create a model just for importing. You don't need to create the app/models/question.rb. Just write the code above in the script responsible for the importing.
UPDATE:
You can use the following function:
def create_class(class_name, superclass, &block)
klass = Class.new superclass, &block
Object.const_set class_name, klass
end
source
File.open("#{RAILS_ROOT}/db/fixtures/#{table_name}.yml", 'r') do |file|
YAML::load(file).each do |record|
model_name = table_name.singularize.camelize
create_class(model_name, ActiveRecod::Base) do
set_table_name table_name.to_sym
end
Kernel.const_get(model_name).create(record)
end
end
To use the connection directly you can use the following:
ActiveRecord::Base.connection.execute("YOUR SQL CODE")
Got it working thanks to #jigfox 's answer. Had to modify a bit for the full implementation now with Rails 4.
table_names = Dir.glob(Rails.root + 'app/models/**.rb').map { |s| Pathname.new(s).basename.to_s.gsub(/\.rb$/,'') }
table_names.each do |table_name|
table_name = table_name.pluralize
path = "#{Rails.root}/db/fixtures/#{table_name}.yml"
if File.exists?(path)
File.open(path, 'r') do |file|
y = YAML::load(file)
if !y.nil? and y
y.each do |record|
model_name = table_name.singularize.camelize
rec = record[1]
rec.tap { |hs| hs.delete("id") }
Kernel.const_get(model_name).create(rec)
end
end
end
end
end
This loads fixtures into the current RAILS_ENV, which, by default, is development.
$ rake db:fixtures:load
I'm developing a "script generator" to automatize some processes at work.
It has a Rails application running on a server that stores all data needed to make the script and generates the script itself at the end of the process.
The problem I am having is how to export the data from the ActiveRecord format to Plain Old Ruby Objects (POROs) so I can deal with them in my script with no database support and a pure-ruby implementation.
I thought about YAML, CSV or something like this to export the data but it would be a painful process to update these structures if the process changes. Is there a simpler way?
Ty!
By "update these structures if the process changes", do you mean changing the code that reads and writes the CSV or YAML data when the fields in the database change?
The following code writes and reads any AR object to/from CSV (requires the FasterCSV gem):
def load_from_csv(csv_filename, poro_class)
headers_read = []
first_record = true
num_headers = 0
transaction do
FCSV.foreach(csv_filename) do |row|
if first_record
headers_read = row
num_headers = headers_read.length
first_record = false
else
hash_values = {}
for col_index in 0...num_headers
hash_values[headers_read[col_index]] = row[col_index]
end
new_poro_obj = poro_class.new(hash_values) # assumes that your PORO has a constructor that accepts a hash. If not, you can do something like new_poro_obj.send(headers_read[col_index], row[col_index]) in the loop above
#work with your new_poro_obj
end
end
end
end
#objects is a list of ActiveRecord objects of the same class
def dump_to_csv(csv_filename, objects)
FCSV.open(csv_filename,'w') do |csv|
#get column names and write them as headers
col_names = objects[0].class.column_names()
csv << col_names
objects.each do |obj|
col_values = []
col_names.each do |col_name|
col_values.push obj[col_name]
end
csv << col_values
end
end
end