I have set up a CSV import for my Ruby App. Everything works fairly well but I'm running into problems when I try to upload a string that then searches for an id to input as the foreign key in the hash. My model for the CSV looks as follows:
class Player < ActiveRecord::Base
belongs_to :team
validates :team_id, presence: true
def self.import(file)
CSV.foreach(file.path, headers: true, :header_converters => :symbol) do |row|
player_hash = row.to_hash
teamname = player_hash[:team]
teamhash = Team.where(:name => teamname).first
hashid = teamhash.id
player_newhash = player_hash.reject!{ |k| k == :team}
player_newhash[:team_id] = hashid
end
Player.create! (player_newhash)
end
end
I'm sure this is where the problem lies. When trying to execute I get the error:
undefined local variable or method `player_newhash' for #
Any help would be greatly appreciated.
player_newhash is a local variable inside your foreach loop - so your create would need to be in there as well:
class Player < ActiveRecord::Base
belongs_to :team
validates :team_id, presence: true
def self.import(file)
CSV.foreach(file.path, headers: true, :header_converters => :symbol) do |row|
player_hash = row.to_hash
teamname = player_hash[:team]
teamhash = Team.where(:name => teamname).first
hashid = teamhash.id
player_newhash = player_hash.reject!{ |k| k == :team}
player_newhash[:team_id] = hashid
Player.create!(player_newhash)
end
end
end
BTW - I got that answer by refactoring for a second to find out what was going on... in case it's helpful my version looked like this:
class Player < ActiveRecord::Base
belongs_to :team
validates :team_id, presence: true
def self.import(file)
CSV.foreach(file.path, headers: true, :header_converters => :symbol) do |row|
player_hash = row.to_hash
player_newhash = player_hash.reject!{ |k| k == :team}
player_newhash[:team_id] = find_team(player_hash[:team]).id
Player.create!(player_newhash)
end
end
private
def find_team(team_name)
Team.where(name: team_name).first
end
end
Related
I have been trying to import a .csv file on my Ruby on Rails application in order to store the data about a Peak, but failed using different methods. The application redirects to the root_url but the database remains empty. The program seems to work, at least it does not provide errors, but no data is imported into the database. This how my peaks_controller looks like:
class PeaksController < ApplicationController
def show
#peak = Peak.find(params[:id])
end
def index
#peaks = Peak.all
end
def import
Peak.import(params[:file])
redirect_to root_url, notice: "Peaks imported."
end
def new
#peak = Peak.new
end
def create
#peak = Peak.new(peak_params)
if #peak.save
redirect_to #peak
else
render 'new'
end
end
def destroy
end
private
def peak_params
params.require(:peak).permit(:name, :altitude, :prominence, :isolation, :key_col, :source, :accessibility, :land_use, :hazard, :longitude, :latitude)
end
end
This is my peak.rb class:
class Peak < ApplicationRecord
validates :altitude, presence: true
validates :prominence, presence: true
validates :isolation, presence: true
validates :key_col, presence: true
validates :source, presence: true
validates :accessibility, presence: true
validates :land_use, presence: true
validates :longitude, presence: true
validates :latitude, presence: true
#non funziona
def self.import(file)
csv = CSV.parse(File.read(file), :headers => true)
csv.each do |row|
p = Peak.new
p.id = row['id']
p.name = row['Name']
p.altitude = row['Altitude']
p.prominence = row['Prominence']
p.isolation = row['Isolation']
p.key_col = row['Key_col']
p.source = row['Source']
p.accessibility = row['Accessibility']
p.land_use = row['Land_use']
p.hazard = row['Hazard']
p.longitude = row['x']
p.latitude = row['y']
p.save
end
end
end
That just does not return anything, so it does not import anything into my database, I have also tried the following import method:
def self.import(file)
csv = CSV.parse(File.read(file), :headers => false)
csv.each do |row|
Peak.create!(row.to_h)
end
end
This is how my database looks like:
class CreatePeaks < ActiveRecord::Migration[6.0]
def change
create_table :peaks do |t|
t.string :name
t.decimal :altitude
t.decimal :prominence
t.decimal :isolation
t.decimal :key_col
t.string :source
t.decimal :accessibility
t.string :land_use
t.string :hazard
t.decimal :longitude
t.decimal :latitude
end
end
end
[And this are the headerds of the .csv file][:1]
I have inclunded "require 'csv'" on my application.rb so the application reads a csv file.
I have also tried to remove the 'id' column from the file and tried with the "row.to_h" but it did not change anything, still can't import the values onto the database.
Do you have any suggestions?
The hard part when doing a mass import is error handling - of which you're doing none. So all those calls to save the records could be failing and you're none the wiser.
You basically have two options:
1. Strict
Wrap the import in a transaction and roll back if any of the records are invalid. This avoids leaving the job half done.
class Peak < ApplicationRecord
# ...
def self.import(file)
csv = CSV.parse(File.read(file), :headers => true)
transaction do
csv.map do |row|
create!(
id: row['id'],
name: row['Name'],
altitude: row['Altitude'],
prominence: row['Prominence'],
isolation: row['Isolation'],
key_col: row['Key_col'],
source: row['Source'],
accessibility: row['Accessibility'],
land_use: row['Land_use'],
hazard: row['Hazard'],
longitude: row['x'],
latitude: row['y']
)
end
end
end
end
class PeaksController < ApplicationController
def import
begin
#peaks = Peak.import(params[:file])
redirect_to root_url, notice: "Peaks imported."
rescue ActiveRecord::RecordInvalid
redirect_to 'somewhere/else', error: "Import failed."
end
end
end
2. Lax
In some scenarios you might want to do lax processing and just keep on going even if some records fail to import. This can be combined with a form that lets the user correct the invalid values.
class Peak < ApplicationRecord
# ...
def self.import(file)
csv = CSV.parse(File.read(file), :headers => true)
peaks = csv.map do |row|
create_with(
name: row['Name'],
altitude: row['Altitude'],
prominence: row['Prominence'],
isolation: row['Isolation'],
key_col: row['Key_col'],
source: row['Source'],
accessibility: row['Accessibility'],
land_use: row['Land_use'],
hazard: row['Hazard'],
longitude: row['x'],
latitude: row['y']
).find_or_create_by(id: row['id'])
end
end
end
class PeaksController < ApplicationController
def import
#peaks = Peak.import(params[:file])
#invalid_peaks = #peaks.reject {|p| p.persisted? }
if #invalid_peaks.none?
redirect_to root_url, notice: "Peaks imported."
else
flash.now[:error] = "import failed"
render 'some_kind_of_form'
end
end
end
I have this import method in my active record which I use to import the csv file. I want to know how to do the error handling of this in the active record.
class SheetEntry < ActiveRecord::Base
unloadable
belongs_to :user
belongs_to :project
belongs_to :task
validate :project_and_task_should_be_active
def self.import(csv_file)
attributes = [:user_id, :project_id, :task_id, :date, :time_spent, :comment]
errors=[]
output = {}
i=0
CSV.foreach(csv_file, headers: true, converters: :date).with_index do |row,j|
entry_hash= row.to_hash
entry_hash['Project'] = SheetProject.where("name= ?" , entry_hash['Project']).pluck(:id)
entry_hash['Task'] = SheetTask.where("name= ?" , entry_hash['Task']).pluck(:id)
entry_hash['Date'] = Time.strptime(entry_hash['Date'], '%m/%d/%Y').strftime('%Y-%m-%d')
entry_hash['Time (Hours)'] = entry_hash['Time (Hours)'].to_f
firstname = entry_hash['User'].split(" ")[0]
lastname = entry_hash['User'].split(" ")[1]
entry_hash['User'] = User.where("firstname=? AND lastname=?",firstname,lastname).pluck(:id)
entry_hash.each do |key,value|
if value.class == Array
output[attributes[i]] = value.first.to_i
else
output[attributes[i]] = value
end
i += 1
end
entry=SheetEntry.new(output)
entry.editing_user = User.current
entry.save!
end
end
def project_and_task_should_be_active
errors.add(:sheet_project, "should be active") unless sheet_project.active?
errors.add(:sheet_task, "should be active") if sheet_task && !sheet_task.active?
end
end
I want to know how to show the error if there is a nil object returned for either entry_hash['Project'] or entry_hash['Task'] or for any of the fields in the csv.
For example: If the user had entered the wrong project or wrong task or wrong date. I want the error to be shown along with the line no and stop the uploading of the csv. Can someone help?
You can use begin and rescue statements to handle errors in any ruby classes.
You can use the rescue block to return the Exception e back to the caller.
However, you cannot call errors.add method to add error because #errors is an instance method which is not accessible inside class method self.import.
def self.import(csv_file)
begin
attributes = [:user_id, :project_id, :task_id, :date, :time_spent, :comment]
errors=[]
output = {}
i=0
CSV.foreach(csv_file, headers: true, converters: :date).with_index do |row,j|
...
end
rescue Exception => e
return "Error: #{e}"
end
end
Good morning! My english is not the best.
I'm trying to import some data from csv file using this model.
class Recibo < ActiveRecord::Base
attr_accessible :id,
:caja_id,
:doctor_id,
:numero_recibo,
:paciente,
:total,
:total_porcentaje_doctor,
:total_porcentaje_clinica,
:total_porcentaje_laboratorio,
:servicio_ids,
:created_at,
:updated_at
belongs_to :caja
belongs_to :doctor
has_many :atencions
has_many :servicios, :through => :atencions
before_save do
servicio_by_id = Servicio.where(:id => servicio_ids)
self.total = servicio_by_id.sum(&:precio)
self.total_porcentaje_doctor = servicio_by_id.sum ('porcentaje_doctor / 100.0 * precio')
self.total_porcentaje_clinica = servicio_by_id.sum ('porcentaje_clinica / 100.0 * precio')
self.total_porcentaje_laboratorio = servicio_by_id.sum ('porcentaje_laboratorio / 100.0 * precio')
end
def self.to_csv
CSV.generate do |csv|
csv << ["id", "caja_id", "doctor_id", "numero_recibo", "paciente", "total", "total_porcentaje_laboratorio",
"total_porcentaje_clinica", "total_porcentaje_doctor", "created_at", "updated_at", "servicio_ids" ]
all.each do |recibo|
recibo.atencions.map(&:servicio_id)
csv << [recibo.id, recibo.caja_id, recibo.doctor_id, recibo.numero_recibo,
recibo.paciente, recibo.total, recibo.total_porcentaje_laboratorio, recibo.total_porcentaje_clinica,
recibo.total_porcentaje_doctor, recibo.created_at, recibo.updated_at, recibo.servicio_ids]
end
end
end
def self.import(file)
CSV.foreach(file.path, headers: true) do |row|
recibo = find_by_id(row["id"]) || new
recibo.attributes = row.to_hash.slice(*accessible_attributes)
recibo.save!
end
end
end
my csv file contain data like this:
id,caja_id,doctor_id,numero_recibo,paciente,total,total_porcentaje_laboratorio,total_porcentaje_clinica,total_porcentaje_doctor,created_at,updated_at,servicio_ids
1,2,3,,Nombre,8,0,4,4,2014-04-21 15:45:29 -0500,2014-05-27 18:58:54 -0500,[1]
2,2,1,,Nombre2,11,0,5.5,5.5,2014-04-21 16:38:32 -0500,2014-05-27 19:28:20 -0500,[1, 8]
The self.import(file) suppose to add the records servicio_ids in the table atencion but it dosen't. I don't know what to do.
Thanks for everything!
When generating your .csv file, instead of:
CSV.generate do |csv|
do:
CSV.generate(force_quotes: true) do |csv|
By default CSV adds commas between the values, which messes up the parsing in your case, because of the commas inside the array elements.
How exactly do you do arithmetic operations in the controller?
I've tried this
def choose
rand_id = rand(Gif.count)
#gif1 = Gif.first(:conditions => [ "id >= ?", rand_id])
#gif2 = Gif.first(:conditions => [ "id >= ?", rand_id])
if #gif1.id == #gif2.id
#gif2 = Gif.first(:order => 'Random()')
end
total = #gif1.votes+#gif2.votes
number_one = #gif1.votes/total*100
number_two = #gif2.votes/total*100
#gif1.update_attribute(:votes, number_one)
#gif2.update_attribute(:votes, number_two)
end
class Gif < ActiveRecord::Base
before_save :default_agree_count
def default_agree_count
self.agree = 1
self.votes = 1
end
VALID_REGEX = /http:\/\/[\S]*\.gif$/
attr_accessible :link, :votes, :agree
acts_as_votable
validates :link, presence: true, format: {with: VALID_REGEX}, uniqueness: {case_sensitive: false}
end
However, it says that +, /, * are all unknown operators. I've also tried doing them within like such #gif1.agree = '#gif1.votes+1' with and without '. Any ideas?
Thanks!
I suppose you are using Acts As Votable gem.
Basically it works as follows:
#post = Post.new(:name => 'my post!')
#post.save
#post.liked_by #user
#post.votes.size # => 1
So try replacing .votes with .votes.size in your code.
E.g.:
total = #gif1.votes.size + #gif2.votes.size
Further to #ilyai's answer (which I +1'd) (I don't have much experience with the Acts As Votable gem), you can perform any calculations you want in your controllers
Here's some refactoring for you:
.first
def choose
Gif.update_votes
end
class Gif < ActiveRecord::Base
before_save :default_agree_count
def default_agree_count
self.agree = 1
self.votes = 1
end
def self.update_votes
rand_id = rand count #-> self.count?
gif = where("id >= ?", rand_id)
gif1 = gif[0]
gif2 = gif[1]
if gif1.id == gif2.id
gif2 = where(order: 'Random()').first
end
total = (gif1.votes) + (gif2.votes)
number_one = ((gif1.votes /total) * 100)
number_two = ((gif2.votes / total) * 100)
gif1.update_attribute(:votes, number_one)
gif2.update_attribute(:votes, number_two)
end
VALID_REGEX = /http:\/\/[\S]*\.gif$/
attr_accessible :link, :votes, :agree
acts_as_votable
validates :link, presence: true, format: {with: VALID_REGEX}, uniqueness: {case_sensitive: false}
end
I've got below code in my model which I want to use for validation:
class Payslip < ActiveRecord::Base
include ActiveModel::Validations
attr_accessor :first_name, :last_name, :annual_salary, :super, :payment_start_date
validates :annual_salary, :super, numericality: { only_integer: true },
presence: true
validates :super, inclusion: { in: 0..50 }
validates :first_name, :last_name, presence: true,
length: { maximum: 100 }
validates_date :payment_start_date
validates :payment_start_date, presence: true
end
I have CSV import from the form and this gets passed over to concern:
module CSV_Manager
extend ActiveSupport::Concern
class << self
def extract_csv(csv_file, headers)
results = []
CSV.foreach(csv_file.path, {headers: false, :encoding => 'UTF-8'}) do |row|
data = row.split(',')
Payslip.first_name = data[0]
Payslip.last_name = data[1]
Payslip.super = data[2]
results.push(row) unless $. == headers.to_i
end
return results
end
def prepare_csv(rows, headers)
csv_file = CSV.generate do |csv|
csv << headers
rows.map { |row| csv << row }
end
return csv_file
end
end
end
I've added Payslip.first_name etc in an attempt to validate and throw an error if not validated.
Is this the right approach?
Here's a rough suggestion on how I would handle the problem you're trying to solve. Feel free to comment if this isn't what you're looking for:
def extract_csv(csv_file, headers)
results = []
CSV.foreach(csv_file.path, {headers: false, :encoding => 'UTF-8'}) do |row|
data = row.split(',')
payslip = Payslip.create({
first_name: data[0],
last_name: data[1],
super: data[2]
})
results.push(row) unless $. == headers.to_i
end
return results
end
Also, super is a reserved keyword, so I would suggest possibly not using it if you an avoid it.