Error handling of csv upload in rails - ruby-on-rails

I have this import method in my active record which I use to import the csv file. I want to know how to do the error handling of this in the active record.
class SheetEntry < ActiveRecord::Base
unloadable
belongs_to :user
belongs_to :project
belongs_to :task
validate :project_and_task_should_be_active
def self.import(csv_file)
attributes = [:user_id, :project_id, :task_id, :date, :time_spent, :comment]
errors=[]
output = {}
i=0
CSV.foreach(csv_file, headers: true, converters: :date).with_index do |row,j|
entry_hash= row.to_hash
entry_hash['Project'] = SheetProject.where("name= ?" , entry_hash['Project']).pluck(:id)
entry_hash['Task'] = SheetTask.where("name= ?" , entry_hash['Task']).pluck(:id)
entry_hash['Date'] = Time.strptime(entry_hash['Date'], '%m/%d/%Y').strftime('%Y-%m-%d')
entry_hash['Time (Hours)'] = entry_hash['Time (Hours)'].to_f
firstname = entry_hash['User'].split(" ")[0]
lastname = entry_hash['User'].split(" ")[1]
entry_hash['User'] = User.where("firstname=? AND lastname=?",firstname,lastname).pluck(:id)
entry_hash.each do |key,value|
if value.class == Array
output[attributes[i]] = value.first.to_i
else
output[attributes[i]] = value
end
i += 1
end
entry=SheetEntry.new(output)
entry.editing_user = User.current
entry.save!
end
end
def project_and_task_should_be_active
errors.add(:sheet_project, "should be active") unless sheet_project.active?
errors.add(:sheet_task, "should be active") if sheet_task && !sheet_task.active?
end
end
I want to know how to show the error if there is a nil object returned for either entry_hash['Project'] or entry_hash['Task'] or for any of the fields in the csv.
For example: If the user had entered the wrong project or wrong task or wrong date. I want the error to be shown along with the line no and stop the uploading of the csv. Can someone help?

You can use begin and rescue statements to handle errors in any ruby classes.
You can use the rescue block to return the Exception e back to the caller.
However, you cannot call errors.add method to add error because #errors is an instance method which is not accessible inside class method self.import.
def self.import(csv_file)
begin
attributes = [:user_id, :project_id, :task_id, :date, :time_spent, :comment]
errors=[]
output = {}
i=0
CSV.foreach(csv_file, headers: true, converters: :date).with_index do |row,j|
...
end
rescue Exception => e
return "Error: #{e}"
end
end

Related

Rails API Does not split Json

Weird problem. If the class at the bottom was a module, split the Json without problems, if it was only methods, also works, but the problem is.. when it is a class, it does not split the Json anymore, and returns an empty array.. however, if being a class, I do a puts the object, it actually puts it..
Any thoughts about why? How can I fix it?
I have this controller:
def index
begin
call_employee_work_locations_api
rescue => ex
render :json => {"service unavailable": "0001" }, :status => :service_unavailable
end
end
I have this service:
def call_employee_work_locations_api
auth = {:username=>ENV["USERNAME"], :password=>ENV["PASSWORD"]}
employee_locations = HTTParty.get(employee_work_Location_url , :basic_auth => auth)
#serialize_work_location(employee_locations)
serializer = EmployeeSerializer.new
serializer.serialize_work_location(employee_locations)
end
I have this builder:
json.array!(#top_locations) do |location|
json.extract! location, :name, :description, :latitude, :longitude
end
I have this class:
class EmployeeSerializer
def serialize_work_location(employee_locations)
employee_locations= JSON.parse(employee_locations)
locations=[]
employee_locations["work_locations"].each do |attributes|
location = Location.new(attributes["latitude"],attributes["longitude"],attributes["description"],attributes["name"])
locations.push(location)
end
employee_locations_selector(locations)
end
def top_office_location_selector(locations, city)
top_locations=[]
locations.each do |office|
if office.name == city[0] then top_locations.push(office) end
if office.name == city[1] then top_locations.push(office) end
end
#top_locations = top_locations
p #top_locations <--- it prints the object perfectly, but does not pass to the view, I get an empty array instead.
end
def employee_locations_selector(locations)
city = locations.each_with_object(Hash.new(0)) { |locations, counts| counts[locations.name] += 1 }.max_by{|k,v| v}
top_office_location_selector(locations, city)
end
end
The instance variable #top_locations is being set within the scope of the EmployeeSerializer class, not your controller. As such it's just a normal instance variable and so Rails knows nothing about it. You can assign the return value of #top_office_location_selector to an instance variable in the controller and it should work.
On a side note, the code would be cleaned up a lot by using #map over #each.

How can I test these RSS parsing service objects?

I have some service objects that use Nokogiri to make AR instances. I created a rake task so that I can update the instances with a cron job. What I want to test is if it's adding items that weren't there before, ie:
Create an Importer with a url of spec/fixtures/feed.xml, feed.xml having 10 items.
Expect Show.count == 1 and Episode.count == 10
Edit spec/fixtures/feed.xml to have 11 items
Invoke rake task
Expect Show.count == 1 and Episode.count == 11
How could I test this in RSpec, or modify my code to be more testable?
# models/importer.rb
class Importer < ActiveRecord::Base
after_create :parse_importer
validates :title, presence: true
validates :url, presence: true
validates :feed_format, presence: true
private
def parse_importer
Parser.new(self)
end
end
# models/show.rb
class Show < ActiveRecord::Base
validates :title, presence: true
validates :title, uniqueness: true
has_many :episodes
attr_accessor :entries
end
# models/episode.rb
class Episode < ActiveRecord::Base
validates :title, presence: true
validates :title, uniqueness: true
belongs_to :show
end
#lib/tasks/admin.rake
namespace :admin do
desc "Checks all Importer URLs for new items."
task refresh: :environment do
#importers = Importer.all
#importers.each do |importer|
Parser.new(importer)
end
end
end
# services/parser.rb
class Parser
def initialize(importer)
feed = Feed.new(importer)
show = Show.where(rss_link: importer.url).first
if show # add new episodes
new_episodes = Itunes::Channel.refresh(feed.origin)
new_episodes.each do |new_episode|
show.episodes.create feed.episode(new_episode)
end
else # create a show and its episodes
new_show = Show.new(feed.show) if (feed && feed.show)
if (new_show.save && new_show.entries.any?)
new_show.entries.each do |entry|
new_show.episodes.create feed.episode(entry)
end
end
end
end
end
# services/feed.rb
class Feed
require "nokogiri"
require "open-uri"
require "formats/itunes"
attr_reader :params, :origin, :show, :episode
def initialize(params)
#params = params
end
def origin
#origin = Nokogiri::XML(open(params[:url]))
end
def format
#format = params[:feed_format]
end
def show
case format
when "iTunes"
Itunes::Channel.fresh(origin)
end
end
def episode(entry)
#entry = entry
case format
when "iTunes"
Itunes::Item.fresh(#entry)
end
end
end
# services/formats/itunes.rb
class Itunes
class Channel
def initialize(origin)
#origin = origin
end
def title
#origin.xpath("//channel/title").text
end
def description
#origin.xpath("//channel/description").text
end
def summary
#origin.xpath("//channel/*[name()='itunes:summary']").text
end
def subtitle
#origin.xpath("//channel/*[name()='itunes:subtitle']/text()").text
end
def rss_link
#origin.xpath("//channel/*[name()='atom:link']/#href").text
end
def main_link
#origin.xpath("//channel/link/text()").text
end
def docs_link
#origin.xpath("//channel/docs/text()").text
end
def release
#origin.xpath("//channel/pubDate/text()").text
end
def image
#origin.xpath("//channel/image/url/text()").text
end
def language
#origin.xpath("//channel/language/text()").text
end
def keywords
keywords_array(#origin)
end
def categories
category_array(#origin)
end
def explicit
explicit_check(#origin)
end
def entries
entry_array(#origin)
end
def self.fresh(origin)
#show = Itunes::Channel.new origin
return {
description: #show.description,
release: #show.release,
explicit: #show.explicit,
language: #show.language,
title: #show.title,
summary: #show.summary,
subtitle: #show.subtitle,
image: #show.image,
rss_link: #show.rss_link,
main_link: #show.main_link,
docs_link: #show.docs_link,
categories: #show.categories,
keywords: #show.keywords,
entries: #show.entries
}
end
def self.refresh(origin)
#show = Itunes::Channel.new origin
return #show.entries
end
private
def category_array(channel)
arr = []
channel.xpath("//channel/*[name()='itunes:category']/#text").each do |category|
arr.push(category.to_s)
end
return arr
end
def explicit_check(channel)
string = channel.xpath("//channel/*[name()='itunes:explicit']").text
if string === "yes" || string === "Yes"
true
else
false
end
end
def keywords_array(channel)
keywords = channel.xpath("//channel/*[name()='itunes:keywords']/text()").text
arr = keywords.split(",")
return arr
end
def entry_array(channel)
arr = []
channel.xpath("//item").each do |item|
arr.push(item)
end
return arr
end
end
class Item
def initialize(origin)
#origin = origin
end
def description
#origin.xpath("*[name()='itunes:subtitle']").text
end
def release
#origin.xpath("pubDate").text
end
def image
#origin.xpath("*[name()='itunes:image']/#href").text
end
def explicit
explicit_check(#origin)
end
def duration
#origin.xpath("*[name()='itunes:duration']").text
end
def title
#origin.xpath("title").text
end
def enclosure_url
#origin.xpath("enclosure/#url").text
end
def enclosure_length
#origin.xpath("enclosure/#length").text
end
def enclosure_type
#origin.xpath("enclosure/#type").text
end
def keywords
keywords_array(#origin.xpath("*[name()='itunes:keywords']").text)
end
def self.fresh(entry)
#episode = Itunes::Item.new entry
return {
description: #episode.description,
release: #episode.release,
image: #episode.image,
explicit: #episode.explicit,
duration: #episode.duration,
title: #episode.title,
enclosure_url: #episode.enclosure_url,
enclosure_length: #episode.enclosure_length,
enclosure_type: #episode.enclosure_type,
keywords: #episode.keywords
}
end
private
def explicit_check(item)
string = item.xpath("*[name()='itunes:explicit']").text
if string === "yes" || string === "Yes"
true
else
false
end
end
def keywords_array(item)
keywords = item.split(",")
return keywords
end
end
end
Before anything else, good for you for using service objects! I've been using this approach a great deal and find POROs preferable to fat models in many situations.
It appears the behavior you're interested in testing is contained in Parser.initialize.
First, I'd create a class method for Parser called parse. IMO, Parser.parse(importer) is clearer about what Parser is doing than is Parser.new(importer). So, it might look like:
#services/parser.rb
class Parser
class << self
def parse(importer)
#importer = importer
#feed = Feed.new(importer)
if #show = Show.where(rss_link: importer.url).first
create_new_episodes Itunes::Channel.refresh(#feed.origin)
else
create_show_and_episodes
end
end # parse
end
end
Then add the create_new_episodes and create_show_and_episodes class methods.
#services/parser.rb
class Parser
class << self
def parse(importer)
#importer = importer
#feed = Feed.new(importer)
if #show = Show.where(rss_link: #importer.url).first
create_new_episodes Itunes::Channel.refresh(#feed.origin)
else
create_show_and_episodes
end
end # parse
def create_new_episodes(new_episodes)
new_episodes.each do |new_episode|
#show.episodes.create #feed.episode(new_episode)
end
end # create_new_episodes
def create_show_and_episodes
new_show = Show.new(#feed.show) if (#feed && #feed.show)
if (new_show.save && new_show.entries.any?)
new_show.entries.each do |entry|
new_show.episodes.create #feed.episode(entry)
end
end
end # create_show_and_episodes
end
end
Now you have a Parser.create_new_episodes method that you can test independently. So, your test might look something like:
require 'rspec_helper'
describe Parser do
describe '.create_new_episodes' do
context 'when an initial parse has been completed' do
before(:each) do
first_file = Nokogiri::XML(open('spec/fixtures/feed_1.xml'))
#second_file = Nokogiri::XML(open('spec/fixtures/feed_2.xml'))
Parser.create_show_and_episodes first_file
end
it 'changes Episodes.count by 1' do
expect{Parser.create_new_episodes(#second_file)}.to change{Episodes.count}.by(1)
end
it 'changes Show.count by 0' do
expect{Parser.create_new_episodes(#second_file)}.to change{Show.count}.by(0)
end
end
end
end
Naturally, you'll need feed_1.xml and feed_2.xml in the spec\fixtures directory.
Apologies for any typos. And, I didn't run the code. So, might be buggy. Hope it helps.

save items in an array to active records

I have an array that looks like this:
[{"id"=>"2", "reply"=>"ok"}, {"id"=>"3", "reply"=>"ok"}, {"id"=>"4", "reply"=>"ok"}, {"id"=>"5"}, {"id"=>"6", "reply"=>"2"}]
now I'm trying to save it like this:
current_user.answers.transaction do
success = params[:answers].map(&:save)
unless sucess.all?
errored = params[:answers].select { |b| !b.errors.blank? }
raise ActiveRecord::Rollback
end
end
but that results in
undefined method `save' for {"id"=>"2", "reply"=>"ok"}:ActionController::Parameters
does anybody know how I can save each item?
class Answer < ActiveRecord::Base
belongs_to :user
belongs_to :question
has_many :comments
validates :reply, :question_id, :week_number, presence: true
end
Yeah, just send it a block... so you can try:
current_user.answers.transaction do
success = params[:answers].map do |hash|
object = Model.new(hash)
object.save
end
unless success.all?
errored = params[:answers].select { |b| !b.errors.blank? }
raise ActiveRecord::Rollback
end
end
Realize that Model needs to be the name of the model you're trying to instantiate objects with.
but, that's only if you're making new objects, which generally won't be the case of you've set the id attribute already
current_user.answers.transaction do
success = params[:answers].map do |hash|
object = Model.find(hash["id"])
object.update_attributes(hash)
end
unless success.all?
errored = params[:answers].select { |b| !b.errors.blank? }
raise ActiveRecord::Rollback
end
end
Hope this helps

How can I lessen the verbosity of my populate method?

I wrote a form object to populate an Order, Billing, and Shipping Address objects. The populate method looks pretty verbose. Since the form fields don't correspond to Address attributes directly, I'm forced to manually assign them. For example:
shipping_address.name = params[:shipping_name]
billing_address.name = params[:billing_name]
Here's the object. Note that I snipped most address fields and validations, and some other code, for brevity. But this should give you an idea. Take note of the populate method:
class OrderForm
attr_accessor :params
delegate :email, :bill_to_shipping_address, to: :order
delegate :name, :street, to: :shipping_address, prefix: :shipping
delegate :name, :street, to: :billing_address, prefix: :billing
validates :shipping_name, presence: true
validates :billing_name, presence: true, unless: -> { bill_to_shipping_address }
def initialize(item, params = nil, customer = nil)
#item, #params, #customer = item, params, customer
end
def submit
populate
# snip
end
def order
#order ||= #item.build_order do |order|
order.customer = #customer if #customer
end
end
def shipping_address
#shipping_address ||= order.build_shipping_address
end
def billing_address
#billing_address ||= order.build_billing_address
end
def populate
order.email = params[:email]
shipping_address.name = params[:shipping_name]
shipping_address.street = params[:shipping_street]
# Repeat for city, state, post, code, etc...
if order.bill_to_shipping_address?
billing_address.name = params[:shipping_name]
billing_address.street = params[:shipping_street]
# Repeat for city, state, post, code, etc...
else
billing_address.name = params[:billing_name]
billing_address.street = params[:billing_street]
# Repeat for city, state, post, code, etc...
end
end
end
Here's the controller code:
def new
#order_form = OrderForm.new(#item)
end
def create
#order_form = OrderForm.new(#item, params[:order], current_user)
if #order_form.submit
# handle payment
else
render 'new'
end
end
Noe I am not interested in accepts_nested_attributes_for, which presents several problems, hence why I wrote the form object.
def populate
order.email = params[:email]
shipping_params = %i[shipping_name shipping_street]
billing_params = order.bill_to_shipping_address? ?
shipping_params : %i[billing_name billing_street]
[[shipping_address, shipping_params], [billing_address, billing_params]]
.each{|a, p|
a.name, a.street = params.at(*p)
}
end
How about
class Order < ActiveRecord::Base
has_one :shipping_address, class_name: 'Address'
has_one :billing_address, class_name: 'Address'
accepts_nested_attributes_for :shipping_address, :billing_address
before_save :clone_shipping_address_into_billing_address, if: [check if billing address is blank]
Then when you set up the form, you can have fields_for the two Address objects, and side step the populate method entirely.
A possible fix would be to use a variable for retrieving those matching params, like so:
def populate
order.email = params[:email]
shipping_address.name = params[:shipping_name]
shipping_address.street = params[:shipping_street]
# etc...
#set a default state
shipping_or_billing = "shipping_"
#or use a ternary here...
shipping_or_billing = "billing_" if order.bill_to_shipping_address?
billing_address.name = params["shipping_or_billing" + "name"]
billing_address.street = params["shipping_or_billing" + "street"]
...
end
Your address classes should probably have a method that would set the values for all the address properties from a hash that it would receive as an argument.
That way your populate method would only check for order.bill_to_shipping_address? and them pass the correct dictionary to the method I'm suggesting.
That method on the other hand, would just assign the values from the hash to the correct properties, without the need for a conditional check.

Bulk insert using one model

I'm trying to create a form using textarea and a submit button that will allow users to do bulk insert. For example, the input would look like this:
0001;MR A
0002;MR B
The result would look like this:
mysql> select * from members;
+------+------+------+
| id | no | name |
+------+------+------+
| 1 | 0001 | MR A |
+------+------+------+
| 2 | 0002 | MR B |
+------+------+------+
I'm very new to Rails and I'm not sure on how to proceed with this one. Should I use attr_accessor? How do I handle failed validations in the form view? Is there any example? Thanks in advance.
Update
Based on MissingHandle's comment, I created a Scaffold and replace the Model's code with this:
class MemberBulk < ActiveRecord::Base
attr_accessor :member
def self.columns
#columsn ||= []
end
def self.column(name, sql_type = nil, default = nil, null = true)
columns << ActiveRecord::ConnectionAdapters::Column.new(name.to_s, default, sql_type.to_s, null)
end
column :data, :text
validates :data, :create_members, :presence => true
def create_members
rows = self.data.split("\r\n")
#member = Array.new
rows.each_with_index { |row, i|
rows[i] = row.strip
cols = row.split(";")
p = Member.new
p.no = cols[0]
p.name = cols[1]
if p.valid?
member << p
else
p.errors.map { |k, v| errors.add(:data, "\"#{row}\" #{v}") }
end
}
end
def create_or_update
member.each { |p|
p.save
}
end
end
I know the code is far from complete, but I need to know is this the correct way to do it?
class MemberBulk < ActiveRecord::Base
#Tells Rails this is not actually tied to a database table
# or is it self.abstract_class = true
# or #abstract_class = true
# ?
abstract_class = true
# members holds array of members to be saved
# submitted_text is the data submitted in the form for a bulk update
attr_accessor :members, :submitted_text
attr_accessible :submitted_text
before_validation :build_members_from_text
def build_members_from_text
self.members = []
submitted_text.each_line("\r\n") do |member_as_text|
member_as_array = member_as_text.split(";")
self.members << Member.new(:number => member_as_array[0], :name => member_as_array[1])
end
end
def valid?
self.members.all?{ |m| m.valid? }
end
def save
self.members.all?{ |m| m.save }
end
end
class Member < ActiveRecord::Base
validates :number, :presence => true, :numericality => true
validates :name, :presence => true
end
So, in this code, members is an array that is a collection of the individual Member objects. And my thinking is that as much as possible, you want to hand off work to the Member class, as it is the class that will actually be tied to a database table, and on which you can expect standard rails model behavior. In order to accomplish this, I override two methods common to all ActiveRecord models: save and valid. A MemberBulk will only be valid if all it's members are valid and it will only count as saved if all of it's members are saved. You should probably also override the errors method to return the errors of it's underlying members, possibly with an indication of which one it is in the submitted text.
In the end I had to change from using Abstract Class to Active Model (not sure why, but it stoppped working the moment I upgrade to Rails v3.1). Here's the working code:
class MemberBulk
include ActiveModel::Validations
include ActiveModel::Conversion
extend ActiveModel::Naming
attr_accessor :input, :data
validates :input, presence: true
def initialize(attributes = {})no
attributes.each do |name, value|
send("#{name}=", value) if respond_to?("#{name}=")
end
end
def persisted?
false
end
def save
unless self.valid?
return false
end
data = Array.new
# Check for spaces
input.strip.split("\r\n").each do |i|
if i.strip.empty?
errors.add(:input, "There shouldn't be any empty lines")
end
no, nama = i.strip.split(";")
if no.nil? or nama.nil?
errors.add(:input, "#{i} doesn't have no or name")
else
no.strip!
nama.strip!
if no.empty? or nama.empty?
errors.add(:input, "#{i} doesn't have no or name")
end
end
p = Member.new(no: no, nama: nama)
if p.valid?
data << p
else
p.errors.full_messages.each do |error|
errors.add(:input, "\"#{i}\": #{error}")
end
end
end # input.strip
if errors.empty?
if data.any?
begin
data.each do |d|
d.save
end
rescue Exception => e
raise ActiveRecord::Rollback
end
else
errors.add(:input, "No data to be processed")
return false
end
else
return false
end
end # def
end

Resources