I've a rake task where I import CSV data into a database via Rails.
I want a specific column (specifically, row[6] below) to be rendered as an integer. However, everything I try returns that value as a string.
Below is the rake task:
require 'csv'
namespace :import_site_csv do
task :create_sites => :environment do
CSV.foreach('../sites.csv', :headers => true) do |row|
row[6] = row[6].to_i
Site.create!(row.to_hash)
end
end
end
Does anyone have an idea how I might do this? Thanks!
You are making one small (but important) mistake here.
When you call CSV.foreach('../sites.csv') each of the rows will be an array of the values in that particular row. That would allow you to access the data you need, in the way you do it now - row[6].
But, when you add the :headers => true option to CSV.foreach, you will not get an array of values (row will not be an array). Instead, it will be a CSV::Row object (docs). As you can read in the documentation:
A CSV::Row is part Array and part Hash. It retains an order for the fields and allows duplicates just as an Array would, but also allows you to access fields by name just as you could if they were in a Hash.
For example, if you have a column with the name Title in the CSV, to get the title in each of the rows, you need to do something like:
CSV.foreach('file.csv', :headers => true) do |row|
puts row['Title']
end
Since I do not know the structure of your CSV, I cannot tell you which key you should use to get the data and convert it to an Integer, but I think that this should give you a good idea of how to proceed.
Related
I have a file b.xls from excel I need to import it to my rails app
I have tried to open it
file = File.read(Rails.root.to_s+'/b.xls')
I have got this
file.encoding => #Encoding:UTF-8
I have few questions:
how to open without this symbols(normal language)?
how to convert this file to a hash?
File pretty large about 5k lines
You must have array of all rows then you can convert it to some hash if you like so.
I would recommend to use a batch_factory gem.
The gem is very simple and relies on the roo gem under the hood.
Here is the code example
require 'batch_factory'
factory = BatchFactory.from_file(
Rails.root.join('b.xlsx'),
keys: [:column1, :column2, ..., :what_ever_column_name]
)
Then you can do
factory.each do |row|
puts row[:column1]
end
You can also omit specifying keys. Then batch_factory will automatically fetch headers from the first row. But your keys would be in russian. Like
factory.each do |row|
puts row['Товар']
end
If you want to hash with product name as key you can do
factory.inject({}) do |hash, row|
hash.merge(row['Товар'] => row)
end
I have some code that is chugging through a set of Rails Active Record models, and setting an attribute based on a related value from a 2D Array.
I am essentially setting a US State abbreviation code in a table of US States which was previously only storing the full names. A library of state names is being used to derive the abbreviations, and it contains a 2D Array with each sub-array having a full name, and an abbreviation (i.e., [['New York', 'NY']['Pennsylvania', 'PA'][etc]]). I compare the state name from each record in the database to each full text name in this Array, then grab the corresponding sibling Array cell when there is a match.
This code works fine, and produces the correct results, but its frumpy looking and not easily understood without reading many lines:
# For the following code, StatesWithNames is an Active Record model, which is
# having a new column :code added to its table.
# Sates::USA represents a 2D Array as: [['StateName', 'NY']], and is used to
# populate the codes for StatesWithNames.
# A comparison is made between StatesWithNames.name and the text name found in
# States::USA, and if there is a match, the abbreviation from States::USA is
# used
if StatesWithNames.any?
StatesWithNames.all.each do |named_state|
if named_state.code.blank?
States::USA.each do |s|
if s[0] == named_state.name
named_state.update_column(:code, s[1])
break
end
end
end
end
end
What is the most Ruby style way of expressing assignments with logic like this? I experimented with a few different procs / blocks, but arrived at even cludgier expressions, or incorrect results. Is there a more simple way to express this in fewer lines and/or if-end conditionals?
Yea, there is a few ifs and checks, that are not needed.
Since it is Rails even though it does not state so in question's tags, you might want to use find_each, which is one of the most efficient way to iterate over a AR collection:
StatesWithNames.find_each do |named_state|
next unless named_state.code.blank?
States::USA.each do |s|
named_state.update_column(:code, s[1]) if s[0] == named_state.name
end
end
Also be aware, that update_column bypasses any validations, and if you wish to keep your objects valid, stick to update!.
And last thing - wrap it all in transaction, so if anything goes wrong all the way - it would rollback any changes.
StatesWithNames.transaction do
StatesWithNames.find_each do |named_state|
next unless named_state.code.blank?
States::USA.each do |s|
named_state.update!(:code, s[1]) if s[0] == named_state.name
end
end
end
You might use a different data structure for this.
With your existing 2D array, you can call to_h on it to get a Hash where
a = [['California', 'CA'], ['Oregon', 'OR']].to_h
=> { 'California' => 'CA', 'Oregon' => 'OR' }
Then in your code you can do
state_hash = States::USA.to_h
if StatesWithNames.any?
StatesWithNames.all.each do |named_state|
if named_state.code.blank?
abbreviation = state_hash[named_state.name]
if !abbreviation.nil?
named_state.update_column(:code, abbreviation)
end
end
end
end
the first thing you want to do is convert the lookup from an array of arrays to a hash.
state_hash = States::USA.to_h
if StatesWithNames.any?
StatesWithNames.all.select{|state| state.code.blank?}.each do |named_state|
named_state.update_column(:code, state_hash[named_state.name]) if state_hash[named_state.name]
end
end
I feel like this is programming 101 stuff, but I am going to swallow my pride and ask for help. I have a CSV that I am processing. Here is a sample...
person_id, name, start_date
1111, busta, 1/1/14
1111, busta, 1/4/14
1111, busta, 1/7/14
2222, mista, 1/3/14
2222, mista, 1/1/14
2222, mista, 1/11/14
...and here is a sample of the code I am using to process the rows...
def self.import(file)
student_start_dates = Hash.new {|hsh, key| hsh[key] = [] }
CSV.foreach(file.tempfile, :headers => true) do |row|
student_start_dates[row["person_id"]] << row["start_date"]
#need something in the loop that says hey...when I find a new person_id send this array to the process method
end
end
def self.process(student)
#process something like 1111 => ["1/1/14", "1/4/14", "1/7/14"]
end
So as you can see from the data each student has multiple start dates associated with them. I am trying to build an array of start_dates for each student. When I find a new person_id, then need to 'do some stuff' with my start_date array. My question is what is the best way to add logic that looks for a change in the person_id as I loop through each row in my csv? I know I could set some sort of flag that gets set when the person_id changes, then based on the state of that flag process my start_date array, and reset the flag. However, I'm tried implementing that without much luck. Or when it did it felt 'dirty'. Just hoping a fresh set of eyes will give me some ideas on cleaner code.
A big part of my issue is the best way to set a flag that says "..when you find a new student (new person_id) then call the process method to find the earliest start date.
If I understand this correctly, you're trying to get a resulting hash that would look something like {1111 => ["1/1/14", "1/4/14", "1/7/14"], 2222 => [...], ...}
If so you could use the built in CSV parser and just construct the hash as you loop over each row.
# Create the hash, the default value will be an array
student_start_dates = Hash.new {|hsh, key| hsh[key] = [] }
CSV.foreach(file_name, :headers => true) do |row|
student_start_dates[row["person_id"]] << row["start_date"]
end
just started learning Rails and have managed to import a csv file into a database, but the price field in the csv has quotes and a comma like this: "560,000"
But if I make the price field as t.integer in the migration file, then add the data, the price gets imported as 560. So, how do I remove the quotes and the comma before importing it? thanks, Adam
edit: here's the rake file:
require 'csv'
task :csv_to_properties => [:environment] do
CSV.foreach("lib/assets/cbmb_sale.csv", :headers => true) do |row|
Property.create!(row.to_hash)
end
end
Try something like:
csvvalue = csvvalue.gsub!(/,/,'').to_i
Cheers!
Thanks for posting your code. I don't do a ton with converting csv's to hashes but something like this will probably work:
Property.create!(row.to_hash.each_pair{|k,v| row.store(k,v.gsub(/,/,'').to_i)})
Pretty ugly but probably pretty close to what you want.
In your code example, assuming the price field is in row element 4:
CSV.foreach("lib/assets/cbmb_sale.csv", :headers => true) do |row|
row[price=4].gsub!(/,/,'')
Property.create!(row.to_hash)
end
The price=4 is just a handy way to document the index value of the price element, it creates a variable called price assigns the value 4 to it, then immediately uses it as the array index.
Since Property.create! is already taking care of the string to integer conversion, we can perform an in-place substitution for the regular expression that contains a comma /,/ for an empty string ''.
Try:
"220,000".scan(/\d+/).join().to_i
So I am trying to parse a list of emails separated by a comma and inputted in a form using the built-in CSV library in rails 3.
Here is the code:
#variable = params[:body]
#csv = CSV.parse(#variable, :col_sep => ",")
#csv.each do |row|
user = User.where(:email => row)
However, the output is something like this:
- - test#hi.com
- ! ' hi#hi.com'
Therefore, each row in the #csv that I am trying to go over is an entire list of emails. How can I separate them? Thanks!
CSV::parse returns an array of arrays if you don't pass a block as an argument. So when you enumerate #csv using #each, the argument row will be an array of strings. Not a single string as your code suggests.
If the email address is the first column of each row in your CSV file, then you would replace
User.where(:email => row)
with
User.where(:email => row[0])
It's not exactly clear what the format of your CSV file is though. If you have multiple email addresses per row, or have the email address in a different column, you'll have to revise the code accordingly.