I'm exporting a CSV from many different sources which makes it very hard to sort before putting it into the CSV.
csv = CSV.generate col_sep: '#' do |csv|
... adding a few columns here
end
Now, it would be awesome if I was able to sort this CSV by the 2nd column. Is that in any way possible?
If you're trying to sort before writing, it depends on your data structure, in which i'll need to see your code a bit more. For reading a csv, you can convert it to hash and sort by header name even:
rows = []
CSV.foreach('mycsvfile.csv', headers: true) do |row|
rows << row.to_h
end
rows.sort_by{ |row| row['last_name'] }
Edit to use sort_by, thanks to max williams.
Here is how you would sort by column number:
rows = []
CSV.foreach('mycsvfile.csv', headers: true) do |row|
# collect each row as an array of values only
rows << row.to_h.values
end
# sort in place by the 2nd column
rows.sort_by! { |row| row[1] }
rows.each do |row|
# do stuff with your now sorted rows
end
Related
I have a CSV document with one column and 1000 rows. Each row has a string of data which is seperated by "|".
For example
BOB|MARLEY|306336|Friday| 9:00AM|02 DIS 2|HELE TP 1|PARRA|JULIA|20 Jul 2018|TOMPSON|TORI|21332|NA|AUS|4214|||0400 000 000|zzz11#bigpond.com|.0000|NULL|NULL|0|QLD|F|2016-06-22 00:00:00.000|
I need to loop through each row then split the string into another array. I then need to loop through each of those arrays.
Currently I have
csv_text = open('https://res.cloudinary.com/thypowerhouse/raw/upload/v1534642033/rackleyswimming/HVL_SCHOOL.csv')
csv = CSV.parse(csv_text, :headers=>true)
csv.each do |row|
new_row = row.map(&:inspect).join
new_row = new_row.delete! '[]'
new_row = new_row.gsub('|', '", "')
new_row = new_row.split(',')
puts new_row
end
Don't know if I'm heading in the right direction?
You can use col_sep to separate the data of each row:
require "csv"
CSV.foreach("HVL_SCHOOL.csv", headers: true, col_sep: "|") do |row|
# Your code here, trait your data
end
Every row on the scope of CSV#foreach (previus example) will be a CSV::Row that can be treated as an array because it has enumerable as included module.
I think with this you can do what you want with this data.
I am trying to print my values in CSV file like following where data is array of hashes.
UPDATES:
CSV.open(fn, "wb") do |csv|
#first rows are always headers and the headers value is generated from the array of hashes
data.each do |name, values|
csv << [name, values.join(",")]
end
and values has data like : true,false,false,false and name is an array with data like: light.
But for some reason my columns are only 2 instead of 5. The values column is concatenated in one column.
How can I achieve multiple columns using above code ?
I think this should work:
CSV.open(fn, "wb") do |csv|
data.each do |name, values|
csv << [name, *values]
end
end
http://ruby-doc.org/core-2.0.0/doc/syntax/calling_methods_rdoc.html#label-Array+to+Arguments+Conversion
I want to export some ActiveRecords in CSV format. After check some tutorials, I found this:
def export_as_csv(equipments)
attributes = %w[id title description category_id]
CSV.generate(headers: true) do |csv|
csv << attributes
equipments.each do |equipment|
csv << equipment.attributes.values_at(*attributes)
end
return csv
end
end
The problem is, I want to manipulate all in memory in my tests(i.e. I don't want to save the file in the disk). So, when I receive this csv object as return value, how I can iterate through rows and columns? I came from Python and so I tried:
csv = exporter.export_as_csv(equipments)
for row in csv:
foo(row)
But obviously didn't work. Also, the equipments are surely not nil.
CSV.generate returns string formatted according csv rules.
So the most obvious way is to parse it and iterate, like:
csv = exporter.expor_as_csv(equipments)
CSV.parse(csv).each do |line|
# line => ['a', 'b', 'c']
end
After some videos, I found that the return was the problem. Returning the CSV I was receiving a CSV object, and not the CSV itself.
I'm parsing a CSV and trying to distinguish between columns in Model and "virtual" columns that'll be added to a JSONB :data column. So far I've got this:
rows = SmarterCSV.process(csv.path)
rows.each do |row|
row.select! { |x| Model.attribute_method?(x) } # this ignores non-matches
Model.create(row)
end
That removes columns from the CSV row that don't match up with Model. Instead, I want to add the data from all those into a column in Model called :data. How can I do that?
Edit
Something like this before the select! maybe?
row[:data] = row.select { |x| !Model.attribute_method?(x) }
There are a number of ways you could do this. One particularly straightforward way is with Hash#slice! from Rails' ActiveSupport extensions, which works like Array#slice! and returns a Hash with those keys that weren't given in its arguments, while preserving the keys that were given:
rows = SmarterCSV.process(csv.path)
attrs = Model.attribute_names.map(&:to_sym)
rows.each do |row|
row[:data] = row.slice!(*attrs)
Model.create(row)
end
P.S. This could probably be filed under "Stupid Ruby Tricks," but if you're using Ruby 2.0+ you can take advantage of the double-splat (**) for this compact construction:
rows.each do |row|
Model.create(data: row.slice!(*attrs), **row)
end
P.P.S. If your CSVs are big and you find yourself having performance concerns (calling create a few thousand times—and the subsequent database INSERTs—ain't cheap), I recommend checking out the activerecord-import gem. It's designed for exactly this sort of thing. With it you'd do something like this:
rows = SmarterCSV.process(csv.path)
attrs = Model.attribute_names.map(&:to_sym)
models = rows.map do |row|
row[:data] = row.slice!(*attrs)
Model.new(row)
end
Model.import(models)
There are other, faster options as well in the activerecord-import docs.
You can try this has_attribute?
row[:data] = row.keep_if { |x| !Model.has_attribute?(x) }
Have you tried:
row[:data] = row.delete_if {|k,v| !Model.attribute_method?(k) }
Model.create(row)
This will remove the elements from the row hash and add the key-value pairs back to the row under a :data key.
I have already CSV file, the content like
a1 a2 a3
1 2 3
4 5 6
5 8 2
Now, What I want, when I read any row i want to add a flag in the csv file like
a1 a2 a3 flag
1 2 3 1
4 5 6 1
5 8 2
the above flag 1 that means this record is inserted in the table.
so How can I add flag in the csv file?
Thanks In Advance
I came up with two ways to append a column(s) to an existing CSV file.
Method 1 late merges the new column by reading the file into an array of hashes, then appending the columns to the end of each row. This method can exhibit anomalies if run multiple times.
require 'csv'
filename = 'test.csv'
# Load the original CSV file
rows = CSV.read(filename, headers: true).collect do |row|
row.to_hash
end
# Original CSV column headers
column_names = rows.first.keys
# Array of the new column headers
additional_column_names = ['flag']
# Append new column name(s)
column_names += additional_column_names
s = CSV.generate do |csv|
csv << column_names
rows.each do |row|
# Original CSV values
values = row.values
# Array of the new column(s) of data to be appended to row
additional_values_for_row = ['1']
values += additional_values_for_row
csv << values
end
end
# Overwrite csv file
File.open(filename, 'w') { |file| file.write(s) }
Method 2 early merges the new column(s) into the row hash. The nicety of this method is it is more compact and avoids duplicate column names if run more than once. This method can also be used to change any existing values in the CSV.
require 'csv'
filename = 'test.csv'
# Load the original CSV file
rows = CSV.read(filename, headers: true).collect do |row|
hash = row.to_hash
# Merge additional data as a hash.
hash.merge('flag' => '0')
# BONUS: Change any existing data here too!
hash.merge('a1' => hash['a1'].to_i + 1 )
end
# Extract column names from first row of data
column_names = rows.first.keys
txt = CSV.generate do |csv|
csv << column_names
rows.each do |row|
# Extract values for row of data
csv << row.values
end
end
# Overwrite csv file
File.open(filename, 'w') { |file| file.write(txt) }
You need to write new CSV file with additional column, and then replace original file with new one.
Not sure if you can append a new column in the same file, but you can append a new row into your csv:
CSV.open('your_csv.csv', 'w') do |csv|
customers.array.each do |row|
csv << row
end
end
Hope this helps.