I have already CSV file, the content like
a1 a2 a3
1 2 3
4 5 6
5 8 2
Now, What I want, when I read any row i want to add a flag in the csv file like
a1 a2 a3 flag
1 2 3 1
4 5 6 1
5 8 2
the above flag 1 that means this record is inserted in the table.
so How can I add flag in the csv file?
Thanks In Advance
I came up with two ways to append a column(s) to an existing CSV file.
Method 1 late merges the new column by reading the file into an array of hashes, then appending the columns to the end of each row. This method can exhibit anomalies if run multiple times.
require 'csv'
filename = 'test.csv'
# Load the original CSV file
rows = CSV.read(filename, headers: true).collect do |row|
row.to_hash
end
# Original CSV column headers
column_names = rows.first.keys
# Array of the new column headers
additional_column_names = ['flag']
# Append new column name(s)
column_names += additional_column_names
s = CSV.generate do |csv|
csv << column_names
rows.each do |row|
# Original CSV values
values = row.values
# Array of the new column(s) of data to be appended to row
additional_values_for_row = ['1']
values += additional_values_for_row
csv << values
end
end
# Overwrite csv file
File.open(filename, 'w') { |file| file.write(s) }
Method 2 early merges the new column(s) into the row hash. The nicety of this method is it is more compact and avoids duplicate column names if run more than once. This method can also be used to change any existing values in the CSV.
require 'csv'
filename = 'test.csv'
# Load the original CSV file
rows = CSV.read(filename, headers: true).collect do |row|
hash = row.to_hash
# Merge additional data as a hash.
hash.merge('flag' => '0')
# BONUS: Change any existing data here too!
hash.merge('a1' => hash['a1'].to_i + 1 )
end
# Extract column names from first row of data
column_names = rows.first.keys
txt = CSV.generate do |csv|
csv << column_names
rows.each do |row|
# Extract values for row of data
csv << row.values
end
end
# Overwrite csv file
File.open(filename, 'w') { |file| file.write(txt) }
You need to write new CSV file with additional column, and then replace original file with new one.
Not sure if you can append a new column in the same file, but you can append a new row into your csv:
CSV.open('your_csv.csv', 'w') do |csv|
customers.array.each do |row|
csv << row
end
end
Hope this helps.
Related
I want to loop over a csv file using CSV.foreach, read the data, perform some operation with it, and write the result to the last column of that row, using the Row object.
So let's say I have a csv with data I need to save to a database using Rails ActiveRecord, I validate the record, if it is valid, I write true in the last column, if not I write the errors.
Example csv:
id,title
1,some title
2,another title
3,yet another title
CSV.foreach(path, "r+", headers: true) do |row|
archive = Archive.new(
title: row["title"]
)
archive.save!
row["valid"] = true
rescue ActiveRecord::RecordInvalid => e
row["valid"] = archive.errors.full_messages.join(";")
end
When I run the code it reads the data, but it does not write anything to the csv. Is this possible?
Is it possible to write in the same csv file?
Using:
Ruby 3.0.4
The row variable in your iterator exists only in memory. You need to write the information back to the file like this:
new_csv = ["id,title,valid\n"]
CSV.foreach(path, 'r+', headers: true) do |row| # error here, see edit note below
row["valid"] = 'foo'
new_csv << row.to_s
end
File.open(path, 'w+') do |f|
f.write new_csv
end
[EDIT] the 'r+' option to foreach is not valid, it should be 'r'
Maybe this is over-engineering things a bit. But I would do the following:
Read the original CSV file.
Create a temporary CSV file.
Insert the updated headers into the temporary CSV file.
Insert the updated records into the temporary CSV file.
Replace the original CSV file with the temporary CSV file.
csv_path = 'archives.csv'
input_csv = CSV.read(csv_path, headers: true)
input_headers = input_csv.headers
# using an UUID to prevent file conflicts
tmp_csv_path = "#{csv_path}.#{SecureRandom.uuid}.tmp"
output_headers = input_headers + %w[errors]
CSV.open(tmp_csv_path, 'w', write_headers: true, headers: output_headers) do |output_csv|
input_csv.each do |archive_data|
values = archive_data.values_at(*input_headers)
archive = Archive.new(archive_data.to_h)
archive.valid?
# error_messages is an empty string if there are no errors
error_messages = archive.errors.full_messages.join(';')
output_csv << values + [error_messages]
end
end
FileUtils.move(tmp_csv_path, csv_path)
I have a CSV document with one column and 1000 rows. Each row has a string of data which is seperated by "|".
For example
BOB|MARLEY|306336|Friday| 9:00AM|02 DIS 2|HELE TP 1|PARRA|JULIA|20 Jul 2018|TOMPSON|TORI|21332|NA|AUS|4214|||0400 000 000|zzz11#bigpond.com|.0000|NULL|NULL|0|QLD|F|2016-06-22 00:00:00.000|
I need to loop through each row then split the string into another array. I then need to loop through each of those arrays.
Currently I have
csv_text = open('https://res.cloudinary.com/thypowerhouse/raw/upload/v1534642033/rackleyswimming/HVL_SCHOOL.csv')
csv = CSV.parse(csv_text, :headers=>true)
csv.each do |row|
new_row = row.map(&:inspect).join
new_row = new_row.delete! '[]'
new_row = new_row.gsub('|', '", "')
new_row = new_row.split(',')
puts new_row
end
Don't know if I'm heading in the right direction?
You can use col_sep to separate the data of each row:
require "csv"
CSV.foreach("HVL_SCHOOL.csv", headers: true, col_sep: "|") do |row|
# Your code here, trait your data
end
Every row on the scope of CSV#foreach (previus example) will be a CSV::Row that can be treated as an array because it has enumerable as included module.
I think with this you can do what you want with this data.
I have a config table as below:
S.NO TableName SourceColumns
1 A a,b,c,d
2 B p,q,r,s,t,u
3 C m,n,o,p,q
4 D x,y,z
Here, result object consists of SourceColumns of each record from the table.
For instance, here it gets the 1 record values. Then I am writing those values to CSV(excel) file. This works.
CSV.open('C:\Actual\Test.csv', 'w') do |csv|
result.each do |eachrow|
csv << ["#{eachrow['a']}","#{eachrow['b']}","#{eachrow['c']}","#{eachrow['d']}"]
end
end
As, I have hardcoded my values in the above query, I am constructing the string per my SourceColumns and passing to query as below:
myformattedstring="#{eachrow['a']}","#{eachrow['b']}","#{eachrow['c']}","#{eachrow['d']}"
CSV.open('C:\Actual\Test.csv', 'w') do |csv|
result.each do |eachrow|
csv << [myformattedstring]
end
end
Now, the data is being treated as string. How, can i pass my string to the CSV so that I can write the actual values
If you want to insert only 1 column in your csv containing myformattedstring, you need to put the double quotes only at the beginning and the end of your string, and interpolate all the #{} inside those quotes, as opposed to repeating the quotes
I am trying to print my values in CSV file like following where data is array of hashes.
UPDATES:
CSV.open(fn, "wb") do |csv|
#first rows are always headers and the headers value is generated from the array of hashes
data.each do |name, values|
csv << [name, values.join(",")]
end
and values has data like : true,false,false,false and name is an array with data like: light.
But for some reason my columns are only 2 instead of 5. The values column is concatenated in one column.
How can I achieve multiple columns using above code ?
I think this should work:
CSV.open(fn, "wb") do |csv|
data.each do |name, values|
csv << [name, *values]
end
end
http://ruby-doc.org/core-2.0.0/doc/syntax/calling_methods_rdoc.html#label-Array+to+Arguments+Conversion
I'm exporting a CSV from many different sources which makes it very hard to sort before putting it into the CSV.
csv = CSV.generate col_sep: '#' do |csv|
... adding a few columns here
end
Now, it would be awesome if I was able to sort this CSV by the 2nd column. Is that in any way possible?
If you're trying to sort before writing, it depends on your data structure, in which i'll need to see your code a bit more. For reading a csv, you can convert it to hash and sort by header name even:
rows = []
CSV.foreach('mycsvfile.csv', headers: true) do |row|
rows << row.to_h
end
rows.sort_by{ |row| row['last_name'] }
Edit to use sort_by, thanks to max williams.
Here is how you would sort by column number:
rows = []
CSV.foreach('mycsvfile.csv', headers: true) do |row|
# collect each row as an array of values only
rows << row.to_h.values
end
# sort in place by the 2nd column
rows.sort_by! { |row| row[1] }
rows.each do |row|
# do stuff with your now sorted rows
end