How to use Rails postgresql connection raw.put_copy_data from string instead of a file - ruby-on-rails

Question

I usually copy data into my postgres database in rails using the following import module.
In this case I am uploading a file that is ready for postgres copy command to take in.
module Import #< ActiveRecord::Base
class Customer
include ActiveModel::Model
include EncodingSupport
attr_accessor :file
validates :file, :presence => true
def process file=nil
file ||= #file.tempfile
ActiveRecord::Base.connection.execute('truncate customers')
conn = ActiveRecord::Base.connection_pool.checkout
raw = conn.raw_connection
raw.exec("COPY customers FROM STDIN WITH (FORMAT CSV, DELIMITER ',', NULL ' ', HEADER true)")
# open up your CSV file looping through line by line and getting the line into a format suitable for pg's COPY...
data = file.open
data::gets
ticker = 0
counter = 0
success_counter = 0
failed_records = []
data.each_with_index do |line, index|
raw.put_copy_data line
counter += 1
end
# once all done...
raw.put_copy_end
while res = raw.get_result do; end # very important to do this after a copy
ActiveRecord::Base.connection_pool.checkin(conn)
return { :csv => false, :item_count => counter, :processed_successfully => counter, :errored_records => failed_records }
end
I have another file now that needs to be formatted correctly though so I have another module that converts it from a text file to a csv file and trims out unnecessary content. Once it is ready I'd like to pass the data to the module above and have postgres take it into the database.
def pg_import file=nil
file ||= #file.tempfile
ticker = 0
counter = 0
col_order = [:warehouse_id, :customer_type_id, :pricelist_id]
data = col_order.to_csv
file.each do |line|
line.strip!
if item_line?(line)
row = built_line
data += col_order.map { |col| row[col] }.to_csv
else
line.empty?
end
ticker +=1
counter +=1
if ticker == 1000
p counter
ticker = 0
end
end
pg_import data
end
My problem is the 'process' method returns data as
"warehouse_id,customer_type_id,pricelist_id\n201,A01,0AA\n201,A02,0AC
which means when I pass it to pg_import I can't iterate over the data. Because it expects it to be in the following format.
[0] "201,A01,0AA\r\n",
[1] "201,A02,0AC\r\n",
[2] "201,A03,oAE\r\n"
What command can I use to convert the string data so that I can iterate over it in the
data.each_with_index do |line, index|
raw.put_copy_data line
counter += 1
end
??
Probably has a really simple solution but just expecting to not be able to use the put_copy_data without having a file to iterate over...

Answer 1

Solved this with this process before and within the loop.
data = CSV.parse(data)
data.shift
data.each_with_index do |line, index|
line = line.to_csv
raw.put_copy_data line
counter += 1
end
Converted the string into an array of arrays with csv. Shifted to get rid of the header row. That allows me to iterate over the arrays. Grab an array and convert that array from an array back into a csv string and then pass it into the database.