I have a simple CSV uploader below that is going row by row and creating a new record (event). I also have the unidecoder gem in use and would like to call the to_ascii method on a field (the description field) that is in every record being created by the CSV uploader. It sounds like it should be simple, but I'm not familiar with iterating through a CSV file.
The uploader:
def self.import(file)
CSV.foreach(file.path, headers: true, encoding: "windows-1252:utf-8") do |row|
Event.create! row.to_hash
end
end
Correct way of implementing this:
def self.import(file)
CSV.foreach(file.path, headers: true, encoding: "windows-1252:utf-8") do |row|
description = row[2]
row[2] = description.to_ascii
Event.create! row.to_hash
end
end
Thanks!
Try this:
CSV.foreach(file.path, headers: true, encoding: "windows-1252:utf-8") do |row|
unless row[1].blank?
description = row[1] # Change 1 for the correct column
row[1] = description.to_ascii
end
Event.create! row
end
If the description is not blank (empty), extract and update the value (description) and then save it.
row is an Array of you comma separated values, for example a CSV file like name, description, address, the row[1] have the value of the description.
Related
I want to write header only 1 time in first row when import data to csv in ruby, but the header is written many time on output file.
job_datas.each do |job_data|
#company_job = job data coverted etc....
save_job_to_csv(#company_job)
end
def save_job_to_csv(job_data)
filepath = "tmp/jobs/jobs.csv"
CSV.open(filepath, "a", :headers => true) do |csv|
if csv.blank?
csv << CompanyJob.attribute_names
end
csv << job_data.attributes.values
end
end
Any one can give me solution? Thank you so much!
You are calling save_job_to_csv the method for each job_data and pushing header every time csv << CompanyJob.attribute_names
filepath = "tmp/jobs/jobs.csv"
CSV.open(filepath, "a", :headers => true) do |csv|
# push header once
csv << CompanyJob.attribute_names
# push every job record
job_datas.each do |job_data|
#company_job = job data coverted etc....
csv << #company_job.attributes.values
end
end
The above script can be created wrapped a method but if you like to write a separate method that just saves the CSV, then you need to refactor the script when you first prepare an array of values holding header and pass it to a method that just saves to CSV.
You could do something similar to this:
def save_job_to_csv(job_data)
filepath = "tmp/jobs/jobs.csv"
unless File.file?(filepath)
File.open(filepath, 'w') do |file|
file.puts(job_data.attribute_names.join(','))
end
end
CSV.open(filepath, "a", :headers => true) do |csv|
csv << job_data.attributes.values
end
end
It just checks beforehand if the file exists and if not it adds the header. If you want tabs as column separators, you just have to change the value for the join function and add the col_sep parameter to CSV.open():
file.puts(job_data.attribute_names.join("\t"))
CSV.open(filepath, "a", :headers => true, col_sep: "\t") do |csv|
I'm new to rails and am trying to process a CSV file, some files will have comments at the start of the CSV file, comments are marked with #. If there a way I can delete these rows? I don't have to just ignore them as I want to save the file without comments.
sample file:
#-----------------------
# report --------------
#-----------------------
Date, transctions
20100923, 34
20200110, 56
Thanks.
The CSV library has a skip_lines options:
When setting an object responding to match, every line matching it is considered a comment and ignored during parsing. When set to a String, it is first converted to a Regexp. When set to nil no line is considered a comment. If the passed object does not respond to match, ArgumentError is thrown.
This should work for you:
CSV.foreach(file, skip_lines: /^#/, headers: true) do |row|
# ...
end
/^#/ matches lines starting with #.
Adding something to #Stefan answer (all credit goes to him for the skip_lines tip), assuming your csv file is input.csv :
require "csv"
CSV.open("output.csv", "wb") do |output_csv|
CSV.foreach("input.csv", skip_lines: /^#/, headers: true) do |row|
# ...
output_csv << row
end
end
This way you will end with a file output.csv without those comments.
EDIT:
If you want also the header, you can do:
CSV.open("output.csv", "wb") do |output_csv|
CSV.foreach("input.csv", skip_lines: /^#/, headers: true).with_index(0) do |row, i|
output_csv << row.headers if i == 0
puts row
output_csv << row
end
end
...It's not as clean as I want but fits your needs ;)
I am trying to import some CSV data but for some reason they are not structured correctly and contain invalid characters. So for example, when I do sale_hash["location"] it returns nil.
Here is the function I am using:
def self.import(file)
CSV.foreach(file.path, headers: true, encoding:'iso-8859-1') do |row|
sale_hash = row.to_hash
p sale_hash
end
end
I get something like this:
{
"date;day;time;location;station;washer;plate_number;O;I;W;total_price;;;;"
=> "23/04/16;SATURDAY;15:11;ALPHAMEGA LIN;GROUNDFLOOR;NIKOLAS APOSTOLIDES;KKY333;YES;NO;NO;Û2",
nil => "95;;;;"
}
All I did was take an excel spreadsheet and export it to CSV. Am I doing something wrong here?
Right so thanks to Jordan and mu is too short, I got it working by adding the following col_sep:
def self.import(file)
csv = CSV.read(file.path, headers: true, col_sep: ";")
csv.each do |row|
sale_hash = row.to_hash
p sale_hash
end
end
In rails 4.2.4, I am trying to extract the data from .csv file and save it to the database. But right now extracted row from the file is in wrong format so that value is not getting save.
require 'csv'
filename = "#{asset.document.path}"
if File.exist?(filename)
file = File.open(filename)
if file
CSV::parse(file)[1..-1].each do |row|
User.create_data(row, admin)
end
end
end
def create_data(row, admin)
usr = User.new
usr.name = row[0] if row[0]
usr.email = row[1] if row[1]
usr.password = row[2] if row[2]
user.save
end
Generated row's data is like ["Sannidhi\tsannidhi#gmail.com\tsannidhi123#\t"]. From this row I am not getting each values separately Eg: row[0], row[1] & row[2] to assign for related database fields.
How can I solve this CSV import issue? Please help me.
Try this:
CSV::parse(file)[1..-1].each do |row|
row.shift.split("\t") unless row.blank?
User.create_data(row, admin)
end
After this, you should be able to access:
row[0] #=> "Sannidhi"
row[1]
row[2]
You CSV file uses tabs as column separators. You can pass your own column separator to CSV as a col_sep option. Even though other 2 answers will do the job, let csv do its job on its own:
CSV::parse(file, col_sep: "\t")[1..-1].each do |row|
User.create_data(row, admin)
end
Also, consider using headers option to use the first line of the file as a header instead of [1..-1]:
CSV::parse(file, col_sep: "\t", headers: 'first_row').each do |row|
User.create_data(row, admin)
end
CSV stands for Comma Separated Value. It seems that your file is separated by spaces instead. Replace the tabs in your file by commas.
I followed railscasts #396 Importing CSV and implemented CSV upload in my rails project.
This is my view file:
<%= form_tag import_customers_path, multipart: true do %>
<%= file_field_tag :file %>
<%= submit_tag "Import" %>
<% end %>
This is my controller action:
def import
current_user.customers.import(params[:file])
redirect_to customers_path, notice: "Users imported."
end
And these are my model methods:
def self.to_csv(options = {})
CSV.generate(options) do |csv|
csv << column_names
all.each do |customer|
csv << customer.attributes.values_at(*column_names)
end
end
end
def self.import(file)
CSV.foreach(file.path, headers: true) do |row|
Customer.create! row.to_hash
end
end
Here I don't want user to include header in CSV. When I replace headers: true with headers: false, I get error:
NoMethodError in CustomersController#import
undefined method `to_hash' for ["abc#wer.com"]:Array
Can anybody tell how to upload CSV files without need of header line?
As far as upload and handling of the CSV file goes, you're very, very close. You just have an issue with reading the rows of data to populate the database with, via the Customer.create! call
It looks like you've been testing with a CSV file that only has a single line of data. With the headers: true, that single line was converted to headers and subsequently ignored in the CSV.foreach iterator. So, in effect, you had no data in the file, and no iterations occurred. If you had two rows of data in the input file, you'd have encountered the error, anyway.
Now, when you use headers: false, that line of data is treated as data. And that's where the issue lies: handling the data isn't done correctly.
Since there's no schema in your question, I'll assume a little bit of leeway on fields; you should be able to extrapolate pretty easily to make it work in your situation. This code shows how it works:
CSV.parse(csv_data, headers: false) do |row|
hash = {
first_name: row[0],
last_name: row[1],
age: row[2],
phone: row[3],
address: row[4]
}
Customer.create!(hash)
end
If you wanted a CSV version with headers, this would work well in this case, and has the benefit of not allowing arbitrary access to columns that shouldn't be assigned from an outside source:
CSV.parse(csv_data, headers: true, header_converters: :symbol) do |row|
hash = {
first_name: row[:first_name],
surname: row[:last_name],
birth_year: Date.today - row[:age],
phone: row[:phone],
street_address: row[:address]
}
Customer.create!(hash)
end
Note that the Customer#to_csv in your model is not quite correct, either. First, it creates the CSV file with a header, so you wouldn't be able to export and then import again with this implementation. Next, the header fields variable column_names is not actually defined in this code. Finally, the code doesn't control the order of columns written to the CSV, which means that the headers and values could possibly go out of sync. A correct (non-header) version of this is very simple:
csv_data = CSV.generate do |csv|
csv.each do |customer|
csv << [customer.first_name, customer.last_name, customer.age, customer.phone, customer.address]
end
end
The header-based version is this:
csv_data = CSV.generate do |csv|
csv << ["First Name","Last Name","Age","Phone","Address"]
csv.each do |customer|
csv << [customer.first_name, customer.last_name, customer.age, customer.phone, customer.address]
end
end
Personally, I'd use the header-based version, because it's far more robust, and it's easy to understand which columns are which. If you've ever received a headerless CSV file and had to figure out how to make sense of it without any keys, you'd know why the header is important.
You could just load the CSV file into an array of arrays and remove the first row:
data = CSV.read("path/to/file.csv")
data = data[1..-1]
However this will store the data as an array of values only.
When you use headers: true it uses a hash where the keys are the column header names.