I have simple csv file that has the following header: 'NYC'.
I use CSV fetch method:
http://ruby-doc.org/stdlib-2.0.0/libdoc/csv/rdoc/CSV/Row.html#method-i-fetch
the problem is that when I use fetch with 'NYC' it works properly but when I use fetch with 'nyc' it returns:
KeyError: key not found: nyc
How can I solve this problem?
There's an option :header_converters. You can set it to:
:downcase Calls downcase() on the header String.
:symbol The header String is downcased, spaces are replaced with underscores, non-word characters are dropped, and finally to_sym() is called.
Example:
require 'CSV'
CSV.parse("NYC\nfoo", headers: true, header_converters: :symbol) do |row|
row[:nyc] #=> "foo"
end
Related
I have a problem since I try to import my CSV created with Numbers on Mac,
Everything worked before on Ubuntu with LibreOffice,
When I try to import my CSV file I have error
unknown attribute 'adress user_id room_type etc...' for Bien.
I think it not detect separators and take the first hearder line rows as one string.
My import function:
def self.import(file)
CSV.foreach(file.path, headers: true) do |row|
#bien = Bien.create! row.to_hash
#bien.save
end
end
I would know how import the file and if I have things to change when I create my CSV on Numbers.
UPDATE
I think you're exactly right, it looks like the separators are not being respected so the header row is showing as one long string. To debug, you can try putting a pry in and running CSV.read(file.path) to see the whole output of the conversion to CSV. Once you do that, you should be able to see what Numbers uses for separators.
This post suggests Numbers uses semicolons as default separators, so if you define your col_sep: ';' as an option, that might do the trick. (Ref: CSV docs).
So, the code would be
def self.import(file)
CSV.foreach(file.path, col_sep: ';', headers: true) do |row|
#bien = Bien.create! row.to_hash
#bien.save
end
end
I'm having problems importing this CSV:
municipality,province,province abbrev,country,region
Vancouver,British Columbia,BC,Canada,Metro Vancouver - North
Specifically, Vancouver is not being returned when I look for its value by its key:
municipality_name = row["municipality"]
Here's the code:
def self.import_csv(file)
CSV.foreach(file, headers: true,
skip_blanks: true,
skip_lines: /^(?:,\s*)+$/,
col_sep: ",") do |row|
municipality_name = row["municipality"]
puts row.to_h
puts "municipality_name: #{municipality_name}"
puts "row[0]: #{row[0]}"
end
end
Here's the output:
irb(main):052:0> Importers::Municipalities.import_csv('tmp/municipalities.csv')
{"municipality"=>"Vancouver", "province"=>"British Columbia", "province abbrev"=>"BC", "country"=>"Canada", "region"=>"Metro Vancouver - North"}
municipality_name:
row['municipality']:
row[0]: Vancouver
Seems like I'm missing something obvious. I thought maybe there was a hidden character in the CSV but turned on hidden characters in Sublime and no dice.
Thanks in advance.
You need to call to_h on the row if you want to access it by its keys. Otherwise, it is an array-like object, accessible by indices.
def self.import_csv(file)
CSV.foreach(file, headers: true,
skip_blanks: true,
skip_lines: /^(?:,\s*)+$/,
col_sep: ",") do |row|
row = row.to_h
municipality_name = row["municipality"]
puts "municipality_name: #{municipality_name}"
end
end
Seems like it was a problem with the CSV and the code works fine. Created a new CSV, typed in the same content, and it worked. Maybe an invisible character that Sublime wasn't showing? Can't verify as I wiped the original CSV that was causing issues.
I have the following two lines of a code that take an uploaded CSV file from params and return a hash of Contact objects. The code works fine when I input a CSV with UTF-8 encoding. If I try to upload a CSV with another type of encoding though, it breaks. How can I adjust the code to detect the encoding of the uploaded file and convert to UTF-8?
CSV::Converters[:blank_to_nil] = lambda { |field| field && field.empty? ? nil : field }
csv = CSV.new(params[:file].tempfile.open, headers: true, header_converters: :symbol, converters: [:all, :blank_to_nil]).to_a.map {|row| row.to_hash }
This question is not a duplicate! I've seen numerous other questions on here revolving around the same encoding issue, but the specifics of those are different than my case. Specifically, I need a way convert the encoding of a TempFile generated from my params hash. Other solutions I've seen involve encoding String and File objects, as well as passing an encoding option to CSV.parse or CSV.open. I've tried those solutions already without success.
I've tried passing in an encoding option to CSV.new, like so:
csv = CSV.new(params[:file].tempfile.open, encoding: 'iso-8859-1:utf-8', headers: true, header_converters: :symbol, converters: [:all, :blank_to_nil]).to_a.map {|row| row.to_hash }
I've also tried this:
csv = CSV.new(params[:file].tempfile.open, encoding: 'iso-8859-1:utf-8', headers: true, header_converters: :symbol, converters: [:all, :blank_to_nil]).to_a.map {|row| row.to_hash }
I've tried adjusting my converter as well, like so:
CSV::Converters[:blank_to_nil] = lambda { |field| field && field.empty? ? nil : field.encode('utf-8') }
I'm looking for a programatic solution here that does not require the user to convert their CSV to the proper encoding.
I've also had to deal with this problem and here is how I finally solved it.
CSV.open(new_csv_file, 'w') do |csv_object|
lines = File.open(uploaded_file).read
lines.each_line do |line|
csv_object << line.encode!("utf-8", "utf-8", invalid: :replace, undef: :replace, replace: '').parse_csv
end
end
CSV.new(File.read(new_csv_file))
Basically go through every line, sanitize it and shove it into a new CSV file.
Hope that leads you and other in the right direction.
You can use filemagic to detect the encoding of a file, although it's not 100% accurate. It bases on system's file command tool, so I'm not sure if it works on windows.
iam new to ruby.i want to remove non numeric characters from phone number parsed from a CSV file.
Here is the code iam using.
require 'csv'
csv_text = File.read('file.csv')
csv = CSV.parse(csv_text, :headers => true)
csv.each do |row|
puts "First Name: #{row['Name']} - HomePhone: #{row['Phone']} - Zip Code: #{row['Zipcode']}"
end
the out put print as Follows
FirstName:Abiel HomePhone:6667-88-76
(In CSV file HomePhone contains non numeric characters.)
I want the out put as FirstName:Abiel HomePhone:66678876
This should work:
row['Phone'].gsub(/[^0-9]/, "")
Yes, or just row['Phone'].gsub(/\D/, "")
where \d means a numeric char, and \D means anything non-numeric.
I have this to parse a CSV file:
csv_file = CSV.parse(
file.read(),
headers: true,
header_converters: :symbol
)
It works, but I want to specify the encoding type so I add: {encoding: 'UTF-8'} into the read method as below:
csv_file = CSV.parse(
file.read({encoding: 'UTF-8'}),
headers: true,
header_converters: :symbol
)
But I get this error: can't convert Hash into Integer
I just can't see what is wrong. I have checked the docs but it says you can pass the encoding in like so but it does need the file as the first argument so it could be stalling there but surely as it already knows what file is is reading it should be ok.
How can I solve this?
Update:
I have updated to the following:
def import
if params[:import_coasters]
file = params[:import_coasters][:file]
Park.import_from_csv(file)
def self.import_from_csv(file)
Park.destroy_all
and
csv_file = CSV.parse(
File.read(file, {encoding: 'UTF-8'}),
headers: true,
header_converters: :symbol
)
But I get the following error:
can't convert ActionDispatch::Http::UploadedFile into String
You're close. Try this:
# Step 1: convert the uploaded file object to a file name
uploaded_file = params[:import_coasters][:file]
file_name = uploaded_file.path
# Step 2: To get the input text and see if it's what you expect
text = File.read(
file_name,
{encoding: 'UTF-8'}
)
# Parse the text
csv_file = CSV.parse(
text,
headers: true,
header_converters: :symbol
)
From the Ruby 1.9.3 docs on IO.read:
"If the last argument is a hash, it specifies option for internal open(). The key would be the following. open_args: is exclusive to others."
http://www.ruby-doc.org/core-1.9.3/IO.html#method-c-read
Also, look at the documentation for UploadedFile because it's not actually a typical Ruby File object:
http://api.rubyonrails.org/classes/ActionDispatch/Http/UploadedFile.html
It is because File.read() expect a filename as the first argument. You are passing ActionDispatch::Http::UploadedFile instead. If you look at the documentation, you'll see that UploadedFile already has a read method, so you can try:
csv_file = CSV.parse(
file.read({encoding: 'UTF-8'}),
headers: true,
header_converters: :symbol
)