I am trying to do the following in IRB:
file = CSV.read('branches.csv', headers:true)
file.each do |branch|
Branch.create(attributes:branch.to_hash)
end
branches.csv contains one header entitled business_name which should map onto the attribute for Branch of the same name, but I see the error:
ActiveRecord::UnknownAttributeError: unknown attribute 'business_name' for Branch.
Strangely, doing Branch.create(business_name:'test') works just fine with no issues.
Update:
I think this has something to do with the encoding of the text in the UTF-8 CSV produced by Excel as suggested in the comments below. Not sure if this IRB gives any clues... but our header title business_name != "business_name"
2.3.3 :348 > file = CSV.read('x.csv', headers:true)
#<CSV::Table mode:col_or_row row_count:165>
2.3.3 :349 > puts file.first.to_hash.first.first
business_name
2.3.3 :350 > file = CSV.read('x.csv', headers:true)
#<CSV::Table mode:col_or_row row_count:165>
2.3.3 :351 > puts file.first.to_hash.first.first == "business_name"
false
Just skip the attributes: part. It is not needed at all, because branch.to_hash already returns exactly the format you describe in your last sentence.
file = CSV.read('branches.csv', headers:true)
file.each { |branch| Branch.create(branch.to_hash) }
Related
I have just wrote a code where I get a csv file passed in argument and treat it line by line ; so far, everything is okay. Now, I would like to secure my code by making sure that what we receive in argument is a .csv file.
I saw in the Ruby doc that it exist a == "--file" option but using it generate an error : the way I understood it, it seems this option only work for the txt files.
Is there a method specific that allowed to check if my file is a csv ? Here some of my code :
if ARGV.empty?
puts "j'ai rien reçu"
# option to check, don't work
elsif ARGV[0].shift == "--file"
# my code so far, whithout checking
else CSV.foreach(ARGV.shift) do |row|
etc, etc...
I think it is unpossible to make a real safe test without additional information.
Just some notes what you can do:
You get a filename in a variable filename.
First, check if it is a file:
File.exist?
Then you could check, if the encoding is correct:
raise "Wrong encoding" unless content.valid_encoding?
Has your csv always the same number of columns? And do you have only one liner?
This can be a possibility to make the next check:
content.each_line{|line|
return false if line.count(sep) < columns - 1
}
This check can be modified for your case, e.g. if you have always an exact number of rows.
In total you can define something like:
require 'csv'
#columns defines the expected numer of columns per line
def csv?(filename, sep: ';', columns: 3)
return false unless File.exist?(filename) #"No file"
content = File.read(filename, :encoding => 'utf-8')
return false unless content.valid_encoding? #"Wrong encoding"
content.each_line{|line|
return false if line.count(sep) < columns - 1
}
CSV.parse(content, :col_sep => sep)
end
if csv = csv?('test.csv')
csv.each do |row|
p row
end
end
You can use ruby-filemagic gem
gem install ruby-filemagic
Usage:
$ irb
irb(main):001:0> require 'filemagic'
=> true
irb(main):002:0> fm = FileMagic.new
=> #<FileMagic:0x7fd4afb0>
irb(main):003:0> fm.file('foo.zip')
=> "Zip archive data, at least v2.0 to extract"
irb(main):004:0>
https://github.com/ricardochimal/ruby-filemagic
Use File.extname() to check the origin file
File.extname("test.rb") #=> ".rb"
Ruby 2.0.0, Rails 4.0.3, Windows 8.1 Update, Roo 1.13.2
I am trying to open an Excel spreadsheet with embedded spaces using Roo. So far, I am unable to do that. I don't really know if this problem is restricted to Roo. If I rename it to eliminate the spaces, I have no problem with it. I tried encoding it, but then it simply said the file doesn't exist. Can I open the file while it contains spaces?
Code sample:
exceptions = [URI::InvalidURIError, IOError]
puts "f is #{f}"
puts "f exist? #{File.exist?(f)}"
begin
xls = Roo::Spreadsheet.open(f)
rescue *exceptions => e
puts e.message
end
encoded_f = URI.encode(f).to_s
puts "encoded_f is #{encoded_f}"
puts "encoded_f exist? #{File.exist?(encoded_f)}"
begin
xls = Roo::Spreadsheet.open(encoded_f)
rescue *exceptions => e
puts e.message
end
gsub_f = f.gsub(" ", "") # Rename file without spaces
File.rename(f, gsub_f)
puts "gsub_f is #{gsub_f}"
puts "gsub_f exist? #{File.exist?(gsub_f)}"
begin
xls = Roo::Spreadsheet.open(gsub_f)
rescue *exceptions => e
puts e.message
end
Output sample:
f is Whitt Report 2014-07-28-0803.xls
f exist? true
bad URI(is not URI?): Whitt Report 2014-07-28-0803.xls
encoded_f is Whitt%20Report%202014-07-28-0803.xls
encoded_f exist? false
file Whitt%20Report%202014-07-28-0803.xls does not exist
gsub_f is WhittReport2014-07-28-0803.xls
gsub_f exist? true
No message is given in the end because the file opens successfully.
This is caused by the way in which the URI module is called in the Roo::Spreadsheet#open method.
I posted a fix to this problem which has now been merged. If you update your Roo gem you should no longer have this issue.
I am using ruby 2.0.0 and rails 4.0.0. I have something similar to this:
require 'net/sftp'
sftp = Net::SFTP.start('ftp.app.com','username', :password => 'password')
sftp.file.open("/path/to/remote/file.csv", "r") do |f|
puts f.gets
end
This opens the file on the FTP site, but it only puts the first line of the csv file. I need to read this file row by row, preferably ignoring the header.
How can I read the file row by row, without downloading the file locally?
I solved this by doing this:
data = sftp.download!("/path/to/remote/file.csv").split(/\r\n/)
data.each do |line|
puts line
end
The proper answer for this would actually be to use the file.eof? value.
The code would look like:
require 'net/sftp'
sftp = Net::SFTP.start('ftp.app.com','username', :password => 'password')
sftp.file.open("/path/to/remote/file.csv", "r") do |f|
while !f.eof?
puts f.gets
end
end
Documentation can be found here
In my case something like this worked:
data = sftp.download!("/path/to/remote/file.csv").split(/\n/).map{ |e| e.split(/,/).map{ |x| x.gsub(/"/, "")} }
data.each do |line|
puts line
end
Will also split each row of the .csv into different array columns and remove any excess of "". Note this is for mac where line breaks are \n.
My rb file reads:
require "csv"
puts "Program1 initialized."
contents = CSV.open "data.csv", headers: true
contents.each do |row|
name = row[4]
puts name
end
...but when i run it in ruby it wont load the program. it gives me the error message about the headers:
syntax error, unexpected ':', expecting $end
contents = CSV.open "data.csv", headers: true
so I'm trying to figure out, why won't ruby let me parse this file? I've tried using other csv files I have and it won't load, and gives me an error message. I'm trying just to get the beginning of the program going! I feel like it has to do with the headers. I've updated as much as I can, mind you I'm using ruby 1.8.7. I read somewhere else that I could try to run the program in irb but it didn't seem like it needed it. so yeah... thank you in advance!!!!
Since you are using this with Ruby 1.8.7, :headers => true won't work in this way.
The simplest way to ignore the headers and get your data is to shift the first row in the data, which would be the headers:
require 'csv'
contents = CSV.open("data.csv", 'r')
contents.shift
contents.each do |row|
name = row[4]
puts name
end
If you do want to use the syntax with headers in ruby 1.8, you would need to use FasterCSV, something similar to this:
require 'fastercsv'
FasterCSV.foreach("data.csv", :headers => true) do |fcsv_obj|
puts fcsv_obj['name']
end
(Refer this question for further read: Parse CSV file with header fields as attributes for each row)
I am a newcomer in Ruby.
I have a sample (input text) like:
Message:
update attributes in file and commit version
----
Modified
I need to put in line the row after "message" tag. Note that this row can be and close with "message" like
Message:update attributes in file and commit version
I've tried like this:
if line =~/Message/
But of course it doesn't search the next row.
Can anyone help me how to catch rows between tags "Message" and "---"
If you know some examples please type a link
Update: the whole code
require 'csv'
data = []
File.foreach("new7.txt") do |line|
line.chomp!
if line =~ /Revision/
data.push [line]
elsif line =~ /Author/
if data.last and not data.last[1]
data.last[1] = line
else
data.push [nil, line]
end
elsif line=~/^Message:(.*)^-/m
if data.last and not data.last[2]
data.last[2] = line
else
data.push [nil, nil, line]
end
end
end
CSV.open('new1.csv', 'w') do |csv|
data.each do |record|
csv << record
end
enter code here
Input file:
Revision: 37407
Author: imakarov
Date: 21 июня 2013 г. 10:23:28
Message:my infomation
dmitry name
Output csv file:
You can use /^Message:(.*)^---/m as your regex. The /m allows you to match across line boundaries. See http://rubular.com/r/FhqiKx0XyI
Update #1: Here's sample output from irb:
Peters-MacBook-Air-2:bot palfvin$ irb
1.9.3p194 :001 > line = "\nMessage:first-line\nsecond-line\n---\nthird-line"
=> "\nMessage:first-line\nsecond-line\n---\nthird-line"
1.9.3p194 :002 > line =~ /^Message:(.*)^-/m
=> 1
1.9.3p194 :003 > $1
=> "first-line\nsecond-line\n"
1.9.3p194 :004 >