I am new to Ruby and I found this https://github.com/drodriguez/reversegeocoding and I think it sounds really cool. So I tried the sample app, installed everything, but when I'm calling
thor geocoder:database
I get this error:
/Users/xyz/.thor/f892c2a1d732c61bbf4ebff2abb70df6:194:in `initialize': wrong number of arguments(2 for 0) (ArgumentError)
Line 194 shows
csv = FasterCSV.new(io, CSV_OPTIONS.merge(COUNTRIES_CSV_OPTIONS))
and the whole method
def insert_countries(db, countries)
ids = Hash.new
country_insert = db.prepare("INSERT INTO countries (name) VALUES (:name)")
open(countries, 'rb') do |io|
io.rewind unless io.read(3) == "\xef\xbb\xbf" # Skip UTF-8 marker
io.readline while io.read(1) == '#' # Skip comments at the start of the file
io.seek(-1, IO::SEEK_CUR) # Unread the last character that wasn't '#'
csv = FasterCSV.new(io, CSV_OPTIONS.merge(COUNTRIES_CSV_OPTIONS))
csv.each do |row|
country_insert.execute :name => row['country']
ids[row['ISO']] = db.last_insert_row_id
end
end
country_insert.close
ids
end
I don't know how to fix this issue and I hope someone can help me with this.
Thanks.
I found the solution. The problem was my Ruby version. Since 1.9.x FasterCSV is no longer supported and now CSV is in the Ruby standard library... Look here for reference https://stackoverflow.com/a/6090840/749242
Related
I have a file b.xls from excel I need to import it to my rails app
I have tried to open it
file = File.read(Rails.root.to_s+'/b.xls')
I have got this
file.encoding => #Encoding:UTF-8
I have few questions:
how to open without this symbols(normal language)?
how to convert this file to a hash?
File pretty large about 5k lines
You must have array of all rows then you can convert it to some hash if you like so.
I would recommend to use a batch_factory gem.
The gem is very simple and relies on the roo gem under the hood.
Here is the code example
require 'batch_factory'
factory = BatchFactory.from_file(
Rails.root.join('b.xlsx'),
keys: [:column1, :column2, ..., :what_ever_column_name]
)
Then you can do
factory.each do |row|
puts row[:column1]
end
You can also omit specifying keys. Then batch_factory will automatically fetch headers from the first row. But your keys would be in russian. Like
factory.each do |row|
puts row['Товар']
end
If you want to hash with product name as key you can do
factory.inject({}) do |hash, row|
hash.merge(row['Товар'] => row)
end
Ruby n00b here. I am scraping the same page twice - but in a slightly different way each time - and exporting them to separate CSV files. I would like to then combine the first column from CSV no.1 and the second column from CSV no.2 to create CSV no.3.
The code to pull CSVs no.1 & 2 works. But add my attempt to combine the two CSVs into the third one (commented-out at the bottom) returns the following error - the two CSVs populate fine, but the third stays blank and the script is in what appears to be an infinite loop. I know these lines shouldn't be at the bottom, but I can't see where else it would go...
alts.rb:45:in `block in <main>': undefined local variable or method `scrapedURLs1' for main:Object (NameError)
from /Users/JammyStressford/.rvm/rubies/ruby-2.0.0-p451/lib/ruby/2.0.0/csv.rb:1266:in `open'
from alts.rb:44:in `<main>'
The code itself:
require 'rubygems'
require 'nokogiri'
require 'open-uri'
require 'csv'
url = "http://www.example.com/page"
page = Nokogiri::HTML(open(url))
CSV.open("results1.csv", "wb") do |csv|
page.css('img.product-card-image').each do |scrape|
product1 = scrape['alt']
page.css('a.product-card-image-link').each do |scrape|
link1 = scrape['href']
scrapedProducts1 = "#{product1}"[0..-7]
scrapedURLs1 = "{link1}"
csv << [scrapedProducts1, scrapedURLs1]
end
end
end
CSV.open("Results2.csv", "wb") do |csv|
page.css('a.product-card-image-link').each do |scrape|
link2 = scrape['href']
page.css('img.product-card-image').each do |scrape|
product2 = scrape['alt']
scrapedProducts2 = "#{product2}"[0..-7]
scrapedURLs2 = "http://www.lyst.com#{link2}"
csv << [scrapedURLs2, scrapedProducts2]
end
end
end
## Here is where I am trying to combine the two columns into a new CSV. ##
## It doesn't work. I suspect that this part should be further up... ##
# CSV.open("productResults3.csv", "wb") do |csv|
# csv << [scrapedURLs1, scrapedProducts2]
#end
puts "upload complete!"
Thanks for reading.
Thank you for sharing your code and your question. I hope my input helps!
Your scrapedURLs1 = "{link}" and scrapedProducts1 = "#{scrape['alt']}"[0..-7] have a 1 on the end but you don't call it at csv << [scrapedProducts, scrapedURLs] THIS IS THE ERROR YOU ARE GETTING
I would recommend combining your first two steps to skip
writing to a file, but into an Array of Arrays and THEN you can write
them to file.
Do you realize that in the example code you've given
scrapedURLs1, scrapedProducts2 would be mixing the wrong urls to
the wrong products. Is that what you mean to do?
Within the commented out code scrapedURLs1, scrapedProducts2 do not exist, they have not been declared. You need to open both files to read with .each do |scrapedURLs1| and then another with .each do |scrapedProducts2| and then those variable will exist because the each Enumerator instantiates them.
Reusing the same |scrape| variable on your inner iteration isn't a good idea. Change the name to something else such as |scrape2| . It "happens" to work because you've already taken what you need in product=scrape['alt'] before the second loop. If you rename the second loop variable you can move the product=scrape['alt'] line into the inner loop and combine them. Example:
# In your code example you may get many links per product.
# If that was your intent then that may be fine.
# This code should get one link per product.
CSV.open("results1.csv", "wb") do |csv|
page.css('img.product-card-image').each do |scrape|
page.css('a.product-card-image-link').each do |scrape2|
# [ product , link ]
csv << [scrape['alt'][0..-7], scrape2['href']]
# NOTE that scrape['alt'][0..-7] and scrape2['href'] are already strings
# so you don't need to use "#{ }"
end
end
end
Side note: Ruby 2.0.0 does not need the line require "rubygems"
If you're working with CSVs I highly recommend you using James Edward Gray II's faster_csv gem. See an example of usage here: https://github.com/JEG2/faster_csv/blob/master/examples/csv_writing.rb
I'm having an issue with FasterCSV and my rake db:seeds migration. I get the error:
"rake aborted! Unquoted fields do not allow \r or \n (line 2)"
on the following seeds.rb data:
require 'csv'
directory = "db/init_data/"
file_name = "gardenzing020812.csv"
path_to_file = directory + file_name
puts 'Loading Plant records'
# Pre-load all Plant records
n=0
CSV.foreach(path_to_file) do |row|
Plant.create! :name => row[1],
:plant_type => row[3],
:group => row[2],
:image_path => row[45],
:height => row[5],
:sow_inside_outside => row[8]
n=n+1
end
I've searched for a solution to this problem and have discovered that for a lot of folks it's a UTF-8 encoding problem. I've tried requiring iconv and :encoding => 'u', but that then gives me the error "invalid byte sequence in UTF-8".
I'm a newbie, and I can't figure out if it's really an encoding issue that I need to crack (which I've been trying to do unsuccessfully and if so, I could really use some guidance) or, more likely I feel, that I've made a simple misstep and done something wrong with the way I've set up seeds.rb and possibly my excel -> csv file. There's no bad or awkward data in the csv file. It's simple one-word strings, text and integers. Please help!
It was as simple as clearing all the formatting off in the csv. Excel seems to have a habit of retaining a lot of the formatting after saving in a csv file, which was causing the failure. After I copied and pasted all the data with no formatting in a new csv file, it was fine.
Use String.encode(universal_newline: true) instead gsub.
It converting CRLF and CR to LF # Always break lines with \n
I do not have enough reputation to comment, but I wanted to say that I have been looking this error across the web day and night for a long time and finally found the solution in the comments, by mu is too short.
I finally got it to work when I put quotes around all of my values.
EDIT: Link to answer!!! Rails FasterCSV "unquoted fields do not allow \r or \n"
I know I've done this before and found a simple set of code, but I cannot remember or find it :(.
I have a text file of records I want to import into my Rails 3 application.
Each line represents a record. Potentially it may be tab delimited for the attributes, but am fine with just a single value as well.
How do I do this?
File.open("my/file/path", "r").each_line do |line|
# name: "Angela" job: "Writer" ...
data = line.split(/\t/)
name, job = data.map{|d| d.split(": ")[1] }.flatten
end
Related topic
What are all the common ways to read a file in Ruby?
You want IO.foreach:
IO.foreach('foo.txt') do |line|
# process the line of text here
end
Alternatively, if it really is tab-delimited, you might want to use the CSV library:
File.open('foo.txt') do |f|
CSV.foreach(f, col_sep:"\t") do |csv_row|
# All parsed for you
end
end
IO.foreach("input.txt") do |line|
out.puts line
# You might be able to use split or something to get attributes
atts = line.split
end
Have you tried using OpenURI (http://ruby-doc.org/stdlib-2.1.2/libdoc/open-uri/rdoc/OpenURI.html)? You would have to make your files accessible from S3.
Or try using de aws-sdk gem (http://aws.amazon.com/sdk-for-ruby).
You can use OpenURI to read remote or local files.
Assuming that your model has an attachment named file:
# If object is stored in amazon S3, access it through url
file_path = record.file.respond_to?(:s3_object) ? record.file.url : record.file.path
open(file_path) do |file|
file.each_line do |line|
# In your case, you can split items using tabs
line.split("\t").each do |item|
# Process item
end
end
end
I have a small ruby application I wrote that's an anagram searcher. It's for learning ruby, but I would like to put it up online for personal use. I have some experience with Rails, and many here have recommended Sinatra. I'm fine with either, but I cannot find any information on how to use a text file instead of a database.
The application is quite simple, validates against a text file of a word list, then finds all anagrams. I have been assuming that this should be quite simple, but I'm stuck on importing that textfile into Rails (or Sinatra if i choose that way). In the Rails project, I have placed the textfile in the lib directory.
Unfortunately, even though the path appears to be correct in Rails, I get an error:
no such file to load -- /Users/court/Sites/cvtest/lib/english.txt
(cvtest is the name of the rails project)
Here is the code. It works great by itself:
file_path = '/Users/court/Sites/anagram/dictionary/english.txt'
input_string = gets.chomp
# validate input to list
if File.foreach(file_path) {|x| break x if x.chomp == input_string}
#break down the word
word = input_string.split(//).sort
# match word
anagrams = IO.readlines(file_path).partition{
|line| line.strip!
(line.size == word.size && line.split(//).sort == word)
}[0]
#list all words except the original
anagrams.each{ |matched_word| puts matched_word unless matched_word == input_string }
#display error if
else
puts "This word cannot be found in the dictionary"
end
Factor the actual functionality (finding the anagrams) into a method. Call that method from your Web app.
In Rails, you'd create a controller action that calls that method instead of ActiveRecord. In Sinatra, you'd just create a route that calls the method. Here's a Sinatra example:
get '/word/:input'
anagrams = find_anagrams(params[:input])
anagrams.join(", ")
end
Then, when you access the http://yourapp.com/word/pool, it will print "loop, polo".
I know the question is marked as answered, but I prefer the following, as it uses query parameters rather than path based parameters, which means you can pass the parameters in using a regular GET form submission:
require 'rubygems'
require 'sinatra'
def find_anagrams word
# your anagram method here
end
get '/anagram' do
#word = params['word']
#anagrams = find_anagrams #word if #word
haml :anagram
end
And the following haml (you could use whatever template language you prefer). This will give you an input form, and show the list of anagrams if a word has been provided and an anagram list has been generated:
%h1
Enter a word
%form{:action => "anagram"}
%input{:type => "text", :name => "word"}
%input{:type => "submit"}
- if #word
%h1
Anagrams of
&= #word
- if #anagrams
%ul
- #anagrams.each do |word|
%li&= word
- else
%p No anagrams found
With sinatra, you can do anything. These examples doesn't even require sinatra, you could roll your own rack interface thing.
require 'rubygems'
require 'sinatra'
require 'yaml'
documents = YAML::load_file("your_data.yml")
Or:
require 'rubygems'
require 'sinatra'
content = Dir[File.join(__DIR__, "content/*.textile)].map {|path|
content = RedCloth(File.read(path)).to_html
}
Etcetera.