I have a file b.xls from excel I need to import it to my rails app
I have tried to open it
file = File.read(Rails.root.to_s+'/b.xls')
I have got this
file.encoding => #Encoding:UTF-8
I have few questions:
how to open without this symbols(normal language)?
how to convert this file to a hash?
File pretty large about 5k lines
You must have array of all rows then you can convert it to some hash if you like so.
I would recommend to use a batch_factory gem.
The gem is very simple and relies on the roo gem under the hood.
Here is the code example
require 'batch_factory'
factory = BatchFactory.from_file(
Rails.root.join('b.xlsx'),
keys: [:column1, :column2, ..., :what_ever_column_name]
)
Then you can do
factory.each do |row|
puts row[:column1]
end
You can also omit specifying keys. Then batch_factory will automatically fetch headers from the first row. But your keys would be in russian. Like
factory.each do |row|
puts row['Товар']
end
If you want to hash with product name as key you can do
factory.inject({}) do |hash, row|
hash.merge(row['Товар'] => row)
end
Related
I created a rake task to import users from a Google Sheet. Therefore I am using the gem 'Roo'. Everything works so far but I can't seem to get it working without importing the first row (headers).
This is my code:
require 'roo'
namespace :import do
desc "Import users from Google Sheet"
task users: :environment do
#counter = 0
url = 'https://docs.google.com/spreadsheets/d/{mycode}/export?format=xlsx'
xlsx = Roo::Spreadsheet.open(url, extension: :xlsx, headers: true)
xlsx.each do |row|
n = User.where(name:row[0]).first
user = User.find_or_create_by(id: n)
user.update(
name:row[0],
country_id:row[6]
)
user.save!
puts user.name
#counter += 1
end
puts "Imported #{#counter} lines."
end
end
Your code says headers: true when you are opening the sheet. Have you tried turning it to false? Or are you saying it does not work when it's set to false?
Also, you are using .each rather differently than the example in the documentation. The doc shows a hash with keys derived from the headers. You are using [n] array notation. Does that work?
EDIT:
Try using .each in a way that's more similar to what the documentation says:
xlsx.each(name: 'Name', country_id: 'Country ID') do |row|
n = User.where(name: row[:name]).first
...
end
The strings 'Name' and 'Country ID' are just examples; they should be the text of whatever column headers have the name and country_id information.
There is a way to skip the headers, it is using the method: each_row_streaming(offset: 1).
It will return an array with rows skipping the header, so you have to get the value using .value method. In documentation specify it for Excelx::Cell objects, but it works for Roo::Spreadsheet objects too.
The documentation example:
xlsx.each_row_streaming(offset: 1) do |row| # Will exclude first (inevitably header) row
puts row.inspect # Array of Excelx::Cell objects
end
I've a rake task where I import CSV data into a database via Rails.
I want a specific column (specifically, row[6] below) to be rendered as an integer. However, everything I try returns that value as a string.
Below is the rake task:
require 'csv'
namespace :import_site_csv do
task :create_sites => :environment do
CSV.foreach('../sites.csv', :headers => true) do |row|
row[6] = row[6].to_i
Site.create!(row.to_hash)
end
end
end
Does anyone have an idea how I might do this? Thanks!
You are making one small (but important) mistake here.
When you call CSV.foreach('../sites.csv') each of the rows will be an array of the values in that particular row. That would allow you to access the data you need, in the way you do it now - row[6].
But, when you add the :headers => true option to CSV.foreach, you will not get an array of values (row will not be an array). Instead, it will be a CSV::Row object (docs). As you can read in the documentation:
A CSV::Row is part Array and part Hash. It retains an order for the fields and allows duplicates just as an Array would, but also allows you to access fields by name just as you could if they were in a Hash.
For example, if you have a column with the name Title in the CSV, to get the title in each of the rows, you need to do something like:
CSV.foreach('file.csv', :headers => true) do |row|
puts row['Title']
end
Since I do not know the structure of your CSV, I cannot tell you which key you should use to get the data and convert it to an Integer, but I think that this should give you a good idea of how to proceed.
just started learning Rails and have managed to import a csv file into a database, but the price field in the csv has quotes and a comma like this: "560,000"
But if I make the price field as t.integer in the migration file, then add the data, the price gets imported as 560. So, how do I remove the quotes and the comma before importing it? thanks, Adam
edit: here's the rake file:
require 'csv'
task :csv_to_properties => [:environment] do
CSV.foreach("lib/assets/cbmb_sale.csv", :headers => true) do |row|
Property.create!(row.to_hash)
end
end
Try something like:
csvvalue = csvvalue.gsub!(/,/,'').to_i
Cheers!
Thanks for posting your code. I don't do a ton with converting csv's to hashes but something like this will probably work:
Property.create!(row.to_hash.each_pair{|k,v| row.store(k,v.gsub(/,/,'').to_i)})
Pretty ugly but probably pretty close to what you want.
In your code example, assuming the price field is in row element 4:
CSV.foreach("lib/assets/cbmb_sale.csv", :headers => true) do |row|
row[price=4].gsub!(/,/,'')
Property.create!(row.to_hash)
end
The price=4 is just a handy way to document the index value of the price element, it creates a variable called price assigns the value 4 to it, then immediately uses it as the array index.
Since Property.create! is already taking care of the string to integer conversion, we can perform an in-place substitution for the regular expression that contains a comma /,/ for an empty string ''.
Try:
"220,000".scan(/\d+/).join().to_i
I know I've done this before and found a simple set of code, but I cannot remember or find it :(.
I have a text file of records I want to import into my Rails 3 application.
Each line represents a record. Potentially it may be tab delimited for the attributes, but am fine with just a single value as well.
How do I do this?
File.open("my/file/path", "r").each_line do |line|
# name: "Angela" job: "Writer" ...
data = line.split(/\t/)
name, job = data.map{|d| d.split(": ")[1] }.flatten
end
Related topic
What are all the common ways to read a file in Ruby?
You want IO.foreach:
IO.foreach('foo.txt') do |line|
# process the line of text here
end
Alternatively, if it really is tab-delimited, you might want to use the CSV library:
File.open('foo.txt') do |f|
CSV.foreach(f, col_sep:"\t") do |csv_row|
# All parsed for you
end
end
IO.foreach("input.txt") do |line|
out.puts line
# You might be able to use split or something to get attributes
atts = line.split
end
Have you tried using OpenURI (http://ruby-doc.org/stdlib-2.1.2/libdoc/open-uri/rdoc/OpenURI.html)? You would have to make your files accessible from S3.
Or try using de aws-sdk gem (http://aws.amazon.com/sdk-for-ruby).
You can use OpenURI to read remote or local files.
Assuming that your model has an attachment named file:
# If object is stored in amazon S3, access it through url
file_path = record.file.respond_to?(:s3_object) ? record.file.url : record.file.path
open(file_path) do |file|
file.each_line do |line|
# In your case, you can split items using tabs
line.split("\t").each do |item|
# Process item
end
end
end
I'm using the rails3-jquery-autocomplete gem on a field with non-unique values, but I want the results it retrieves to be duplicate-free. Any ideas on how to accomplish this?
I had the same problem in my project https://github.com/marciomr/Terra-Livre and I solved it doing the following:
I installed rails3-jquery-autocomplete as a plugin in vendor/plugin directory
I changed the file helpers.rb like this:
def json_for_autocomplete(items, method, extra_data)
json = items.collect do |item| # here I put the result in a variable
hash = {"label" => item.send(method), "value" => item.send(method)} #here I removed the id
extra_data.each do |datum|
hash[datum] = item.send(datum)
end if extra_data
hash
end
json.uniq # this line is new
end
I removed the id from the json file and then retrieved uniq values.
Since I didn't need the id it worked fine for me. I think if I need the id I can put it in extra_data, but I am not sure.
I have just forked the project with this alteration: git://github.com/marciomr/rails3-jquery-autocomplete.git
Since I ran into this myself, I thought I would record my own solution for posterity, since it does not require editing the gem's source. This is for the officially maintained fork of the gem: https://github.com/bigtunacan/rails-jquery-autocomplete.
You can handle the json encoding directly via the autocomplete block in the controller, which we can leverage to change the array of records.
Here is an example in which we get a unique list of schools that students go to:
autocomplete :student, :school do |items|
ActiveSupport::JSON.encode( items.uniq{ |i| i["value"] } )
end
"items" is an array of hashes, which by default contain an id, a label, and a value, so this passes only unique values into the json encoder (of your choice).