I have a simple CSV import where the provided file is already broken (UTF characters) (German).
e.g.: The list has :
G%C3%B6tterbote
where as the right name should be
Götterbote
I'm trying to force the encoding when importing the CSV.
My Import Action
def import
Player.import(params[:file])
redirect_to players_path, notice: "Players Imported successfully"
end
My Import Method
def self.import(file)
SmarterCSV.process(file.path) do |row|
Player.create(row.first)
end
end
I found out that this converts the String successfully, but couldn't implement it successfully:
u = "G%C3%B6tterbote"
=> "G%C3%B6tterbote"
u1 = CGI::unescape(u).force_encoding('UTF-8')
=> "Götterbote"
So basically i need something like a before_action (i guess)..
You don't need a before action.
You need a pre-prossessor, well actually you need to pre-prossess yourself.
Your CSV comes with columns. Column 0, 1, 2, 3 etc (since you don't use headers).
So, for your text columns, let's call them for the sake of the example columns 1, 3, 5.
def self.import(file)
text_cols=[1,3,5] #for example
SmarterCSV.process(file.path) do |row|
text_cols.each do |column|
row[column]=CGI::unescape(row[column]).force_encoding('UTF-8')
end
Player.create(row)
end
end
Or simply, for your particular case:
def self.import(file)
SmarterCSV.process(file.path) do |row|
row.first=CGI::unescape(row.first).force_encoding('UTF-8')
Player.create(row.first)
end
end
Related
My github address https://github.com/ParkHyunDo/ImportCsv
I am studying how to import an excel file using roo. The import works fine, but everything is blank.
Like this....
Here is my code
product.rb
class Product < ActiveRecord::Base
acts_as_xlsx
def self.import(file)
spreadsheet = open_spreadsheet(file)
header = spreadsheet.row(1)
(2..spreadsheet.last_row).each do |i|
row = Hash[[header, spreadsheet.row(i)].transpose]
product = find_by_id(row["id"]) || new
product.attributes = row.to_hash.slice(accepts_nested_attributes_for)
product.save!
end
end
def self.open_spreadsheet(file)
case File.extname(file.original_filename)
# You're using a tab seperated file, so specify seperator as a tab with \t
when ".csv" then Roo::CSV.new(file.path, csv_options: {col_sep: "\t"})
when ".xls" then Roo::Excel.new(file.path)
when ".xlsx" then Roo::Excelx.new(file.path)
else raise "Unknown file type: #{file.original_filename}"
end
end
end
products_controller.rb
def import
Product.import(params[:file])
redirect_to root_url, notice: "Products imported."
end
Please help me!
This line seems strange:
product.attributes = row.to_hash.slice(accepts_nested_attributes_for)
The class method accepts_nested_attributes_for has a completely different purpose than listing the attribute names of Product. But you could use attribute_names. Try this:
product.attributes = row.to_hash.stringify_keys.slice(*attribute_names)
Note that stringify_keys might be unnecessary, depending on how exactly the hash returned by row.to_hash looks. Also note that slice takes a list of attribute names, not an array. The asterisk * allows us to use the elements of an array as individual arguments to a function.
I have the ability to import people details via CSV as entries.
If the spreadsheet contains no id on that row, it creates the entry, otherwise it updates the entry according to the id number.
Ultimately, what I want to do is to have the 'CreatedAt' field update when importing the entry if there is no id, otherwise, leave it alone (each entry only needs 'CreatedAt' to be set once).
If there already is a 'CreatedAt' set and we are just updating, it needs to do nothing.
This is the code I have so far:
controller file:
def import
Votsphonebook.import(params[:file])
redirect_to root_url, notice: "Entries updated!"
end
model file:
def self.import(file)
CSV.foreach(file.path, headers: true) do |row|
votsphonebook_hash = row.to_hash
votsphonebook = Votsphonebook.where(id: votsphonebook_hash["id"])
if votsphonebook.count == 1
votsphonebook.first.update_attributes(votsphonebook_hash)
else
Votsphonebook.create!(votsphonebook_hash)
end
end
end
I just need an entry point in my code where I can take the current row it is looking at and run the if statement.
Thank you
The updated_at and created_at columns are automatically populated by Active Record. There are no additional steps required to achieve what you want.
If you want to capture the current time on a custom column you could do this:
def self.import(file)
CSV.foreach(file.path, headers: true) do |row|
votsphonebook_hash = row.to_hash
votsphonebook = Votsphonebook.find(votsphonebook_hash["id"])
if votsphonebook
votsphonebook.update_attributes(votsphonebook_hash.merge(your_custom_column: Time.now))
else
Votsphonebook.create!(votsphonebook_hash)
end
end
end
Please note I refactored your finder a little bit for clarity.
I've followed Railscasts and the similar GoRails videos--searched SO for [rails] [csv] export to no avail. There's a similar but hard-to-follow question/answer called Two Export Buttons to CSV in Rails. I am able to add a single button to export my table records to csv (have chosen all records for this button).
I want to add a second (or nth) button to my view page to export a subset of table records...e.g. Event.where(country: some_country) in addition to the current (working fine) button/download that exports all records of a model. I thought this would be a common use of a csv export.
Here's what I have, working backwards:
Events Index view: I have a download button to export to CSV (added helper code for the glyph image)
<%= link_to glyph(:save), "events.csv", class: "btn btn-success" %>
If I want a second action, I need a different route than events.csv, right? So, I tried a new button changing "events.csv" to #events_us.csv.
Events Controller: have defined the 'what' and respond as all records (using ransack and will_paginate gems too):
def index
#search = Event.ransack(params[:q])
#events = #search.result.paginate(:page => params[:page], :per_page => 50)
#events_all = Event.all
respond_to do |format|
format.html # need to have html
format.csv { send_data #events_all.to_csv, filename: "Events-#{Date.today}.csv" }
end
end
Should I add a second format.csv in the respond_to do...? Tried that (i.e. defined a different instance variable like #events_us = Event.where(country: "US"), to no avail. Seems weird to have two format.csvs though. And, I get No route matches [GET] "/#events_us.csv"
I probably don't need to say much more, as I'm all kinds of lost on this one.
Event Model: use all scope, csv library, etc to organize the csv:
def self.to_csv
column_names = Event.column_names.map(&:to_s) - %w[id created_at updated_at]
CSV.generate(headers: true) do |csv|
csv << column_names
all.each do |record|
csv << record.attributes.values_at(*column_names)
end
end
end
If I have a second respond_to action in the controller, could I have a second method like self.to_csv_us?
def self.to_csv_us
column_names = Event.column_names.map(&:to_s) - %w[id created_at updated_at]
CSV.generate(headers: true) do |csv|
csv << column_names
#events_us.each do |record|
csv << record.attributes.values_at(*column_names)
end
end
end
My objective is to transform a hash array to a csv file.
This is my controller:
respond_to do |format|
format.html
format.csv { send_data #comsumptions.to_csv }
end
#comsumptions is a hash array:
[{"x"=>76,
"y"=>"example",
"z"=>2015,
"consumption"=>#<BigDecimal:7fea4a1cadb8,'0.5382857142E4',18(27)>},
{"x"=>76,
"y"=>"example2",
"z"=>2015,
"consumption"=>#<BigDecimal:7fea4a1ca7c8,'0.5437E4',9(27)>},(..)
I want to create a CSV file with 2 specific columns, "consumption" and "z".
When I did this with these 3 lines comment the output is a file with all the #consumptions. How can I select these 2 columns and transform in a cv file?
def self.to_csv
CSV.generate(headers: true ) do |csv|
#csv << column_names
#all.each do |product|
# csv << product.attributes.values_at(*column_names)
end
end
end
From your feedback, I think the best way here is creating a csv view file in your views. For example, if your html file is comsumptions.html.erb, then your csv view file should be comsumptions.csv.ruby
# comsumptions.csv.ruby
require 'csv'
CSV.generate do |csv|
csv << ['consumption', 'z']
#comsumptions.each do |c|
csv << [ c['consumption'].to_s, c['z'] ]
end
end
And we need to change the controller too. Remove respond_to part or modify it as follows
respond_to do |format|
format.html
format.csv
end
I already tested on my localhost, and this should work!
require 'csv'
#comsumptions =
[{"x"=>76,
"y"=>"example",
"z"=>2015},
{"x"=>76,
"y"=>"example2",
"z"=>2015}]
class << #comsumptions
def to_csv (*keys)
keys = first.keys if keys.empty?
CSV.generate(headers: keys, write_headers: true) do |csv|
each do |e|
csv << e.values_at(*keys)
end
end
end
end
p #comsumptions.to_csv("x","y")
p #comsumptions.to_csv()
This solution heavily inspired from the one from Van Huy works fine for any array oh hashes as long as the hashes have all the same keys, undefined behaviour to be expected otherwise
What's not clear to me is where do you put the def self.to_csv method, inside the controller? That is wrong. What you want to do is either augment the #consumptions object, augment the Array class, or define a method in the controller. My example does augment the #consumptions object, you can put all that stuff inside the controller method and it should work.
I followed railscasts #396 Importing CSV and implemented CSV upload in my rails project.
This is my view file:
<%= form_tag import_customers_path, multipart: true do %>
<%= file_field_tag :file %>
<%= submit_tag "Import" %>
<% end %>
This is my controller action:
def import
current_user.customers.import(params[:file])
redirect_to customers_path, notice: "Users imported."
end
And these are my model methods:
def self.to_csv(options = {})
CSV.generate(options) do |csv|
csv << column_names
all.each do |customer|
csv << customer.attributes.values_at(*column_names)
end
end
end
def self.import(file)
CSV.foreach(file.path, headers: true) do |row|
Customer.create! row.to_hash
end
end
Here I don't want user to include header in CSV. When I replace headers: true with headers: false, I get error:
NoMethodError in CustomersController#import
undefined method `to_hash' for ["abc#wer.com"]:Array
Can anybody tell how to upload CSV files without need of header line?
As far as upload and handling of the CSV file goes, you're very, very close. You just have an issue with reading the rows of data to populate the database with, via the Customer.create! call
It looks like you've been testing with a CSV file that only has a single line of data. With the headers: true, that single line was converted to headers and subsequently ignored in the CSV.foreach iterator. So, in effect, you had no data in the file, and no iterations occurred. If you had two rows of data in the input file, you'd have encountered the error, anyway.
Now, when you use headers: false, that line of data is treated as data. And that's where the issue lies: handling the data isn't done correctly.
Since there's no schema in your question, I'll assume a little bit of leeway on fields; you should be able to extrapolate pretty easily to make it work in your situation. This code shows how it works:
CSV.parse(csv_data, headers: false) do |row|
hash = {
first_name: row[0],
last_name: row[1],
age: row[2],
phone: row[3],
address: row[4]
}
Customer.create!(hash)
end
If you wanted a CSV version with headers, this would work well in this case, and has the benefit of not allowing arbitrary access to columns that shouldn't be assigned from an outside source:
CSV.parse(csv_data, headers: true, header_converters: :symbol) do |row|
hash = {
first_name: row[:first_name],
surname: row[:last_name],
birth_year: Date.today - row[:age],
phone: row[:phone],
street_address: row[:address]
}
Customer.create!(hash)
end
Note that the Customer#to_csv in your model is not quite correct, either. First, it creates the CSV file with a header, so you wouldn't be able to export and then import again with this implementation. Next, the header fields variable column_names is not actually defined in this code. Finally, the code doesn't control the order of columns written to the CSV, which means that the headers and values could possibly go out of sync. A correct (non-header) version of this is very simple:
csv_data = CSV.generate do |csv|
csv.each do |customer|
csv << [customer.first_name, customer.last_name, customer.age, customer.phone, customer.address]
end
end
The header-based version is this:
csv_data = CSV.generate do |csv|
csv << ["First Name","Last Name","Age","Phone","Address"]
csv.each do |customer|
csv << [customer.first_name, customer.last_name, customer.age, customer.phone, customer.address]
end
end
Personally, I'd use the header-based version, because it's far more robust, and it's easy to understand which columns are which. If you've ever received a headerless CSV file and had to figure out how to make sense of it without any keys, you'd know why the header is important.
You could just load the CSV file into an array of arrays and remove the first row:
data = CSV.read("path/to/file.csv")
data = data[1..-1]
However this will store the data as an array of values only.
When you use headers: true it uses a hash where the keys are the column header names.