Ruby 2.2, Ruby on Rails 4.2
I'm genarating some CSV data in Ruby on Rails, and want empty fields to be empty, like ,, not like ,"", .
I wrote codes like below:
somethings_cotroller.rb
def get_data
respond_to do |format|
format.html
format.csv do
#data = SheetRepository.accounts_data
send_data render_to_string, type: :csv
end
end
end
somethings/get_data.csv.ruby
require 'csv'
csv_str = CSV.generate do |csv|
csv << [1,260,37335,'','','','','','']
...
end
And this generates CSV file like this.
get_data.csv
1,260,37335,"","","","","",""
I want CSV data like below.
1,260,37335,,,,,,
It seems like Ruby adds "" automatically.
How can I do this??
In order to get CSV to output an empty column, you need to tell it that nothing is in the column. An empty string, in ruby, is still something, you'll need to replace those empty strings with nil in order to get the output you want:
csv_str = CSV.generate do |csv|
csv << [1,260,37335,'','','','','',''].map do |col|
col.respond_to?(:empty?) && col.empty? ? nil : col
end
end
# => 1,260,37335,,,,,,
In rails you can clean that up by making use of presence, though this will blank out false as well:
csv_str = CSV.generate do |csv|
csv << [1,260,37335,'',false, nil,'','',''].map(&:presence)
end
# => 1,260,37335,,,,,,
The CSV documentation shows an option that you can use for this case. There are not examples but you can guess what it does.
The only consideration is, you need to send an array of Strings, otherwise, you will get a NoMethodError
csv_str = CSV.generate(write_empty_value: nil) do |csv|
csv << [1,260,37335,'','','','','','', false, ' ', nil].map(&:to_s)
end
=> "1,260,37335,,,,,,,false, ,\n"
The benefit of this solution is, you preserve the false.
I resolved by myself!
in somethings_controller.rb
send_data render_to_string.gsub("\"\"",""), type: :csv
Related
I want to write header only 1 time in first row when import data to csv in ruby, but the header is written many time on output file.
job_datas.each do |job_data|
#company_job = job data coverted etc....
save_job_to_csv(#company_job)
end
def save_job_to_csv(job_data)
filepath = "tmp/jobs/jobs.csv"
CSV.open(filepath, "a", :headers => true) do |csv|
if csv.blank?
csv << CompanyJob.attribute_names
end
csv << job_data.attributes.values
end
end
Any one can give me solution? Thank you so much!
You are calling save_job_to_csv the method for each job_data and pushing header every time csv << CompanyJob.attribute_names
filepath = "tmp/jobs/jobs.csv"
CSV.open(filepath, "a", :headers => true) do |csv|
# push header once
csv << CompanyJob.attribute_names
# push every job record
job_datas.each do |job_data|
#company_job = job data coverted etc....
csv << #company_job.attributes.values
end
end
The above script can be created wrapped a method but if you like to write a separate method that just saves the CSV, then you need to refactor the script when you first prepare an array of values holding header and pass it to a method that just saves to CSV.
You could do something similar to this:
def save_job_to_csv(job_data)
filepath = "tmp/jobs/jobs.csv"
unless File.file?(filepath)
File.open(filepath, 'w') do |file|
file.puts(job_data.attribute_names.join(','))
end
end
CSV.open(filepath, "a", :headers => true) do |csv|
csv << job_data.attributes.values
end
end
It just checks beforehand if the file exists and if not it adds the header. If you want tabs as column separators, you just have to change the value for the join function and add the col_sep parameter to CSV.open():
file.puts(job_data.attribute_names.join("\t"))
CSV.open(filepath, "a", :headers => true, col_sep: "\t") do |csv|
I have created a CSV downloader in a controller like this
format.csv do
#records = Model.all
headers['Content-Disposition'] = "attachment; filename=\"products.csv\""
headers['Content-Type'] ||= 'text/csv'
end
Now I want to create server sent events to download CSV from this for optimising purpose. I know I can do this in Rails using ActionController::Live but I have have no experience with it.
Can some one explain to me how I can
Query records as batches
Add records to stream
Handle sse from browser side
Write records to CSV files
Correct me if any of my assumptions are wrong. Help me do this in a better way. Thanks.
Mongoid automatically query your records in batches (More info over here)
To add your records to a CSV file, you should do something like:
records = MyModel.all
# By default batch_size is 100, but you can modify it using .batch_size(x)
result = CSV.generate do |csv|
csv << ["attribute1", "attribute2", ...]
records.each do |r|
csv << [r.attribute1, r.attribute2, ...]
end
end
send_data result, filename: 'MyCsv.csv'
Remember that send_data is an ActionController method!
I think you don´t need SSE for generating a CSV. Just include ActionController::Live into the controller to use the response.stream.write iterating your collection:
include ActionController::Live
...
def some_action
format.csv do
# Needed for streaming to workaround Rack 2.2 bug
response.headers['Last-Modified'] = Time.now.httpdate
headers['Content-Disposition'] = "attachment; filename=\"products.csv\""
headers['Content-Type'] ||= 'text/csv'
[1,2,3,4].each do |i| # --> change it to iterate your DB records
response.stream.write ['SOME', 'thing', "interesting #{i}", "#{Time.zone.now}"].to_csv
sleep 1 # some fake delay to see chunking
end
ensure
response.stream.close
end
end
Try it with curl or similar to see the output line by line:
$ curl -i http://localhost:3000/test.csv
My objective is to transform a hash array to a csv file.
This is my controller:
respond_to do |format|
format.html
format.csv { send_data #comsumptions.to_csv }
end
#comsumptions is a hash array:
[{"x"=>76,
"y"=>"example",
"z"=>2015,
"consumption"=>#<BigDecimal:7fea4a1cadb8,'0.5382857142E4',18(27)>},
{"x"=>76,
"y"=>"example2",
"z"=>2015,
"consumption"=>#<BigDecimal:7fea4a1ca7c8,'0.5437E4',9(27)>},(..)
I want to create a CSV file with 2 specific columns, "consumption" and "z".
When I did this with these 3 lines comment the output is a file with all the #consumptions. How can I select these 2 columns and transform in a cv file?
def self.to_csv
CSV.generate(headers: true ) do |csv|
#csv << column_names
#all.each do |product|
# csv << product.attributes.values_at(*column_names)
end
end
end
From your feedback, I think the best way here is creating a csv view file in your views. For example, if your html file is comsumptions.html.erb, then your csv view file should be comsumptions.csv.ruby
# comsumptions.csv.ruby
require 'csv'
CSV.generate do |csv|
csv << ['consumption', 'z']
#comsumptions.each do |c|
csv << [ c['consumption'].to_s, c['z'] ]
end
end
And we need to change the controller too. Remove respond_to part or modify it as follows
respond_to do |format|
format.html
format.csv
end
I already tested on my localhost, and this should work!
require 'csv'
#comsumptions =
[{"x"=>76,
"y"=>"example",
"z"=>2015},
{"x"=>76,
"y"=>"example2",
"z"=>2015}]
class << #comsumptions
def to_csv (*keys)
keys = first.keys if keys.empty?
CSV.generate(headers: keys, write_headers: true) do |csv|
each do |e|
csv << e.values_at(*keys)
end
end
end
end
p #comsumptions.to_csv("x","y")
p #comsumptions.to_csv()
This solution heavily inspired from the one from Van Huy works fine for any array oh hashes as long as the hashes have all the same keys, undefined behaviour to be expected otherwise
What's not clear to me is where do you put the def self.to_csv method, inside the controller? That is wrong. What you want to do is either augment the #consumptions object, augment the Array class, or define a method in the controller. My example does augment the #consumptions object, you can put all that stuff inside the controller method and it should work.
There is CSV-export of some objects (such as tasks, contacts, etc) in my application. It just renders CSV-file like this:
respond_to do |format|
format.html
format.csv { render text: Task.to_csv } # I have self.to_csv def in model
end
It generates a CSV file when I go to '/tasks.csv' without a problem.
Now I want to export all the objects and zip them. I'm using rubyzip gem to create zip-files. Now my code for creating zip-file with all the CSVs looks like that:
Zip::ZipFile.open("#{path_to_file}.zip", Zip::ZipFile::CREATE) do |zipfile|
zipfile.file.open("tasks.csv", "w") { |f| f << open("http://#{request.host}:#{request.port.to_s}/tasks.csv").read }
# the same lines for contacts and other objects
end
But it seems that there is something wrong with it because it's executing for a long time (I'm getting Timeout::Error even if there is just one line in CSV) and the resulting zip-archive contains something broken.
How can I save my "/tasks.csv", "/contacts.csv", etc as a file on server (inside of zip-archive in this case)?
I did it! The code is:
Zip::ZipFile.open("#{path_to_file}.zip", Zip::ZipFile::CREATE) do |zipfile|
zipfile.file.open("tasks.csv", "w") do |f|
CSV.open(f, "w") do |csv|
CSV.parse(Task.to_csv) { |row| csv << row }
end
end
end
I'm attempting to DRY up a method I've been using for a few months:
def export(imagery_requests)
csv_string = FasterCSV.generate do |csv|
imagery_requests.each do |ir|
csv << [ir.id, ir.service_name, ir.description, ir.first_name, ir.last_name, ir.email,
ir.phone_contact, ir.region, ir.imagery_type, ir.file_type, ir.pixel_type,
ir.total_images, ir.tile_size, ir.progress, ir.expected_date, ir.high_priority,
ir.priority_justification, ir.raw_data_location, ir.service_overviews,
ir.is_def, ir.isc_def, ir.special_instructions, ir.navigational_path,
ir.fyqueue, ir.created_at, ir.updated_at]
end
end
# send it to the browser with proper headers
send_data csv_string,
:type => 'text/csv; charset=iso-8859-1; header=present',
:disposition => "attachment; filename=requests_as_of-#{Time.now.strftime("%Y%m%d")}.csv"
end
I figured it would be a LOT better if instead of specifying EVERY column manually, I did something like this:
def export(imagery_requests)
csv_string = FasterCSV.generate do |csv|
line = []
imagery_requests.each do |ir|
csv << ir.attributes.values.each do |i|
line << i
end
end
end
# send it to the browser with proper headers
send_data csv_string,
:type => 'text/csv; charset=iso-8859-1; header=present',
:disposition => "attachment; filename=requests_as_of-#{Time.now.strftime("%Y%m%d")}.csv"
end
That should be creating an array of arrays. It works just fine in the Rails console. But in the production environment, it just produces garbage output. I'd much rather make this method extensible so I can add more fields to the ImageryRequest model at a later time. Am I going about this all wrong?
I'm guessing that it probably works in the console when you do it for just one imagery_request, yes?
But when you do multiple it fails?
Again I'm guessing that's because you never reset line to be an empty array again. So you're continually filling a single array.
Try the simple way first, to check it works, then start going all << on it then:
csv_string = FasterCSV.generate do |csv|
imagery_requests.each do |ir|
csv << ir.attributes.values.clone
end
end
PS - in the past I've even used clone on my line-by-line array, just to be sure I wasn't doing anything untoward with persisted stuff...