There is CSV-export of some objects (such as tasks, contacts, etc) in my application. It just renders CSV-file like this:
respond_to do |format|
format.html
format.csv { render text: Task.to_csv } # I have self.to_csv def in model
end
It generates a CSV file when I go to '/tasks.csv' without a problem.
Now I want to export all the objects and zip them. I'm using rubyzip gem to create zip-files. Now my code for creating zip-file with all the CSVs looks like that:
Zip::ZipFile.open("#{path_to_file}.zip", Zip::ZipFile::CREATE) do |zipfile|
zipfile.file.open("tasks.csv", "w") { |f| f << open("http://#{request.host}:#{request.port.to_s}/tasks.csv").read }
# the same lines for contacts and other objects
end
But it seems that there is something wrong with it because it's executing for a long time (I'm getting Timeout::Error even if there is just one line in CSV) and the resulting zip-archive contains something broken.
How can I save my "/tasks.csv", "/contacts.csv", etc as a file on server (inside of zip-archive in this case)?
I did it! The code is:
Zip::ZipFile.open("#{path_to_file}.zip", Zip::ZipFile::CREATE) do |zipfile|
zipfile.file.open("tasks.csv", "w") do |f|
CSV.open(f, "w") do |csv|
CSV.parse(Task.to_csv) { |row| csv << row }
end
end
end
Related
I have a doubt about showing a generated CSV file to the user (with a large amount of data). So here is the task I have to do.
App: I have a film that has many characters.
Task:
allow users to upload characters via CSV (ok, done)
if there are errors, show them for each row (ok, done)
in the results page, also show a link to a new CSV file only with the remaining characters - the ones that couldn’t be created (I’m stuck here)
Here is part of my code (upload method):
def upload
saved_characters = []
characters_with_errors = []
errors = {}
begin
CSV.parse(params[:csv].read, **csv_options) do |row|
row_hash = clear_input(row.to_h)
new_character = Character.new(row_hash)
if new_character.save
add_images_to(new_character, row)
saved_characters << new_character
else
characters_with_errors << new_character
errors[new_character.name] = new_character.errors.full_messages.join(', ')
end
end
rescue CSV::MalformedCSVError => e
errors = { 'General error': e.message }.merge(errors)
end
#upload = {
errors: errors,
characters: saved_characters,
characters_with_errors: characters_with_errors
}
end
The issue: large amount of data
In the end, the upload.html.erb almost everything works fine, it shows the results and errors per column BUT I’m not sure how create a link on this page to send the user to the new CSV file (only with characters with errors). If the link sends the user to another method / GET endpoint (for the view with CSV format), how can I send such a large amount of data (params won’t work because they will get too long)? What would be the best practice here?
You can use a session variable to store the data, and then redirect to a new action to download the file. In the new action, you can get the data from the session variable, and then generate the CSV file.
For example, In the upload action, you can do something like this:
session[:characters_with_errors] = characters_with_errors
redirect_to download_csv_path
In the download_csv action, you can do something like this:
characters_with_errors = session[:characters_with_errors]
session[:characters_with_errors] = nil
respond_to do |format|
format.csv { send_data generate_csv(characters_with_errors) }
end
In the generate_csv method, you can do something like this:
def generate_csv(characters_with_errors)
CSV.generate do |csv|
csv << ['name', 'age' ]
characters_with_errors.each do |character|
csv << [character.name, character.age]
end
end
end
Another option, you can use a temporary file to store the data and then send the user to the new CSV file. Here is an example:
def upload
saved_characters = []
characters_with_errors = []
errors = {}
begin
CSV.parse(params[:csv].read, **csv_options) do |row|
row_hash = clear_input(row.to_h)
new_character = Character.new(row_hash)
if new_character.save
add_images_to(new_character, row)
saved_characters << new_character
else
characters_with_errors << new_character
errors[new_character.name] = new_character.errors.full_messages.join(', ')
end
end
rescue CSV::MalformedCSVError => e
errors = { 'General error': e.message }.merge(errors)
end
#upload = {
errors: errors,
characters: saved_characters,
characters_with_errors: characters_with_errors
}
respond_to do |format|
format.html
format.csv do
# Create a temporary file
tmp = Tempfile.new('characters_with_errors')
# Write the CSV data to the temporary file
tmp.write(characters_with_errors.to_csv)
# Send the user to the new CSV file
send_file tmp.path, filename: 'characters_with_errors.csv'
# Close the temporary file
tmp.close
end
end
end
I have a Ruby on Rails application running with Mongodb. I have a model called Label and I have added some data to it.
I am trying to generate a CSV file from the index method in the Labels controller.
I added the following code in the index method:
respond_to do |format|
format.html
format.csv { send_data #labels.to_csv, filename: "labels-#{Date.today}.csv" }
end
I added a method to_csv in the controller:
def self.to_csv(options={})
attributes = %w{name manufacturing_date digital_signature1}
#all_labels=Label.all
csv << attributes
#all_labels.each do |label|
csv << attributes.map{ |attr| label.send(attr) }
end
end
end
But when I actually generate a CSV file, it gives me a file where there is no data. I get six column names like:
#<Label:0x00007f637ac2e130>, #<Label:0x00007f637ac2e090>, #<Label:0x00007f637ac2dfc8>, #<Label:0x00007f637ac2db18>, #<Label:0x00007f637ac2d9d8>, #<Label:0x00007f637ac2d668>
What am I doing wrong?
If you are following https://www.codementor.io/#victor_hazbun/export-records-to-csv-files-ruby-on-rails-vda8323q0 the to_csv method should be added to the model, not to the controller.
I am trying to write a helper method that can download a CSV file from S3 storage, read the first few rows of the file and then save those first few rows to a new local file.
All is working well when I include the helper in the rails console and call the methods on the object, but when calling it in exactly the same way through the controller, the local file contains all of the rows from the S3 file, rather than just the first few.
My code, in the helper file (I've replaced AWS credentials with comments for the purpose of posting the question):
def download_file(data_source)
s3 = Aws::S3::Client.new(#API keys etc.)
File.open(data_source.file.data['id'], 'wb') do |file|
reap = s3.get_object({ bucket:#Bucket Name, key: 'store/' + data_source.file.data['id'] }, target: file)
end
end
def reduce_csv(filename)
data = CSV.open(filename, 'r') { |csv| csv.first(3) }
csv_string = CSV.generate do |csv|
data.each do |d|
csv << d
end
end
File.open('test.csv', 'wb') do |file|
file << csv_string
end
end
def make_small_data_source(data_source)
download_file(data_source)
reduce_csv(data_source.file.data['id'])
end
And in the controller:
if #data_source.save
make_small_data_source(#data_source)
Any ideas would be much appreciated!
Ruby 2.2, Ruby on Rails 4.2
I'm genarating some CSV data in Ruby on Rails, and want empty fields to be empty, like ,, not like ,"", .
I wrote codes like below:
somethings_cotroller.rb
def get_data
respond_to do |format|
format.html
format.csv do
#data = SheetRepository.accounts_data
send_data render_to_string, type: :csv
end
end
end
somethings/get_data.csv.ruby
require 'csv'
csv_str = CSV.generate do |csv|
csv << [1,260,37335,'','','','','','']
...
end
And this generates CSV file like this.
get_data.csv
1,260,37335,"","","","","",""
I want CSV data like below.
1,260,37335,,,,,,
It seems like Ruby adds "" automatically.
How can I do this??
In order to get CSV to output an empty column, you need to tell it that nothing is in the column. An empty string, in ruby, is still something, you'll need to replace those empty strings with nil in order to get the output you want:
csv_str = CSV.generate do |csv|
csv << [1,260,37335,'','','','','',''].map do |col|
col.respond_to?(:empty?) && col.empty? ? nil : col
end
end
# => 1,260,37335,,,,,,
In rails you can clean that up by making use of presence, though this will blank out false as well:
csv_str = CSV.generate do |csv|
csv << [1,260,37335,'',false, nil,'','',''].map(&:presence)
end
# => 1,260,37335,,,,,,
The CSV documentation shows an option that you can use for this case. There are not examples but you can guess what it does.
The only consideration is, you need to send an array of Strings, otherwise, you will get a NoMethodError
csv_str = CSV.generate(write_empty_value: nil) do |csv|
csv << [1,260,37335,'','','','','','', false, ' ', nil].map(&:to_s)
end
=> "1,260,37335,,,,,,,false, ,\n"
The benefit of this solution is, you preserve the false.
I resolved by myself!
in somethings_controller.rb
send_data render_to_string.gsub("\"\"",""), type: :csv
My objective is to transform a hash array to a csv file.
This is my controller:
respond_to do |format|
format.html
format.csv { send_data #comsumptions.to_csv }
end
#comsumptions is a hash array:
[{"x"=>76,
"y"=>"example",
"z"=>2015,
"consumption"=>#<BigDecimal:7fea4a1cadb8,'0.5382857142E4',18(27)>},
{"x"=>76,
"y"=>"example2",
"z"=>2015,
"consumption"=>#<BigDecimal:7fea4a1ca7c8,'0.5437E4',9(27)>},(..)
I want to create a CSV file with 2 specific columns, "consumption" and "z".
When I did this with these 3 lines comment the output is a file with all the #consumptions. How can I select these 2 columns and transform in a cv file?
def self.to_csv
CSV.generate(headers: true ) do |csv|
#csv << column_names
#all.each do |product|
# csv << product.attributes.values_at(*column_names)
end
end
end
From your feedback, I think the best way here is creating a csv view file in your views. For example, if your html file is comsumptions.html.erb, then your csv view file should be comsumptions.csv.ruby
# comsumptions.csv.ruby
require 'csv'
CSV.generate do |csv|
csv << ['consumption', 'z']
#comsumptions.each do |c|
csv << [ c['consumption'].to_s, c['z'] ]
end
end
And we need to change the controller too. Remove respond_to part or modify it as follows
respond_to do |format|
format.html
format.csv
end
I already tested on my localhost, and this should work!
require 'csv'
#comsumptions =
[{"x"=>76,
"y"=>"example",
"z"=>2015},
{"x"=>76,
"y"=>"example2",
"z"=>2015}]
class << #comsumptions
def to_csv (*keys)
keys = first.keys if keys.empty?
CSV.generate(headers: keys, write_headers: true) do |csv|
each do |e|
csv << e.values_at(*keys)
end
end
end
end
p #comsumptions.to_csv("x","y")
p #comsumptions.to_csv()
This solution heavily inspired from the one from Van Huy works fine for any array oh hashes as long as the hashes have all the same keys, undefined behaviour to be expected otherwise
What's not clear to me is where do you put the def self.to_csv method, inside the controller? That is wrong. What you want to do is either augment the #consumptions object, augment the Array class, or define a method in the controller. My example does augment the #consumptions object, you can put all that stuff inside the controller method and it should work.