I have a rails project which is exporting some csv data into excel. On some instances excel is outputting special characters.
ex.
test 1 & test 2 & test 2
reads in excel as
test 1 & test 2 & test 2Â
My default CSV encoding is set to UTF-8 and I have played around with a number of other encoding settings although none of them have seemed to solve this issue.
Here is where the csv gets generated.
<% headers = default_headers %>
<%= CSV.generate_line(headers).strip %>
<% #activities.each do |activity| %>
<% has_permission = #_controller.is_activity_permissioned_to_user?(activity, current_user) %>
<% row = activity_generate_csv_row_data(activity, headers, has_permission, preferences) %>
<%= CSV.generate_line(row).gsub(/\A""/,'').strip.html_safe %>
<% end %>
and in my controller.
format.csv {
headers['Content-Disposition'] = "attachment; filename=\"
{report_name}.csv\""
headers['Content-Type'] ||= 'text/csv'
}
Every solution i've tried has failed. I really just can't figure out how to fix this.
Try to concatenate a Byte Order Mark string with your CSV string, like:
BOM = "\uFEFF"
def some_csv_string_generating_func
...
return BOM + a_csv_string
end
This will make Excel show the CSV file correctly.
Also, I would advice against having all the CSV generation logic/code on the view, but on a helper class/module or the like.
Related
When a click on the 'View CSV' link on my site the csv downloads but the content of the csv is in Chinese instead of English. How do I ensure the language of what I'm passing through the send_data function doesn't change? Thanks for your help!
I have the following function in my controller:
def get_data_visit
#csv_string = CSV.generate do |csv|
csv << ["row", "of", "CSV", "data"]
end
send_data Iconv.conv('iso-8859-1//IGNORE', 'utf-8', #csv_string), filename: "something.csv"
end
And the following function in my view:
<%= link_to "View CSV", get_data_visit_admin_stats_path %>
Ie. ["row", "of", "CSV", "data"] becomes 潲ⱷ景䌬噓搬瑡 and I would like it to stay in English.
I had to ensure libre office was reading utf-8 (changed in a drop down), then it worked.
I am bumping into an issue in my Rails app where a .csv export of data from a table is not formatting correctly.
First of all, here is the code that generates the .csv export...
report_controller.erb
class Admin::ReportController < ApplicationController
def index
#report = Report.all
respond_to do |format|
format.html
format.csv do
headers['Content-Disposition'] = "attachment;
filename=\"report-summary.csv\""
headers['Content-Type'] ||= 'text/csv'
end
end
end
end
index.csv.erb
<%- headers = ["Report Id", "Datetime", "Latitude", "Longitude", "Report Type"] -%>
<%= CSV.generate_line headers -%>
<%- #report.each do |report| -%>
<%- row = [report.id, report.datetime, report.latitude, report.longitude, report.report_type.report_type] -%>
<%= CSV.generate_line row -%>
<%- end -%>
As I was building this out, everything was working fine, until I got to the last column in the table, 'Report Type'. Here's what happens when that gets added:
Notice how that first entry has the HTML " leading and trailing? But the next one doesn't? And how the first one is distributed across additional headerless columns? And how the last one doesn't have the " but does have & in the middle (for an ampersand, of course)?
All of these values are stored as text in a ReportType table and they come through as strings in Rails and, as far as I can tell, there is absolutely nothing different about their data types anywhere, yet they are behaving differently when pulled out of the table.
If I take 'Report Type' out of the export, see how lovely my table is?
No skipped lines or anything!
It seems to me that these HTML characters - especially the leading and trailing double quotes - are causing at least some of the trouble here. I tried using .gsub('"', '') to remove them, but this has no effect.
Has anyone encountered this before or have any insight here?
Thanks to #tadman for pointing me in the right direction on this - I changed this line:
<%= CSV.generate_line row -%>
by adding raw like so:
<%=raw CSV.generate_line row -%>
And that did the trick. Whew!
Still, if anyone has other solutions for this, I would like to hear them.
I have an application that allows users to enter a string, I parse it, store it in the database for historical purposes, and return some messages.
In the controller, this is how I build the messages
#messages << name + " was not created"
In the view, this is the line that it's crashing on
<% #messages.each do |msg| %>
<li> <b><%= msg %></b></li> <--- this line
<% end %>
Doing a search on the issue resulted in several solutions and explanations of why the problem occurs.
I am properly handling encoding in several places:
My application by default converts things to UTF8.
When I type in chinese characters and I render the specific token in the controller, it displays what I typed in.
When I render the concatenated string above, it displays the correct string
The database is set to UTF-8 encoding
Other pages are able to render the text correctly (they fetch from the database and display it directly without any sort of string manipulation on my part)
The issue disappears when I comment out "this line" in the View, but I don't understand what is wrong with it.
If I write this, following another suggestion, it works
<li> <b><%= msg.force_encoding("utf-8") %></b></li>
But I don't like it, since I shouldn't be having to "force" any encodings when ideally everything going in should be UTF-8 or properly converted to UTF-8, and the views can assume everything they are rendering is proper UTF-8.
I suspect that the problem is the way I am concatenating the string:
#messages << name + " was not created"
If I do a force encoding like this
#messages.size.times do |i|
#messages[i] = #messages[i].force_encoding("UTF-8")
end
That also works.
What is the proper way to concantenate strings?
What is the proper way to concantenate strings?
Using #mb_chars everywhere seems to solve such kind of issues:
#messages << name.mb_chars + " was not created"
And
<% #messages.each do |msg| %>
<li><b><%= msg.mb_chars %></b></li>
<% end %>
I'm attempting to upload a csv file, parse it, and spit out a file for S3 or just pass to view. I use a file_field_tag to upload the csv. I thought file_field_tag passes an object that is a subclass of IO and would have all ruby IO methods such as "each_line". I can call "read" on the object (method of IO class) but not "each_line"... so how can I iterate over each line of a file_field_tag upload?
create method of my controller as:
#csv_file = params[:csv_file]
My show view which throws a no "each_line" method error:
<% #csv_file.each_line do |line| %>
<%= line %>
<% end %>
Yet I can use
<%= #csv_file.read(100) %>
I'm really confused what methods a file_field_tag upload params[] has... each_line, gets don't work... I can't seem to find a list of what I can use.
EDIT
I worked around this by doing:
#csv_file = params[:csv_file].read.to_s
then iterated through with:
<% #sp_file.each_line do |line| %>
<%= line %>
<% end %>
EDIT 2
The file being uploaded has repeats the header after lines which don't contain a comma (don't ask)... So I find lines without a comma and call .gets (in my rb script independent of rails). Unfortunately I get an error about gets being a private method I can't call. Which goes back to my initial issue being. Aren't files a sub class of IO with IO methods like read_lines & gets?
#file_out = []
#file_in.each_line do |line|
case line
when /^[^,]+$/
#comp = line.to_s.strip
comp_header = #file_in.gets.strip.split('')
#file_out.push(#comp)
end
end
When you post a 'file_field' , the param returned to the controller has some special magic hooked in.
I.e. in your case you could this
<%= "The following file was uploaded #{params[:csv_file].original_filename}" %>
<%= "It's content type was #{params[:csv_file].content_type}" %>
<%= "The content of the file is as follows: #{params[:csv_file].read}" %>
So those are the three special methods you can call on params[:csv_file], or any params posted as the result of a successful 'file_field_tag' or 'f.file_field' in a view
Just remember that those are the three extra special things you can to to a param posted as a result of a file_field:
original_filename
content_type
read
You've clearly figured out how to do the read, the original_filename and content_type may help you out in the future.
EDIT
OK, so all you have is the read method, which will read the contents of the file uploaded.
contents = params[:csv_file].read
So now contents is a string containing the contents of the file, but nothing else is known about that file EXCEPT that it's a csv file. We know that csvs are delimited with '\r' (I think, I've done a lot of work with parsing csv's, but I'm too lazy to go check)
so you could do this:
contents = params[:csv_file].read
contents.split("\r").each do |csvline|
???
end
EDIT 2
so here is the take away from this post
When you post a file_field to a controller, the most common thing to do with the contents of the uploaded file is 'read' it into a ruby String. Any additional processing of the uploaded contents must be done on that ruby String returned from the 'read'.
In this particular case, if the uploaded file is ALWAYS a CSV, you can just assume the CSV and start parsing it accordingly. If you expect multiple upload formats, you have to deal with that, for example:
contents = params[:csv_file].read
case params[:csv_file].content_type
when 'txt/csv'
contents.split("\r").each do |csvline|
???
end
when 'application/pdf'
???
end
We need to verify that download links work. And because the files are quite large, we need to verify that we don't have outdated, useless files sitting in our download directory. So we have a download management page that validates that every database file_name has a file named the same in the download directory. The page then verifies that every file in our directory has a file_name in our database.
in the controller:
#documents = Document.find(:all, :order => "section asc, sub_section asc, position asc, name asc")
#files = Dir.glob("public/downloads/*").sort
FIRST VALIDATION: in my view to validate that there is a file for every document record in the database:
<% #documents.each do |d| -%>
<% if #files.include?("public/downloads/" + d.file_name)
clr = "Green"
else
clr = "Red"
end %>
... color coded print routine ...
<% end %>
SECOND VALIDATION: in my view to validate that every file has a document record in the datebase:
<% #files.each do |f| -%>
<% str = f.gsub(/^.*\//, '')
if #documents.include?(str)
clr = "Green"
else
clr = "Red"
end %>
... color coded print routine ...
<% end %>
with my small test document list, the printed database file_names match exactly with the file names printed from our download directory. however, the include test for the second validation is not working. i haven't been able to figure out why.
thanks!
This code:
if #documents.include?(str)
should look like this:
if #documents.detect{|doc| doc.file_name == str}
Instead of #documents.include?(str) use #documents.any? { |d| d.file_name == str}
The problem is that #documents is not a collection of file_names, it is a collection of Document instances. You should try something like this instead:
#documents.map(&:file_name).include?(str)
This will collect all file_names and then check if str matches any of them.