Rails 4: incompatible character encodings: UTF-8 and ASCII-8BIT - ruby-on-rails

I have an application that allows users to enter a string, I parse it, store it in the database for historical purposes, and return some messages.
In the controller, this is how I build the messages
#messages << name + " was not created"
In the view, this is the line that it's crashing on
<% #messages.each do |msg| %>
<li> <b><%= msg %></b></li> <--- this line
<% end %>
Doing a search on the issue resulted in several solutions and explanations of why the problem occurs.
I am properly handling encoding in several places:
My application by default converts things to UTF8.
When I type in chinese characters and I render the specific token in the controller, it displays what I typed in.
When I render the concatenated string above, it displays the correct string
The database is set to UTF-8 encoding
Other pages are able to render the text correctly (they fetch from the database and display it directly without any sort of string manipulation on my part)
The issue disappears when I comment out "this line" in the View, but I don't understand what is wrong with it.
If I write this, following another suggestion, it works
<li> <b><%= msg.force_encoding("utf-8") %></b></li>
But I don't like it, since I shouldn't be having to "force" any encodings when ideally everything going in should be UTF-8 or properly converted to UTF-8, and the views can assume everything they are rendering is proper UTF-8.
I suspect that the problem is the way I am concatenating the string:
#messages << name + " was not created"
If I do a force encoding like this
#messages.size.times do |i|
#messages[i] = #messages[i].force_encoding("UTF-8")
end
That also works.
What is the proper way to concantenate strings?

What is the proper way to concantenate strings?
Using #mb_chars everywhere seems to solve such kind of issues:
#messages << name.mb_chars + " was not created"
And
<% #messages.each do |msg| %>
<li><b><%= msg.mb_chars %></b></li>
<% end %>

Related

Rails CSV exporting into excel produces special  and &amp characters

I have a rails project which is exporting some csv data into excel. On some instances excel is outputting special characters.
ex.
test 1 & test 2 & test 2
reads in excel as
test 1 & test 2 & test 2 
My default CSV encoding is set to UTF-8 and I have played around with a number of other encoding settings although none of them have seemed to solve this issue.
Here is where the csv gets generated.
<% headers = default_headers %>
<%= CSV.generate_line(headers).strip %>
<% #activities.each do |activity| %>
<% has_permission = #_controller.is_activity_permissioned_to_user?(activity, current_user) %>
<% row = activity_generate_csv_row_data(activity, headers, has_permission, preferences) %>
<%= CSV.generate_line(row).gsub(/\A""/,'').strip.html_safe %>
<% end %>
and in my controller.
format.csv {
headers['Content-Disposition'] = "attachment; filename=\"
{report_name}.csv\""
headers['Content-Type'] ||= 'text/csv'
}
Every solution i've tried has failed. I really just can't figure out how to fix this.
Try to concatenate a Byte Order Mark string with your CSV string, like:
BOM = "\uFEFF"
def some_csv_string_generating_func
...
return BOM + a_csv_string
end
This will make Excel show the CSV file correctly.
Also, I would advice against having all the CSV generation logic/code on the view, but on a helper class/module or the like.

Rails - .csv export formatting strings incorrectly

I am bumping into an issue in my Rails app where a .csv export of data from a table is not formatting correctly.
First of all, here is the code that generates the .csv export...
report_controller.erb
class Admin::ReportController < ApplicationController
def index
#report = Report.all
respond_to do |format|
format.html
format.csv do
headers['Content-Disposition'] = "attachment;
filename=\"report-summary.csv\""
headers['Content-Type'] ||= 'text/csv'
end
end
end
end
index.csv.erb
<%- headers = ["Report Id", "Datetime", "Latitude", "Longitude", "Report Type"] -%>
<%= CSV.generate_line headers -%>
<%- #report.each do |report| -%>
<%- row = [report.id, report.datetime, report.latitude, report.longitude, report.report_type.report_type] -%>
<%= CSV.generate_line row -%>
<%- end -%>
As I was building this out, everything was working fine, until I got to the last column in the table, 'Report Type'. Here's what happens when that gets added:
Notice how that first entry has the HTML " leading and trailing? But the next one doesn't? And how the first one is distributed across additional headerless columns? And how the last one doesn't have the " but does have &amp in the middle (for an ampersand, of course)?
All of these values are stored as text in a ReportType table and they come through as strings in Rails and, as far as I can tell, there is absolutely nothing different about their data types anywhere, yet they are behaving differently when pulled out of the table.
If I take 'Report Type' out of the export, see how lovely my table is?
No skipped lines or anything!
It seems to me that these HTML characters - especially the leading and trailing double quotes - are causing at least some of the trouble here. I tried using .gsub('"', '') to remove them, but this has no effect.
Has anyone encountered this before or have any insight here?
Thanks to #tadman for pointing me in the right direction on this - I changed this line:
<%= CSV.generate_line row -%>
by adding raw like so:
<%=raw CSV.generate_line row -%>
And that did the trick. Whew!
Still, if anyone has other solutions for this, I would like to hear them.

Rails with AJAX trimming contents of field

I have a model called Content with two fields 'before' and 'after'.
Before creating the record, I do some preprocessing on field 'before' and store it in 'after'. No problem here.
Problem is when I load the show page the contents in 'before' display incomplete. I'm trimming white spaces after a period if there are 2 or more...
e.g.
"this is. some text. with many spaces. after the periods."
The view shows me that 'before' looks like this:
"this is. some text. with many spaces. after the periods."
If I look at the .json version the 'before' field contains the original format - why then does the rails view (non-json) present the 'before' as being "trimmed"?
This is my _content.html.erb
<li><%= content.before %></li>
<li><%= content.after %></li>
This is my show.js.erb
$("#main").html("<%= escape_javascript(render #content) %>");
This is the show definition in contents_controller.rb
respond_to :html, :json
def show
#content = Content.find(params[:id])
respond_with(#content)
end
This is the pre-processing in my model content.rb
class Content < ActiveRecord::Base
before_validation :format_params
def format_params
rm_spaces = /\.\s{2,}/ # regex to rm extra spaces
new_format = '. ' # only one space after period
self.after = before.gsub(rm_spaces, new_format)
end
end
And here are some screenshots of what I mean.
Edit
I believe this is the code that is getting executed since I tried Swards recommendation and it did not work.
This is the code for my highlighting function in my contents_helper.rb
def highlight_changes(text)
highlighter = '<span style= "background: yellow">\1</span>'
matcher = /(\.\s{2,})/
text.gsub(matcher, highlighter).html_safe
end
This is the code in show.html.rb
<p>
<strong>Before:</strong>
<!-- call the helper method to highlight changes made -->
<%= highlight_changes #content.before %>
</p>
You can put a .before class on the before li and add this to css to see it as it was entered.
<li class="before"><%= content.before %></li>
<li><%= content.after %></li>
in the css
.before {
white-space: pre;
}
I think it is the browser that omits extra spaces. You can use to make a whitespace instead.
To transform your string you could try
content.before.gsub(' ', ' ').html_safe
This replaces each space with and then tells rails not to escape the html characters ( ).
Warning:If the before field is input by users then using html_safe is dangerous as it could allow users to inject malicious code.

each_element not behaving as expected

When I'm in irb and I do something like this:
node_list.each_element { |e| puts e.text }
It works and prints one line of text per element returned (plus I think it returns the xml object). However, when I head over to rails and have things moving between controllers, helpers, views and layouts it just dumps the xml object.
I should mention that for good reasons I'm using rails 1.2.3 and ruby 1.8.7.
Gratzi!
So the issue your having is that puts writes to console and not the template. Also, in ruby the a method will return by default its last object. So your method as written will loop through #child1, print each's text to the console, and then return the #child1 object at which point your erb tags of <%= %> tells it print the object (#child1 in this case)
You have two options, either you can move it out into the template:
<% tol_get_names(#child1) do |e| %> #note just <% and not <%=
<%= e.text %>
<% end %>
Or build your method so that it loops through building a string and then returns that string instead of the original object:
def tol_get_names(child)
texts = [] #empty array for all our values
child.each_element { |e|
texts << e.text #add the text string to our array
end
texts.join(", ") #build a string and seperate it with a comma
end
Several ways you could write this type of method, but this is my usual way.

Rails 3 / Controller / Flash hash

I want to be able to pass multiple messages to the flash hash, inside of my controller, and have them display nicely together, e.g., in a bulleted list. The way I've devised to do this is to create a helper function in my Application Controller, which formats an array into a bulleted list, which I then pass to, in my case, flash[:success]. This is clearly not the Rails Way because, i.a., my bulleted list gets encoded. That is, instead of getting:
Message 1
Message 2
I get:
<ul><li>Message 1</li><li>Message 2</li></ul>
I'm sure I could figure out a way to raw() the output, but isn't there a simple way to get something like this working? Perhaps there's an option to pass to flash[]? Something else?
I used render_to_string and a partial instead of a helper to achieve something similar.
# app/controller/dogs_controller.rb
def create
#dog = Dog.new(params[:dog])
#messages=[]
if #dog.save
#messages << "one"
#messages << "two"
flash[:notice] = render_to_string( :partial => "bulleted_flash")
redirect_to(dogs_path)
else
render :action => 'new
end
end
Then I format the array of flash messages in an HTML list
# app/views/dogs/_bulleted_flash.html.erb
<ol>
<% #messages.each do |msg| %>
<li><%= msg %></li>
<% end %>
</ol>
Which produces the following HTML
# http://0.0.0.0:3000/dogs
<body>
<div id="flash_notice">
<ul>
<li>one</li>
<li>two</li>
</ul>
</div>
...
</body>
If you need to continue using a helper then I think you need to append the html_safe method to your string to prevent it from being encoded (which rails 3 does by default). Here is a question showing how to use html_safe in a similar fashion
If you are using Rails3, try the raw method.
raw(my_html_string)
And it won't escape the html. Oh, sorry, I just read your last sentence. Check out this information, "Rails 3 flash message problems", it looks like it may be what you are looking for:
http://www.ruby-forum.com/topic/215108
Usually I would ask for more information about your views and layouts in this situation, because scaffolding doesn't display flash[:success] by default.
The way I solve this is to totally redo my flash messages usually, by making the flash[:whatever] an array every time, and in my layout handling that array instead of just the notice. Remember that flash is just a Hash, you're just setting values.
However if you just want to do this with the setup you have now (helper putting the HTML inside the flash[:success]), you can change the way that the flash messages are displayed in your layout file. By default they are just use <%= flash[:success] %>, which automatically escapes HTML. To make it not do that for the flash messages, change it to <%=raw flash[:success] %>

Resources