Why is my Rails app on Heroku soemtimes displaying apostrophes as HTML entities? - ruby-on-rails

Sometimes we have jobs whose name has an apostrophe in it. I always want those apostrophes to display as ' , never as their HTML entity (').
The apostrophe displays correctly on most pages most of the time. But in some instances, the apostrophe displays as the HTML entity instead. Here's a screen-capture showing the behavior:
The apostrophes in the "Notes" field (a textarea) display correctly, but not in the "Job name" and "Display as" fields. Luckily, the apostrophes display correctly on the public-facing side, and I only see this behavior on the admin-side.
This sentence is going to sound insane, but stick with me, here: When I look at a page's source code where this problem occurs, it looks like the leading ampersand in the apostrophe's html entity is being replaced with the HTML entity for ampersand, thus becoming '
Here's a gist of the form's code.
When I look up this job's record in console, the job name and display name are "Job's Got An Apostrophe", so I know (think?) I'm not storing the HTML entity in my database. My database.yml specifies unicode encoding. It's a PostgreSQL 9.2.7 database. Not sure what other information is needed to help resolve this, if any.

Related

In Reflected XSS, why do we need to sanitize single quote, double quote, ampersand, and backslash

Based on this article
https://resources.infosecinstitute.com/topic/how-to-prevent-cross-site-scripting-attacks/
Reflected XXS happens when data injected is reflected in the response. I get the idea that if I, for example, have a search box in my page and the search term inputted by a user is displayed in the page, someone could write as a search term:
<script>alert('x');</script>
and that would be read as regular HTML element in the page that displays the response.
But lets say greater than and less than are already blocked in input (meaning they wouldn't be able to put in script tags or any tag), what's the issue if I allow single quote, double quote, ampersand, and backslash reflected in the response. I'm trying to make sense of it but I am not sure if I am understanding correctly.
Today the web stack is big and complex with many languages. We have HTML, CSS, JavaScript, VB-Script, SVG, URLs…
Each with its own rules for:
Encoding
Quoting
Commenting
Escaping
Also, each one can be nested inside each other:
And just replacing <> fixes some issues, but not all of them as you don't know where you data will end up, is it in HTML? as a HTML Attribute? inside a JavaScript string? Each one needs different encoding to become safe.
So, the world is a bit more complicated.....

For variables names for text components ending with a dot(.) , itextpdf appends .widget to them

For variable names that i am declaring using pdf expro, having dot(.) character at the end, the itext is appending "Widget" to them. IF my variable name on pdf template for a textbox is "Payment transaction no." then itextpdf makes it as "Payment Transaction No.Widget"
Why does this happen ? and suggestions about handling it, because as a part of data validation i need to pick up pdf field names and validate them with those saved in form configuration settings in the database.
Is there any other such kind of wierd possibility that might occur ? Please provide those cases.

Trouble encoding form data with accents

I have a basic form that asks the user to enter some text in a regular html input control and I am also using jquery-tokeninput to allow users to choose tags from a pre-filled list.
One of the tags in the pre-filled list happens to be the word café, which I have got from the server and populating the tag list by calling
mytaglist.push({id: 'café', name: 'café'});
The problem is that when they attempt to enter a word like 'café' as a tag, asp.net mvc rejects the input saying that:
A potentially dangerous Request.Form value was detected from the client (articleTags="café").
Inspection using firebug shows my post data to be something like:
UserName=neo&category=&Title=caf%C3%A9&Text=sometext&articleTags=caf%26%23233%3B&IsAgreedTerms=true
As you can see, Title has value caf%C3%A9 which is correct but articleTags has value caf%26%23233%3B which I was not expecting.
I absolutely need to make sure that café (and not some encoded value) appears on the screen.
How can I make sure that I send the correct post data and still display café everytime?
Should I change how my server sends the text?
sending the data via Html.Raw solves the problem..

Grails: User inputs formatted string, but formatting not preserved

I am just starting a very basic program in Grails (never used it before, but it seems to be very useful).
What I have so far is:
in X.groovy,
a String named parameters, with constraint of maximum length 50000 and a couple other strings and dates, etc.
in XController.groovy,
static scaffold = X;
It displays the scaffold UI (very handy!), and I can add parameter strings and the other objects associated with it.
My problem is that the parameters string is a long string with formatting that is pasted in by the user. When it is displayed on the browser, however, it does not retain any carriage returns.
What is the best way to go about this? I'm a very beginner at Grails and still have lots and lots of learning to do on this account. Thanks.
The problem is that the string is being displayed using HTML which doesn't parse \n into a new line by default. You need to wrap the text in <pre> (see: http://www.w3schools.com/tags/tag_pre.asp) or replace the \n with <br/> tags to display it correctly to the user.

Encoding of XHTML and & (ampersand)

My website is XHTML Transitional compliant except for one thing: the & (ampersand) in the URL are written as it is, instead of &
That is, all the URLs in my pages are usually like this:
Foo
But XHTML validator generates this error:
cannot generate system identifier for general entity "y"
... and it wants the URL to be written like this:
Foo
The problem is that Internet Explorer and Firefox don't handle the URL correctly and ignore the y parameter. How can I make this link work and validate correctly?
It seems to me that it is impossible to write XHTML pages if the browsers don't work with strict encoded XHTML URLs.
Do you want to see in action? See the difference between these two links (copy and paste them as they are):
http://stackoverflow.com/search?q=ff&sort=newest
and
http://stackoverflow.com/search?q=ff&sort=newest
I have just tried this. What you attempted to do is correct. In HTML if you are writing a link the & characters should be encoded as & You would only encode the & as %26 if you wanted a parameter value to contain an ampersand. I just wrote a simple HTML page that contained a link: Click me
and it worked fine: default2.aspx received the parameters intended and the source passed validation.
The encoding of & as & is required in HTML, not in the link. When the browser sees the & in the HTML source for a link it will interpret it as an ampersand and the link target will be as intended. If you paste a URL into your browser address bar it does not expect it to be HTML and does not try to interpret any HTML encoding that it may contain. This is why your example links that you suggest we should copy/paste into a browser don't work and why we wouldn't expect them to work.
If you post a bit more of your actual code we might be able to see what you have done wrong, but you appear to be heading the right direction by using & in your anchor tags.
It was my fault: the hyperlink control already encoded &, so my URL http://foo?x=1&y=2 was encoded to http://foo?x=1&amp;y=2
Normally the &amp inside the URL is correctly handled by browsers, as you stated.
You could use & instead of & in your URL within your page.
That should allow it to be validated as strict XHTML...
Foo
Note, if used by an ASP.NET Request.QueryString function, the query string doesn't use XML encoding; it uses URL encoding:
/mypath/mypage?b=%26stuff
So you need to provide a function translating '&' into %26.
Note: in that case, Server.URLEncode(”neetu & geetu”), which would produce neetu+%26+geetu, is not what you want, since you need to translate & into %26, not just '&'. You must add a replace() call applied to URLEncode result, in order to replace '%26amp;' by '%26'.
To be even more thorough: use &, a numeric character reference.
Because & is a character entity reference:
Character entity references are defined in the markup language
definition. This means, for example, that for HTML only a specific
range of characters (defined by the HTML specification) can be
represented as character entity references (and that includes only a
small subset of the Unicode range).
That's coming from the wise people at W3C (read this for more).
Of course, this is not a very big deal, but the suggestion of W3C is that the numeric one will be valid and useable everywhere and always, while the named one is 'fine' for HTML but nothing more.
The problem is worse than you think - try it in Safari. &amp; gets converted to &#38; and the hash ends the URL.
The correct answer is to not output XHTML - there's no reason that justifies spending more time on development and alienating Mac users.

Resources