I just noticed that the respond method in controllers is returning HTML responses in the ISO-8859-1 charset (which garbles my unicode characters). It uses UTF-8 if I set the format to JSON. The render method also uses UTF-8.
I'm using Grails 2.4.4 and the Tomcat plugin v. 7.0.55 in development mode without overriding web.xml. Both grails.converters.encoding and grails.views.gsp.encoding are set to UTF-8. I have <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> in my template if that influences anything.
I've also tried forcing the charset by using respond myInstance, [encoding: "UTF-8"] but it didn't change anything.
Is there something I am not seeing or have I hit a bug?
EDIT: my config.groovy contains the following mime type definitions:
grails.mime.types = [ // the first one is the default format
all: '*/*', // 'all' maps to '*' or the first available format in withFormat
atom: 'application/atom+xml',
css: 'text/css',
csv: 'text/csv',
form: 'application/x-www-form-urlencoded',
html: ['text/html','application/xhtml+xml'],
js: 'text/javascript',
json: ['application/json', 'text/json'],
multipartForm: 'multipart/form-data',
rss: 'application/rss+xml',
text: 'text/plain',
hal: ['application/hal+json','application/hal+xml'],
xml: ['text/xml', 'application/xml']
]
It would seem this is a Grails bug, I've narrowed down the specific case when it happens: you need to have a static responseFormats = ['html', ...] limitation on the controller to trigger it. The fact that Grails' generate-restful-controller includes the responseFormats block automatically makes developers even more likely to encounter this issue.
I've filed a bug report.
EDIT: to keep the responseFormats block but still have UTF-8 responses, it's possible to set the encoding manually, perhaps like this:
def beforeInterceptor = {
response.characterEncoding = 'UTF-8' //workaround for https://jira.grails.org/browse/GRAILS-11830
}
Related
I'm attempting to upgrade to Turbo from Turbolinks and I've found that the client is not rendering redirects for form submissions.
Versions:
rails 6.1.4
hotwire-rails 0.1.2
#hotwired/turbo-rails 7.0.0-beta.8
I've ignored the incompatibility between Turbo and Devise for now - just trying to get regular forms working without having to disable Turbo on them.
Here's an example action:
def update
authorize #label
#label.update(label_params)
if #label.save
redirect_to document_labels_path(document_id: #document.id)
else
render :new, status: :unprocessable_entity
end
end
Here's a rendered form:
<form class="simple_form new_label" id="label_form" novalidate="novalidate" action="/documents/72/labels" accept-charset="UTF-8" method="post">
...
</form>
When submitting a valid form, the server will say Processing by LabelsController#create as TURBO_STREAM and correctly serve a 302. It will then serve the 200 for the redirect location. The browser however is left just looking at the submitted form. Changing the redirect status to 303 doesn't change anything.
I added a console.log for every Turbo event:
document.addEventListener("turbo:load", function () {
console.log('TURBO:LOAD')
})
document.addEventListener("turbo:click", function () {
console.log('TURBO:CLICK')
})
document.addEventListener("turbo:before-visit", function () {
console.log('TURBO:BEFORE-VISIT')
})
document.addEventListener("turbo:visit", function () {
console.log('TURBO:VISIT')
})
document.addEventListener("turbo:submit-start", function () {
console.log('TURBO:SUBMIT-START')
})
document.addEventListener("turbo:before-fetch-request", function () {
console.log('TURBO:BEFORE-FETCH-REQUEST')
})
document.addEventListener("turbo:before-fetch-response", function () {
console.log('TURBO:BEFORE-FETCH-RESPONSE')
})
document.addEventListener("turbo:submit-end", function (event) {
console.log('TURBO:SUBMIT-END')
// event.detail
})
document.addEventListener("turbo:before-cache", function () {
console.log('TURBO:BEFORE-CACHE')
})
document.addEventListener("turbo:before-stream-render", function () {
console.log('TURBO:BEFORE-STREAM-RENDER')
})
document.addEventListener("turbo:render", function () {
console.log('TURBO:RENDER')
})
This is what the output is for a successful form submission:
TURBO:BEFORE-FETCH-REQUEST
TURBO:SUBMIT-START
TURBO:BEFORE-FETCH-RESPONSE
TURBO:SUBMIT-END
There is no render event. Investigating event.detail.fetchResponse.response for turbo:submit-end it seems to be perfectly aware that the client should redirect, it just didn't.
Response {type: "basic", url: "http://lvh.me:3000/documents/72/labels", redirected: true, status: 200, ok: true, …}
body: (...)
bodyUsed: true
headers: Headers {}
ok: true
redirected: true
status: 200
statusText: "OK"
type: "basic"
url: "http://lvh.me:3000/documents/72/labels"
__proto__: Response
Update: It is actually performing the redirect and the server is generating the response. The issue is that the client is not rendering the redirect response.
What is happening here is that your application is specifying that it prefers turbo-stream responses over text/html responses. If you were to look at your request headers for the redirect page, you'll likely see the following:
Accept: text/vnd.turbo-stream.html, text/html, application/xhtml+xml
As a result, Rails returns the data with the first type it recognizes, which is text/vnd.turbo-stream.html. Turbo in your browser sees this and, since it's not interpretable as a Turbo Stream, unhelpfully ignores it quietly.
The solution (workaround?) is to make sure you are redirecting to the html version of your page:
redirect_to document_labels_path(document_id: #document.id, format: :html)
This will return the page with a Content-Type of text/html, and Turbo will replace the whole page with the contents.
Jeff's answer is correct but I wanted to share the specific fix for the issue I was having.
If you use HAML or Slim, I've seen it on more than one codebase where developers rename all template files .haml instead of .html.haml (same for Slim). It's never bitten me before using Turbo, but without .html in the filename, part of Rails won't know what format to serve a response in, so it defaults to the request format.
Turbo makes a turbostream request when submitting a form, but if the response is a redirect, it expects it to be text/html in order to render it. If it receives a turbostream response to a redirect request, Turbo just sits there doing nothing with no console errors or warnings (terrible default behavior IMO).
So if your templates do not include .html, just add it back and Turbo will render redirects. You may still need status: :see_other.
More information:
https://github.com/hotwired/turbo-rails/issues/122
https://github.com/hotwired/turbo-rails/issues/287
Adding to the excellent answer of Jeff Seifert.
If you don't need turbo streams, you may also unregister the turbo-stream content type altogether by putting this into an initializer e.g. config/initializers/turbo.rb:
Rails.application.config.after_initialize do
Mime::Type.unregister(:turbo_stream)
end
I want to use non latin symbols in the Suave, for example, cyrillic, but getting the weird result
MCVE
open Suave
open Suave.Filters
open Suave.Operators
open Suave.Successful
let app =
choose [
GET >=> OK "Привет, Мир!"
]
startWebServer defaultConfig app
Result
So, the Q is - how to fix it?
For text only responses you need to set the mine type encoding: >=> setMimeType "text/plain; charset=utf-8"
Set the Content-Type header to the mime type given. Remember that it should include the encoding of your content. So for example, specifying a mimeType value of 'application/json; charset=utf-8' is strongly recommended (but replace 'json' with your own MIME type, of course ;))
That looks like UTF-8 being interpreted as Latin-1. Try adding >=> setMimeType "text/html; charset=utf-8" to your app and see if that makes the browser treat your UTF-8 as actual UTF-8 instead of defaulting to the incorrect Latin-1.
i am submitting a value "02AP0040¶" in my grails 3 application, when the form is submitted via ajax the value in the controller is not altered but when the form is submitted normally the value is changed to "02AP0040¶"  is added. I have changed the mime types in application.yml, form: application/x-www-form-urlencoded to form: multipart/form-data Part of my configuration file (application.yml) is this :
grails:
applicationVersion: '#info.app.version#'
mime:
disable:
accept:
header:
userAgents:
- Gecko
- WebKit
- Presto
- Trident
types:
all: '*/*'
atom: application/atom+xml
css: text/css
csv: text/csv
form: multipart/form-data
html:
- text/html
- application/xhtml+xml
js: text/javascript
json:
- application/json
- text/json
multipartForm: multipart/form-data
pdf: application/pdf
rss: application/rss+xml
text: text/plain
hal:
- application/hal+json
- application/hal+xml
xml:
- text/xml
- application/xml
file.extensions: true
use.accept.header: false
urlmapping:
cache:
maxsize: 1000
controllers:
upload:
maxFileSize: 2097152 #2MB
maxRequestSize: 2097152 #2MB
defaultScope: prototype
converters:
encoding: UTF-8
views:
default:
codec: none
gsp:
encoding: UTF-8
htmlcodec: xml
codecs:
expression: none
scriptlets: html
taglib: none
staticparts: none
The submitting form is also submitting the correct value to the controller but when it comes to the controller the value is altered.
I have found a solution might not be the perfect one but solved my problem.
I have added enctype attribute as <form action="" method="post" enctype=multipart/form-data> to each form that submits such ascii characters. I tried to change the mime type in application.yml but it didn't work so i used javascript in my main page to add the enctype to every form which currently solved the issue.
What i found was, even though the method is post, the default form enctype is application/x-www-form-urlencoded which treats the request body as queryparameters and hence encode/escape the ascii characters so for to transfer these kind of characters we have to use enctype as multipart/form-data which doesn't encode the content.
I recently upgrade from grails 2.2 to 2.3.1. My controller used to bind data to an command object at controller parameters. After I upgraded to 2.3.1, the binding seems not working and have '[Fatal Error] :-1:-1: Premature end of file.' output to console.
it works fine at 2.2 like this
def home(ACommand cmd) {}
after upgrading, it outputs 'Premature end of file' before it goes to the action and skip the action method and going to home.gsp view directly.
I also tried new an instance inside the action and use bindData(cmd, params). When I step through the action, bindData method produced the same message but can continue and generate view models and pass to home.gsp.
Does anyone happen to know what might cause the problem? Thanks.
Grails 2.3 includes a new data binding mechanism which has additional features. If you need to access the legacy spring data binding mechanism use this configuration in Config.groovy
grails.databinding.useSpringBinder=true
Eventually if you feel the need to use the latest data binder then a transition to use the new features would be needed.
Thanks dmahapatro. I tried it before but does not work.
FYI, I fixed it by rearranging content negotiations at Config.groovy.
Changing from
mime.types = [ xml: ['text/xml', 'application/xml'],
text: 'text/plain',
js: 'text/javascript',
rss: 'application/rss+xml',
atom: 'application/atom+xml',
css: 'text/css',
csv: 'text/csv',
all: '*/*',
json: 'text/json',
html: ['text/html','application/xhtml+xml']
]
to
mime.types = [
all: '*/*',
atom: 'application/atom+xml',
css: 'text/css',
csv: 'text/csv',
form: 'application/x-www-form-urlencoded',
html: ['text/html','application/xhtml+xml'],
js: 'text/javascript',
json: ['application/json', 'text/json'],
multipartForm: 'multipart/form-data',
rss: 'application/rss+xml',
text: 'text/plain',
hal: ['application/hal+json','application/hal+xml'],
xml: ['text/xml', 'application/xml']
]
Solves the problem.
Not sure why the order matters, but I think it is caused by Grails 2.3 databinding intend to parse request body and bind to my command object and lead to an xml parser error.
I have code like this:
Request: require('request').Request
_makeCall:function(callback){
Request({
url: 'URL-TO-API',
contentType: "application/x-www-form-urlencoded; charset=iso-8859-1",
content: {
op: 'OPERATION-TO-CALL,
password: 'super-sec,
user: 'me#gmail.com'
},
onComplete: function (response) {
if(response.status == 200){
callback(response.text);
}
else{
callback('');
}
}
});
req.post();
}
The API will return a XML struct encoded in ISO-8859-1. The returned data, in response.text, will contain Swedish characters like ö, ä and ö. Unfortunately characters like this will be displayed as �.
The html page I use in the Panel, where the text is displayed, look like this:
<!DOCTYPE html>
<html>
<head>
<meta charset="iso-8859-1">
….
I'm really stuck here, anyone have any idea on handling this encoding issue?
XMLHttpRequest (which Request is most likely based on) defaults to UTF-8 encoding. For a different encoding the response of your server needs to specify the charset in the Content-Type header, e.g.:
Content-Type: text/html; charset=iso-8859-1
This will make sure the response is converted to Unicode correctly. <meta> tags won't help.