The Problem:
I'm about to implement language localization to an already very large ipad application that was built using sencha touch wrapped in phonegap. I have the english and spanish translations in json files.
What I'm Planning on Doing:
I plan to load the json files into a sencha touch store, creating a global object. Then in each place where I call text that is displayed, i will replace the text with a call to the global object.
My question(s):
Is there an easier way to implement language localization with my
setup?
Will I run into issues with native sencha stuff (like datepickers)?
When loading/reloading language json files, will I have performance
issues (require a webview reload?, sencha object resizing issues,
etc)
edit 1 : Useful Related Info:
For those going down this road, it quickly becomes useful to write a simple phonegap plugin to get the ipad/iphone device's language setting into your javascript. This requires a plugin, which will look something like this:
Javascript :
part 1:
PhoneGap.exec("PixFileDownload.getSystemLanguage");
part 2(callback Function):
setLanguage(returnedLanguage)
{
GlobalVar.CurrentLanguage = returnedLanguage; //GloablVar.CurrentLanguage already defined
}
Objective C:
-(void)getSystemLanguage:(NSMutableArray*)paramArray withDict:(NSMutableDictionary*)options
{
/*Plugin Details
PhoneGap.exec("PixFileDownload.getSystemLanguage");
Returns Language Code
*/
NSUserDefaults* defs = [NSUserDefaults standardUserDefaults];
NSArray* languages = [defs objectForKey:#"AppleLanguages"];
NSString *language = [languages objectAtIndex:0];
NSLog(#"####### This is the language code%#",language);
NSString *jsCallBack;
jsCallBack = [NSString stringWithFormat:#"setLanguage('%#');",language];
[self.webView stringByEvaluatingJavaScriptFromString:jsCallBack];
}
edit 2: character encoding
When adding additional language characters to a sencha project (or any webview phonegap project), ensure that you have the proper encoding specified in the index file. This is the tag i needed.
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
I've finished this language localization plugin. It's not amazing, but it worked better than I originally speculated. Here are the short answers to each of the questions.
1- Is there an easier way to implement language localization with my
setup?
Not that i'm aware of. The comment by Stuart provided this link Sencha-touch localization. Use a store or a global JSON object? which had some good information on one way that you can use class overrides. I didn't like this approach, because it spread my language translations into different classes. But certainly, if you are doing something simple, or you want something that could be more powerful, perhaps you should investigate that.
2- Will I run into issues with native sencha stuff (like datepickers)?
I ended up specifically leaving "datepickers" in english for now. But everything else was really relatively easy to customize. Almost every graphical UI element can have it's text altered.
3- When loading/reloading language json files, will I have performance
issues (require a webview reload?, sencha object resizing issues,
etc).
The method that i employed (see below) worked exceptionally well in regards to performance. The one issue that you have is right when you switch the languages, you need that specific page to reload. Sencha handled resizing without any flaws, except where I had been foolish and statically set sizes.
Some of what I did was described in edits to the question. Here is a detailed overview of my solution. (warning, it's not the most elegant solution.)
Instead of using a pure JSON file, I ended up just using a javascript function. This isn't the greatest solution because it requires some minimal maintenance, but JSON parsing with phonegap/sencha isn't the best. (I get JSON files from translater's, and quickly paste into the javascript file. Takes around 2 minutes, see further explanation below).
Language.js
function setLanguage(language)
{
if(language == "en")
{
//console.log("inside if Language == en");
GlobalLanguage.CurrentLanguage = language;
GlobalLanguage.ID = {"glossary": [
{
//CONVERTED JSON
about : 'About',
checking_for_updates : 'Checking for updates...(This may take a few minutes.)'
//Any additional translations
}
]};
}
if (language == "es"){
//console.log("inside language == es");
GlobalLanguage.CurrentLanguage = language;
GlobalLanguage.ID = {"glossary": [
{
//CONVERTED JSON
about : 'Acerca de ',
checking_for_updates : 'Verificando actualizaciones... (Capas que demore algunos minutos).'
//Any additional translations
}]};
}
if (language == "pt"){
//console.log("inside language == pt");
GlobalLanguage.CurrentLanguage = language;
GlobalLanguage.ID = {"glossary": [
{
//CONVERTED JSON
about : 'Sobre',
checking_for_updates : 'Verificando se há atualizações... (pode demorar alguns minutos.)'
//Any additional translations
}]};
}
}
As you can see, this file allows for 3 languages (Portugese, English, and Spanish). After setting the language, you could access each localized string anywhere in your object. For example, if you need to access the word "about" simply use:
GlobalLanguage.ID.glossary[0]["about"]
This will access the GlobalLanguage object which will have whatever language loaded into the properties. So throughout your project, you could have these calls. However, I'd recommend taking it one step further
function langSay(languageIdentifier){
// console.log("inside langSay");
if(!GlobalLanguage.ID.glossary[0][languageIdentifier]){
return "[! LANGUAGE EXCEPTION !]";
}
else{
return GlobalLanguage.ID.glossary[0][languageIdentifier];
}
}
This protects you from having language exceptions and having your program crash without knowing where (you may have hundreds or thousands of properties being set in that language.js file). So now simply :
langSay("about")
One additional note about formatting from JSON. The format you want your language files in is:
languageIdentifier : 'Translation',
languageIdentifier : 'Translation',
languageIdentifier : 'Translation'
I used Excel for the formatting. Also the languageIdentifiers are unique identifiers without spaces. I recommend just using Excel to format first 3 to 4 words word1_word2_word3_word4 of the english translation.
word1_word2_word3 : 'word1 word2 word3'
Hope this helps you out! I'd be happy to respond to any questions
Related
The wiktionary entry for faint lies at https://en.wiktionary.org/wiki/faint
The wikitext for the etymology section is:
From {{inh|en|enm|faynt}}, {{m|enm|feynt||weak; feeble}}, from
{{etyl|fro|en}} {{m|fro|faint}}, {{m|fro|feint||feigned; negligent;
sluggish}}, past participle of {{m|fro|feindre}}, {{m|fro|faindre||to
feign; sham; work negligently}}, from {{etyl|la|en}}
{{m|la|fingere||to touch, handle, usually form, shape, frame, form in
thought, imagine, conceive, contrive, devise, feign}}.
It contains various templates of the form {{xyz|...}}
I would like to parse them and get the text output as it shows on the page:
From Middle English faynt, feynt (“weak; feeble”), from Old French
faint, feint (“feigned; negligent; sluggish”), past participle of
feindre, faindre (“to feign; sham; work negligently”), from Latin
fingere (“to touch, handle, usually form, shape, frame, form in
thought, imagine, conceive, contrive, devise, feign”).
I have about 10000 entries extracted from the freely available dumps of wiktionary here.
To do this, my thinking is to extract templates and their expansions (in some form). To explore the possibilites I've been fiddling with the lua scripting facility on mediawiki. By trying various queries inside the debug console on edit pages of modules, like here:
https://en.wiktionary.org/w/index.php?title=Module:languages/print&action=edit
mw.log(p)
>> table
mw.logObject(p)
>> table#1 {
["code_to_name"] = function#1,
["name_to_code"] = function#2,
}
p.code_to_name("aaa")
>>
p.code_to_name("ab")
>>
But, I can't even get the function calls right. p.code_to_name("aaa") doesn't return anything.
The code that presumably expands the templates for the etymology section is here:
https://en.wiktionary.org/w/index.php?title=Module:etymology/templates
How do I call this code correctly?
Is there a simpler way to achieve my goal of parsing wikitext templates?
Is there some function available in mediawiki that I can call like "parse-wikitext("text"). If so, how do I invoke it?
To expand templates (and other stuff) in wikitext, use frame.preprocess, which is called as a method on a frame object. To get a frame object, use mw.getCurrentFrame. For instance, type = mw.getCurrentFrame():preprocess('{{l|en|word}}') in the console to get the wikitext resulting from {{l|en|word}}. That currently gives <span class="Latn" lang="en">[[word#English|word]]</span>.
You can also use the Expandtemplates action in the MediaWiki API ( https://en.wiktionary.org/w/api.php?action=expandtemplates&text={{l|en|word}}), or the Special:ExpandTemplates page, or JavaScript (if you open the browser console while browsing a Wiktionary page):
new mw.Api().get({
action: 'parse',
text: '{{l|en|word}}',
title: mw.config.values.wgPageName,
}).done(function (data) {
const wikitext = data.parse.text['*'];
if (wikitext)
console.log(wikitext);
});
If the mw.api library hasn't already been loaded and you get a TypeError ("mw.Api is not a constructor"):
mw.loader.using("mediawiki.api", function() {
// Use mw.Api here.
});
So these are some of the ways to expand templates.
I'm working on a Grails / Backbone / Handlebars application that's a front end to a much larger legacy Java system in which (for historical & customizability reasons) internationalization messages are deep in a database hidden behind a couple of SOAP services which are in turn hidden behind various internal Java libraries. Getting at these messages from the Grails layer is easy and works fine.
What I'm wondering, though, is how to get (for instance) internationalized labels into my Handlebars templates.
Right now, I'm using GSP fragments to generate the templates, including a custom tag that gets the message I'm interested in, something like:
<li><myTags:message msgKey="title"/> {{title}}</li>
However, for performance and code layout reasons I want to get away from GSP templates and get them into straight HTML. I've looked a little into client-side internationalization options such as i18n.js, but they seem to depend on the existence of a messages file I haven't got. (I could generate it, possibly, but it would be ginormous and expensive.)
So far the best thing I can think of is to wedge the labels into the Backbone model as well, so I'd end up with something like
<li>{{titleLabel}} {{title}}</li>
However, this really gets away from the ideal of building the Backbone models on top of a nice clean RESTful JSON API -- either the JSON returned by the RESTful service is cluttered up with presentation data (i.e., localized labels), or I have to do additional work to inject the labels into the Backbone model -- and cluttering up the Backbone model with presentation data seems wrong as well.
I think what I'd like to do, in terms of clean data and clean APIs, is write another RESTful service that takes a list of message keys and similar, and returns a JSON data structure containing all the localized messages. However, questions remain:
What's the best way to indicate (probably in the template) what message keys are needed for a given view?
What's the right format for the data?
How do I get the localized messages into the Backbone views?
Are there any existing Javascript libraries that will help, or should I just start making stuff up?
Is there a better / more standard alternative approach?
I think you could create quite an elegant solution by combining Handelbars helpers and some regular expressions.
Here's what I would propose:
Create a service which takes in a JSON array of message keys and returns an JSON object, where keys are the message keys and values are the localized texts.
Define a Handlebars helper which takes in a message key (which matches the message keys on the server) and outputs an translated text. Something like {{localize "messageKey"}}. Use this helper for all template localization.
Write a template preprocessor which greps the message keys from a template and makes a request for your service. The preprocessor caches all message keys it gets, and only requests the ones it doesn't already have.
You can either call this preprocessor on-demand when you need to render your templates, or call it up-front and cache the message keys, so they're ready when you need them.
To optimize further, you can persist the cache to browser local storage.
Here's a little proof of concept. It doesn't yet have local storage persistence or support for fetching the texts of multiple templates at once for caching purposes, but it was easy enough to hack together that I think with some further work it could work nicely.
The client API could look something like this:
var localizer = new HandlebarsLocalizer();
//compile a template
var html = $("#tmpl").html();
localizer.compile(html).done(function(template) {
//..template is now localized and ready to use
});
Here's the source for the lazy reader:
var HandlebarsLocalizer = function() {
var _templateCache = {};
var _localizationCache = {};
//fetches texts, adds them to cache, resolves deferred with template
var _fetch = function(keys, template, deferred) {
$.ajax({
type:'POST',
dataType:'json',
url: '/echo/json',
data: JSON.stringify({
keys: keys
}),
success: function(response) {
//handle response here, this is just dummy
_.each(keys, function(key) { _localizationCache[key] = "(" + key + ") localized by server"; });
console.log(_localizationCache);
deferred.resolve(template);
},
error: function() {
deferred.reject();
}
});
};
//precompiles html into a Handlebars template function and fetches all required
//localization keys. Returns a promise of template.
this.compile = function(html) {
var cacheObject = _templateCache[html],
deferred = new $.Deferred();
//cached -> return
if(cacheObject && cacheObject.ready) {
deferred.resolve(cacheObject.template);
return deferred.promise();
}
//grep all localization keys from template
var regex = /{{\s*?localize\s*['"](.*)['"]\s*?}}/g, required = [], match;
while((match = regex.exec(html))) {
var key = match[1];
//if we don't have this key yet, we need to fetch it
if(!_localizationCache[key]) {
required.push(key);
}
}
//not cached -> create
if(!cacheObject) {
cacheObject = {
template:Handlebars.compile(html),
ready: (required.length === 0)
};
_templateCache[html] = cacheObject;
}
//we have all the localization texts ->
if(cacheObject.ready) {
deferred.resolve(cacheObject.template);
}
//we need some more texts ->
else {
deferred.done(function() { cacheObject.ready = true; });
_fetch(required, cacheObject.template, deferred);
}
return deferred.promise();
};
//translates given key
this.localize = function(key) {
return _localizationCache[key] || "TRANSLATION MISSING:"+key;
};
//make localize function available to templates
Handlebars.registerHelper('localize', this.localize);
}
We use http://i18next.com for internationalization in a Backbone/Handlebars app. (And Require.js which also loads and compiles the templates via plugin.)
i18next can be configured to load resources dynamically. It supports JSON in a gettext format (supporting plural and context variants).
Example from their page on how to load remote resources:
var option = {
resGetPath: 'resources.json?lng=__lng__&ns=__ns__',
dynamicLoad: true
};
i18n.init(option);
(You will of course need more configuration like setting the language, the fallback language etc.)
You can then configure a Handlebars helper that calls i18next on the provided variable (simplest version, no plural, no context):
// namespace: "translation" (default)
Handlebars.registerHelper('_', function (i18n_key) {
i18n_key = Handlebars.compile(i18n_key)(this);
var result = i18n.t(i18n_key);
if (!result) {
console.log("ERROR : Handlebars-Helpers : no translation result for " + i18n_key);
}
return new Handlebars.SafeString(result);
});
And in your template you can either provide a dynamic variable that expands to the key:
<li>{{_ titleLabeli18nKey}} {{title}}</li>
or specify the key directly:
<li>{{_ "page.fancy.title"}} {{title}}</li>
For localization of datetime we use http://momentjs.com (conversion to local time, formatting, translation etc.).
I'm using the Laravel Lang class for localization of my web app. I've added two languages to the languages array in application/config/application.php. This changes the default language it uses for localization to whatever the first part of the URI indicates (e.g. bla.com/en/bla and bla.com/co/bla). Now I need to be able to check what the current default language is in my view. However, the Lang class provides no way of checking this as far as I've been able to figure out, as the Lang::$language variable is protected. Is there any way of checking this apart from manually parsing the URI?
The cleanest way to know the current language of your website in Laravel appears to be :
Lang::locale();
https://laravel.com/api/5.8/Illuminate/Translation/Translator.html#method_locale
It's different than this command that will return the default language of your website :
Config::get('app.locale');
An alternative, a bit shorter way could be using something like this:
app()->getLocale()
The advantage of this is that IDEs such as PHPStorm recognize this function and can help you develop much faster.
BenjaminRH's answer is very good, and his suggested approach works perfectly. I've improved the snippet a bit. Now it detects the browser language and checks if that language is supported according to the application's config file.
It's a quick hack, but it works on my app. Note that the application language is also set now. Feel free to use ore improve it.
Route::filter('before', function()
{
// current uri language ($lang_uri)
$lang_uri = URI::segment(1);
// Set default session language if none is set
if(!Session::has('language'))
{
// use lang in uri, if provided
if(in_array($lang_uri, Config::get('application.languages')))
{
$lang = $lang_uri;
}
// detect browser language
elseif(isset(Request::server('http_accept_language')))
{
$headerlang = substr(Request::server('http_accept_language'), 0, 2);
if(in_array($headerlang, Config::get('application.languages')))
{
// browser lang is supported, use it
$lang = $headerlang;
}
// use default application lang
else
{
$lang = Config::get('application.language');
}
}
// no lang in uri nor in browser. use default
else
{
// use default application lang
$lang = Config::get('application.language');
}
// set application language for that user
Session::put('language', $lang);
Config::set('application.language', $lang);
}
// session is available
else
{
// set application to session lang
Config::set('application.language', Session::get('language'));
}
// prefix is missing? add it
if(!in_array($lang_uri, Config::get('application.languages')))
{
return Redirect::to(URI::current());
}
// a valid prefix is there, but not the correct lang? change app lang
elseif(in_array($lang_uri, Config::get('application.languages')) AND $lang_uri != Config::get('application.language'))
{
Session::put('language', $lang_uri);
Config::set('application.language', $lang_uri);
}
});
In the newer Laravel versions, you can get the current language with:
Config::get('app.locale');
This would work fine
lang="{{ app()->getLocale() }}"
I've figured out a solution to the language problem (thanks to nickstr on the IRC and the accepted answer to this question). It involves storing the current language as a session variable, which is updated when the language uri segment is changed.
Route::filter('before', function()
{
// Do stuff before every request to your application...
// Default language ($lang) & current uri language ($lang_uri)
$lang = 'he';
$lang_uri = URI::segment(1);
// Set default session language if none is set
if(!Session::has('language'))
{
Session::put('language', $lang);
}
// Route language path if needed
if($lang_uri !== 'en' && $lang_uri !== 'he')
{
Return Redirect::to($lang.'/'.URI::current());
}
// Set session language to uri
elseif($lang_uri !== Session::get('language'))
{
Session::put('language', $lang_uri);
}
});
This might help.
Config::get('application.language')
You can use
https://github.com/mcamara/laravel-localization
Laravel Localization uses the URL given for the request. In order to achieve this purpose, a group should be added into the routes.php file. It will filter all pages that must be localized.
// app/routes.php
Route::group(array('prefix' => LaravelLocalization::setLanguage()), function()
{
/** ADD ALL LOCALIZED ROUTES INSIDE THIS GROUP **/
Route::get('/', function()
{
return View::make('hello');
});
Route::get('test',function(){
return View::make('test');
});
});
/** OTHER PAGES THAT SHOULD NOT BE LOCALIZED **/
Once this group is added to the routes file, an user can access to all languages added into the 'languagesAllowed' ('en' and 'es' for default, look at the config section to change that option). For example, an user can now access to two different languages, using the following addresses:
http://laravel.com/en
http://laravel.com/es
http://laravel.com
I use App::getLocale() which is probably the most supported way as the App::setLocale('EN') method is used in the documentation.
You can use this method everywhere. If it throughs an error somewhere, you can use \App::... to make it work.
I'm using Laravel 5.0.
Your can get current language in laravel blade by:
{{Lang::locale()}}
The Lang class is specifically for outputting the correct language and as you say manages the language internally.
Looking through the API there is no method to help you directly with this and parsing the URI to get the language would seem the appropriate course of action.
You can always just do this to retrieve the language string in the URI:
$language = URI::segment(1);
Examining Laravel Requests
First, apologies as I realize this is only tangentially related to parser programming.
I've spend hours looking for a text file containing something like the following but with hundreds (hopefully thousands) of sub-entries. A complete biological classification file would be perfect. A massive version of the following would be great as my parser parses simple tabbed files:
TL,DR - I need a massive single-file hierarchical data set something like the following:
Kindoms
Monera
Protista
Fungi
Plants
Animals
Porifera
Sponges
Coelenterates
Hydra
Coral
Jellyfish
Platyhelminthes
Flatworms
Flukes
Nematodes
Roundworms
Tapeworms
Chordates
Urochordataes
Cephalochordates
Vertebrates
Fish
Amphibians
Reptiles
Birds
Mammals
The best I've been able to find are tree-of-life images (from which I transcribed the sample data set above). A single file with a TON of real data would be awesome. It doesn't have to be a biological classification data set, but I would really like the data to reflect something in the real-world. (My parser feeds a menu - would be great if the remainder of my testing was with a data set that actually meant something!) Even if the file is not tabbed but the data was fairly easily regex'ed to a tabbed format... that would be great.
Any ideas? Thanks!
It is possible that the xml layout was changed since the last answer but the code submitted above is no longer accurate. The resulting dump is extraneous. Some of the nodes have aliases (denoted as 'othername') that are reported as distinct nodes themselves.
I used the script below to generate the correct dump.
<?php
$reader = new XMLReader();
$reader->open('http://tolweb.org/onlinecontributors/app?service=external&page=xml/TreeStructureService&node_id=1'); //15963 is the primates index
$set=-1;
while ($reader->read()) {
switch ($reader->nodeType) {
case (XMLREADER::ELEMENT):
if ($reader->name == "OTHERNAMES"){
$set=1;
}
if ($reader->name == "NODES"){
$set=-1;
}
if ($reader->name == "NODE"){
$set=-1;
}
if ($reader->name == "NAME" AND $set == -1){
echo str_repeat("\t", $reader->depth - 2); //repeat tabs for depth
$node = $reader->expand();
echo $node->textContent . "\n";
}
break;
}
}
?>
This turned out to be such a pain in the ass. I finally tracked down a data feed from "The Tree of Life Web Project" at tolweb.org. I made the php script below to provide the basic functionality my post was looking for.
Change the node_id to have it print a tabbed representation of any of tolweb.org's data - just take the id from the page you're browsing on their site and change the node_id below.
Be aware though - their data feeds serve up large files, so definitely download the file to your own server (and change the "open" method below to point to the local file) if you're going to hit it more than once or twice.
More info on tolweb.org data feeds can be found here:
http://tolweb.org/tree/home.pages/downloadtree.html
<?php
$reader = new XMLReader();
$reader->open('http://tolweb.org/onlinecontributors/app?service=external&page=xml/TreeStructureService&node_id=15963'); //15963 is the primates index
while ($reader->read()) {
switch ($reader->nodeType) {
case (XMLREADER::ELEMENT):
if ($reader->name == "NAME"){
echo str_repeat("\t", $reader->depth - 2); //repeat tabs for depth
$node = $reader->expand();
echo $node->textContent . "\n";
}
break;
}
}
?>
is it possible to get the value of an entity
<!ENTITY gatwayError "Gateway error">
using javascript? For now I reference them in my xul file using
&gatewayError;
UPDATE: In my ff-sidebar.xul within the <page> I have
<stringbundleset id="stringbundleset">
<stringbundle id="strings"
src="chrome://myaddon/locale/de/sidebar.properties"/>
</stringbundleset>
In my ff-sidebar.js I do on click:
var strbundle = document.getElementById("strings");
var localizedString = strbundle.getString("test");
This gives me following error
Should it not be
var strbundle = document.getElementById("stringbundleset");
This gives me no error but no result too.
Basically what Neil posted there is what you need to do (minus first paragraph rant :P )
Here's an example (basically digest from Neil's links):
Your XUL file:
<stringbundleset id="strbundles">
<stringbundle id="strings" src="chrome://yourextension/locale/something.properties"/>
</stringbundleset>
Your something.properties (there you define your localized strings key=value). Of course you can have as many files as you want/need:
something=Some text for localization
something2=Some more text
Your js file:
var strbundle = document.getElementById("strings");
var localizedString = strbundle.getString("something");
Hope this helps.
This works for small numbers of entities. For instance, menuitems sometimes have two entities with slightly different text depending on what the menuitem will be used for, and the correct entity is then copied to the label. The worst abuse of this was for the Delete menuitem in Thunderbird and SeaMonkey's mail windows, which had labels for unsubscribing from newsgroups, deleting folders, cancelling news posts, deleting single or multiple messages, or undeleting single or multiple messages from folders using the IMAP mark as delete model. Phew!
If you have lots of locale data then the best thing is to put it in its own .properties file and read it using a <stringbundle>. If your script doesn't have access to a <stringbundle> element it is also possible to manually retrieve an nsIStringBundle from the nsIStringBundleService.