localising postal / physical address display from database fields - localization

Can anyone point me to a list of international postal / residential / delivery address format templates that use some kind of parseable standard vocabulary for address parts?
The ideal list contains a country code then a format using replaceable tokens so I can substitute database address fields into a template to produce something printable in the local format.
for example
NZ | [first_name] [family_name]\n[company_name]\n[street_address]\n[city] [post_code]\n[country]
AU | [first_name] [family_name]\n[company_name]\n[street_address]\n[city]\n[state] [post_code]\n[country]
US | etc
UK | etc
Background: I used to have a simple freetext field to accept addresses. Moving to support vCard download, which requires addresses to be broken down into specific fields. Thats all fine: we can do the migration. I'm looking for a way to display the fields in the "correct" order for each country. thanks for your help!

This MSDN page has the information in the format you need and seems accurate, but covers only 33 countries. Maybe they are enough.
The Universal Postal Union offers all the information you need for a lot of countries here. This is top quality information; however, it is split across as many PDF documents as there are countries and is not in the format you need.
This page provides the information in a slightly more accessible form. As far as I can judge, it is accurate (and contains a lot of valuable info), but I can't speak to its quality nor its currentness.

Google have a JSON-based API that they use for their Android address input field library that contains this kind of formatting information.
The field you'd be interested in is fmt. There doesn't seem to be any formal documentation on the format they use, but a proposal to include this information as part of the Unicode CLDR has matching fields (scroll down to "Detailed Breakdown of elements"); there are also some clues in Google's libaddressinput source code.

Related

Parsing six figure Ordnance Survey Grid Reference input

An extensive search has produced no answer to the question, "Is there a class or function that parses input for soundness relating to UK Ordnance Survey Grid References".
The UK is mapped by the UK Ordnance Survey who produce detailed maps of the United Kingdom with many types of referencing. One of these is the commonly used six figure Grid Reference, we are at SO896804.
We already use a postcode (zip) checker to make sure that the information entered into the postcode field is sound, but we can't find the same for the OS Grid Reference.
Does such a Grid Reference function exist, or do we down tools and write one?
Thank you.
Since you tagged this "parsing", rather than using some GIS-like tag, I'd say that a reasonably valid OS grid reference corresponds to the regular expression:
(H[PTUWXYZ]|N[ABCDFGHJKLMNORQSTUWXYZ]|OV|S[CDEHJKMNOPRSTUVWXYZ]|T[AFGLMQRV])([0-9]{6})
If you were prepared to accept 4-digit 100ha blocks as well as the six-digit 1ha blocks, you could replace the second parenthesized expression with ([0-9]{4}|[0-9]{2}).
Of course, some of the two-letter blocks are almost completely marine. You could almost certainly ignore OV, for example. NW contains a little bit of Dumfries & Galloway (Portpatrick, NW995545), and a much larger but irrelevant part of Northern Ireland.

Information for localization

Is there a list with complete information about caracteristics like:
currency
date and time (including if it is 12 or 24 hours) format
measurement units (distance, speed, temperature...)
preferred language
masks for phone and local documents
timezones (at least the main ones / variations if daylight saving time is applicable)
decimal and thousand separators
for countries around the world?
I am doing it myself, however, as it takes too long to gather the data, I tought maybe someone have already have it done.
Don't reinvent the wheel.
Start with CLDR, the Common Locale Data Repository (http://cldr.unicode.org/)
Or if you want to honor the locale preferences in your application, use standard I18N APIs (from you platform, whatever that is, or a popular library, like ICU, http://site.icu-project.org/)
For currencies you can rely on international standard ISO 4217. It also refers to the country code of each currency code. This website provides this dataset for download.
For date formats, the best reference seems to be wikipedia.
The measurement units is a very complex domain, because you need to know which dimension you measure (speed, distance, volume, ...) and the units (paper size in cm is not the same as road distance in km). Here you have some lists per type of units, but not per country. This website shows a list of system of measurements in use per country. You'll see that fortunately ùany of them share the metric system, so taht you could use an approach "by exception" documenting yourself only on the remaining ones".
For languages, you have international standard ISO 639 or IANA , but it's country independent. You can look at reference lists for locale such as here: it associates a language code to a country code, so that you could complete the standard information. Note that some countries have several language, and you cannot and should not decide which one is preferred.
For telephone masks, there is only an international list of prefix. The usage vary greately accross countries. Some have fixed format, some use variable formats, some have zone prefixes and some not. Sometimes there is even no clear standard in the country and there are several coexisting usages. I'm not aware of any global list of these.
For timezones around the world, you could have a look at IANA which is extremely comprehensive.
For decimal and thousand separators, it's not an international standard. Again I'd suggest to refer to Wikipedia

Parsing a full name into its constituents

We are in need of developing a back end application that can parse a full name into
Prefix (Dr. Mr. Ms. etc)
First Name
Last Name
Middle Name
etc
Challenge here is that it has to support names of multiple countries and languages. One assumption that we have is we will always get a country and language along with the full name as input.
The full name may come in any format. For the same country / language combination, it may come in with first name last name or the reverse. Comma will not be a part of the Full Name.
Is is feasible? We are also open to any commercially available software.
I think this is impossible. Consider Ralph Vaughan Williams. His family name is "Vaughan Williams" and his first name is "Ralph". Contrast this with Charles Villiers Stanford, whose family name is "Stanford", with first name "Charles" and middle name "Villiers".
Both are English-speaking composers from England, so country and language information is not sufficient to establish the correct parsing logic.
Since the OP was open to any commercially available offering...
The "IBM InfoSphere Global Name Analytics" appears to be a commercial solution satisfying the original request for the parsing of a [free-form unstructured] personal name [full name]; apparently with a degree of certainty in regards to resolving some of the name ambiguity issues alluded to in other responses.Note: I have no personal experience nor association with the product, I had merely encountered this discussion and the following reference links while re-investigating effectively the same concern as described by the OP. HTH.
A general product documentation link:
http://publib.boulder.ibm.com/infocenter/gnrgna/v4r1m0/topic/com.ibm.gnr.gna.ic.doc/topics/gnr_gna_con_gnaoverview.html
Refer to the "Parsing names using NameParser" at
http://publib.boulder.ibm.com/infocenter/gnrgna/v4r1m0/topic/com.ibm.gnr.gna.ic.doc/topics/gnr_np_con_parsingnamesusingnameparser.html
The NameParser is a component API for the product per
http://publib.boulder.ibm.com/infocenter/gnrgna/v4r1m0/topic/com.ibm.gnr.gna.ic.doc/topics/gnr_gnm_con_logicalarchitecturecapis.html
Refer to the "Parsing names using IBM NameWorks" at
http://publib.boulder.ibm.com/infocenter/gnrgna/v4r1m0/topic/com.ibm.gnr.gna.ic.doc/topics/gnr_gnm_con_parsingnamesusingnameworks.html
"IBM NameWorks combines the individual IBM InfoSphere Global Name Recognition components into a single, unified, easy-to-use application programming interface (API), and also extends this functionality to Java applications and as a Web service"
http://publib.boulder.ibm.com/infocenter/gnrgna/v4r1m0/topic/com.ibm.gnr.gna.ic.doc/topics/gnr_gnm_con_logicalarchitecturenwapis.html
To clarify why I think this answers the question, ameliorating some of the previous alluded difficulties in accomplishing the task... If I understood correctly what I read, the APIs use the "NameHunter Server" to search the "IBM InfoSphere Global Name Data Archive (NDA)" which is described as "a collection of nearly one billion names from around the world, along with gender and country of association for each name. This large repository of name information powers the algorithms and rules that IBM InfoSphere Global Name Recognition products use to categorize, classify, parse, genderize , and match names."
FWiW I also ran across a "Name Parser" which uses a database of ~140K names as noted at:
http://www.melissadata.com/dqt/websmart-web-services.htm
The only reasonable approach is to avoid having to do so in the first place. The most obvious (and common) way to do that is to have the user enter the title, first/given name, last/family name, suffix, etc., separately from each other, rather than attempting to parse them out of a single string.
Ask yourself: do you really need the different parts of a name? Parsing names is inherently un-doable, since different cultures use different conventions (e.g. "middle name" is a typical USA-ism) and some small percentage of names will always be treated wrongly.
It is much preferable to treat a name as an "atomic" not-splittable entity.
Here are two free PHP name parsing libraries for those on a budget:
https://code.google.com/p/php-name-parser/
http://jasonpriem.org/human-name-parse/
And here is a Javasript library in Node package manager:
https://npmjs.org/package/name-parser
I wrote a simple human name parser in javascript as an npm module:
https://www.npmjs.org/package/humanparser
humanparser
Parse a human name string into salutation, first name, middle name, last name, suffix.
Install
npm install humanparser
Usage
var human = require('humanparser');
var fullName = 'Mr. William R. Jenkins, III'
, attrs = human.parseName(fullName);
console.log(attrs);
//produces the following output
{ saluation: 'Mr.',
firstName: 'William',
suffix: 'III',
lastName: 'Jenkins',
middleName: 'R.',
fullName: 'Mr. William R. Jenkins, III' }
A basic algorithm could do the following:
First see if incoming string starts with a title such as Mrs and remove it if it does, checking against a fixed list of titles.
If there is one space left and one space exactly, assume first word is first name and second word is surname (which will be incorrect at times)
To go beyond that would be lots of work, see How to parse full names to identify avenues for improvement and see these involved IBM docs for further implementation clues
"Ashton Jordan" "Jordan Ashton" -- u can't tell which is the surname and which is the give name.
Also people in South India apparently don't have a surname. The same with Sherpas in the Himalayas.
But say you have a huge list of all surnames (which are never used as given names) then maybe you can use that to identify other parts of the name (Salutations/Given/Middle/Jr/Sr/I/II/...) And if there is ambiguity your name-parser could ask for human input.
As others have explained, the problem is not solvable. The best approach I can think of to storing names is storing the full name, followed by the start (and potentially also ending) offsets into a "primary collating subfield" which the person entering the name could have indicated by highlighting it or such. For example
John Robert Miller, Jr.
where the boldface is indicating what was marked as the "primary collating subfield". This range would then be moved to the beginning of the string when generating the collating key.
Of course this approach alone may not be sufficient if you also want to support titles (and ignoring them for collation purposes)...

Guide to international postal address formats

Where can I find a guide to different postal address formats that are used in the major countries in the world?
For example, in the U.S. one format is:
street_number street_name street_type
city, state zipcode
But in Germany it might be:
street_name street_number
postcode city
I have not found any definitive resource (there may not be one), but have found a several, which do not always agree. The first two seem helpful--though they seem to focus on addressing mail from the US to other countries. The third, a book on MSDN, varied significantly from the others and may be out of date.
Address Doctor Address Formats
Columbia U. FRANK'S COMPULSIVE GUIDE TO POSTAL ADDRESSES
Developing International Software Appendix V International Address Formats
Lots of information here for many countries (first hit on google for "international postal address format")
Have a look at this Wikipedia page: http://en.wikipedia.org/wiki/Address_(geography)
There does seem to be a definitive resource here: Universal Postal Union Postal Addressing Systems in Member Countries.
Each country's postal service submits its addressing format to the Union Postale Universelle (Universal Postal Union). The Union has this information available for download and it is the only authoritative resource on the subject matter. The postal services around the world use it.
This material is available here: English; French.
If the links become unavailable, try searching on the Union's website for "addressing".

What products support 3-digit region subtags, e.g., es-419 for Latin-American Spanish?

What products support 3-digit region subtags, e.g., es-419 for Latin-American Spanish?
Are web browsers, translation tools and translators familiar with these numeric codes in addition to the more common "es" or "es-ES"?
I've already visited the following pages:
W3C Choosing a Language Tag
W3C Language tags in HTML and XML
RFC 5646 Tags for Identifying Languages
Microsoft National Language Support (NLS) API Reference
I doubt that many products like that exist. It seems that some main stream programming languages (I have tested C# and Java) does not support these tags, therefore it would be quite hard to develop programs that does so.
BTW. NLS API Reference that you have provided, does not contain region tag for any of the LCID definition. And if you think of it for the moment, knowing how Locale Identifier is built, there is no way to support it now, actually. Implementation change would be required (they should use some reserved bits, I suppose).
I don't think we will see support for region tags in foreseeable future.
Edit
I saw that Microsoft assigned LCID of value -1 and -2 to "European Union 1" and "European Union 2" respectively. However I don't think it is related.

Resources