BlackBerry maps location document with multiple points - blackberry

When I implement BlackBerry maps in my application using the code from RIM in How To - Invoke BlackBerry Maps, my map works fine with Ontario points, but when I use
Latitude 13.083333 and Longitude: 80.283333, it does not work.
What about the dot? On skipping the dot, it shows the incorrect position.
String document = "<lbs>
<location lon='-8030000' lat='4326000' label='Kitchener, ON' description='Kitchener, Ontario, Canada' />
<location lon='-7569792' lat='4542349' label='Ottawa, ON' description='Ottawa, Ontario, Canada' />
";


The right way would be to multiply LAT & LONG by 100,000 and cast it to an integer value,
something like this:
// LAT & LONG times 100,000, converted to integer
int lon = (int) (80.283333 * 100000);
int lat = (int) (13.083333 * 100000);
String label = "Label of point on map";
String desc = "Description of point when clicked";
// create the location element
StringBuffer sb = new StringBuffer();
sb.append("<location");
sb.append(" lon='").append(String.valueOf(lon)).append("'");
sb.append(" lat='").append(String.valueOf(lat)).append("'");
sb.append(" label='").append(label).append("'");
sb.append(" description='").append(desc).append("'");
sb.append(" />");

Related

Find people who were near to the user's last location

I'm trying to find people who were near to a user's last location in last 24 hours for a simple recommendation engine. My data set is simple for test purposes:
What is wrong with my query? It's returning user Ayse multiple times, but I don't want same person twice or more.
Here is my queries:
var neo4j = require('neo4j-driver').v1;
var driver = neo4j.driver("bolt://localhost", neo4j.auth.basic("neo4j", "neo4j"));
var session = driver.session();
function runQuery(query) {
session.run(query, {}).then(function(result){
console.log("finish")
result.records.forEach(function(record) {
console.log(record._fields)
})
}).catch(console.log);
}
//runQuery("MATCH (n) DETACH DELETE n" )
function newUser(user) {
runQuery("CREATE (:User{name:'"+user+"'})")
}
function newLocation(user, lat, lon) {
var q =
"MATCH (u:User{name:'"+user+"'}) "+
"CREATE (l:Location{lat: "+lat+", lon: "+lon+", created_at: TIMESTAMP()}) "+
"CREATE (u)-[:HAVE_BEEN]->(l) "
runQuery(q)
}
/*
newUser("Ozgur")
newUser("Fatma")
newUser("Ayse")
*/
/*
newLocation("Ozgur", 38.134972, 26.96681)
newLocation("Ozgur", 37.239972, 25.96681)
newLocation("Ozgur", 38.334972, 16.96681)
*/
/*
newLocation("Ayse", 38.294972, 26.76681)
newLocation("Ayse", 37.639972, 25.66681)
newLocation("Ayse", 35.134972, 18.96681)
*/
/*
newLocation("Fatma", 31.114972, 21.76681)
newLocation("Fatma", 31.139972, 21.66681)
*/
function findUserRecommendation(user) {
// get user recommendations
var q =
"MATCH "+
//get user locations for last 24 hours (me_loc)
"(me:User{name:'"+user+"'})-[:HAVE_BEEN]->(me_loc:Location), " +
// strangers (recommendations)
"(l:Location)<-[:HAVE_BEEN]-(stranger:User) " +
"WHERE "+
//except me
"NOT me = stranger "+
// nearest location algorithm based on KM
"AND 2 * 6371 * "+
"asin("+
"sqrt("+
"haversin(radians(me_loc.lat - l.lat)) + "+
"cos(radians(me_loc.lat)) * "+
"cos(radians(l.lat)) * haversin(radians(me_loc.lon - l.lon))"+
")"+
// 500 km
") < 500 " +
// last 24 hours of they location
//"AND l.created_at > (TIMESTAMP() - 604800) "+
// last 24 hours of user's location
//"AND me_loc.created_at > (TIMESTAMP() - 604800)" +
// return every stranger's name and last location once
// but returning Ayse's name and different location more than once
"RETURN DISTINCT stranger.name, l.lat, l.lon"
runQuery(q)
}
findUserRecommendation("Ozgur")

DISTINCT outside of an aggregation just ensures that you won't get duplicate result rows, it doesn't limit you to the first row for each DISTINCT element (it actually applies to all non-aggregated elements in the row as a unit, like using a tuple as a key). You'll have to aggregate the lat/long and then figure out which one of them to keep. Assuming you just want any single lat/long, try this in place of your return line:
WITH stranger, COLLECT(l) AS locs
WITH stranger, HEAD(locs) AS l
RETURN stranger.name, l.lat, l.lon
You can replace the HEAD() call with a REDUCE() if you want to pick out a single lat/long based on particular criteria, like most recent or closest.

SQL CLR User Defined Function (C#) adds null character (\0) in between every existing character in String being returned

This one has kept me stumped for a couple of days now.
It's my first dabble with CLR & UDF ...
I have created a user defined function that takes a multiline String as input, scans it and replaces a certain line in the string with an alternative if found. If it is not found, it simply appends the desired line at the end. (See code)
The problem, it seems, comes when the final String (or Stringbuilder) is converted to an SqlString or SqlChars. The converted, returned String always contains the Nul character as every second character (viewing via console output, they are displayed as spaces).
I'm probably missing something fundamental on UDF and/or CLR.
Please Help!!
Code (I leave in the commented Stringbuilder which was my initial attempt... changed to normal String in a desperate attempt to find the issue):
[Microsoft.SqlServer.Server.SqlFunction]
[return: SqlFacet(MaxSize = -1, IsFixedLength = false)]
//public static SqlString udf_OmaChangeJob(String omaIn, SqlInt32 jobNumber) {
public static SqlChars udf_OmaChangeJob(String omaIn, SqlInt32 jobNumber) {
if (omaIn == null || omaIn.ToString().Length <= 0) return new SqlChars("");
String[] lines = Regex.Split(omaIn.ToString(), "\r\n");
Regex JobTag = new Regex(#"^JOB=.+$");
//StringBuilder buffer = new StringBuilder();
String buffer = String.Empty;
bool matched = false;
foreach (var line in lines) {
if (!JobTag.IsMatch(line))
//buffer.AppendLine(line);
buffer += line + "\r\n";
else {
//buffer.AppendLine("JOB=" + jobNumber);
buffer += ("JOB=" + jobNumber + "\r\n");
matched = true;
}
}
if (!matched) //buffer.AppendLine("JOB=" + jobNumber);
buffer += ("JOB=" + jobNumber) + "\r\n";
//return new SqlString(buffer.ToString().Replace("\0",String.Empty)) + "blablabla";
// buffer = buffer.Replace("\0", "|");
return new SqlChars(buffer + "\r\nTheEnd");
}

I know in my experiences, the omaIn parameter should be of type SqlString and when you go to collect its value/process it, set a local variable:
string omaString = omaIn != SqlString.Null ? omaIn.Value : string.empty;
Then when you return on any code path, to rewrap the string in C#, you'd need to set
return omaString == string.empty ? new SqlString.Null : new SqlString(omaString);
I have had some fun wrestling matches learning the intricate hand-off between local and outbound types, especially with CLR TVFs.
Hope that can help!

Lucene.net proximity search

Does anybody have any experience with having lucene.net index latitude and longitude values then return an ordered set of results based on distance from a single point?
Will the Lucene.Net.Spatial library help me at all with this?

A little late to the party but yes, the Spatial library is the place to start with this. The basics behind it are to:
1) Add Lat and Long fields to your document
doc.Add(new Field("Latitude",
NumericUtils.DoubleToPrefixCoded(Latitude),
Field.Store.YES, Field.Index.NOT_ANALYZED));
doc.Add(new Field("Longitude",
NumericUtils.DoubleToPrefixCoded(Longitude),
Field.Store.YES, Field.Index.NOT_ANALYZED));
2) Create plotters for the each tier of granularity that your search needs to support
IProjector projector = new SinusoidalProjector();
var ctp = new CartesianTierPlotter(0, projector,
Fields.LocationTierPrefix);
StartTier = ctp.BestFit(MaxKms);
EndTier = ctp.BestFit(MinKms);
Plotters = new Dictionary<int, CartesianTierPlotter>();
for (var tier = StartTier; tier <= EndTier; tier++)
{
Plotters.Add(tier, new CartesianTierPlotter(tier,
projector,
Fields.LocationTierPrefix));
}
3) Use your plotters to index your document for each tier
private static void AddCartesianTiers(double latitude,
double longitude,
Document document)
{
for (var tier = StartTier; tier <= EndTier; tier++)
{
var ctp = Plotters[tier];
var boxId = ctp.GetTierBoxId(latitude, longitude);
document.Add(new Field(ctp.GetTierFieldName(),
NumericUtils.DoubleToPrefixCoded(boxId),
Field.Store.YES,
Field.Index.NOT_ANALYZED_NO_NORMS));
}
}
With your document indexed you can move onto building a query. This example uses a ConstantScoreQuery but you can swap that out for your ranged scoring:
/* Builder allows us to build a polygon which we will use to limit
* search scope on our cartesian tiers, this is like putting a grid
* over a map */
var builder = new CartesianPolyFilterBuilder(Fields.LocationTierPrefix);
/* Bounding area draws the polygon, this can be thought of as working
* out which squares of the grid over a map to search */
var boundingArea = builder.GetBoundingArea(Latitude,
Longitude,
DistanceInKilometres * ProductSearchEngine.KmsToMiles);
/* We refine, this is the equivalent of drawing a circle on the map,
* within our grid squares, ignoring the parts the squares we are
* searching that aren't within the circle - ignoring extraneous corners
* and such */
var distFilter = new LatLongDistanceFilter(boundingArea,
DistanceInKilometres * KmsToMiles,
Latitude,
Longitude,
ProductSearchEngine.Fields.Latitude,
ProductSearchEngine.Fields.Longitude);
/* We add a query stating we will only search against products that have
* GeoCode information */
var query = new TermQuery(new Term(Fields.HasGeoCode,
FieldFlags.HasField));
/* Add our filter, this will stream through our results and
* determine eligibility */
masterQuery.Add(new ConstantScoreQuery(distanceFilter),
BooleanClause.Occur.MUST);
All of this is taken from a blog post I just wrote whilst looking at a similar problem. You can see it at http://www.leapinggorilla.com/Blog/Read/1005/spatial-search-in-lucenenet

How to parse freeform street/postal address out of text, and into components

We do business largely in the United States and are trying to improve user experience by combining all the address fields into a single text area. But there are a few problems:
The address the user types may not be correct or in a standard format
The address must be separated into parts (street, city, state, etc.) to process credit card payments
Users may enter more than just their address (like their name or company with it)
Google can do this but the Terms of Service and query limits are prohibitive, especially on a tight budget
Apparently, this is a common question:
PHP script to parse address?
How do I parse the free format address to save into the DataBase
java postal address parser
More efficient way to extract address components
How can i show a pre populated postal address in contacts screen with street, city, zip on android
PHP regexp US address
Is there a way to isolate an address from the text around it and break it into pieces? Is there a regular expression to parse addresses?

I saw this question a lot when I worked for an address verification company. I'm posting the answer here to make it more accessible to programmers who are searching around with the same question. The company I was at processed billions of addresses, and we learned a lot in the process.
First, we need to understand a few things about addresses.
Addresses are not regular
This means that regular expressions are out. I've seen it all, from simple regular expressions that match addresses in a very specific format, to this:
/\s+(\d{2,5}\s+)(?![a|p]m\b)(([a-zA-Z|\s+]{1,5}){1,2})?([\s|,|.]+)?(([a-zA-Z|\s+]{1,30}){1,4})(court|ct|street|st|drive|dr|lane|ln|road|rd|blvd)([\s|,|.|;]+)?(([a-zA-Z|\s+]{1,30}){1,2})([\s|,|.]+)?\b(AK|AL|AR|AZ|CA|CO|CT|DC|DE|FL|GA|GU|HI|IA|ID|IL|IN|KS|KY|LA|MA|MD|ME|MI|MN|MO|MS|MT|NC|ND|NE|NH|NJ|NM|NV|NY|OH|OK|OR|PA|RI|SC|SD|TN|TX|UT|VA|VI|VT|WA|WI|WV|WY)([\s|,|.]+)?(\s+\d{5})?([\s|,|.]+)/i
... to this where a 900+ line-class file generates a supermassive regular expression on the fly to match even more. I don't recommend these (for example, here's a fiddle of the above regex, that makes plenty of mistakes). There isn't an easy magic formula to get this to work. In theory and by theory, it's not possible to match addresses with a regular expression.
USPS Publication 28 documents the many formats of addresses that are possible, with all their keywords and variations. Worst of all, addresses are often ambiguous. Words can mean more than one thing ("St" can be "Saint" or "Street") and there are words that I'm pretty sure they invented. (Who knew that "Stravenue" was a street suffix?)
You'd need some code that really understands addresses, and if that code does exist, it's a trade secret. But you could probably roll your own if you're really into that.
Addresses come in unexpected shapes and sizes
Here are some contrived (but complete) addresses:
1) 102 main street
Anytown, state
2) 400n 600e #2, 52173
3) p.o. #104 60203
Even these are possibly valid:
4) 829 LKSDFJlkjsdflkjsdljf Bkpw 12345
5) 205 1105 14 90210
Obviously, these are not standardized. Punctuation and line breaks not guaranteed. Here's what's going on:
Number 1 is complete because it contains a street address and a city and state. With that information, there's enough to identify the address, and it can be considered "deliverable" (with some standardization).
Number 2 is complete because it also contains a street address (with secondary/unit number) and a 5-digit ZIP code, which is enough to identify an address.
Number 3 is a complete post office box format, as it contains a ZIP code.
Number 4 is also complete because the ZIP code is unique, meaning that a private entity or corporation has purchased that address space. A unique ZIP code is for high-volume or concentrated delivery spaces. Anything addressed to ZIP code 12345 goes to General Electric in Schenectady, NY. This example won't reach anyone in particular, but the USPS would still be able to deliver it.
Number 5 is also complete, believe it or not. With just those numbers, the full address can be discovered when parsed against a database of all possible addresses. Filling in the missing directionals, secondary designator, and ZIP+4 code is trivial when you see each number as a component. Here's what it looks like, fully expanded and standardized:
205 N 1105 W Apt 14
Beverly Hills CA 90210-5221
Address data is not your own
In most countries that provide official address data to licensed vendors, the address data itself belongs to the governing agency. In the US, the USPS owns the addresses. The same is true for Canada Post, Royal Mail, and others, though each country enforces or defines ownership a little differently. Knowing this is important, since it usually forbids reverse-engineering the address database. You have to be careful how to acquire, store, and use the data.
Google Maps is a common go-to for quick address fixes, but the TOS is rather prohibitive; for example, you can't use their data or APIs without showing a Google Map, and for non-commercial purposes only (unless you pay), and you can't store the data (except for temporary caching). Makes sense. Google's data is some of the best in the world. However, Google Maps does not verify the address. If an address does not exist, it will still show you where the address would be if it did exist (try it on your own street; use a house number that you know doesn't exist). This is useful sometimes, but be aware of that.
Nominatim's usage policy is similarly limiting, especially for high volume and commercial use, and the data is mostly drawn from free sources, so it isn't as well maintained (such is the nature of open projects) -- however, this may still suit your needs. It is supported by a great community.
The USPS itself has an API, but it goes down a lot and comes with no guarantees nor support. It might also be hard to use. Some people use it sparingly with no problems. But it's easy to miss that the USPS requires that you use their API only for confirming addresses to ship through them.
People expect addresses to be hard
Unfortunately, we've conditioned our society to expect addresses to be complicated. There's dozens of good UX articles all over the Internet about this, but the fact is, if you have an address form with individual fields, that's what users expect, even though it makes it harder for edge-case addresses that don't fit the format the form is expecting, or maybe the form requires a field it shouldn't. Or users don't know where to put a certain part of their address.
I could go on and on about the bad UX of checkout forms these days, but instead I'll just say that combining the addresses into a single field will be a welcome change -- people will be able to type their address how they see fit, rather than trying to figure out your lengthy form. However, this change will be unexpected and users may find it a little jarring at first. Just be aware of that.
Part of this pain can be alleviated by putting the country field out front, before the address. When they fill out the country field first, you know how to make your form appear. Maybe you have a good way to deal with single-field US addresses, so if they select United States, you can reduce your form to a single field, otherwise show the component fields. Just things to think about!
Now we know why it's hard; what can you do about it?
The USPS licenses vendors through a process called CASS™ Certification to provide verified addresses to customers. These vendors have access to the USPS database, updated monthly. Their software must conform to rigorous standards to be certified, and they don't often require agreement to such limiting terms as discussed above.
There are many CASS-Certified companies that can process lists or have APIs: Melissa Data, Experian QAS, and SmartyStreets to name a few.
(Due to getting flak for "advertising" I've truncated my answer at this point. It's up to you to find a solution that works for you.)
The Truth: Really, folks, I don't work at any of these companies. It's not an advertisement.

There are many street address parsers. They come in two basic flavors - ones that have databases of place names and street names, and ones that don't.
A regular expression street address parser can get up to about a 95% success rate without much trouble. Then you start hitting the unusual cases. The Perl one in CPAN, "Geo::StreetAddress::US", is about that good. There are Python and Javascript ports of that, all open source. I have an improved version in Python which moves the success rate up slightly by handling more cases. To get the last 3% right, though, you need databases to help with disambiguation.
A database with 3-digit ZIP codes and US state names and abbreviations is a big help. When a parser sees a consistent postal code and state name, it can start to lock on to the format. This works very well for the US and UK.
Proper street address parsing starts from the end and works backwards. That's how the USPS systems do it. Addresses are least ambiguous at the end, where country names, city names, and postal codes are relatively easy to recognize. Street names can usually be isolated. Locations on streets are the most complex to parse; there you encounter things such as "Fifth Floor" and "Staples Pavillion". That's when a database is a big help.

UPDATE: Geocode.xyz now works worldwide. For examples see https://geocode.xyz
For USA, Mexico and Canada, see geocoder.ca.
For example:
Input: something going on near the intersection of main and arthur kill rd new york
Output:
<geodata>
<latt>40.5123510000</latt>
<longt>-74.2500500000</longt>
<AreaCode>347,718</AreaCode>
<TimeZone>America/New_York</TimeZone>
<standard>
<street1>main</street1>
<street2>arthur kill</street2>
<stnumber/>
<staddress/>
<city>STATEN ISLAND</city>
<prov>NY</prov>
<postal>11385</postal>
<confidence>0.9</confidence>
</standard>
</geodata>
You may also check the results in the web interface or get output as Json or Jsonp. eg. I'm looking for restaurants around 123 Main Street, New York

No code? For shame!
Here is a simple JavaScript address parser. It's pretty awful for every single reason that Matt gives in his dissertation above (which I almost 100% agree with: addresses are complex types, and humans make mistakes; better to outsource and automate this - when you can afford to).
But rather than cry, I decided to try:
This code works OK for parsing most Esri results for findAddressCandidate and also with some other (reverse)geocoders that return single-line address where street/city/state are delimited by commas. You can extend if you want or write country-specific parsers. Or just use this as case study of how challenging this exercise can be or at how lousy I am at JavaScript. I admit I only spent about thirty mins on this (future iterations could add caches, zip validation, and state lookups as well as user location context), but it worked for my use case: End user sees form that parses geocode search response into 4 textboxes. If address parsing comes out wrong (which is rare unless source data was poor) it's no big deal - the user gets to verify and fix it! (But for automated solutions could either discard/ignore or flag as error so dev can either support the new format or fix source data.)
/*
address assumptions:
- US addresses only (probably want separate parser for different countries)
- No country code expected.
- if last token is a number it is probably a postal code
-- 5 digit number means more likely
- if last token is a hyphenated string it might be a postal code
-- if both sides are numeric, and in form #####-#### it is more likely
- if city is supplied, state will also be supplied (city names not unique)
- zip/postal code may be omitted even if has city & state
- state may be two-char code or may be full state name.
- commas:
-- last comma is usually city/state separator
-- second-to-last comma is possibly street/city separator
-- other commas are building-specific stuff that I don't care about right now.
- token count:
-- because units, street names, and city names may contain spaces token count highly variable.
-- simplest address has at least two tokens: 714 OAK
-- common simple address has at least four tokens: 714 S OAK ST
-- common full (mailing) address has at least 5-7:
--- 714 OAK, RUMTOWN, VA 59201
--- 714 S OAK ST, RUMTOWN, VA 59201
-- complex address may have a dozen or more:
--- MAGICICIAN SUPPLY, LLC, UNIT 213A, MAGIC TOWN MALL, 13 MAGIC CIRCLE DRIVE, LAND OF MAGIC, MA 73122-3412
*/
var rawtext = $("textarea").val();
var rawlist = rawtext.split("\n");
function ParseAddressEsri(singleLineaddressString) {
var address = {
street: "",
city: "",
state: "",
postalCode: ""
};
// tokenize by space (retain commas in tokens)
var tokens = singleLineaddressString.split(/[\s]+/);
var tokenCount = tokens.length;
var lastToken = tokens.pop();
if (
// if numeric assume postal code (ignore length, for now)
!isNaN(lastToken) ||
// if hyphenated assume long zip code, ignore whether numeric, for now
lastToken.split("-").length - 1 === 1) {
address.postalCode = lastToken;
lastToken = tokens.pop();
}
if (lastToken && isNaN(lastToken)) {
if (address.postalCode.length && lastToken.length === 2) {
// assume state/province code ONLY if had postal code
// otherwise it could be a simple address like "714 S OAK ST"
// where "ST" for "street" looks like two-letter state code
// possibly this could be resolved with registry of known state codes, but meh. (and may collide anyway)
address.state = lastToken;
lastToken = tokens.pop();
}
if (address.state.length === 0) {
// check for special case: might have State name instead of State Code.
var stateNameParts = [lastToken.endsWith(",") ? lastToken.substring(0, lastToken.length - 1) : lastToken];
// check remaining tokens from right-to-left for the first comma
while (2 + 2 != 5) {
lastToken = tokens.pop();
if (!lastToken) break;
else if (lastToken.endsWith(",")) {
// found separator, ignore stuff on left side
tokens.push(lastToken); // put it back
break;
} else {
stateNameParts.unshift(lastToken);
}
}
address.state = stateNameParts.join(' ');
lastToken = tokens.pop();
}
}
if (lastToken) {
// here is where it gets trickier:
if (address.state.length) {
// if there is a state, then assume there is also a city and street.
// PROBLEM: city may be multiple words (spaces)
// but we can pretty safely assume next-from-last token is at least PART of the city name
// most cities are single-name. It would be very helpful if we knew more context, like
// the name of the city user is in. But ignore that for now.
// ideally would have zip code service or lookup to give city name for the zip code.
var cityNameParts = [lastToken.endsWith(",") ? lastToken.substring(0, lastToken.length - 1) : lastToken];
// assumption / RULE: street and city must have comma delimiter
// addresses that do not follow this rule will be wrong only if city has space
// but don't care because Esri formats put comma before City
var streetNameParts = [];
// check remaining tokens from right-to-left for the first comma
while (2 + 2 != 5) {
lastToken = tokens.pop();
if (!lastToken) break;
else if (lastToken.endsWith(",")) {
// found end of street address (may include building, etc. - don't care right now)
// add token back to end, but remove trailing comma (it did its job)
tokens.push(lastToken.endsWith(",") ? lastToken.substring(0, lastToken.length - 1) : lastToken);
streetNameParts = tokens;
break;
} else {
cityNameParts.unshift(lastToken);
}
}
address.city = cityNameParts.join(' ');
address.street = streetNameParts.join(' ');
} else {
// if there is NO state, then assume there is NO city also, just street! (easy)
// reasoning: city names are not very original (Portland, OR and Portland, ME) so if user wants city they need to store state also (but if you are only ever in Portlan, OR, you don't care about city/state)
// put last token back in list, then rejoin on space
tokens.push(lastToken);
address.street = tokens.join(' ');
}
}
// when parsing right-to-left hard to know if street only vs street + city/state
// hack fix for now is to shift stuff around.
// assumption/requirement: will always have at least street part; you will never just get "city, state"
// could possibly tweak this with options or more intelligent parsing&sniffing
if (!address.city && address.state) {
address.city = address.state;
address.state = '';
}
if (!address.street) {
address.street = address.city;
address.city = '';
}
return address;
}
// get list of objects with discrete address properties
var addresses = rawlist
.filter(function(o) {
return o.length > 0
})
.map(ParseAddressEsri);
$("#output").text(JSON.stringify(addresses));
console.log(addresses);
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<textarea>
27488 Stanford Ave, Bowden, North Dakota
380 New York St, Redlands, CA 92373
13212 E SPRAGUE AVE, FAIR VALLEY, MD 99201
1005 N Gravenstein Highway, Sebastopol CA 95472
A. P. Croll & Son 2299 Lewes-Georgetown Hwy, Georgetown, DE 19947
11522 Shawnee Road, Greenwood, DE 19950
144 Kings Highway, S.W. Dover, DE 19901
Intergrated Const. Services 2 Penns Way Suite 405, New Castle, DE 19720
Humes Realty 33 Bridle Ridge Court, Lewes, DE 19958
Nichols Excavation 2742 Pulaski Hwy, Newark, DE 19711
2284 Bryn Zion Road, Smyrna, DE 19904
VEI Dover Crossroads, LLC 1500 Serpentine Road, Suite 100 Baltimore MD 21
580 North Dupont Highway, Dover, DE 19901
P.O. Box 778, Dover, DE 19903
714 S OAK ST
714 S OAK ST, RUM TOWN, VA, 99201
3142 E SPRAGUE AVE, WHISKEY VALLEY, WA 99281
27488 Stanford Ave, Bowden, North Dakota
380 New York St, Redlands, CA 92373
</textarea>
<div id="output">
</div>

For U.S. address parsing, I prefer using the usaddress package that is available in pip.
python3 -m pip install usaddress
Usage sample:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# address_parser.py
import sys
from usaddress import tag
from json import dumps, loads
if __name__ == '__main__':
tag_mapping = {
'Recipient': 'recipient',
'AddressNumber': 'addressStreet',
'AddressNumberPrefix': 'addressStreet',
'AddressNumberSuffix': 'addressStreet',
'StreetName': 'addressStreet',
'StreetNamePreDirectional': 'addressStreet',
'StreetNamePreModifier': 'addressStreet',
'StreetNamePreType': 'addressStreet',
'StreetNamePostDirectional': 'addressStreet',
'StreetNamePostModifier': 'addressStreet',
'StreetNamePostType': 'addressStreet',
'CornerOf': 'addressStreet',
'IntersectionSeparator': 'addressStreet',
'LandmarkName': 'addressStreet',
'USPSBoxGroupID': 'addressStreet',
'USPSBoxGroupType': 'addressStreet',
'USPSBoxID': 'addressStreet',
'USPSBoxType': 'addressStreet',
'BuildingName': 'addressStreet',
'OccupancyType': 'addressStreet',
'OccupancyIdentifier': 'addressStreet',
'SubaddressIdentifier': 'addressStreet',
'SubaddressType': 'addressStreet',
'PlaceName': 'addressCity',
'StateName': 'addressState',
'ZipCode': 'addressPostalCode',
}
try:
address, _ = tag(' '.join(sys.argv[1:]), tag_mapping=tag_mapping)
except:
with open('failed_address.txt', 'a') as fp:
fp.write(sys.argv[1] + '\n')
print(dumps({}))
else:
print(dumps(dict(address)))
Running address_parser.py:
python3 address_parser.py 9757 East Arcadia Ave. Saugus MA 01906
{"addressStreet": "9757 East Arcadia Ave.", "addressCity": "Saugus", "addressState": "MA", "addressPostalCode": "01906"}

I'm late to the party, but here is an Excel VBA script I wrote years ago for Australia. It can be easily modified to support other Countries. I've made a GitHub repository of the C# code here. I've hosted it on my site and you can download it here: http://jeremythompson.net/Rocks/ParseAddress.xlsm
Strategy
For any country with a PostCode that's numeric or can be matched with a RegEx my strategy works very well:
First we detect the First and Surname which are assumed to be the top line. Its easy to skip the name and start with the address by unticking the checkbox (called 'Name is top row' as shown below).
Next its safe to expect the Address consisting of the Street and Number come before the Suburb and the St, Pde, Ave, Av, Rd, Cres, loop, etc is a separator.
Detecting the Suburb vs the State and even Country can trick the most sophisticated parsers as there can be conflicts. To overcome this I use a PostCode look up based on the fact that after stripping Street and Apartment/Unit numbers as well as the PoBox,Ph,Fax,Mobile etc, only the PostCode number will remain. This is easy to match with a regEx to then look up the suburb(s) and country.
Your National Post Office Service will provide a list of post codes with Suburbs and States free of charge that you can store in an excel sheet, db table, text/json/xml file, etc.
Finally, since some Post Codes have multiple Suburbs we check which suburb appears in the Address.
Example
VBA Code
DISCLAIMER, I know this code is not perfect, or even written well however its very easy to convert to any programming language and run in any type of application. The strategy is the answer depending on your country and rules, take this code as an example:
Option Explicit
Private Const TopRow As Integer = 0
Public Sub ParseAddress()
Dim strArr() As String
Dim sigRow() As String
Dim i As Integer
Dim j As Integer
Dim k As Integer
Dim Stat As String
Dim SpaceInName As Integer
Dim Temp As String
Dim PhExt As String
On Error Resume Next
Temp = ActiveSheet.Range("Address")
'Split info into array
strArr = Split(Temp, vbLf)
'Trim the array
For i = 0 To UBound(strArr)
strArr(i) = VBA.Trim(strArr(i))
Next i
'Remove empty items/rows
ReDim sigRow(LBound(strArr) To UBound(strArr))
For i = LBound(strArr) To UBound(strArr)
If Trim(strArr(i)) <> "" Then
sigRow(j) = strArr(i)
j = j + 1
End If
Next i
ReDim Preserve sigRow(LBound(strArr) To j)
'Find the name (MUST BE ON THE FIRST ROW UNLESS CHECKBOX UNTICKED)
i = TopRow
If ActiveSheet.Shapes("chkFirst").ControlFormat.Value = 1 Then
SpaceInName = InStr(1, sigRow(i), " ", vbTextCompare) - 1
If ActiveSheet.Shapes("chkConfirm").ControlFormat.Value = 0 Then
ActiveSheet.Range("FirstName") = VBA.Left(sigRow(i), SpaceInName)
Else
If MsgBox("First Name: " & VBA.Mid$(sigRow(i), 1, SpaceInName), vbQuestion + vbYesNo, "Confirm Details") = vbYes Then ActiveSheet.Range("FirstName") = VBA.Left(sigRow(i), SpaceInName)
End If
If ActiveSheet.Shapes("chkConfirm").ControlFormat.Value = 0 Then
ActiveSheet.Range("Surname") = VBA.Mid(sigRow(i), SpaceInName + 2)
Else
If MsgBox("Surame: " & VBA.Mid(sigRow(i), SpaceInName + 2), vbQuestion + vbYesNo, "Confirm Details") = vbYes Then ActiveSheet.Range("Surname") = VBA.Mid(sigRow(i), SpaceInName + 2)
End If
sigRow(i) = ""
End If
'Find the Street by looking for a "St, Pde, Ave, Av, Rd, Cres, loop, etc"
For i = 1 To UBound(sigRow)
If Len(sigRow(i)) > 0 Then
For j = 0 To 8
If InStr(1, VBA.UCase(sigRow(i)), Street(j), vbTextCompare) > 0 Then
'Find the position of the street in order to get the suburb
SpaceInName = InStr(1, VBA.UCase(sigRow(i)), Street(j), vbTextCompare) + Len(Street(j)) - 1
'If its a po box then add 5 chars
If VBA.Right(Street(j), 3) = "BOX" Then SpaceInName = SpaceInName + 5
If ActiveSheet.Shapes("chkConfirm").ControlFormat.Value = 0 Then
ActiveSheet.Range("Street") = VBA.Mid(sigRow(i), 1, SpaceInName)
Else
If MsgBox("Street Address: " & VBA.Mid(sigRow(i), 1, SpaceInName), vbQuestion + vbYesNo, "Confirm Details") = vbYes Then ActiveSheet.Range("Street") = VBA.Mid(sigRow(i), 1, SpaceInName)
End If
'Trim the Street, Number leaving the Suburb if its exists on the same line
sigRow(i) = VBA.Mid(sigRow(i), SpaceInName) + 2
sigRow(i) = Replace(sigRow(i), VBA.Mid(sigRow(i), 1, SpaceInName), "")
GoTo PastAddress:
End If
Next j
End If
Next i
PastAddress:
'Mobile
For i = 1 To UBound(sigRow)
If Len(sigRow(i)) > 0 Then
For j = 0 To 3
Temp = Mb(j)
If VBA.Left(VBA.UCase(sigRow(i)), Len(Temp)) = Temp Then
If ActiveSheet.Shapes("chkConfirm").ControlFormat.Value = 0 Then
ActiveSheet.Range("Mobile") = VBA.Mid(sigRow(i), Len(Temp) + 2)
Else
If MsgBox("Mobile: " & VBA.Mid(sigRow(i), Len(Temp) + 2), vbQuestion + vbYesNo, "Confirm Details") = vbYes Then ActiveSheet.Range("Mobile") = VBA.Mid(sigRow(i), Len(Temp) + 2)
End If
sigRow(i) = ""
GoTo PastMobile:
End If
Next j
End If
Next i
PastMobile:
'Phone
For i = 1 To UBound(sigRow)
If Len(sigRow(i)) > 0 Then
For j = 0 To 1
Temp = Ph(j)
If VBA.Left(VBA.UCase(sigRow(i)), Len(Temp)) = Temp Then
'TODO: Detect the intl or national extension here.. or if we can from the postcode.
If ActiveSheet.Shapes("chkConfirm").ControlFormat.Value = 0 Then
ActiveSheet.Range("Phone") = VBA.Mid(sigRow(i), Len(Temp) + 3)
Else
If MsgBox("Phone: " & VBA.Mid(sigRow(i), Len(Temp) + 3), vbQuestion + vbYesNo, "Confirm Details") = vbYes Then ActiveSheet.Range("Phone") = VBA.Mid(sigRow(i), Len(Temp) + 3)
End If
sigRow(i) = ""
GoTo PastPhone:
End If
Next j
End If
Next i
PastPhone:
'Email
For i = 1 To UBound(sigRow)
If Len(sigRow(i)) > 0 Then
'replace with regEx search
If InStr(1, sigRow(i), "#", vbTextCompare) And InStr(1, VBA.UCase(sigRow(i)), ".CO", vbTextCompare) Then
Dim email As String
email = sigRow(i)
email = Replace(VBA.UCase(email), "EMAIL:", "")
email = Replace(VBA.UCase(email), "E-MAIL:", "")
email = Replace(VBA.UCase(email), "E:", "")
email = Replace(VBA.UCase(Trim(email)), "E ", "")
email = VBA.LCase(email)
If ActiveSheet.Shapes("chkConfirm").ControlFormat.Value = 0 Then
ActiveSheet.Range("Email") = email
Else
If MsgBox("Email: " & email, vbQuestion + vbYesNo, "Confirm Details") = vbYes Then ActiveSheet.Range("Email") = email
End If
sigRow(i) = ""
Exit For
End If
End If
Next i
'Now the only remaining items will be the postcode, suburb, country
'there shouldn't be any numbers (eg. from PoBox,Ph,Fax,Mobile) except for the Post Code
'Join the string and filter out the Post Code
Temp = Join(sigRow, vbCrLf)
Temp = Trim(Temp)
For i = 1 To Len(Temp)
Dim postCode As String
postCode = VBA.Mid(Temp, i, 4)
'In Australia PostCodes are 4 digits
If VBA.Mid(Temp, i, 1) <> " " And IsNumeric(postCode) Then
If ActiveSheet.Shapes("chkConfirm").ControlFormat.Value = 0 Then
ActiveSheet.Range("PostCode") = postCode
Else
If MsgBox("Post Code: " & postCode, vbQuestion + vbYesNo, "Confirm Details") = vbYes Then ActiveSheet.Range("PostCode") = postCode
End If
'Lookup the Suburb and State based on the PostCode, the PostCode sheet has the lookup
Dim mySuburbArray As Range
Set mySuburbArray = Sheets("PostCodes").Range("A2:B16670")
Dim suburbs As String
For j = 1 To mySuburbArray.Columns(1).Cells.Count
If mySuburbArray.Cells(j, 1) = postCode Then
'Check if the suburb is listed in the address
If InStr(1, UCase(Temp), mySuburbArray.Cells(j, 2), vbTextCompare) > 0 Then
'Set the Suburb and State
ActiveSheet.Range("Suburb") = mySuburbArray.Cells(j, 2)
Stat = mySuburbArray.Cells(j, 3)
ActiveSheet.Range("State") = Stat
'Knowing the State - for Australia we can get the telephone Ext
PhExt = PhExtension(VBA.UCase(Stat))
ActiveSheet.Range("PhExt") = PhExt
'remove the phone extension from the number
Dim prePhone As String
prePhone = ActiveSheet.Range("Phone")
prePhone = Replace(prePhone, PhExt & " ", "")
prePhone = Replace(prePhone, "(" & PhExt & ") ", "")
prePhone = Replace(prePhone, "(" & PhExt & ")", "")
ActiveSheet.Range("Phone") = prePhone
Exit For
End If
End If
Next j
Exit For
End If
Next i
End Sub
Private Function PhExtension(ByVal State As String) As String
Select Case State
Case Is = "NSW"
PhExtension = "02"
Case Is = "QLD"
PhExtension = "07"
Case Is = "VIC"
PhExtension = "03"
Case Is = "NT"
PhExtension = "04"
Case Is = "WA"
PhExtension = "05"
Case Is = "SA"
PhExtension = "07"
Case Is = "TAS"
PhExtension = "06"
End Select
End Function
Private Function Ph(ByVal Num As Integer) As String
Select Case Num
Case Is = 0
Ph = "PH"
Case Is = 1
Ph = "PHONE"
'Case Is = 2
'Ph = "P"
End Select
End Function
Private Function Mb(ByVal Num As Integer) As String
Select Case Num
Case Is = 0
Mb = "MB"
Case Is = 1
Mb = "MOB"
Case Is = 2
Mb = "CELL"
Case Is = 3
Mb = "MOBILE"
'Case Is = 4
'Mb = "M"
End Select
End Function
Private Function Fax(ByVal Num As Integer) As String
Select Case Num
Case Is = 0
Fax = "FAX"
Case Is = 1
Fax = "FACSIMILE"
'Case Is = 2
'Fax = "F"
End Select
End Function
Private Function State(ByVal Num As Integer) As String
Select Case Num
Case Is = 0
State = "NSW"
Case Is = 1
State = "QLD"
Case Is = 2
State = "VIC"
Case Is = 3
State = "NT"
Case Is = 4
State = "WA"
Case Is = 5
State = "SA"
Case Is = 6
State = "TAS"
End Select
End Function
Private Function Street(ByVal Num As Integer) As String
Select Case Num
Case Is = 0
Street = " ST"
Case Is = 1
Street = " RD"
Case Is = 2
Street = " AVE"
Case Is = 3
Street = " AV"
Case Is = 4
Street = " CRES"
Case Is = 5
Street = " LOOP"
Case Is = 6
Street = "PO BOX"
Case Is = 7
Street = " STREET"
Case Is = 8
Street = " ROAD"
Case Is = 9
Street = " AVENUE"
Case Is = 10
Street = " CRESENT"
Case Is = 11
Street = " PARADE"
Case Is = 12
Street = " PDE"
Case Is = 13
Street = " LANE"
Case Is = 14
Street = " COURT"
Case Is = 15
Street = " BLVD"
Case Is = 16
Street = "P.O. BOX"
Case Is = 17
Street = "P.O BOX"
Case Is = 18
Street = "PO BOX"
Case Is = 19
Street = "POBOX"
End Select
End Function

How to generate unique (short) URL folder name on the fly...like Bit.ly

I'm creating an application which will create a large number of folders on a web server, with files inside of them.
I need the folder name to be unique. I can easily do this with a GUID, but I want something more user friendly. It doesn't need to be speakable by users, but should be short and standard characters (alphas is best).
In short: i'm looking to do something like Bit.ly does with their unique names:
www.mydomain.com/ABCDEF
Is there a good reference on how to do this? My platform will be .NET/C#, but ok with any help, references, links, etc on the general concept, or any overall advice to solve this task.

Start at 1. Increment to 2, 3, 4, 5, 6, 7,
8, 9, a, b...
A, B, C...
X, Y, Z, 10, 11, 12, ... 1a, 1b,
You get the idea.
You have a synchronized global int/long "next id" and represent it in base 62 (numbers, lowercase, caps) or base 36 or something.

I'm assuming that you know how to use your web server's redirect capabilities. If you need help, just comment :).
The way I would do it would be generating a random integer (between the integer values of 'a' and 'z'); converting it into a char; appending it to a string; and repeating until we reach the needed length. If it generates a value already in the database, repeat the process. If it was unique, store it in the database with the name of the actual location and the name of the alias.
This is a bit hack-like because it assumes that 'a' through 'z' are actually in sequence in their integer values.
Best I could think of :(.

In Perl, without modules so you can translate more easly.
sub convert_to_base {
my ($n, $b) = #_;
my #digits;
while ($n) {
my $digits = $n % $b;
unshift #digits, $digit;
$n = ($n - $digit) / $b;
}
unshift #digits, 0 if !#digits;
return #digits;
}
# Whatever characters you want to use.
my #digit_set = ( '0'..'9', 'a'..'z', 'A'..'Z' );
# The id of the record in the database,
# or one more than the last id you generated.
my $id = 1;
my $converted =
join '',
map { $digit_set[$_] }
convert_to_base($id, 0+#digits_set);

I needed something similar to what you're trying to accomplish. I retooled my code to generate folders so try this. It's setup for a console app, but you can use it in a website also.
private static void genRandomFolders()
{
string basepath = "C:\\Users\\{username here}\\Desktop\\";
int count = 5;
int length = 8;
List<string> codes = new List<string>();
int total = 0;
int i = count;
Random rnd = new Random();
while (i-- > 0)
{
string code = RandomString(rnd, length);
if (!codes.Exists(delegate(string c) { return c.ToLower() == code.ToLower(); }))
{
//Create directory here
System.IO.Directory.CreateDirectory(basepath + code);
}
total++;
if (total % 100 == 0)
Console.WriteLine("Generated " + total.ToString() + " random folders...");
}
Console.WriteLine();
Console.WriteLine("Generated " + total.ToString() + " total random folders.");
}
public static string RandomString(Random r, int len)
{
//string str = "ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890"; //uppercase only
//string str = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890"; //All
string str = "abcdefghjkmnpqrstuvwxyz123456789"; //Lowercase only
StringBuilder sb = new StringBuilder();
while ((len--) > 0)
sb.Append(str[(int)(r.NextDouble() * str.Length)]);
return sb.ToString();
}

Resources