Detecting regional settings (List Separator) from web - ruby-on-rails

After having the unpleasant surprise that Comma Seperated Value (CSV) files are not necessarily comma-separated, I'm trying to find out if there is any way to detect what the regional settings list separator value is on the client machine from http request.
Scenario is as follows: A user can download some data in CSV format from web site (RoR, if it matters). That CSV file is generated on the fly, sent to the user, and most of the time double-clicked and opened in MS Excel on Windows machine at the destination. Now, if the user has ',' set as the list separator, the data is properly arranged in columns, but if any other separator (';' is widely used here) is set, it all just gets thrown into a single column. So, is there any way to detect what separator is used on the client machine, and generate the file accordingly?
I have a sinking feeling that it is not, but I'd like to be sure before I pass the 'can't be done, sorry' line to the customer :)

Here's a JavaScript solution that I just wrote based on the method shown here:
function getListSeparator() {
var list = ['a', 'b'], str;
if (list.toLocaleString) {
str = list.toLocaleString();
if (str.indexOf(';') > 0 && str.indexOf(',') == -1) {
return ';';
}
}
return ',';
}
The key is in the toLocaleString() method that uses the system list separator.
You could use JavaScript to get the list separator and set it in a cookie which you could then detect from your server.
I checked all the Windows Locales, and it seems that the default list separator is virtually always either ',' or ';'. For some locales the drop-down list in the Control Panel offers both options; for others it offers just ','. One locale, Divehi, has a strange character that I've not seen before as the list separator, and, for any locale, it is possible for the user to enter any string they want as the list separator.
Putting random strings as the separator in a CSV file sounds like trouble to me, so my function above will only return either a ';' or a '.', and it will only return a ';' if it can't find a ',' in the Array.toLocaleString string. I'm not entirely sure about whether array.toLocaleString has a format that's guaranteed across browsers, hence the indexOf checks rather than picking out a character at a specific index.
Using Array.toLocaleString to get the list separator works on IE6, IE7, and IE8, but unfortunately it doesn't seem to work on Firefox, Safari, Opera, and Chrome (or at least the versions of those browsers on my computer): they all seem to separate array items with a comma, irrespective of the Windows "list separator" setting.
Also worth noting that by default Excel seems to use the system "decimal separator" when it's parsing numbers out of CSV files. Yuk. So, if you're localizing the list separator you might want to localize the decimal separator too.

I think everyone should use Calc from OpenOffice - it asks when you open a file about encoding, column separators and other. I don't know answer for your question, but maybe you can try to send data in html tables or in xml - excel should read both of them correctly. From my experience it isn't easy to export data to excel. Few weeks ago I have problem with it and after few hours of work I asked a person, who couldn't open my csv file in excel, about version. It was Excel 98...
Take a look on html example and xml.

The simplier version of getListSeparator function, enabling any character to be a separator:
function getListSeparator_bis()
{
var list = ['a', 'b'];
return(list.toLocaleString().charAt(1));
}// getListSeparator_bis
Just set any char (f.e. '#') as list separator in your OS and try the code as above. The appropriate char (i.e. '#' if set as sugested) is returned.

Could you just have the users with non comma separators set a profile kind of option and then generate CSVs based on user settings with the default being commas?

Toms, as far as I'm aware there is no way of achieving what you're after. The most you can do is try and detect the user locale and map it against a database of locales/list separators, altering the list separator in the .CSV file as a result.

Related

TFS code search find <Button>

I want to find all usages of my react component in code.
I tried <Button> but Special chars <> are not supported.
Tried "Button" and i get "Button" and button with lowercase as results as well.
So exact match is also not supported.
Is there is a way to find a string exactly without any additional results?
Unfortunately, search symbols (<> and "" in your scenario) is not supported in code search.
In tfs the symbol "" is used for finding an exact match to a set of words by enclosing your search terms in double-quotes. For example, "Client not found".
Is there is a way to find a string exactly without any additional
results?
Yes, but it seems a little complex, just reference my answer in another thread:Is there a way to make TFS code search recognize the "#" symbol?
Checked for some characters in code search. You can't use the symbol
characters except * and ? as part of your search query, which
including below characters: . , : ; / \ ` ' " # = ! # $ & + ^ | ~ < >
( ) { } [ ]. The search will simply ignore these symbols.
But you can use wildcard characters * and ? to broaden your search.
You can use wildcard characters anywhere in your search string except
as a prefix in a simple search string or a query that uses a code type
filter. For example, you cannot use a search query such as
*RequestHandler or class:?RequestHandler. However, you can use prefix wildcards with the other search filter functions; for
example, the
search query strings file:*RequestHandler.cs and repo:?Handlers are
valid.
Please see Broaden your search with wildcards for details.
If you want to search the strings including these symbol exactly(such
as '#' here), you can code search with other strings (eg,
testexample.com here) to narrow down the scope first, then copy the
specific code to text editor which support the symbols (eg,
Notepad++), then search stings with the symbol characters.
Besides, if you are using Git, another workaround is using the code
search tool Hound: a lightning fast code search tool, it supports
the symbol characters. Reference this thread to use it:
How can I publish source code (Visual Studio) on a intranet?
Also, there is a User Voice here to suggest the feature, you can go and vote it up to achieve that in future.

Neo4j - search like query with non english characters

Is there an option in neo4j to write a select query with where clause, that ignores non-latin characters ?
MATCH (places:Place)
WHERE (places.name =~ '.*(?ui)Fabergé.*')
RETURN places
I have place with Fabergé name in graph and i want to find it when user type Fabergé or Faberge without this special character.
I'm not aware of an easy way to do this directly with a regex match in Cypher.
One possible workaround is to store the string in question in a normalized form in a second property e.g. place.name_normalized and then compare it with the normalized search string. Of course normalization needs to be done on client side, see another SO question on how to achive this: Remove diacritical marks (ń ǹ ň ñ ṅ ņ ṇ ṋ ṉ ̈ ɲ ƞ ᶇ ɳ ȵ) from Unicode chars

Microsoft Access: Complex string search to update field in another table

I have a table that is linked to Access to return the results of emails into a folder. All of the emails being returned will be answering the same questions. I need to parse this email body text from this table and update several fields of another table with this data. The problem is that the linked table brings the text in super messy. Even though I have the email that is being returned all nicely formatted in a table, it comes back into access a hot mess full of extra spacing. I want to open a recordset based on the linked table (LinkTable), and then parse the LinkTable.Body field somehow so I can update another table with clean data. The data that is coming back into LinkTable looks like this:
Permit? (Note: if yes, provide specific permit type in Additional Requirements section)
No
Phytosanitary Certificate? (Note: if recommended, input No and complete Additional Requirements section)
Yes
Additional Requirements: if not applicable, indicate NA or leave blank (Type of permit required, container labeling, other agency documents, other)
Double containment, The labeling or declaration must provide the following information: -The kind, variety, and origin of each lot of seed -The designation “hybrid” when the lot contains hybrid seed -If the seed was treated, the name of the substance or p
The answer of the first two should either be yes or no, so I figured I could set up code with case statements and based on a match I should place yes or no in the corresponding field in my real table (not sure how to deal with the extra spaces here), The third one could have any number of responses, but it is the last question so anything after the "(Type of permit required, container labeling, other agency documents, other)" could be taken and placed in the other table. Does anyone have any ideas how I could set this up? I am at a bit of a loss, especially with how to deal with all of the extra spaces and how to grab all of the text after the Additional Requirements paragraph. Thank you in advance!
My select statement to get the body text looks like this:
Set rst1 = db.OpenRecordset("SELECT Subject, Contents FROM LinkTable WHERE Subject like '*1710'")
There are multiple ways to do this, one is using Instr() and Len() to find beginning and end of the fixed questions, then Mid() to extract the answers.
But I think using Split() is easier. It's best explained with commented code.
Public Sub TheParsing()
' A string constant that you expect to never come up in the Contents, used as separator for Split()
Const strSeparator = "##||##"
Dim rst1 As Recordset
Dim S As String
Dim arAnswers As Variant
Dim i As Long
S = Nz(rst1!Contents, "")
' Replace all the constant parts (questions) with the separator
S = Replace(S, "Permit? (Note: if yes, provide specific permit type in Additional Requirements section)", strSeparator)
' etc. for the other questions
' Split the remaining string into a 0-based array with the answers
arAnswers = Split(S, strSeparator)
' arAnswers(0) contains everything before the first question (probably ""), ignore that.
' Check that there are 3 answers
If UBound(arAnswers) <> 3 Then
' Houston, we have a problem
Stop
Else
For i = 1 To 3
' Extract each answer
S = arAnswers(i)
' Remove whitespace: CrLf, Tab
S = Replace(S, vbCrLf, "")
S = Replace(S, vbTab, "")
' Trim the remaining string
S = Trim(S)
' Now you have the cleaned up string and can use it
Select Case i
Case 1: strPermit = S
Case 2: strCertificate = S
Case 3: strRequirements = S
End Select
Next i
End If
rst1.MoveNext
' etc
End Sub
This will fail if the constant parts (the questions) have been altered. But so will all other straightforward methods.

Open Search Server v1.4 select query special characters

We're using Open Search Server v1.4. When the user enters a search for the text "Refrigerator temperature chart (5" we create a URL something like
http://10.192.16.160:8080/services/rest/select/search/<indexname/json?login=<login>&key=<apikey>template=search&query=Refrigerator%20temperature%20chart%20%285&start=0&rows=1000&filter=fileType%3afile&lang=ENGLISH
This fails with ...
HTTP Status 500 - org.apache.cxf.interceptor.Fault:
com.jaeksoft.searchlib.SearchLibException:
com.jaeksoft.searchlib.query.ParseException:
org.apache.lucene.queryParser.ParseException: Cannot parse
'content:(Refrigerator temperature chart (5) OR content:("Refrigerator
temperature chart (5") OR
So adding an escape character %5C before the open bracket fixes this query like so...
http://10.192.16.160:8080/services/rest/select/search/<indexname/json?login=<login>&key=<apikey>template=search&query=Refrigerator%20temperature%20chart%20%5C%285&start=0&rows=1000&filter=fileType%3afile&lang=ENGLISH
Can someone point me to some documentation that lists all the special characters that can be used in an Open Search select query that need to be escaped when entered as part of the search string?
Yes you are right, characters listed in section "Escaping Special Characters" in the page you linked also need to be escaped in OpenSearchServer.
We recently released a fix allowing to escape those characters in query of type Search (field) for Searched fields configured with a pattern mode.
Previously escaping of characters was only available in query of type Search (pattern).
(more information of these two kind of queries here: http://www.opensearchserver.com/documentation/tutorials/functionalities.html#two-kinds-of-queries)
Regards,
Alexandre
I believe Open Search Server is based on Lucene. The query syntax for the Lucene engine is described here...
http://lucene.apache.org/core/2_9_4/queryparsersyntax.html
Lucene supports escaping special characters that are part of the query
syntax. The current list special characters are
+ - && || ! ( ) { } [ ] ^ " ~ * ? : \
To escape these character use the \ before the character. For example
to search for (1+1):2 use the query:
\(1\+1\)\:2

Double slashes in urls in Outlook

The url of our search page is build like this:
http://www.example.com/results/name/John/city/Miami/gender/Male
This would display every male named John in Miami.
When one of the filters is left empty, the url would be something like this:
http://www.example.com/results/name/John/city//gender/Male
So there are two slashes in the url.
Outlook doesn't seem to like this. When you click on the second url, it removes one of the two slashes. This leaves the url like this:
http://www.example.com/results/name/John/city/gender/Male
Persons names John in the city 'gender'..
What would be the best way to fix this problem?
This is a bug in Microsoft Office.
URLs with two consecutive slashes are allowed by RFC 2396, but they're not commonly used. As the RFC says (extract from Appendix A):
abs_path = "/" path_segments
path_segments = segment *( "/" segment )
segment = *pchar *( ";" param )
Note that segment is defined as containing ZERO OR MORE characters. (You might argue that this is a spec bug, and it shouldn't be allowed... but it is)
As you've discovered, Microsoft Office will "fix" URLs containing double slashes. This is apparently a deliberate feature for "cleanliness and consistency". There is no way to override or disable it. Source.
So, as other people have suggested, you're probably going to have to change the way the server formats URLs.
Try to replace (one of) the slashes with ASCII code 2F (decimal 47).
"Regkey" should help: You need to implement this key
Path: "HKEY_CURRENT_USER\Software\Microsoft\Office\Common"
Name: "AllowConsecutiveSlashesInUrlPathComponent"
Type: "REG_DWORD"
Value: "1"
For implementation & more detail see here.
The standard is to collapse the two slashes into one, so there is no way to prevent this from happening. It might be a good idea to put something between those slashes to indicate to your search page that that field is blank.
Alternatively, you could change the search page to use a query string such as this:
http://www.example.com/results?name=John&city=&gender=Male
You can use - instead of blank segment. For example:
http://www.example.com/results/name/John/city/-/gender/Male
I agree with Peter, just replace the second "/" with "%2F" or "%2f" is enough.
If you must use slashes, consider fixing this on the server side. Create a list of keywords (city|results|...) and, if a slash is followed by one of the keywords, treat it as an empty entry. (edited) Double slashes should not be treated as one, but as you found out, some applications "fix" this.
An alternative and standard way of fixing this is using a placeholder, normally a dot, because it has no special meaning:
http://example.com/results/name/./city/amsterdam

Resources