Backslash read and write and F# interactive console - f#

Edit: whats the difference between reading a backslash from a file and writing it to the interactive window vs writing directly the string to the interactive window ?
For example
let toto = "Adelaide Gu\u00e9nard"
toto;;
the interactive window prints "Adelaide Guénard".
Now if I save a txt file with the single line Adelaide Gu\u00e9nard . And read it in:
System.IO.File.ReadAllLines(#"test.txt")
The interactive window prints [|"Adelaide Gu\u00e9nard"|]
What is the difference between these 2 statements in terms of the interactive window printing ?

As far as I know, there is no library that would decode the F#/C# escaping of string for you, so you'll have to implement that functionality yourself. There was a similar question on how to do that in C# with a solution using regular expressions.
You can rewrite that to F# like this:
open System
open System.Globalization
open System.Text.RegularExpressions
let regex = new Regex (#"\\[uU]([0-9A-F]{4})", RegexOptions.IgnoreCase)
let line = "Adelaide Gu\\u00e9nard"
let line = regex.Replace(line, fun (m:Match) ->
(char (Int32.Parse(m.Groups.[1].Value, NumberStyles.HexNumber))).ToString())
(If you write "some\\u00e9etc" then you're creating string that contains the same thing as what you'd read from the text file - if you use single backslash, then the F# compiler interprets the escaping)

It uses the StructuredFormat stuff from the F# PowerPack. For your string, it's effectively doing printfn toto;;.
You can achieve the same behaviour in a text file as follows:
open System.IO;;
File.WriteAllText("toto.txt", toto);;
The default encoding used by File.WriteAllText is UTF-8. You should be able to open toto.txt in Notepad or Visual Studio and see the é correctly.
Edit: If wanted to write the content of test.txt to another file in the clean F# interactive print, how would i proceed ?
It looks like fsi is being too clever when printing the contents of test.txt. It's formatting it as a valid F# expression, complete with quotes, [| |] brackets, and a Unicode character escape. The string returned by File.ReadAllLines doesn't contain any of these things; it just contains the words Adelaide Guénard.
You should be able to take the array returned by File.ReadAllLines and pass it to File.WriteAllLines, without the contents being mangled.

Related

How to read string stored in hdf5 format files by DM

I am scripting with DM and would like to read hdf5 file format.
I borrowed Tore Niermann's gms_HDF5_Plug-In (hdf5_GMS2X_amd64.dll) and his CMD_import_hdf5.s script. It use h5_read_dataset(filename, datapath) to read a image dataset.
I am trying to figure out the way to read a string info stored in the same file. I am particular interested to read the angle stored in string as shown in this figure.Demonstrated string to read. The h5_read_dataset(filename, datapath) function doesn't work for reading string.
There is a help file (hdf5_plugin.chm) with a list of functions but unfortunately I can't open them to see more info.
hdf5_plugin.chm showing the function list.
I suppose the right function to read strings should be something like h5_read_attr() or h5_info() but I didn't test them out. DM always says the two functions doesn't exist.
After reading out the angle by string, I will also need a bit help to convert the string to a double datatype.
Thank you.
Converting String to Number is done with the Val() command.
There is no integer/double/float concept for variables in DM-script, all are just number. ( This is different for images, where you can define the numeric type. Also: For file-inport/export a type differntiation can be made using the taggroup streaming commands in the other answer. )
Example script:
string numStr = "1.234e-2"
number num = val( numStr )
ClearResults()
Result( "\n As string:" + numStr )
Result( "\n As value:" + num )
Result( "\n As value, formatted:" + Format(num,"%3.2f") )
Potential answer regarding the .chm files: When you download (or email) .chm files in Windows, the OS classifies them as "potentially dagerouse" (because it could contain executable HTML code, I think). As a result, these files can not be shown by default. However, you can right-click these files and "unblock" them in the file properties.
Example:
I think this will be most likely a question specific to that plugin and not general DM scripting. So it might be better to contact the plugin-author directly.
The alternative (not good) solution would be to "rewrite" your own HDF5 file-reader, if you know the file-format. For this you would need the "Streaming" commands of the DM script language and browse through the (binary?) source file to the apropriate file location. The starting point for reading on this in the F1 help documentation would be here:

Is it possible to test a lexer made in JFlex without writing a parser?

I am beginning to use JFlex and I want to try to write a lexer first, and then move onto the parser. However, it seems like there is no way to test your JFlex lexer without writing a parser in CUP as well.
All I want to do is write a lexer, give it an input file and then output the lexemes to check that it read everything correctly. Later I would like to output the tokens, but lexemes would be a good start.
Yes It is possible to write a standalone scanner. You can read the details on this page. If you specify %standalone directive, it will add main method to the generated class. You can mention input files as command line arguments to run this program. jflex tar comes with an examples directory you can find one standalone example inside examples/standalone-maven/src/main/jflex directory. For quick reference I am posting one example code here
/**
This is a small example of a standalone text substitution scanner
It reads a name after the keyword name and substitutes all occurences
of "hello" with "hello <name>!". There is a sample input file
"sample.inp" provided in this directory
*/
package de.jflex.example.standalone;
%%
%public
%class Subst
%standalone
%unicode
%{
String name;
%}
%%
"name " [a-zA-Z]+ { name = yytext().substring(5); }
[Hh] "ello" { System.out.print(yytext()+" "+name+"!"); }

String Stream in Prolog?

I have to work with some SWI-Prolog code that opens a new stream (which creates a file on the file system) and pours some data in. The generated file is read somewhere else later on in the code.
I would like to replace the file stream with a string stream in Prolog so that no files are created and then read everything that was put in the stream as one big string.
Does SWI-Prolog have string streams? If so, how could I use them to accomplish this task? I would really appreciate it if you could provide a small snippet. Thank you!
SWI-Prolog implements memory mapped files. Here is a snippet from some old code of mine, doing both write/read
%% html2text(+Html, -Text) is det.
%
% convert from html to text
%
html2text(Html, Text) :-
html_clean(Html, HtmlDescription),
new_memory_file(Handle),
open_memory_file(Handle, write, S),
format(S, '<html><head><title>html2text</title></head><body>~s</body></html>', [HtmlDescription]),
close(S),
open_memory_file(Handle, read, R, [free_on_close(true)]),
load_html_file(stream(R), [Xml]),
close(R),
xpath(Xml, body(normalize_space), Text).
Another option is using with_output_to/2 combined with current_output/1:
write_your_output_to_stream(Stream) :-
format(Stream, 'example output\n', []),
format(Stream, 'another line', []).
str_out(Codes) :-
with_output_to(codes(Codes), (
current_output(Stream),
write_your_output_to_stream(Stream)
)).
Usage example:
?- portray_text(true), str_out(C).
C = "example output
another line"
Of course, you can choose between redirecting output to atom, string, list of codes (as per example above), etc., just use the corresponding parameter to with_output_to/2:
with_output_to(atom(Atom), ... )
with_output_to(string(String), ... )
with_output_to(codes(Codes), ... )
with_output_to(chars(Chars), ... )
See with_output_to/2 documentation:
http://www.swi-prolog.org/pldoc/man?predicate=with_output_to/2
Later on, you could use open_string/2, open_codes_stream/2 and similar predicates to open string/list of codes as an input stream to read data.

F# FSI - Is there a limit to the script length? String Literal appears too long

On transferring a F# file -- Jira.fs to a script file -- Jira.fsx
I am encountering a problem with a constant string that uses triple quotes and within it contains double quotes (example below)
Jira.fs = success
Jira.fsx = fails with error FS0010: Unexpected symbol ':' in expression. Expected '}' or other token.
Warnings include - Warning: line too long, ignoring some characters
Is there a limit to .fsx scripts?
If so is there a recommended way around this? Just read in the string from a file?
Code below: thanks
[<Literal>]
let childIssueSchema = """ {"expand":"renderedFields,names,schema,transitions,operations,editmeta,changelog","id":"18043","self":"https://atlassian.au.MyCompany.net/jira/rest/api/latest/issue/18043","key":"DPMITPRODU-141","fields":{"progress":{"progress":0,"total":0},"summary":"APC - Calculator link on AOL ","customfield_10560":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/customFieldOption/10340","value":"Yes","id":"10340"},"customfield_11067":null,"timetracking":{},"customfield_11066":null,"issuetype":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/issuetype/6","id":"6","description":"A user story","iconUrl":"https://atlassian.au.MyCompany.net/jira/images/icons/sales.gif","name":"User Story","subtask":false},"customfield_10562":null,"customfield_11069":null,"customfield_11068":null,"customfield_11160":"4837","customfield_11161":null,"customfield_11660":null,"timespent":null,"reporter":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/user?username=lkaligotla","name":"lkaligotla","emailAddress":"lkaligotla#MyCompany.com.au","avatarUrls":{"16x16":"https://atlassian.au.MyCompany.net/jira/secure/useravatar?size=small&avatarId=10102","48x48":"https://atlassian.au.MyCompany.net/jira/secure/useravatar?avatarId=10102"},"displayName":"Lakshmi Kaligotla","active":true},"created":"2013-12-31T14:39:09.457+1100","updated":"2014-01-02T09:32:57.023+1100","customfield_10041":null,"priority":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/priority/3","iconUrl":"https://atlassian.au.MyCompany.net/jira/images/icons/priority_major.gif","name":"Medium","id":"3"},"description":"The current Age pension Calculator Icon must lead to a landing page with content and Start Calculator Button \r\n\r\nOn Click of start Calculator button calculator Home Page with instructions and start calculator button ","customfield_10002":null,"customfield_10003":null,"customfield_10040":null,"issuelinks":[{"id":"13177","self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/issueLink/13177","type":{"id":"10010","name":"CrossProjectLink","inward":"part of","outward":"contains","self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/issueLinkType/10010"},"inwardIssue":{"id":"18036","key":"DPMITPROJ-35","self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/issue/18036","fields":{"summary":"APC - Calculator","status":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/status/10003","description":"User has placed this item in the queue","iconUrl":"https://atlassian.au.MyCompany.net/jira/images/icons/status_visible.gif","name":"In Queue","id":"10003"},"priority":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/priority/3","iconUrl":"https://atlassian.au.MyCompany.net/jira/images/icons/priority_major.gif","name":"Medium","id":"3"},"issuetype":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/issuetype/5","id":"5","description":"A big user story that needs to be broken down.","iconUrl":"https://atlassian.au.MyCompany.net/jira/download/resources/com.pyxis.greenhopper.jira:greenhopper-webactions/images/ico_epic.png","name":"Epic","subtask":false}}}}],"customfield_10000":null,"customfield_10765":null,"subtasks":[],"customfield_10767":null,"status":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/status/10016","description":"","iconUrl":"https://atlassian.au.MyCompany.net/jira/images/icons/status_open.gif","name":"Backlog","id":"10016"},"labels":[],"workratio":-1,"project":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/project/DPMITPRODU","id":"11433","key":"DPMITPRODU","name":"DPMIT-Products","avatarUrls":{"16x16":"https://atlassian.au.MyCompany.net/jira/secure/projectavatar?size=small&pid=11433&avatarId=11680","48x48":"https://atlassian.au.MyCompany.net/jira/secure/projectavatar?pid=11433&avatarId=11680"}},"environment":null,"customfield_10053":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/customFieldOption/10030","value":"Yes","id":"10030"},"aggregateprogress":{"progress":0,"total":0},"customfield_10050":"A link is available and on click leads to Calculor Landing page\r\nhttps://adviseronlineportal.com.au/Agepensioncalculator","components":[],"comment":{"startAt":0,"maxResults":0,"total":0,"comments":[]},"timeoriginalestimate":null,"customfield_10461":null,"customfield_10460":null,"customfield_11963":[{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/customFieldOption/11441","value":"False","id":"11441"}],"customfield_10360":null,"votes":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/issue/DPMITPRODU-141/votes","votes":0,"hasVoted":false},"customfield_10261":null,"customfield_10262":null,"customfield_10263":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/customFieldOption/10061","value":"Yes","id":"10061"},"fixVersions":[],"resolution":null,"resolutiondate":null,"aggregatetimeoriginalestimate":null,"customfield_10161":null,"customfield_10160":null,"duedate":null,"customfield_10020":null,"customfield_10060":"4793","watches":{"self":"https://atlassian.au.MyCompany.net/jira/rest/api/2/issue/DPMITPRODU-141/watchers","watchCount":1,"isWatching":false},"customfield_10162":null,"worklog":{"startAt":0,"maxResults":0,"total":0,"worklogs":[]},"assignee":null,"attachment":[],"aggregatetimeestimate":null,"versions":[],"timeestimate":null,"customfield_10030":null,"customfield_10031":null,"aggregatetimespent":null}} """
type ChildJsonSchema = FSharp.Data.JsonProvider< childIssueSchema >
I don't think there is a limit on the size of a script file (not sure what would happen around 2GB, but that does not sound like a realistic scenario), but for some reason (not sure why!) there is a limit on a single line length.
This is a bit unfortunate when you just want to copy & paste sample for a JSON type provider, but if you're using """ quotes, then you can just have a newline character in the string and it will still be treated as a single constant string:
[<Literal>]
let childIssueSchema = """{"foo":1,
"bar":42}"""
type ChildJsonSchema = FSharp.Data.JsonProvider<childIssueSchema>
That said, in case you have long and complex samples, it might be better to save them to files and just point the type provider to a file. Assuming you have childIssue.json in the same folder as the source file, you should be able to write:
type ChildJsonSchema = FSharp.Data.JsonProvider<"childIssue.json">

How to read .docx file using F#

How can I read a .docx file using F#. If I use
System.IO.File.ReadAllText("D:/test.docx")
It is returning me some garbage output with beep sounds.
Here is a F# snippet that may give you a jump-start. It successfully extracts all text contents of a Word2010-created .docx file as a string of concatenated lines:
open System
open System.IO
open System.IO.Packaging
open System.Xml
let getDocxContent (path: string) =
use package = Package.Open(path, FileMode.Open)
let stream = package.GetPart(new Uri("/word/document.xml", UriKind.Relative)).GetStream()
stream.Seek(0L, SeekOrigin.Begin) |> ignore
let xmlDoc = new XmlDocument()
xmlDoc.Load(stream)
xmlDoc.DocumentElement.InnerText
printfn "%s" (getDocxContent #"..\..\test.docx")
In order to make it working do not forget to reference WindowsBase.dll in your VS project.
.docx files follow Open Packaging Convention specifications. At the lowest level, they are .ZIP files. To read it programmatically, see example here:
A New Standard For Packaging Your Data
Packages and Parts
Using F#, it's the same story, you'll have to use classes in the System.IO.Packaging Namespace.
System.IO.File.ReadAllText has type of string -> string.
Because a .docx file is a binary file, it's probable that some of the chars in the strings have the bell character. Rather than ReadAllText, look into Word automation, the Packaging, or the OpenXML APIs
Try using the OpenXML SDK from Microsoft.
Also on the linked page is the Microsoft tool that you can use to decompile the office 2007 files. The decompiled code can be quite lengthy even for simple documents though so be warned. There is a big learning curve associated with OpenXML SDK. I'm finding it quite difficult to use.

Resources