F# reading and writing to and from unmanaged data - f#

I'm exploring low level programming in F#, I do know about the risk and general "don't do this", that is not what I'm asking for.
I'm currently trying the basic stuff of just getting things to work.
right now I having the problem that I got some simple type (code below) and I put some data into the memory and trying to read that data as the struct. but it keeps thronging the System.Reflection.TargetInvocationException saying I'm trying to read protected memory?
#nowarn "9"
open FSharp.NativeInterop
open System.Runtime.InteropServices
type msgtype = OK = 0 | ERR = 1
[<type: Struct; StructLayout(LayoutKind.Explicit)>]
type msg =
[<field: FieldOffsetAttribute(0); MarshalAs(UnmanagedType.I1)>]
val tp : byte
[<field: FieldOffsetAttribute(1); MarshalAs(UnmanagedType.I4)>]
val content : int
let memory =
Marshal.AllocHGlobal(1024)
let read n =
let adr = memory
let ptr = NativePtr.ofNativeInt<_> memory
NativePtr.read ptr
let write item adr =
let adr' = memory + adr
let ptr = NativePtr.ofNativeInt adr'
NativePtr.write ptr item
write 1uy memory
write 654 (memory+1n)
let m : msg = read memory
Marshal.FreeHGlobal memory

I solved the problem my self.
Apparently the write function was the problem.
I changed it to
let write add item =
NativePtr.write (NativePtr.ofNativeInt<_> add) item
and it just worked.
can say if it is the declarings that course to problem only that somehow it is different?

Related

writing binary files with streams

How to download image files via streams in the temp directory, I've following code and I'm stuck and need guidance with the seek and the count part. There are some wrapper approaches but I'm looking specifically for while loop approach for RAM efficiency reasons.
Writing
let tempFileName = Path.GetTempFileName()
let request = WebRequest.CreateHttp "http://example.com/image.png"
use response = request.GetResponse() :?> HttpWebResponse
use stream = response.GetResponseStream()
let buffer = Array.zeroCreate 1024
use reader = new BinaryReader(stream)
use memoryStream = new MemoryStream()
use fileStream = new FileStream(tempFileName, FileMode.Open)
while not (reader.PeekChar() <> -1) do
fileStream.Write(reader.ReadBytes(1024), 0, 1024)
return Ok (tempFileName)
First of all, I notice that although you're creating a buffer array, you're not actually using it. Second, when I look at the BinaryReader documentation, and specifically the documentation for the ReadBytes method, I notice that it takes an int parameter and returns a byte array. This must mean that it's allocating a new array every time, which seems to be the opposite of what you intend (since you mention RAM efficiency, I assume that what you actually want is to re-use the same buffer each time).
And one other observation: the ReadBytes method says that it might return an array smaller than the requested size, if there were fewer bytes available. Your code currently isn't handling that case.
All of these can be fixed, though, by switching to the BinaryReader.Read(byte[], int, int) method instead. With this method, your while loop would look something like the following:
while not (reader.PeekChar() <> -1) do
let bytesRead = reader.Read(buffer, 0, 1024)
fileStream.Write(buffer, 0, bytesRead)
And now that we're keeping track of how many bytes were read by each Read operation, we can get rid of the PeekChar call and save ourselves some time (calling PeekChar on something you're downloading is not without cost since the library has to download the next byte, then save it somewhere so it can be returned the next time you call Read). We can do that by checking how many bytes were read at the previous call: if it was 0, then that means we're at the end of the stream. To do this, we'll move the bytesRead variable out of the loop, which means making it a mutable variable that we'll re-use every time through the loop:
let mutable bytesRead = -1
while not (bytesRead = 0) do
bytesRead <- reader.Read(buffer, 0, 1024)
fileStream.Write(buffer, 0, bytesRead)
Or if you want to be slightly more explicit about the fact that you're skipping Write if bytesRead is 0, you could add an if block:
let mutable bytesRead = -1
while not (bytesRead = 0) do
bytesRead <- reader.Read(buffer, 0, 1024)
if bytesRead > 0 then
fileStream.Write(buffer, 0, bytesRead)
That last if statement isn't strictly necessary, though: FileStream.Write should just return without doing anything if it's asked to write 0 bytes. However, since that's not documented anywhere that I could find, I added the if statement in this last code sample just to be on the safe side.
As of .NET 4.6.2, there is System.IO.Stream#CopyTo method:
namespace FSharpBasics
module ImageCrawler =
open System.Net
open System.IO
open System.Text.RegularExpressions
let private myurl = "https://cdn.pixabay.com/photo/2016/07/06/15/29/math-1500720_960_720.jpg"
let crawler (url: string) =
let fileName = Regex.Match(url, #"\/([^\/]+)$", RegexOptions.RightToLeft).Groups.[1].Value
let request = WebRequest.CreateHttp url
let response = request.GetResponse()
use s = response.GetResponseStream()
use w = File.Create fileName
s.CopyTo w
w.Flush true
[<EntryPoint>]
let main argv =
printfn "JPEG file will be saved"
crawler myurl
printf "Saved"
0

How to convert a data array to a record using idiomatic F#

I'm trying to create a communication library that interacts with hardware. The protocol is made up of byte arrays with a header (source/destination address, command number, length) and a command specific payload. I'm creating Record Types for each of the commands to make them more user friendly.
Is there a more idiomatic way of converting an array to a record than
let data = [0;1]
type Rec = {
A : int
B : int
}
let convert d =
{
A = d.[0]
B = d.[1]
}
This can become very tedious when the records are much larger.
A few comments:
You record type definition is bogus - there should be no = in there. I assume you want
type Rec = {
A : int
B : int
}
You mentioned byte arrays, but your data value is a List. Accessing List items by index is expensive (O(n)) and should be avoided. If you meant to declare it as an array, the syntax is let data = [|0;1|]
But I wonder if records are the right fit here. If your goal is to have a single function that accepts a byte array and returns back various strongly-typed interpretations of that data, then a discriminated union might be best.
Maybe something along these lines:
// various possible command types
type Commands =
| Command1 of byte * int // maybe payload of Command1 is known to be an int
| Command2 of byte * string // maybe payload of Command1 is known to be a string
// active pattern for initial data decomposition
let (|Command|) (bytes : byte[]) =
(bytes.[0], bytes.[1], Array.skip 2 bytes)
let convert (bytes : byte[]) =
match bytes with
| Command(addr, 1uy, [| intData |]) ->
Command1(addr, int intData)
| Command(addr, 2uy, strData) ->
Command2(addr, String(Text.Encoding.ASCII.GetChars(strData)))
| _ ->
failwith "unknown command type"
// returns Command1(0x10, 42)
convert [| 0x10uy; 0x01uy; 0x2Auy |]
// returns Command2(0x10, "foobar")
convert [| 0x10uy; 0x02uy; 0x66uy; 0x6Fuy; 0x6Fuy; 0x62uy; 0x61uy; 0x72uy |]

How to use SQLDriverConnect from f#

I am porting a VB.NET application to F# as an experiment. The VB program uses SQLDriverConnect, so I need to call it from F#. I cannot get the pinvoke/extern declaration to work properly. The call to SQLDriver connect always returns -2, SQL_INVALID_HANDLE, instead of prompting for a connection as expected.
Anybody know how to get this to work?
open System
open System.Runtime.InteropServices
open System.Text
[<DllImport("odbc32.dll")>]
extern Int16 SQLAllocEnv(IntPtr& EnvironmentHandle);
[<DllImport("odbc32.dll")>]
extern Int16 SQLDriverConnect(IntPtr hdbc, IntPtr hwnd, string szConnStrIn,
Int16 cbConnStrIn, StringBuilder szConnStrOut,
Int16 cbConnStrOutMax, Int16& pcbConnStrOut,
UInt16 fDriverCompletion)
let getConnectionString () =
let SQL_DRIVER_PROMPT = 2us
let mutable henv = IntPtr(0)
let mutable csLen = 0s
let rc1 = SQLAllocEnv &henv
assert (rc1 = 0s)
let csOut = new StringBuilder(1024)
let rc2 = SQLDriverConnect(henv, IntPtr.Zero, "", 0s, csOut, 1024s, &csLen, SQL_DRIVER_PROMPT)
assert (rc2 = 0s)
csOut.ToString()
[<EntryPoint>]
let main argv =
printfn "Connection string: %s" (getConnectionString())
0 // return an integer exit code
I don't do F# but in C you call SQLAllocEnv (or SQLAllocHandle) to create an environment handle, then you call SQLSetEnvAttr to set the version of ODBC you want, then SQLAllocConnect (or SQLAllocHandle) to allocate a connection handle and lastly call SQLDriverConnect with the connection handle. Your code looks to be passing an environment handle to SQLDriverConnect but SQLDriverConnect needs a connection handle hence SQL_INVALID_HANDLE.

How to pass F# a string and get the result back in c# [duplicate]

This question already has answers here:
Call F# code from C#
(4 answers)
Closed 8 years ago.
I am SQL developer and am really new to both F# and C#. I need help on how to pass a string to f# function below and to return the result from F# to C#.
Description of project:
I am using stanford postagger to tag a sentence with the parts of speech.
Reference link from where i copied this code.
(http://sergey-tihon.github.io/Stanford.NLP.NET/StanfordPOSTagger.html)
module File1
open java.io
open java.util
open edu.stanford.nlp.ling
open edu.stanford.nlp.tagger.maxent
// Path to the folder with models
let modelsDirectry =
__SOURCE_DIRECTORY__ + #'..\stanford-postagger-2013-06-20\models\'
// Loading POS Tagger
let tagger = MaxentTagger(modelsDirectry + 'wsj-0-18-bidirectional-nodistsim.tagger')
let tagTexrFromReader (reader:Reader) =
let sentances = MaxentTagger.tokenizeText(reader).toArray()
sentances |> Seq.iter (fun sentence ->
let taggedSentence = tagger.tagSentence(sentence :?> ArrayList)
printfn "%O" (Sentence.listToString(taggedSentence, false))
)
// Text for tagging
let text = System.Console.ReadLine();
tagTexrFromReader <| new StringReader(text)
it won't matter if C# or F# - do make a function that gets a string and returns ... let
s say an int, you just need something like this (put it in some MyModule.fs):
namespace MyNamespace
module MyModule =
// this is your function with one argument (a string named input) and result of int
let myFun (input : string) : int =
// do whatever you have to
5 // the value of the last line will be your result - in this case a integer 5
call it in from C#/.net with
int result = MyNamespace.MyModule.myFun ("Hallo");
I hope this helps you out a bit
For your example this would be:
let myFun (text : string) =
use reader = new StringReader(text)
tagTexrFromReader reader
as you'll have this in the module File1 you can just call it with var res = Fiel1.myFun(text);
BTW: use is in there because StringReader is IDisposable and using use F# will dispose the object when you exit the scope.
PS: is tagTexrFromReader a typo?

F# lazy eval from stream reader?

I'm running into a bug in my code that makes me think that I don't really understand some of the details about F# and lazy evaluation. I know that F# evaluates eagerly and therefore am somewhat perplexed by the following function:
// Open a file, then read from it. Close the file. return the data.
let getStringFromFile =
File.OpenRead("c:\\eo\\raw.txt")
|> fun s -> let r = new StreamReader(s)
let data = r.ReadToEnd
r.Close()
s.Close()
data
When I call this in FSI:
> let d = getStringFromFile();;
System.ObjectDisposedException: Cannot read from a closed TextReader.
at System.IO.__Error.ReaderClosed()
at System.IO.StreamReader.ReadToEnd()
at <StartupCode$FSI_0134>.$FSI_0134.main#()
Stopped due to error
This makes me think that getStringFromFile is being evaluated lazily--so I'm totally confused. I'm not getting something about how F# evaluates functions.
For a quick explanation of what's happening, lets start here:
let getStringFromFile =
File.OpenRead("c:\\eo\\raw.txt")
|> fun s -> let r = new StreamReader(s)
let data = r.ReadToEnd
r.Close()
s.Close()
data
You can re-write the first two lines of your function as:
let s = File.OpenRead(#"c:\eo\raw.txt")
Next, you've omitted the parentheses on this method:
let data = r.ReadToEnd
r.Close()
s.Close()
data
As a result, data has the type unit -> string. When you return this value from your function, the entire result is unit -> string. But look what happens in between assigning your variable and returning it: you closed you streams.
End result, when a user calls the function, the streams are already closed, resulting in the error you're seeing above.
And don't forget to dispose your objects by declaring use whatever = ... instead of let whatever = ....
With that in mind, here's a fix:
let getStringFromFile() =
use s = File.OpenRead(#"c:\eo\raw.txt")
use r = new StreamReader(s)
r.ReadToEnd()
You don't read from your file. You bind method ReadToEnd of your instance of StreamReader to the value data and then call it when you call getStringFromFile(). The problem is that the stream is closed at this moment.
I think you have missed the parentheses and here's the correct version:
// Open a file, then read from it. Close the file. return the data.
let getStringFromFile =
File.OpenRead("c:\\eo\\raw.txt")
|> fun s -> let r = new StreamReader(s)
let data = r.ReadToEnd()
r.Close()
s.Close()
data

Resources