A couple days ago, I posted a question about deserialization with enums in F#.
The question is here: Deserialization in F# vs. C#
The answer pointed to some code written by Isaac Abraham, at: https://gist.github.com/isaacabraham/ba679f285bfd15d2f53e
However I am facing another problem:
If the object to deserialize to has an object of type 'enum option', the deserialization will fail, whereas it'll work if the type is just 'enum'.
A minimal example:
type TestType =
| A = 0
| B = 1
type TestObjectA =
{
test : TestType
}
type TestObjectB =
{
test : TestType option
}
let x = "{\"test\":\"A\"}"
let TestA = Deserialize<TestObjectA> x // will work
let TestB = Deserialize<TestObjectB> x // will fail
and the large deserialization code is at: https://pastebin.com/95JZLa6j
I put the whole code in a fiddle: https://dotnetfiddle.net/0Vc0Rh
but it can't be run from there since the F# version they support will not accept the 'object' keyword.
So, my question is: why can't I use the option type on an enum, but it works on other types? As a side note, since I'm quite new to F#, I'm not fully understanding Isaac's code, although I spent some time going through it and trying to troubleshoot it.
My understanding is that this line:
|> Seq.map (fun (value, propertyInfo) -> Convert.ChangeType(value, propertyInfo.PropertyType))
will try to convert the type to the right enum, but not to the enum option.
As a bonus question, is there a working solution that does full idiomatic deserialization with enums? (without going through null types)
open System.IO
type TestType =
| A = 0
| B = 1
type TestObjectB =
{
test : TestType option
}
let jsonSerializeToString obj =
use writer = new StringWriter()
let ser = new Newtonsoft.Json.JsonSerializer()
ser.Formatting <- Newtonsoft.Json.Formatting.Indented
ser.Serialize(writer, obj)
writer.ToString()
let jsonDeserializeFromString str =
Newtonsoft.Json.JsonConvert.DeserializeObject<TestObjectB>(str)
let Test obj =
let str = jsonSerializeToString obj
let obj' = jsonDeserializeFromString str
obj'
[<EntryPoint>]
let main argv =
{ test = Some TestType.B } |> Test |> ignore
{ test = None } |> Test |> ignore
0
Note: if you need to serialize a large collection of objects, then stream them to a file instead of an in-memory string to avoid an OutOfMemoryException. Like use writer = File.CreateText(filePath).
As a bonus question, is there a working solution that does full
idiomatic deserialization with enums?
I use the Microsoft.FsharpLu.Json package in production and find it works quite well for serializing and deserializing between "plain" javascript and idiomatic F#. Note Microsoft.FsharpLu.Json relies on Newtonsoft.Json under the hood.
Below is an example with your types and your test string, using Expecto for tests.
namespace FsharpLuJsonTest
open Newtonsoft.Json
open Microsoft.FSharpLu.Json
open Expecto
open Expecto.Flip
// Setup for FSharpLu.Json
type JsonSettings =
static member settings =
let s = JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
MissingMemberHandling = MissingMemberHandling.Ignore)
s.Converters.Add(CompactUnionJsonConverter())
s
static member formatting = Formatting.None
type JsonSerializer = With<JsonSettings>
// Your example
type TestType =
| A = 0
| B = 1
type TestObjectA = { test : TestType }
type TestObjectB = { test : TestType option }
module Tests =
let x = """{"test":"A"}"""
[<Tests>]
let tests =
testList "Deserialization Tests" [
testCase "To TestObjectA" <| fun _ ->
JsonSerializer.deserialize x
|> Expect.equal "" { TestObjectA.test = TestType.A }
testCase "To TestObjectB" <| fun _ ->
JsonSerializer.deserialize x
|> Expect.equal "" { TestObjectB.test = Some TestType.A }
]
module Main =
[<EntryPoint>]
let main args =
runTestsInAssembly defaultConfig args
As you can see FsharpLu.Json supports Discriminated Unions and option types out of the box in the way you prefer. FsharpLu.Json is a less flexible solution than some others like Chiron (which allow for much more customisation) but I tend to prefer the opinionated approach of FsharpLu.Json.
I haven't used it personally, but the new FSharp.SystemText.Json library with the JsonUnionEncoding.ExternalTag setting should work roughly the same way FsharpLu.Json does. That library uses Microsoft's new System.Text.Json library under the hood rather than Newtonsoft.Json.
Related
For a while F# has supported the ability to auto-quote using [<ReflectedDefinitionAttribute>]. Is there anything similar for laziness?
e.g.
member __.Quoted ([<ReflectedDefinitionAttribute>] quotation:Expr<'T>) = ...
member __.Thunked ([<LazyAttribute>] thunk:Lazy<'T>) = ...
I suppose I could use something like
member __.Quoted ([<ReflectedDefinitionAttribute>] quotation:Expr<'T>) =
Lazy (evaluate (<# fun () -> %quotation #>)) // evaluate using Unquote or similar
But wouldn't this be costly?
UPDATE:
I found a hack, it's not exactly what I would like but it give the correct behavior.
type Signal = Signal with
member __.Return x = x
member __.Delay (f:unit -> _) = f
let a = Signal { return randint }
let b = Signal { return randint }
let c = Signal { return a() + b() }
There is nothing like the ReflectedDefinition attribute for automatically turning things into delayed Lazy<'T> computations.
You are right that automatically quoting the argument achieves something like this. You could use the (very limited) LeafExpressionConverter.EvaluateQuotation to do this for some limited kinds of expressions, but as you note, this would be inefficient. The following is a proof of concept though (but you cannot call custom functions in the branches as this uses LINQ expressions):
open Microsoft.FSharp.Quotations
open Microsoft.FSharp.Linq.RuntimeHelpers
type A =
static member If<'T>(c:bool,
[<ReflectedDefinition>] t:Expr<'T>,
[<ReflectedDefinition>] f:Expr<'T>) =
if c then LeafExpressionConverter.EvaluateQuotation t :?> 'T
else LeafExpressionConverter.EvaluateQuotation f :?> 'T
A.If(1 = 2, 0, 1)
In practice, I think a more reasonable approach is to just use the built-in Lazy<'T> values. F# has a (not widely known) lazy keyword that gives you a bit nicer syntax for creating those:
let iff c (t:Lazy<_>) (f:Lazy<_>) =
if c then t.Value else f.Value
iff (1 = 2)
(lazy (printfn "true"; 41))
(lazy (printfn "false"; 42))
I'm trying to understand the reader monad transformer. I'm using FSharpPlus and try to compile the following sample which first reads something from the reader environment, then performs some async computation and finally combines both results:
open FSharpPlus
open FSharpPlus.Data
let sampleReader = monad {
let! value = ask
return value * 2
}
let sampleWorkflow = monad {
do! Async.Sleep 5000
return 4
}
let doWork = monad {
let! envValue = sampleReader
let! workValue = liftAsync sampleWorkflow
return envValue + workValue
}
ReaderT.run doWork 3 |> Async.RunSynchronously |> printfn "Result: %d"
With this I get a compilation error at the line where it says let! value = ask with the following totally unhelpful (at least for me) error message:
Type constraint mismatch when applying the default type 'obj' for a type inference variable. No overloads match for method 'op_GreaterGreaterEquals'.
Known return type: Async
Known type parameters: < obj , (int -> Async) >
It feels like I'm just missing some operator somewhere, but I can't figure it out.
Your code is correct, but F# type inference is not that smart in cases like this.
If you add a type annotation to sampleReader it will compile fine:
let sampleReader : ReaderT<int,Async<_>> = monad {
let! value = ask
return value * 2
}
// val sampleReader : FSharpPlus.Data.ReaderT<int,Async<int>> =
// ReaderT <fun:sampleReader#7>
Update:
After reading your comments.
If what you want is to make it generic, first of all your function has to be declared inline otherwise type constraints can't be applied:
let inline sampleReader = monad ...
But that takes you to the second problem: a constant can't be declared inline (actually there is a way but it's too complicated) only functions can.
So the easiest is to make it a function:
let inline sampleReader () = monad ...
And now the third problem the code doesn't compile :)
Here again, you can give type inference a minimal hint, just to say at the call site that you expect a ReaderT<_,_> will be enough:
let inline sampleReader () = monad {
let! value = ask
return value * 2
}
let sampleWorkflow = monad {
do! Async.Sleep 5000
return 4
}
let doWork = monad {
let! envValue = sampleReader () : ReaderT<_,_>
let! workValue = liftAsync sampleWorkflow
return envValue + workValue
}
ReaderT.run doWork 3 |> Async.RunSynchronously |> printfn "Result: %d"
Conclusion:
Defining a generic function is not that trivial task in F#.
If you look into the source of F#+ you'll see what I mean.
After running your example you'll see all the constraints being generated and you'll probably noted how the compile-time increased by making your function inline and generic.
These are all indications that we're pushing F# type system to the limits.
Although F#+ defines some ready-to-use generic functions, and these functions can sometimes be combined in such a way that you create your own generic functions, that's not the goal of the library, I mean you can but then you're on your own, in some scenarios like exploratory development it might make sense.
I want to create a sequence of 'candles' ( a simple structure) for performing testing on financial data. I have downloaded some financial data from a broker and they are in a csv format.
I have the following code, using type providers:
type DukasCandles = CsvProvider<"C:\Users\**\Documents\Visual Studio 2017\Projects\FI\FI\Data\schema.csv" >
let row2Candle (myTicker: string ) (mySide: Side) (aRow:DukasCandles.Row) : Candle =
{Open = aRow.Open ;
Close = aRow.Close;
Low = aRow.Low
High = aRow.High
StartTime = aRow.LocalTime
Volume = aRow.Volume
Ticker = myTicker
Side = mySide }
let sortedCandles aTicker aParsedFile =
aParsedFile
|> Seq.map ( row2Candle aTicker Side.Bid )
|> Seq.sortBy ( fun candle -> candle.StartTime)
let fileContent = inputFile |> DukasCandles.Load
let rows = fileContent.Rows
let dataContent = rows|> (sortedCandles "EUR_USD")
It is just a toy example to test my understanding of type providers. I have it in a FSX script after the necessary boilerplate for declaring the file name and open the necessary modules.
Now the question: if I highlight this as it is and try to execute in F# interactive, then type
dataContent;;
I get the following message
val it : seq<Candle> =
Error: Couldn't parse row 1057 according to schema: Expecting DateTime in LocalTime, got 13.01.2016 00:00:00.000
The odd thing is the following:
suppose I type the following code in F# interactive:
fileContent;;
get the answer and then refer to it through the 'it' keyword, as in:
it.Rows |> sortedCandles "EUR_USD";;
Then the code is executed with no problems.
Why do we have such an inconsistent behaviour? Any idea?
Any help highly appreciated. Thank you.
I am trying to set the message formatter for a message in F#. In C# I can have:
foreach (System.Messaging.Message message in messages)
{
message.Formatter = new XmlMessageFormatter(new String[] { "System.String,mscorlib" });
string body = message.Body.ToString();
Console.WriteLine(body);
}
which works just fine. I now want to do the same thing in F# and have:
let mList = messageQueue.GetAllMessages()
let xt = [| "System.String,mscorlib" |]
for m in mList do
m.Formatter = XmlMessageFormatter(xt)
which causes this error at compile time:
Error 2 This expression was expected to have type
IMessageFormatter
but here has type
XmlMessageFormatter
I suspect I am missing a basic concept in F#. What am I doing wrong?
--EDIT--
latkin's answer worked perfectly. Just in case anyone else is interested, here is the full working program in F#:
open System.Messaging
[<EntryPoint>]
let main argv =
printfn "%A" argv
let messageQueue = new MessageQueue(".\private$\Twitter")
messageQueue.MessageReadPropertyFilter.SetAll();
let mList = messageQueue.GetAllMessages()
let xt = [| "System.String,mscorlib" |]
for m in mList do
m.Formatter <- XmlMessageFormatter(xt)
printfn "%s " (m.Body.ToString())
0 // return an integer exit code
When you are assigning a mutable value, the operator is <-, not =. In F# = is only used for initial bindings, otherwise it's used as the Boolean equality operator (like C-family ==). Some docs here.
You want
let mList = messageQueue.GetAllMessages()
let xt = [| "System.String,mscorlib" |]
for m in mList do
m.Formatter <- XmlMessageFormatter(xt)
No casting is needed in this case.
The error comes up because the compiler thinks you are trying to compare a IMessageFormatter to a XmlMessageFormatter.
F# doesn't have implicit casts like C# does, so it doesn't automatically upcast your XmlMessageFormatter to the IMessageFormatter used by the Formatter property.
There was a similar question a couple of days ago with more information about this:
F# return ICollection
I have a FileReader class whose job is to read and process text files using a StreamReader. To facilitate unit testing, I'd like to provide a type parameter to this class so that I can swap the StreamReader for a FakeReader that doesn't actually interact with the file system (and maybe throws exceptions such as OutOfMemory, so I can test the error handling in FileReader).
Ideally, I'd like to define FileReader something like this (trivialized for clarity):
type FileReader<'Reader> =
member this.Read file =
use sr = new 'Reader(file)
while not sr.EndOfStream do
printfn "%s" <| sr.ReadLine()
and simply define FakeReader to have a constructor that takes the file name, the EndOfStream property getter, the ReadLine() method, and the (empty) Dispose() method. However, F# has several complaints about this type definition, including "Calls to object constructors on type parameters cannot be given arguments." Since StreamReader has no default constructor, this approach seems like a no-go.
So far the only way I've gotten this to work is to inherit FakeReader from StreamReader:
type FakeReader() =
inherit StreamReader("") with
override this.ReadLine() = "go away"
member this.EndOfStream = false
member this.Dispose() = ()
and use a factory method that returns either a new FakeReader or a new StreamReader as appropriate.
type ReaderType = Fake | SR
let readerFactory (file : string, readerType) =
match readerType with
| Fake -> new FakeReader() :> StreamReader
| SR -> new StreamReader(file)
type FileReader(readertype) =
member this.Read file =
use sr = readerFactory(file, readertype)
while not sr.EndOfStream do
printfn "%s" <| sr.ReadLine()
This seems a lot less elegant. Is there a way to do this with a type parameter? Thanks to all.
Using a function that creates a reader object (as suggested by MizardX) is the direct answer to your question. However, I'd maybe consider using a different abstraction than TextReader). As Ankur mentioned in a comment, you could use a more functional approach.
If you're just reading lines of text from the input using TextReader, you could use a seq<string> type instead. The FileReader type may actually be just a function taking seq<string> (although that may be oversimplification... it depends).
This makes it more "functional" - in functional programming, you're often transforming data structures using functions, which is exactly what this example does:
open System.IO
/// Creates a reader that reads data from a file
let readFile (file:string) = seq {
use rdr = new StreamReader(file)
let line = ref ""
while (line := rdr.ReadLine(); !line <> null) do
yield !line }
/// Your function that processes the input (provided as a sequence)
let processInput input =
for s in input do
printfn "%s" s
readFile "input.txt" |> processInput
To test the processInput function, you could then create a new seq<string> value. This is significantly easier than implementing a new TextReader class:
let testInput = seq {
yield "First line"
yield "Second line"
raise <| new System.OutOfMemoryException() }
testInput |> processInput
You could pass in a function that constructs and returns an object of your desired type.
type FileReader(f : string -> TextReader) =
member this.Read file =
use sr = f file
while sr.Peek() <> -1 do
printfn "%s" <| sr.ReadLine()
type FakeReader() =
inherit StringReader("")
override this.ReadLine() = "go away"
override this.Peek() = 0
let reader1 = new FileReader(fun fn -> new StreamReader(fn) :> _)
let reader2 = new FileReader(fun fn -> new FakeReader() :> _)
Cast was necessary because I dropped the generic type-argument, but the actual type can be inferred.