I wrote this script from some resources I found. It's working but I some files I have problem. I am new in F# so how can I change line with FileHelpersException to get exact line where is the problem? Thanks
// Learn more about F# at http://fsharp.net
// See the 'F# Tutorial' project for more help.
open FileHelpers
open System
[<DelimitedRecord(",")>]
type CsvRecord =
class
val field1 : string
val field2 : string
val field3 : int
new () = {
field1 = ""
field2 = ""
field3 = 0
}
end
[<EntryPoint>]
let main argv =
use file = System.IO.File.CreateText("result.txt")
let engine = new FileHelperEngine<CsvRecord>()
engine.Encoding <- new Text.UTF8Encoding()
let res =
try
engine.ReadFile("test.csv")
with
| :? FileHelpersException -> Array.empty<CsvRecord>
for record in res do
fprintfn file "%s" record.field1
printf "DONE!"
let s = Console.ReadLine()
0 // return an integer exit code
I suggest that you use CsvTypeProvider instead. When there's a mismatch the error message states the line which has the error
open FSharp.Data
[<EntryPoint>]
let main argv =
use file = System.IO.File.CreateText("result.txt")
let csv = new CsvProvider<"test.csv">()
for record in csv.Data do
fprintfn file "%s" record.field1
If you want to ignore the lines with errors, just pass IgnoreErrors=true as an extra parameter to CsvProvider
This question is about the FileHelpers library you are using, not F#, so looking at the docs for that might help. In this case you can check for ConvertException instead of FileHelpersException, which contains members that give you more details about the member.
try
engine.ReadFile("test.csv")
with
| :? ConvertException as ex ->
printfn "ERROR: Line %d Col %d" ex.LineNumber ex.ColumnNumber
Array.empty<CsvRecord>
I agree with Gustavo though, you might find it easier to use the CsvTypeProvider.
Related
I've got an application that I've built in SAFE-Stack using websockets, more or less following the approach here: https://github.com/CompositionalIT/safe-sockets
It works fine but the Elmish debugger doesn't like the type of WsSender in this example:
type ConnectionState =
| DisconnectedFromServer
| ConnectedToServer of WsSender
| Connecting
member this.IsConnected =
match this with
| ConnectedToServer _ -> true
| DisconnectedFromServer | Connecting -> false
and WsSender = Msg -> Unit
giving the following error message in the Browser Console:
Can anyone tell me how to go about fixing this issue? (Assuming it's fixable and that I've diagnosed the problem correctly.) Thanks.
you see this error because of Elmish.Debugger using Thoth.Json to serialize your Msg/Model to a JSON format.
The type WsSender can't be represented in a JSON format because it is a function. So Thoth.Json is asking you to explain how it should encode this type.
You can do that by creating what is called an extraCoder like that:
In your case, you will have to create a fake encoder/decoder "just" to make the Debugger happy.
module CustomEncoders =
let wsSenderEncoder (_ : WsSender) = Encode.string "WsSender function"
let wsSenderDecoder = Decode.fail "Decoding is not supported for WsSender type"
let myExtraCoders =
Extra.empty
|> Extra.withCustom wsSenderEncoder wsSenderDecoder
let modelEncoder = Encode.Auto.generateEncoder(extra = myExtraCoders)
let modelDecoder = Decode.Auto.generateDecoder(extra = myExtraCoders)
In your Program creation, you should replace Program.withDebugger by Program.withDebuggerCoders and give it the encoder and decoder you created.
Program.withDebuggerCoders CustomEncoders.modelEncoder CustomEncoders.modelDecoder
I had a bit of a play around to try and come up with something that would make it easier to have multiple extra coders if required. This seems to work - thought it might be helpful to others.
module CustomEncoders =
let inline addDummyCoder<'b> extrasIn =
let typeName = string typeof<'b>
let simpleEncoder(_ : 'b) = Encode.string (sprintf "%s function" typeName)
let simpleDecoder = Decode.fail (sprintf "Decoding is not supported for %s type" typeName)
extrasIn |> Extra.withCustom simpleEncoder simpleDecoder
let inline buildExtras<'a> extraCoders =
let myEncoder:Encoder<'a> = Encode.Auto.generateEncoder(extra = extraCoders)
let myDecoder:Decoder<'a> = Decode.Auto.generateDecoder(extra = extraCoders)
(myEncoder, myDecoder)
type TestType = Msg -> Unit
type TestType2 = string -> Unit
let extras = Extra.empty
|> CustomEncoders.addDummyCoder<TestType>
|> CustomEncoders.addDummyCoder<TestType2>
|> CustomEncoders.buildExtras<Model.Model>
#if DEBUG
open Elmish.Debug
open Elmish.HMR
#endif
Program.mkProgram Model.init Model.update View.render
|> Program.withSubscription subs
#if DEBUG
|> Program.withConsoleTrace
#endif
|> Program.withReactBatched "elmish-app"
#if DEBUG
|> Program.withDebuggerCoders (fst extras) (snd extras)
#endif
|> Program.run
If anyone has a better idea of how to do it, I'd be happy to update this answer with their suggestions. Also, the apostrophe in the generic type seems to upset the code prettifier above - do I need to do something to fix that?
A couple days ago, I posted a question about deserialization with enums in F#.
The question is here: Deserialization in F# vs. C#
The answer pointed to some code written by Isaac Abraham, at: https://gist.github.com/isaacabraham/ba679f285bfd15d2f53e
However I am facing another problem:
If the object to deserialize to has an object of type 'enum option', the deserialization will fail, whereas it'll work if the type is just 'enum'.
A minimal example:
type TestType =
| A = 0
| B = 1
type TestObjectA =
{
test : TestType
}
type TestObjectB =
{
test : TestType option
}
let x = "{\"test\":\"A\"}"
let TestA = Deserialize<TestObjectA> x // will work
let TestB = Deserialize<TestObjectB> x // will fail
and the large deserialization code is at: https://pastebin.com/95JZLa6j
I put the whole code in a fiddle: https://dotnetfiddle.net/0Vc0Rh
but it can't be run from there since the F# version they support will not accept the 'object' keyword.
So, my question is: why can't I use the option type on an enum, but it works on other types? As a side note, since I'm quite new to F#, I'm not fully understanding Isaac's code, although I spent some time going through it and trying to troubleshoot it.
My understanding is that this line:
|> Seq.map (fun (value, propertyInfo) -> Convert.ChangeType(value, propertyInfo.PropertyType))
will try to convert the type to the right enum, but not to the enum option.
As a bonus question, is there a working solution that does full idiomatic deserialization with enums? (without going through null types)
open System.IO
type TestType =
| A = 0
| B = 1
type TestObjectB =
{
test : TestType option
}
let jsonSerializeToString obj =
use writer = new StringWriter()
let ser = new Newtonsoft.Json.JsonSerializer()
ser.Formatting <- Newtonsoft.Json.Formatting.Indented
ser.Serialize(writer, obj)
writer.ToString()
let jsonDeserializeFromString str =
Newtonsoft.Json.JsonConvert.DeserializeObject<TestObjectB>(str)
let Test obj =
let str = jsonSerializeToString obj
let obj' = jsonDeserializeFromString str
obj'
[<EntryPoint>]
let main argv =
{ test = Some TestType.B } |> Test |> ignore
{ test = None } |> Test |> ignore
0
Note: if you need to serialize a large collection of objects, then stream them to a file instead of an in-memory string to avoid an OutOfMemoryException. Like use writer = File.CreateText(filePath).
As a bonus question, is there a working solution that does full
idiomatic deserialization with enums?
I use the Microsoft.FsharpLu.Json package in production and find it works quite well for serializing and deserializing between "plain" javascript and idiomatic F#. Note Microsoft.FsharpLu.Json relies on Newtonsoft.Json under the hood.
Below is an example with your types and your test string, using Expecto for tests.
namespace FsharpLuJsonTest
open Newtonsoft.Json
open Microsoft.FSharpLu.Json
open Expecto
open Expecto.Flip
// Setup for FSharpLu.Json
type JsonSettings =
static member settings =
let s = JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
MissingMemberHandling = MissingMemberHandling.Ignore)
s.Converters.Add(CompactUnionJsonConverter())
s
static member formatting = Formatting.None
type JsonSerializer = With<JsonSettings>
// Your example
type TestType =
| A = 0
| B = 1
type TestObjectA = { test : TestType }
type TestObjectB = { test : TestType option }
module Tests =
let x = """{"test":"A"}"""
[<Tests>]
let tests =
testList "Deserialization Tests" [
testCase "To TestObjectA" <| fun _ ->
JsonSerializer.deserialize x
|> Expect.equal "" { TestObjectA.test = TestType.A }
testCase "To TestObjectB" <| fun _ ->
JsonSerializer.deserialize x
|> Expect.equal "" { TestObjectB.test = Some TestType.A }
]
module Main =
[<EntryPoint>]
let main args =
runTestsInAssembly defaultConfig args
As you can see FsharpLu.Json supports Discriminated Unions and option types out of the box in the way you prefer. FsharpLu.Json is a less flexible solution than some others like Chiron (which allow for much more customisation) but I tend to prefer the opinionated approach of FsharpLu.Json.
I haven't used it personally, but the new FSharp.SystemText.Json library with the JsonUnionEncoding.ExternalTag setting should work roughly the same way FsharpLu.Json does. That library uses Microsoft's new System.Text.Json library under the hood rather than Newtonsoft.Json.
Is there a way to make fsi print the documentation for a function?
I can retrieve the type by just evaluating the function as a value but I would like the know what the xml docs say if they exist.
Without the use case I have 2 suggestions. If this is just about help then use and FSX file.
If you are actually wanting to print out the docs in FSI you could use the following code to print out the docs for a member. Since the xml comments are not stored in the dll, this code checks if the xml doc exists in the location of the dll and loads that in if so.
#r "packages/Newtonsoft.Json/lib/net45/Newtonsoft.Json.dll"
open System.IO
open System.Xml
open System.Reflection
let loadXml (path:string) =
if (File.Exists(path)) then
let xml = new XmlDocument()
xml.Load(path)
Some xml
else None
let xmlForMember (maybeXml:XmlDocument option) (mi:MemberInfo) =
let path = sprintf "M:%s.%s" mi.DeclaringType.FullName mi.Name
match maybeXml with
| None -> None
| Some xml -> xml.SelectSingleNode("//member[starts-with(#name, '" + path + "')]") |> Some
let docFrom (node:XmlNode option) =
match node with
| None -> "No docs available"
| Some n -> n.InnerXml
Usage would be something like this but you could neaten and package this up for your needs:
let t = typedefof<Newtonsoft.Json.JsonSerializer>
let assembly = t.Assembly
let dllPath = assembly.Location
printfn "Assembly location: %s" dllPath
let expectedXmlPath = Path.ChangeExtension(dllPath, ".xml")
printfn "Expected xml: %s" expectedXmlPath
let xmlDoc = expectedXmlPath |> loadXml
let mi = t.GetMember("Create").[0]
let docNode = mi |> xmlForMember xmlDoc
docNode |> docFrom |> printfn "%s"
Hope this can get you started.
I need to write a Deedle FrameData (including "ID" column and additional "Delta" column with blank entries) to CSV. While I can generate a 2D array of the FrameData, I am unable to write it correctly to a CSV file.
module SOQN =
open System
open Deedle
open FSharp.Data
// TestInput.csv
// ID,Alpha,Beta,Gamma
// 1,no,1,hi
// ...
// TestOutput.csv
// ID,Alpha,Beta,Gamma,Delta
// 1,"no","1","hi",""
// ...
let inputCsv = #"D:\TestInput.csv"
let outputCsv = #"D:\TestOutput.csv"
let (df:Frame<obj,string>) = Frame.ReadCsv(inputCsv, hasHeaders=true, inferTypes=false, separators=",", indexCol="ID")
// See http://www.fssnip.net/sj/title/Insert-Deedle-frame-into-Excel
let data4Frame (frame:Frame<_,_>) = frame.GetFrameData()
// See http://www.fssnip.net/sj/title/Insert-Deedle-frame-into-Excel
let boxOptional obj =
match obj with
| Deedle.OptionalValue.Present obj -> box (obj.ToString())
| _ -> box ""
// See http://www.fssnip.net/sj/title/Insert-Deedle-frame-into-Excel
let frameToArray (data:FrameData) =
let transpose (array:'T[,]) =
Array2D.init (array.GetLength(1)) (array.GetLength(0)) (fun i j -> array.[j, i])
data.Columns
|> Seq.map (fun (typ, vctr) -> vctr.ObjectSequence |> Seq.map boxOptional |> Array.ofSeq)
|> array2D
|> transpose
let main =
printfn ""
printfn "Output Deedle FrameData To CSV"
printfn ""
let dff = data4Frame df
let rzlt = frameToArray dff
printfn "rzlt: %A" rzlt
do
use writer = new StreamWriter(outputCsv)
writer.WriteLine("ID,Alpha,Beta,Gamma,Delta")
// writer.WriteLine rzlt
0
[<EntryPoint>]
main
|> ignore
What am I missing?
I would not use FrameData to do this - frame data is mostly internal and while there are some legitimate uses for it, I don't think it makes sense for this task.
If you simply want to add an empty Delta column to your input CSV, then you can do this:
let df : Frame<int, _> = Frame.ReadCsv("C:/temp/test-input.csv", indexCol="ID")
df.AddColumn("Delta", [])
df.SaveCsv("C:/temp/test-output.csv", ["ID"])
This does almost everything you need - it writes the ID column and the extra Delta column.
The only caveat is that it does not add the extra quotes around the data. This is not required by the CSV specification unless you need to escape a comma in a column and I don't think there is an easy way to get Deedle to do this.
So, I think then you'd have to write your own writing to a CSV file. The following shows how to do this, but it does not correctly escape quotes and commas (which is why you should use SaveCsv even if it does not put in the quotes when they're not needed):
use writer = new StreamWriter("C:/temp/test-output.csv")
writer.WriteLine("ID,Alpha,Beta,Gamma,Delta")
for key, row in Series.observations df.Rows do
writer.Write(key)
for value in Series.valuesAll row do
writer.Write(",")
writer.Write(sprintf "\"%O\"" (if value.IsSome then value.Value else box ""))
writer.WriteLine()
You can get the example of writing to csv from source of the library (it uses FrameData there)
After adding wrapper:
type FrameData with
member frameData.SaveCsv(path:string, ?includeRowKeys, ?keyNames, ?separator, ?culture) =
use writer = new StreamWriter(path)
writeCsv writer (Some path) separator culture includeRowKeys keyNames frameData
you could write like this:
dff.SaveCsv outputCsv
This is C# version:
public static IEnumerable<string> ReadLinesEnumerable(string path) {
using ( var reader = new StreamReader(path) ) {
var line = reader.ReadLine();
while ( line != null ) {
yield return line;
line = reader.ReadLine();
}
}
}
But directly translating needs a mutable variable.
If you're using .NET 4.0, you can just use File.ReadLines.
> let readLines filePath = System.IO.File.ReadLines(filePath);;
val readLines : string -> seq<string>
open System.IO
let readLines (filePath:string) = seq {
use sr = new StreamReader (filePath)
while not sr.EndOfStream do
yield sr.ReadLine ()
}
To answer the question whether there is a library function for encapsulating this pattern - there isn't a function exactly for this, but there is a function that allows you to generate sequence from some state called Seq.unfold. You can use it to implement the functionality above like this:
new StreamReader(filePath) |> Seq.unfold (fun sr ->
match sr.ReadLine() with
| null -> sr.Dispose(); None
| str -> Some(str, sr))
The sr value represents the stream reader and is passed as the state. As long as it gives you non-null values, you can return Some containing an element to generate and the state (which could change if you wanted). When it reads null, we dispose it and return None to end the sequence. This isn't a direct equivalent, because it doesn't properly dispose StreamReader when an exception is thrown.
In this case, I would definitely use sequence expression (which is more elegant and more readable in most of the cases), but it's useful to know that it could be also written using a higher-order function.
let lines = File.ReadLines(path)
// To check
lines |> Seq.iter(fun x -> printfn "%s" x)
On .NET 2/3 you can do:
let readLines filePath = File.ReadAllLines(filePath) |> Seq.cast<string>
and on .NET 4:
let readLines filePath = File.ReadLines(filePath);;
In order to avoid the "System.ObjectDisposedException: Cannot read from a closed TextReader." exception, use:
let lines = seq { yield! System.IO.File.ReadLines "/path/to/file.txt" }