I am making a compiler from (a small subset of) Java to CIL (MSIL) in F# and I was thinking about writing some unit tests for the actual compilation part. Is there a way I could run ilasm on the generated IL code and the run the .exe file from a unit test? I am using NUnit.
Concretely, is there a way to do the following?
[<Test>]
member this.TestSimpleMain () =
let fileName = __SOURCE_DIRECTORY__.Replace("code", "examples\SimpleMain.cil")
// run ilasm on fileName -> should produce ...\SimpleMain.exe
let actualResult = // run ...\SimpleMain.exe file
Assert.That(actualResult , Is.EqualTo("Program returned value 5!")))
If you want to use ilasm make sure you create SimpleMain.cil in IL assembly format (text representation of CIL).
Then you should create PE (exe/dll) file from your il file.
here is an example for the nunit test with output and exit code verification:
[<Test>]
member this.TestSimpleMain() =
// Code that creates SimpleMain.il
let p1 = new System.Diagnostics.Process()
p1.StartInfo.FileName <- "C:/Windows/Microsoft.NET/Framework64/v4.0.30319/ilasm.exe"
p1.StartInfo.Arguments <- "SimpleMain.il /exe /output=SimpleMain.exe"
p1.Start()
p1.WaitForExit()
let p2 = new System.Diagnostics.Process()
p2.StartInfo.FileName <- "SimpleMain.exe"
p2.StartInfo.RedirectStandardOutput <- true // For output verification
p2.Start()
p2.WaitForExit()
let resultA = p2.ExitCode // For exit code verification
let resultB = p2.StandardOutput.ReadToEnd() // For output verification
Assert.That(result, Is.EqualTo(expectedResult))
Related
A couple days ago, I posted a question about deserialization with enums in F#.
The question is here: Deserialization in F# vs. C#
The answer pointed to some code written by Isaac Abraham, at: https://gist.github.com/isaacabraham/ba679f285bfd15d2f53e
However I am facing another problem:
If the object to deserialize to has an object of type 'enum option', the deserialization will fail, whereas it'll work if the type is just 'enum'.
A minimal example:
type TestType =
| A = 0
| B = 1
type TestObjectA =
{
test : TestType
}
type TestObjectB =
{
test : TestType option
}
let x = "{\"test\":\"A\"}"
let TestA = Deserialize<TestObjectA> x // will work
let TestB = Deserialize<TestObjectB> x // will fail
and the large deserialization code is at: https://pastebin.com/95JZLa6j
I put the whole code in a fiddle: https://dotnetfiddle.net/0Vc0Rh
but it can't be run from there since the F# version they support will not accept the 'object' keyword.
So, my question is: why can't I use the option type on an enum, but it works on other types? As a side note, since I'm quite new to F#, I'm not fully understanding Isaac's code, although I spent some time going through it and trying to troubleshoot it.
My understanding is that this line:
|> Seq.map (fun (value, propertyInfo) -> Convert.ChangeType(value, propertyInfo.PropertyType))
will try to convert the type to the right enum, but not to the enum option.
As a bonus question, is there a working solution that does full idiomatic deserialization with enums? (without going through null types)
open System.IO
type TestType =
| A = 0
| B = 1
type TestObjectB =
{
test : TestType option
}
let jsonSerializeToString obj =
use writer = new StringWriter()
let ser = new Newtonsoft.Json.JsonSerializer()
ser.Formatting <- Newtonsoft.Json.Formatting.Indented
ser.Serialize(writer, obj)
writer.ToString()
let jsonDeserializeFromString str =
Newtonsoft.Json.JsonConvert.DeserializeObject<TestObjectB>(str)
let Test obj =
let str = jsonSerializeToString obj
let obj' = jsonDeserializeFromString str
obj'
[<EntryPoint>]
let main argv =
{ test = Some TestType.B } |> Test |> ignore
{ test = None } |> Test |> ignore
0
Note: if you need to serialize a large collection of objects, then stream them to a file instead of an in-memory string to avoid an OutOfMemoryException. Like use writer = File.CreateText(filePath).
As a bonus question, is there a working solution that does full
idiomatic deserialization with enums?
I use the Microsoft.FsharpLu.Json package in production and find it works quite well for serializing and deserializing between "plain" javascript and idiomatic F#. Note Microsoft.FsharpLu.Json relies on Newtonsoft.Json under the hood.
Below is an example with your types and your test string, using Expecto for tests.
namespace FsharpLuJsonTest
open Newtonsoft.Json
open Microsoft.FSharpLu.Json
open Expecto
open Expecto.Flip
// Setup for FSharpLu.Json
type JsonSettings =
static member settings =
let s = JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
MissingMemberHandling = MissingMemberHandling.Ignore)
s.Converters.Add(CompactUnionJsonConverter())
s
static member formatting = Formatting.None
type JsonSerializer = With<JsonSettings>
// Your example
type TestType =
| A = 0
| B = 1
type TestObjectA = { test : TestType }
type TestObjectB = { test : TestType option }
module Tests =
let x = """{"test":"A"}"""
[<Tests>]
let tests =
testList "Deserialization Tests" [
testCase "To TestObjectA" <| fun _ ->
JsonSerializer.deserialize x
|> Expect.equal "" { TestObjectA.test = TestType.A }
testCase "To TestObjectB" <| fun _ ->
JsonSerializer.deserialize x
|> Expect.equal "" { TestObjectB.test = Some TestType.A }
]
module Main =
[<EntryPoint>]
let main args =
runTestsInAssembly defaultConfig args
As you can see FsharpLu.Json supports Discriminated Unions and option types out of the box in the way you prefer. FsharpLu.Json is a less flexible solution than some others like Chiron (which allow for much more customisation) but I tend to prefer the opinionated approach of FsharpLu.Json.
I haven't used it personally, but the new FSharp.SystemText.Json library with the JsonUnionEncoding.ExternalTag setting should work roughly the same way FsharpLu.Json does. That library uses Microsoft's new System.Text.Json library under the hood rather than Newtonsoft.Json.
I'm working through the Embedding F# Interactive example from here but like this post, I'm having an issue with the following line throwing an exception:
let fsiSession = FsiEvaluationSession.Create(fsiConfig, allArgs, inStream, outStream, errStream)
The exception thrown is:
"System.Exception: 'Error creating evaluation session: StopProcessingExn None'"
My project is being run from VS2017 Enterprise, setup as a simple F# console app, with the Target Framework as .NET Core 2.0. The version of FSharp.Compiler.Service downloaded from nuget is 17.0.1 and FSharp.Core is 4.2.0.
The Program.Fs file code I'm running is here (a direct port of the example):
open System
open System.IO
open System.Text
open Microsoft.FSharp.Compiler.Interactive.Shell
[<EntryPoint>]
let main argv =
let sbOut = new StringBuilder()
let sbErr = new StringBuilder()
let inStream = new StringReader("")
let outStream = new StringWriter(sbOut)
let errStream = new StringWriter(sbErr)
// Build command line arguments & start FSI session
let argv = [| "C:\\Program Files (x86)\\Microsoft SDKs\\F#\\4.1\\Framework\\v4.0\\fsi.exe" |]
let allArgs = Array.append argv [|"--noninteractive"|]
let fsiConfig = FsiEvaluationSession.GetDefaultConfiguration()
let fsiSession = FsiEvaluationSession.Create(fsiConfig, allArgs, inStream, outStream, errStream)
/// Evaluate expression & return the result
let evalExpression text =
match fsiSession.EvalExpression(text) with
| Some value -> printfn "%A" value.ReflectionValue
| None -> printfn "Got no result!"
evalExpression "42+1" // prints '43'
Console.ReadLine() |> ignore
0 // return integer
I already tried to add the files FSharp.Core.optdata and FSharp.Core.sigdata in my bin folder (bin\Debug\netcoreapp2.0) as mentioned here and here, but without success. By the way, my bin folder does not contain the file FSharp.Core.dll.
I also tried to publish my app and add the .optdata and .sigdata files manually in the publish folder, but without success either.
Any thoughts would be appreciated.
I want to create a sequence of 'candles' ( a simple structure) for performing testing on financial data. I have downloaded some financial data from a broker and they are in a csv format.
I have the following code, using type providers:
type DukasCandles = CsvProvider<"C:\Users\**\Documents\Visual Studio 2017\Projects\FI\FI\Data\schema.csv" >
let row2Candle (myTicker: string ) (mySide: Side) (aRow:DukasCandles.Row) : Candle =
{Open = aRow.Open ;
Close = aRow.Close;
Low = aRow.Low
High = aRow.High
StartTime = aRow.LocalTime
Volume = aRow.Volume
Ticker = myTicker
Side = mySide }
let sortedCandles aTicker aParsedFile =
aParsedFile
|> Seq.map ( row2Candle aTicker Side.Bid )
|> Seq.sortBy ( fun candle -> candle.StartTime)
let fileContent = inputFile |> DukasCandles.Load
let rows = fileContent.Rows
let dataContent = rows|> (sortedCandles "EUR_USD")
It is just a toy example to test my understanding of type providers. I have it in a FSX script after the necessary boilerplate for declaring the file name and open the necessary modules.
Now the question: if I highlight this as it is and try to execute in F# interactive, then type
dataContent;;
I get the following message
val it : seq<Candle> =
Error: Couldn't parse row 1057 according to schema: Expecting DateTime in LocalTime, got 13.01.2016 00:00:00.000
The odd thing is the following:
suppose I type the following code in F# interactive:
fileContent;;
get the answer and then refer to it through the 'it' keyword, as in:
it.Rows |> sortedCandles "EUR_USD";;
Then the code is executed with no problems.
Why do we have such an inconsistent behaviour? Any idea?
Any help highly appreciated. Thank you.
I wrote this script from some resources I found. It's working but I some files I have problem. I am new in F# so how can I change line with FileHelpersException to get exact line where is the problem? Thanks
// Learn more about F# at http://fsharp.net
// See the 'F# Tutorial' project for more help.
open FileHelpers
open System
[<DelimitedRecord(",")>]
type CsvRecord =
class
val field1 : string
val field2 : string
val field3 : int
new () = {
field1 = ""
field2 = ""
field3 = 0
}
end
[<EntryPoint>]
let main argv =
use file = System.IO.File.CreateText("result.txt")
let engine = new FileHelperEngine<CsvRecord>()
engine.Encoding <- new Text.UTF8Encoding()
let res =
try
engine.ReadFile("test.csv")
with
| :? FileHelpersException -> Array.empty<CsvRecord>
for record in res do
fprintfn file "%s" record.field1
printf "DONE!"
let s = Console.ReadLine()
0 // return an integer exit code
I suggest that you use CsvTypeProvider instead. When there's a mismatch the error message states the line which has the error
open FSharp.Data
[<EntryPoint>]
let main argv =
use file = System.IO.File.CreateText("result.txt")
let csv = new CsvProvider<"test.csv">()
for record in csv.Data do
fprintfn file "%s" record.field1
If you want to ignore the lines with errors, just pass IgnoreErrors=true as an extra parameter to CsvProvider
This question is about the FileHelpers library you are using, not F#, so looking at the docs for that might help. In this case you can check for ConvertException instead of FileHelpersException, which contains members that give you more details about the member.
try
engine.ReadFile("test.csv")
with
| :? ConvertException as ex ->
printfn "ERROR: Line %d Col %d" ex.LineNumber ex.ColumnNumber
Array.empty<CsvRecord>
I agree with Gustavo though, you might find it easier to use the CsvTypeProvider.
How do I execute F# code from a string in a compiled F# program?
Here's a little script that uses the FSharp CodeDom to compile a string into an assembly and dynamically load it into the script session.
It uses a type extension simply to allow useful defaults on the arguments (hopefully let bound functions will support optional, named and params arguments in the near future.)
#r "FSharp.Compiler.dll"
#r "FSharp.Compiler.CodeDom.dll"
open System
open System.IO
open System.CodeDom.Compiler
open Microsoft.FSharp.Compiler.CodeDom
let CompileFSharpString(str, assemblies, output) =
use pro = new FSharpCodeProvider()
let opt = CompilerParameters(assemblies, output)
let res = pro.CompileAssemblyFromSource( opt, [|str|] )
if res.Errors.Count = 0 then
Some(FileInfo(res.PathToAssembly))
else None
let (++) v1 v2 = Path.Combine(v1, v2)
let defaultAsms = [|"System.dll"; "FSharp.Core.dll"; "FSharp.Powerpack.dll"|]
let randomFile() = __SOURCE_DIRECTORY__ ++ Path.GetRandomFileName() + ".dll"
type System.CodeDom.Compiler.CodeCompiler with
static member CompileFSharpString (str, ?assemblies, ?output) =
let assemblies = defaultArg assemblies defaultAsms
let output = defaultArg output (randomFile())
CompileFSharpString(str, assemblies, output)
// Our set of library functions.
let library = "
module Temp.Main
let f(x,y) = sin x + cos y
"
// Create the assembly
let fileinfo = CodeCompiler.CompileFSharpString(library)
// Import metadata into the FSharp typechecker
#r "0lb3lphm.del.dll"
let a = Temp.Main.f(0.5 * Math.PI, 0.0) // val a : float = 2.0
// Purely reflective invocation of the function.
let asm = Reflection.Assembly.LoadFrom(fileinfo.Value.FullName)
let mth = asm.GetType("Temp.Main").GetMethod("f")
// Wrap weakly typed function with strong typing.
let f(x,y) = mth.Invoke(null, [|box (x:float); box (y:float)|]) :?> float
let b = f (0.5 * Math.PI, 0.0) // val b : float = 2.0
To use this in a compiled program you would need the purely reflective invocation.
Of course this is a toy compared to a full scripting API that many of us in the community have urgently requested.
best of luck,
Danny
Are you looking for an Eval function?
You might want to try looking at this blog post:
http://fsharpnews.blogspot.com/2007/02/symbolic-manipulation.html
If you read in your expressions into these kind of symbolic datastructures, they are pretty easy to evaluate.
Or, perhaps you are looking for scripting support:
http://blogs.msdn.com/chrsmith/archive/2008/09/12/scripting-in-f.aspx
If you really want dynamic compilation, you could do it with the F# CodeDom provider.
There has been movement on this front. You can now compile using the FSharp.Compiler.Service
simple sample using FSharp.Compiler.Service 5.0.0 from NuGet
open Microsoft.FSharp.Compiler.SimpleSourceCodeServices
let compile (codeText:string) =
let scs = SimpleSourceCodeServices()
let src,dllPath =
let fn = Path.GetTempFileName()
let fn2 = Path.ChangeExtension(fn, ".fs")
let fn3 = Path.ChangeExtension(fn, ".dll")
fn2,fn3
File.WriteAllText(src,codeText)
let errors, exitCode = scs.Compile [| "fsc.exe"; "-o"; dllPath; "-a";src; "-r"; "WindowsBase"; "-r" ;"PresentationCore"; "-r"; "PresentationFramework" |]
match errors,exitCode with
| [| |],0 -> Some dllPath
| _ ->
(errors,exitCode).Dump("Compilation failed")
File.Delete src
File.Delete dllPath
None
then it's a matter of Assembly.LoadFrom(dllPath) to get it into the current app domain.
followed by reflection based-calls into the dll (or possibly Activator.CreateInstance)
Sample LinqPad Usage