I'm trying to build a system that is similar to FsBolero (TryWebassembly), Fable Repl and many more that uses Fsharp.Compiler.Services.
So I expect it is feasible to achieve my goals but I encountered a problem that I hope is only a result of my lack of experience with that realm of software development
I'm implementing a service that gives user the power to write custom algorithms (DSL) in the context of the domain system.
The code to compile come as a plain raw string that is fully correct F# code.
Sample DSL algorithm looks like:
let code = """
module M
open Lifespace
open Lifespace.LocationPricing
let alg (pricing:LocationPricing) =
let x=pricing.LocationComparisions.CityLevel.Transportation
(8.*x.PublicTransportationStation.Data+ x.RailwayStation.Data+ 5.*x.MunicipalBikeStation.Data) / 14.
"""
that code compiles correctly via CompileToDynamicAssembly. I also provided proper reference to my domain *.dll via -r Fsc parameter.
And here comes my problems as next I have the generated dynamic assembly and want to invoke that algorithm.
I do it with reflection (is there any other way?) with
f.Invoke(null, [|arg|]) when arg is of type LocationPricing and comes from main/hosting project reference.
The Invoke doesn't work because I have error:
Cannot cast LocationPricing to LocationPricing
I had the same problem when tried to use F# interactive services, the error was similar:
Cannot cast [A]LocationPricing to [B]LocationPricing
I'm aware I have two same dlls in the context and F# does have extern alias syntax to solve it.
But other mentioned public systems somehow deals with that or I'm doing it wrongly.
I will look at code of Bolero and FableRepl but it will definately take some time to understand the pitfalls.
Update: Full code (Azure Function)
namespace AzureFunctionFSharp
open System.IO
open System.Text
open Microsoft.Azure.WebJobs
open Microsoft.Azure.WebJobs.Extensions.Http
open Microsoft.AspNetCore.Http
open Microsoft.AspNetCore.Mvc
open Microsoft.Extensions.Logging
open FSharp.Compiler.SourceCodeServices
open Lifespace.LocationPricing
module UserCodeEval =
type CalculationResult = {
Value:float
}
type Error = {
Message:string
}
[<FunctionName("UserCodeEvalSampleLocation")>]
let Run([<HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)>] req: HttpRequest, log: ILogger , [<Blob("ranks/short-ranks.json", FileAccess.Read)>] myBlob:Stream)=
log.LogInformation("F# HTTP trigger function processed a request.")
// confirm valid domain dll location
// for a in System.AppDomain.CurrentDomain.GetAssemblies() do
// if a.FullName.Contains("wrometr.lam.to.ranks") then log.LogInformation(a.Location)
// let code = req.Query.["code"].ToString()
// replaced just to show how the user algorithm can looks like
let code =
"""
module M
open Lifespace
open Lifespace.LocationPricing
open Math.MyStatistics
open MathNet.Numerics.Statistics
let alg (pricing:LocationPricing) =
let x= pricing.LocationComparisions.CityLevel.Transportation
(8.*x.PublicTransportationStation.Data+ x.RailwayStation.Data+ 5.*x.MunicipalBikeStation.Data) / 14.
"""
use reader = new StreamReader(myBlob, Encoding.UTF8)
let content = reader.ReadToEnd()
let encode x = LocationPricingStore.DecodeArrayUnpack x
let pricings = encode content
let checker = FSharpChecker.Create()
let fn = Path.GetTempFileName()
let fn2 = Path.ChangeExtension(fn, ".fsx")
let fn3 = Path.ChangeExtension(fn, ".dll")
File.WriteAllText(fn2, code)
let errors, exitCode, dynAssembly =
checker.CompileToDynamicAssembly(
[|
"-o"; fn3;
"-a"; fn2
"-r";#"C:\Users\longer\azure.functions.compiler\bin\Debug\netstandard2.0\bin\MathNet.Numerics.dll"
"-r";#"C:\Users\longer\azure.functions.compiler\bin\Debug\netstandard2.0\bin\Thoth.Json.Net.dll"
// below is crucial and obtained with AppDomain resolution on top, comes as a project reference
"-r";#"C:\Users\longer\azure.functions.compiler\bin\Debug\netstandard2.0\bin\wrometr.lam.to.ranks.dll"
|], execute=None)
|> Async.RunSynchronously
let assembly = dynAssembly.Value
// get one item to test the user algorithm works in the funtion context
let arg = pricings.[0].Data.[0]
let result =
match assembly.GetTypes() |> Array.tryFind (fun t -> t.Name = "M") with
| Some moduleType ->
moduleType.GetMethods()
|> Array.tryFind (fun f -> f.Name = "alg")
|>
function
| Some f -> f.Invoke(null, [|arg|]) |> unbox<float>
| None -> failwith "Function `f` not found"
| None -> failwith "Module `M` not found"
// end of azure function, not important in the problem context
let res = req.HttpContext.Response
match String.length code with
| 0 ->
res.StatusCode <- 400
ObjectResult({ Message = "No Good, Please provide valid encoded user code"})
| _ ->
res.StatusCode <-200
ObjectResult({ Value = result})
**Update: changing data flow **
To move forward I resigned to use domain types in both places. Instead I do all logic in domain assembly and only pass primitives (strings) to reflected invocation. I'm also suprised a lot that caching still works everytime I do compilation on each Azure Function call. I will experiment as well with FSI, in theory it should be faster than reflection but with additional burden to pass parameters to evaluations
In your example, the code that runs inside your dynamically compiled assembly and the code calling it need to share a type LocationPricing. The error you are seeing typically means that you somehow ended up with different assembly loaded in the process that is calling the dynamically compiled code and the code actually running the computation.
It is hard to say exactly why this happened, but you should be able to check whether this is indeed the case by looking at assemblies loaded in the current App Domain. Say that your shared assembly is MyAssembly. You can run:
for a in System.AppDomain.CurrentDomain.GetAssemblies() do
if a.FullName.Contains("MyAssembly") then printfn "%s" a.Location
If you were using F# Interactive Services, then a trick to fix this is to start an FSI session and then send an interaction to the service that loads the assembly from the right place. Something along those lines:
let myAsm = System.AppDomain.CurrentDomain.GetAssemblies() |> Seq.find (fun asm ->
asm.FullName.Contains("MyAssembly"))
fsi.EvalInteraction(sprintf "#r #\"%s\"" myAsm.Location)
Related
I'm working on a side project and I'm using Hopac for the first time. I ran into an odd (to me) compilation issue that I haven't been able to grok. I suspect that I'm the problem here, and not Hopac.
The program is supposed to be a simple console app that consumes notifications from various services. Here's the problematic module:
module Provider
open System
open System.IO
open Hopac
open BitThicket.NotificationHelper.Core
open BitThicket.NotificationHelper.Providers
let defaultProviderTypes =
[| typeof<GitHub.GitHubNotificationProvider> |]
type Provider = {
getCh : Ch<Providers.INotification seq>
}
let giveLatest ch latest =
Ch.give
let start config logger (providerType:Type) = Job.delay <| fun () ->
let providerImpl = Activator.CreateInstance(providerType) :?> Providers.INotificationProvider
let p = { getCh = Ch() }
let rec server =
let latest = providerImpl.GetLatestNotificationsAsync(None) |> Job.fromAsync
latest >>= Ch.give p.getCh // error here
}
Job.start server
In this case, the compiler complains: Expecting a type supporting the operator '>>=' but given a function type. You may be missing an argument to a function.
Similarly, if I use a slightly different syntax:
// ...
let rec server =
let latest = providerImpl.GetLatestNotificationsAsync(None) |> Job.fromAsync
latest >>= fun l -> Ch.give p.getCh l // error here
// ...
In this case, the error is: This function takes too many arguments, or is used in a context where a function is not expected.
I asked haf about his in slack, and his suggestion was to check for alternative definitions of >>=. The tooling doesn't really do much to help me figure that one out, but the only namespace/module I have opened that defines >>= is Hopac (the BitThicket ones are just trivially simple namespaces with some type definitions in them).
What am I doing wrong here?
I'm looking at the source code, and I see that the bind operator is actually defined in Hopac.Infixes, not in Hopac.
It's trying to make a TypeProvider for Xamarin.Forms, but has been plagued by FS2024 error.
Parse own library from the XAML of Xamarin.Forms
Assign x:Name to Propertis
`F#
type MainPage = Moonmile.XamarinFormsTypeProvider.XAML<"MainPage.xaml">
// made btn1 and text1 propertis
type MainPageEx(target:MainPage) =
let mutable count = 0
do
// When set event to btn.Clicked, happen FS2024 error.
// If this event is comment out, it success build.
target.btn1.Clicked.Add( fun e ->
count <- count + 1
target.btn1.Text <- "Clicked " + count.ToString())
// Property is success
member this.CurrentPage
with get() = target.CurrentPage
When you are referring to a property, build & operation you can normally.
But the internal class of Xamarin.Forms like Button.Clicked, If you try to access to, it is the build error.
Sample code for error
https://github.com/moonmile/SimpleEventTypeProvider
Making code for XamarinFormsTypeProvider
github.com/moonmile/XamarinFormsTypeProvider
Maybe, I suspect inconsistencies and is happening in the part of the generation of a Native TypeProvider and Xamrin.Forms.Core a PCL.
F# Compiler for F# 3.1 (Open Source Edition)
Freely distributed under the Apache 2.0 Open Source License my error!!!
isMscorlib: true
name: "System.Runtime"
PrimaryAssembly.DotNetCore.Name: "System.Runtime"
PrimaryAssembly.Mscorlib.Name: "mscorlib"
parameter error FS2024: Static linking may not use assembly that targets different profile.
It's to operate the property they work properly, and to MVVM perhaps.
Butt I am trying to implement a way to be assigned to Button.Clicked events
as shown in the codebehide-like buildings if possible.
Would there workaround or what?
In the case of XAML in WPF, How can such seems to work well.
github.com/fsprojects/FsXaml
This answer isn't guaranteed to be correct, but it should help at least point you in the right direction.
The first thing to do is to make sure that you have installed the latest Visual F# Tools Build, as this adds the FSharp.Core that is compatible with the PCL profiles (You can find it here: (https://visualfsharp.codeplex.com/). Once that is installed, you will want to reference either the Profile78, or Profile259 FSharp.Core.dll (On my machine, these are found at: "C:\Program Files (x86)\Reference Assemblies\Microsoft\FSharp.NETPortable\2.3.5.0", and "C:\Program Files (x86)\Reference Assemblies\Microsoft\FSharp.NETPortable\2.3.5.1" respectively).
Once you have that installed, the next thing to do is make sure that your PCL projects have the following in their project files (This tells MSBuild / xBuild that the projects are PCL libraries, and that they are F# Projects):
<ProjectTypeGuids>{786C830F-07A1-408B-BD7F-6EE04809D6DB};{F2A71F9B-5D33-465A-A702-920D77279786}</ProjectTypeGuids>
Once that is done, you will need to select either Profile78, or Profile259 (I would recommend 78, as the current Xamarin.Forms nuget package doesn't support 259).
Once that is done, then you should be able to build and run and it should get rid of the error.
Thank you for my question.
Meybe,
When it build TypeProvider, F# compiler use classes in mscorlib.
When it resolve the type of btn1.Clicked event, the F# comiler use type in System.Runtime.
I think for that, and it can not be resolved at build time
Try, if you attach the Clicked Event using reflection, it has moved successfully on Android build through.
I seem, if it I use only shard classes in mscorlib and System.Rutime, I can build no FS2024 error.
type MainPage = Moonmile.XamarinFormsTypeProvider.XAML<"MainPage.xaml">
type MainPageEx() as this =
inherit BindObject<MainPage>(new MainPage())
// Add handlder by reflection
let AddHandler(target:obj, eventName:string, eventMethod: obj*obj -> unit ) =
let hdr = Action<obj,obj>( fun s e -> eventMethod(s,e))
let ei = target.GetType().GetRuntimeEvent(eventName)
let dt = ei.AddMethod.GetParameters().[0].ParameterType
let handler = new Action<obj,obj>(fun s e -> hdr.Invoke( s, new EventArgs() ))
let handlerInvoke = handler.GetType().GetRuntimeMethod("Invoke", [|typeof<obj>; typeof<Type[]>|])
let dele = handlerInvoke.CreateDelegate( dt, handler )
let add = new Func<Delegate, EventRegistrationToken> ( fun t ->
let para = ei.AddMethod.GetParameters()
let ret = ei.AddMethod.Invoke( target, [|t|])
if ret <> null then
ret :?> EventRegistrationToken
else
new EventRegistrationToken()
)
let remove = new Action<EventRegistrationToken>( fun t -> ei.RemoveMethod.Invoke(target, [|t|]) |> ignore )
// WindowsRuntimeMarshal.AddEventHandler<Delegate>(add, remove, dele)
add.Invoke( dele ) |> ignore
()
let mutable count = 0
do
(* // build error
target.btn1.Clicked.Add( fun e ->
count <- count + 1
target.btn1.Text <- "Clicked " + count.ToString())
*)
// add handler by reflection
AddHandler( base.Target.btn1, "Clicked", this.ButtonClick )
()
member this.CurrentPage
with get() = this.Target.CurrentPage
member this.ButtonClick(s,e) =
count <- count + 1
base.Target.text1.Text <- "clicked " + count.ToString()
I'm using the FSharp.Data typeproviders.
I would like to make a function that has a parameter that sets the typeprovider's sample string or file location.
let SyncIt url sample converter storer =
async {
url
|> MakeRequestAsync
|> Async.RunSynchronously
|> JsonProvider< sample >.Parse
|> Seq.iter (converter >> storer)
}
If a call the JsonProvider in the module with
[<Literal>]
let sample = """{"name":"Peter","age":9}"""
type provider = JsonProvider<sample>
works fine. Why can't I pass it as a parameter? I know it has something to do with the reference being clear at compile time, but cannot figure out how to get around it other than declaring each providers explicitly.
A function cannot take a value of static parameter as an argument, because the value has to be determined at compile time. This means that if you write:
let [<Literal>] sample = """{"name":"Peter","age":9}"""
let parseHtml html = JsonProvider<sample>.Parse(html)
... everything is fine, because the compiler knows that sample is a constant (and the compiler knows its value) and so it can instantiate the type provider (during compilation) to generate types. If you write something like:
let parseHtml sample html = JsonProvider<sample>.Parse(html)
... then the compiler cannot know what the value of sample may be at runtime and so it cannot generate the required types at compile time. Type providers do not "exist" at runtime, so the types cannot be generated on the fly (This would not be really useful, because the point of type providers is to give you some compile-time safety guarantees).
Your example. In your case, it might make sense to take the specific JsonProvider<sample>.Parse function as an argument instead:
let SyncIt url parse storer =
async {
url
|> MakeRequestAsync
|> Async.RunSynchronously
|> parse
|> Seq.iter (converter >> storer)
}
This way, the caller can specify the static parameter to a type provider and then call your function to do the synchronization:
let [<Literal>] sample = """{"name":"Peter","age":9}"""
SyncIt url (JsonProvider<sample>.Parse) storer
Although, it is not entirely clear to me why you need a type provider here. The point of providers is to give you nice types that you can use to access a concrete source of data. If your converter and storer work on any JSON data file, then you might be able to get the thing done using just JSON parser (also in F# Data).
Asynchronous block. Also, note that your code is not really running asynchronously - to make it asynchronous, you need to download the URL using let! operation:
let SyncIt url parser storer =
async {
let wc = new WebClient()
let! html = wc.AsyncDownloadString(url)
parser html
|> Seq.iter (converter >> storer)
}
I have created a simple generating type provider that takes the path to an assembly at reorganizes the types, to bring them under the type providers namespace, (sort of Internalising if you like).
The link to the code concerned is here
https://github.com/colinbull/Playground
Now the types seem to be provided correctly,
let[<Literal>]assemblyPath = #"D:\Appdev\Playground\SimpleLib\bin\Debug\SimpleLib.dll"
type T = Inno.InternalisingProvider<assemblyPath>
type C = T.Class1
[<EntryPoint>]
let main argv =
let c = new C()
printfn "Result: %A" c.X
System.Console.ReadLine() |> ignore
0
since the displays in VS without any reported errors. However when I compile this assembly the IL seems to get emitted incorrectly with the following error.
Error 1: A problem occurred writing the binary 'obj\Debug\TypeProviders.Tests.exe': Error in pass3 for type Program, error: Error in GetMethodRefAsMethodDefIdx for mref = ".ctor", error: Exception of type 'Microsoft.FSharp.Compiler.AbstractIL.ILBinaryWriter+MethodDefNotFound' was thrown. FSC 1 1 TypeProviders.Tests
Examples given for generated types given in samples pack doesn't seem to have any StaticParameters defined which requires a type with the provided type name to be returned. In this case how do I emit the types in the provided assembly? Currently I am doing the following
let provideAssembly (reqType:ProvidedTypeDefinition) assemblyPath =
let name = Path.GetFileName(assemblyPath)
let providedAssembly = ProvidedAssembly.RegisterGenerated(assemblyPath)
for t in providedAssembly.GetExportedTypes() do
let ty = createGeneratedType t
ty.SetAssembly(providedAssembly)
reqType.AddMember(ty)
reqType
Thanks in advance
Disclaimer: browser compiled solutions
I believe here you don't need to create generated types that will wrap exiting types => this should work
let provideAssembly (reqType:ProvidedTypeDefinition) assemblyPath =
let existingAssembly = Assembly.LoadFrom(assemblyPath)
for ty in providedAssembly.GetExportedTypes() do
reqType.AddMember(ty)
reqType
You can also try this one:
let provideAssembly (reqType:ProvidedTypeDefinition) assemblyPath =
reqType.AddAssemblyTypesAsNestedTypesDelayed(fun() -> Assembly.LoadFrom assemblyPath)
reqType
this one will preserve namespace so declaration of type C will look like
type C = T.SimpleLib.Class1
I'm trying to use async workflows in F# to fetch several web requests.
However, some of my requests are occasionally returning errors (e.g. http 500), and I don't know how to handle this. It appears like my F# program gets stuck in an infinite loop when running in the debugger.
I'm probably missing some stuff, cause examples I seen didn't compile out of the box. First thing I found that helped was this bit of code:
type System.Net.WebRequest with
member req.GetResponseAsync() =
Async.BuildPrimitive(req.BeginGetResponse, req.EndGetResponse)
and then I have my bit of code to fetch the requests, which is pretty standard from examples I've seen:
let async_value = async {
let req = WebRequest.Create(url)
let! rsp = req.GetResponseAsync()
return (rsp :?> HttpWebResponse).StatusCode
}
and then I try to get the result:
let status = Async.RunSynchronously(async_value)
But when I run my program in debugger, it breaks at req.EndGetResponse because server returned internal server error 500. If I keep just continuing execution, it gets in a funky loop, breaking at req.EndGetResponse (sometimes several in a row), and at let status = Async.RunSynchronously(async_value).
How do I get around the exception problem so I can get my status code? Also, do I need the type thing I did above? Or am I missing some library/dll for F#/VS 2010 Beta 1, of which this is already a part of?
I actually run several requests in parallel, using Async.RunSynchronously(Async.Parallel(my_array_of_async_values)), though I don't think that is related to the exception issue I'm having.
The fact the examples I've come across only use Async.Run rather than Async.RunSynchronously is probably an indicator I'm missing something... =/
It's now called 'AsyncGetResponse' (no longer 'GetResponseAsync'). And 'Run' was renamed to 'RunSynchronously'. So I don't think you're missing anything substantial here, just name changes in the latest release.
What are your debugger settings with regard to "Tools\Options\Debugging\General\Enable Just My Code" and "Debug\Exceptions" (e.g. set to break when any first-chance CLR exception is thrown or not)? I am unclear if your question involves the program behavior, or the VS tooling behavior (sounds like the latter). This is further confounded by the fact that breakpoint/debugging 'locations' in F# Beta1 have some bugs, especially regarding async workflows, which means that the behavior you see in the debugger may look a little strange even if the program is executing properly...
Are you using VS2008 CTP or VS2010 Beta1?
In any case, it appears the exception due to a 500 response is expected, this is how WebRequest works. Here's a short demo program:
open System
open System.ServiceModel
open System.ServiceModel.Web
[<ServiceContract>]
type IMyContract =
[<OperationContract>]
[<WebGet(UriTemplate="/Returns500")>]
abstract Returns500 : unit -> unit
[<OperationContract>]
[<WebGet(UriTemplate="/Returns201")>]
abstract Returns201 : unit -> unit
type MyService() =
interface IMyContract with
member this.Returns500() =
WebOperationContext.Current.OutgoingResponse.StatusCode <-
System.Net.HttpStatusCode.InternalServerError
member this.Returns201() =
WebOperationContext.Current.OutgoingResponse.StatusCode <-
System.Net.HttpStatusCode.Created
let addr = "http://localhost/MyService"
let host = new WebServiceHost(typeof<MyService>, new Uri(addr))
host.AddServiceEndpoint(typeof<IMyContract>, new WebHttpBinding(), "") |> ignore
host.Open()
open System.Net
let url500 = "http://localhost/MyService/Returns500"
let url201 = "http://localhost/MyService/Returns201"
let async_value (url:string) =
async {
let req = WebRequest.Create(url)
let! rsp = req.AsyncGetResponse()
return (rsp :?> HttpWebResponse).StatusCode
}
let status = Async.RunSynchronously(async_value url201)
printfn "%A" status
try
let status = Async.RunSynchronously(async_value url500)
printfn "%A" status
with e ->
printfn "%s" (e.ToString())
You can use try...with inside the async to catch exceptions:
let async_value =
async {
let req = WebRequest.Create("http://unknown")
try
let! resp = req.AsyncGetResponse()
return "success"
with
| :? WebException as e -> return "failure"
}