F# async web request, handling exceptions - f#

I'm trying to use async workflows in F# to fetch several web requests.
However, some of my requests are occasionally returning errors (e.g. http 500), and I don't know how to handle this. It appears like my F# program gets stuck in an infinite loop when running in the debugger.
I'm probably missing some stuff, cause examples I seen didn't compile out of the box. First thing I found that helped was this bit of code:
type System.Net.WebRequest with
member req.GetResponseAsync() =
Async.BuildPrimitive(req.BeginGetResponse, req.EndGetResponse)
and then I have my bit of code to fetch the requests, which is pretty standard from examples I've seen:
let async_value = async {
let req = WebRequest.Create(url)
let! rsp = req.GetResponseAsync()
return (rsp :?> HttpWebResponse).StatusCode
}
and then I try to get the result:
let status = Async.RunSynchronously(async_value)
But when I run my program in debugger, it breaks at req.EndGetResponse because server returned internal server error 500. If I keep just continuing execution, it gets in a funky loop, breaking at req.EndGetResponse (sometimes several in a row), and at let status = Async.RunSynchronously(async_value).
How do I get around the exception problem so I can get my status code? Also, do I need the type thing I did above? Or am I missing some library/dll for F#/VS 2010 Beta 1, of which this is already a part of?
I actually run several requests in parallel, using Async.RunSynchronously(Async.Parallel(my_array_of_async_values)), though I don't think that is related to the exception issue I'm having.
The fact the examples I've come across only use Async.Run rather than Async.RunSynchronously is probably an indicator I'm missing something... =/

It's now called 'AsyncGetResponse' (no longer 'GetResponseAsync'). And 'Run' was renamed to 'RunSynchronously'. So I don't think you're missing anything substantial here, just name changes in the latest release.
What are your debugger settings with regard to "Tools\Options\Debugging\General\Enable Just My Code" and "Debug\Exceptions" (e.g. set to break when any first-chance CLR exception is thrown or not)? I am unclear if your question involves the program behavior, or the VS tooling behavior (sounds like the latter). This is further confounded by the fact that breakpoint/debugging 'locations' in F# Beta1 have some bugs, especially regarding async workflows, which means that the behavior you see in the debugger may look a little strange even if the program is executing properly...
Are you using VS2008 CTP or VS2010 Beta1?
In any case, it appears the exception due to a 500 response is expected, this is how WebRequest works. Here's a short demo program:
open System
open System.ServiceModel
open System.ServiceModel.Web
[<ServiceContract>]
type IMyContract =
[<OperationContract>]
[<WebGet(UriTemplate="/Returns500")>]
abstract Returns500 : unit -> unit
[<OperationContract>]
[<WebGet(UriTemplate="/Returns201")>]
abstract Returns201 : unit -> unit
type MyService() =
interface IMyContract with
member this.Returns500() =
WebOperationContext.Current.OutgoingResponse.StatusCode <-
System.Net.HttpStatusCode.InternalServerError
member this.Returns201() =
WebOperationContext.Current.OutgoingResponse.StatusCode <-
System.Net.HttpStatusCode.Created
let addr = "http://localhost/MyService"
let host = new WebServiceHost(typeof<MyService>, new Uri(addr))
host.AddServiceEndpoint(typeof<IMyContract>, new WebHttpBinding(), "") |> ignore
host.Open()
open System.Net
let url500 = "http://localhost/MyService/Returns500"
let url201 = "http://localhost/MyService/Returns201"
let async_value (url:string) =
async {
let req = WebRequest.Create(url)
let! rsp = req.AsyncGetResponse()
return (rsp :?> HttpWebResponse).StatusCode
}
let status = Async.RunSynchronously(async_value url201)
printfn "%A" status
try
let status = Async.RunSynchronously(async_value url500)
printfn "%A" status
with e ->
printfn "%s" (e.ToString())

You can use try...with inside the async to catch exceptions:
let async_value =
async {
let req = WebRequest.Create("http://unknown")
try
let! resp = req.AsyncGetResponse()
return "success"
with
| :? WebException as e -> return "failure"
}

Related

How to use results of FSharp.Compiler.Services

I'm trying to build a system that is similar to FsBolero (TryWebassembly), Fable Repl and many more that uses Fsharp.Compiler.Services.
So I expect it is feasible to achieve my goals but I encountered a problem that I hope is only a result of my lack of experience with that realm of software development
I'm implementing a service that gives user the power to write custom algorithms (DSL) in the context of the domain system.
The code to compile come as a plain raw string that is fully correct F# code.
Sample DSL algorithm looks like:
let code = """
module M
open Lifespace
open Lifespace.LocationPricing
let alg (pricing:LocationPricing) =
let x=pricing.LocationComparisions.CityLevel.Transportation
(8.*x.PublicTransportationStation.Data+ x.RailwayStation.Data+ 5.*x.MunicipalBikeStation.Data) / 14.
"""
that code compiles correctly via CompileToDynamicAssembly. I also provided proper reference to my domain *.dll via -r Fsc parameter.
And here comes my problems as next I have the generated dynamic assembly and want to invoke that algorithm.
I do it with reflection (is there any other way?) with
f.Invoke(null, [|arg|]) when arg is of type LocationPricing and comes from main/hosting project reference.
The Invoke doesn't work because I have error:
Cannot cast LocationPricing to LocationPricing
I had the same problem when tried to use F# interactive services, the error was similar:
Cannot cast [A]LocationPricing to [B]LocationPricing
I'm aware I have two same dlls in the context and F# does have extern alias syntax to solve it.
But other mentioned public systems somehow deals with that or I'm doing it wrongly.
I will look at code of Bolero and FableRepl but it will definately take some time to understand the pitfalls.
Update: Full code (Azure Function)
namespace AzureFunctionFSharp
open System.IO
open System.Text
open Microsoft.Azure.WebJobs
open Microsoft.Azure.WebJobs.Extensions.Http
open Microsoft.AspNetCore.Http
open Microsoft.AspNetCore.Mvc
open Microsoft.Extensions.Logging
open FSharp.Compiler.SourceCodeServices
open Lifespace.LocationPricing
module UserCodeEval =
type CalculationResult = {
Value:float
}
type Error = {
Message:string
}
[<FunctionName("UserCodeEvalSampleLocation")>]
let Run([<HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)>] req: HttpRequest, log: ILogger , [<Blob("ranks/short-ranks.json", FileAccess.Read)>] myBlob:Stream)=
log.LogInformation("F# HTTP trigger function processed a request.")
// confirm valid domain dll location
// for a in System.AppDomain.CurrentDomain.GetAssemblies() do
// if a.FullName.Contains("wrometr.lam.to.ranks") then log.LogInformation(a.Location)
// let code = req.Query.["code"].ToString()
// replaced just to show how the user algorithm can looks like
let code =
"""
module M
open Lifespace
open Lifespace.LocationPricing
open Math.MyStatistics
open MathNet.Numerics.Statistics
let alg (pricing:LocationPricing) =
let x= pricing.LocationComparisions.CityLevel.Transportation
(8.*x.PublicTransportationStation.Data+ x.RailwayStation.Data+ 5.*x.MunicipalBikeStation.Data) / 14.
"""
use reader = new StreamReader(myBlob, Encoding.UTF8)
let content = reader.ReadToEnd()
let encode x = LocationPricingStore.DecodeArrayUnpack x
let pricings = encode content
let checker = FSharpChecker.Create()
let fn = Path.GetTempFileName()
let fn2 = Path.ChangeExtension(fn, ".fsx")
let fn3 = Path.ChangeExtension(fn, ".dll")
File.WriteAllText(fn2, code)
let errors, exitCode, dynAssembly =
checker.CompileToDynamicAssembly(
[|
"-o"; fn3;
"-a"; fn2
"-r";#"C:\Users\longer\azure.functions.compiler\bin\Debug\netstandard2.0\bin\MathNet.Numerics.dll"
"-r";#"C:\Users\longer\azure.functions.compiler\bin\Debug\netstandard2.0\bin\Thoth.Json.Net.dll"
// below is crucial and obtained with AppDomain resolution on top, comes as a project reference
"-r";#"C:\Users\longer\azure.functions.compiler\bin\Debug\netstandard2.0\bin\wrometr.lam.to.ranks.dll"
|], execute=None)
|> Async.RunSynchronously
let assembly = dynAssembly.Value
// get one item to test the user algorithm works in the funtion context
let arg = pricings.[0].Data.[0]
let result =
match assembly.GetTypes() |> Array.tryFind (fun t -> t.Name = "M") with
| Some moduleType ->
moduleType.GetMethods()
|> Array.tryFind (fun f -> f.Name = "alg")
|>
function
| Some f -> f.Invoke(null, [|arg|]) |> unbox<float>
| None -> failwith "Function `f` not found"
| None -> failwith "Module `M` not found"
// end of azure function, not important in the problem context
let res = req.HttpContext.Response
match String.length code with
| 0 ->
res.StatusCode <- 400
ObjectResult({ Message = "No Good, Please provide valid encoded user code"})
| _ ->
res.StatusCode <-200
ObjectResult({ Value = result})
**Update: changing data flow **
To move forward I resigned to use domain types in both places. Instead I do all logic in domain assembly and only pass primitives (strings) to reflected invocation. I'm also suprised a lot that caching still works everytime I do compilation on each Azure Function call. I will experiment as well with FSI, in theory it should be faster than reflection but with additional burden to pass parameters to evaluations
In your example, the code that runs inside your dynamically compiled assembly and the code calling it need to share a type LocationPricing. The error you are seeing typically means that you somehow ended up with different assembly loaded in the process that is calling the dynamically compiled code and the code actually running the computation.
It is hard to say exactly why this happened, but you should be able to check whether this is indeed the case by looking at assemblies loaded in the current App Domain. Say that your shared assembly is MyAssembly. You can run:
for a in System.AppDomain.CurrentDomain.GetAssemblies() do
if a.FullName.Contains("MyAssembly") then printfn "%s" a.Location
If you were using F# Interactive Services, then a trick to fix this is to start an FSI session and then send an interaction to the service that loads the assembly from the right place. Something along those lines:
let myAsm = System.AppDomain.CurrentDomain.GetAssemblies() |> Seq.find (fun asm ->
asm.FullName.Contains("MyAssembly"))
fsi.EvalInteraction(sprintf "#r #\"%s\"" myAsm.Location)

Avoid exception causes to stop Mono.zip immediately

Is it possible to avoid that if one mono in mono.zip throws exception all other monos are stopping immediately? I want them to end normally and perhaps to handle the erroneous one by something like „.doOnError“ or „.continueOnError. Is that a way to go?
Regards
Bernado
Yes, it's possible. You can use Mono.zipDelayError. As you can understand from the method's name, it delays errors from the Monos. If several Monos error, their exceptions are combined.
If you have to get the combined result anyway, zipDelayError is not the solution. Use the zip operator and handle the error case with a fallback operator like onErrorResume or retry on the zipped Mono or any upstream one.
I stated that my question is answered but it is not yet. The following example states my case: some mono will fail, but i want the result as the error too. i expected the follwoing code as to run to completion but it fails:
Mono<String> error = Mono.error(new RuntimeException());
error = error.onErrorResume(throwable -> Mono.just("hell0"));
Mono<String> test = Mono.just("test");
Mono<String> test1 = Mono.just("test1");
Mono<String> test2 = Mono.just("test2");
List<Mono<String>> monolist = new ArrayList<>();
monolist.add(test);
monolist.add(test1);
monolist.add(test2);
monolist.add(error);
Mono<Long> zipDelayError = Mono.zipDelayError(monolist, arrayObj -> Arrays.stream(arrayObj).count());
System.out.println(zipDelayError.block());

f# SqlDataProvider .Net Core 2.0 - enlisting in ambient is not supported

By day I'm a C# programmer, but an F# enthusiast.
whist doing some tutorial (suave) I stumbled upon this error
System.NotSupportedException
HResult=0x80131515
Message=Enlisting in Ambient transactions is not supported.
Source=System.Data.SqlClient
StackTrace:
at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
at System.Data.SqlClient.SqlConnection.Open()
at FSharp.Data.Sql.Providers.MSSqlServerProvider.FSharp-Data-Sql-Common-ISqlProvider-ProcessUpdates(IDbConnection con, ConcurrentDictionary`2 entities, TransactionOptions transactionOptions, FSharpOption`1 timeout)
at <StartupCode$FSharp-Data-SqlProvider>.$SqlRuntime.DataContext.f#1-52(SqlDataContext __, IDbConnection con, Unit unitVar0)
at FSharp.Data.Sql.Runtime.SqlDataContext.FSharp-Data-Sql-Common-ISqlDataContext-SubmitPendingChanges()
at Program.main(String[] argv) in C:\Users\M_R_N\source\repos\ConsoleApp2\ConsoleApp2\Program.fs:line 34
yet the code seems so trivial, I cant believe it doesn work, we seem to be able to read data from a (SQL express) database, but not write to it (or at least not delete, I've not tried adding). I don't really know what an ambient transaction is, I'm actually not to concerned about transactional behaviour I simply want to select some data, update it or delete it.
This is all the code....
open System
open FSharp.Data.Sql
[<Literal>]
let ConnectionString =
"Data Source=(localdb)\ProjectsV13;Initial Catalog=suavemusicstore;Integrated Security=SSPI;Connect Timeout=30;Encrypt=False;TrustServerCertificate=True;ApplicationIntent=ReadWrite;MultiSubnetFailover=False"
type Sql =
SqlDataProvider<
ConnectionString = ConnectionString,
DatabaseVendor = Common.DatabaseProviderTypes.MSSQLSERVER>
type DbContext = Sql.dataContext
type Album = DbContext.``dbo.AlbumsEntity``
type Genre = DbContext.``dbo.GenresEntity``
let getAlbum id (ctx : DbContext) : Album option =
query {
for album in ctx.Dbo.Albums do
where (album.AlbumId = id)
select album
} |> Seq.tryHead
[<EntryPoint>]
let main argv =
let ctx = Sql.GetDataContext()
match (getAlbum 2 ctx) with
| Some(album) ->
album.Delete()
ctx.SubmitUpdates() // EXCEPTION thrown here
0
| _ -> 0
is there a workaround? its the first time I've used type providers and core, yet it seems that you cannot write a simple CRUD app.
this HAS been reported elsewhere, mostly in C# EF apps, where I think there is more scope to work around the problem (maybe).
Any ideas how to work around it? I've tried upgrading/downgrading various nugget packages, to no avail
I've had the same issue not that long ago on my toy F# project. I did not find any proper solution and ended up ignoring transactions entirely. This doesn't solve the underlying issue, but for me it was enough (the project is mainly for learning purposes).
let TransactionOptions = {IsolationLevel = IsolationLevel.DontCreateTransaction; Timeout = TimeSpan.FromSeconds(1.0)}
let dbContext = Sql.GetDataContext(TransactionOptions)
I don't know if the above works, but it seems like a reasonable workaround.
I DID fix my problem, but by reverting to a framework based console app template rather than the core one.

Does the >>= operator not take a function?

I'm working on a side project and I'm using Hopac for the first time. I ran into an odd (to me) compilation issue that I haven't been able to grok. I suspect that I'm the problem here, and not Hopac.
The program is supposed to be a simple console app that consumes notifications from various services. Here's the problematic module:
module Provider
open System
open System.IO
open Hopac
open BitThicket.NotificationHelper.Core
open BitThicket.NotificationHelper.Providers
let defaultProviderTypes =
[| typeof<GitHub.GitHubNotificationProvider> |]
type Provider = {
getCh : Ch<Providers.INotification seq>
}
let giveLatest ch latest =
Ch.give
let start config logger (providerType:Type) = Job.delay <| fun () ->
let providerImpl = Activator.CreateInstance(providerType) :?> Providers.INotificationProvider
let p = { getCh = Ch() }
let rec server =
let latest = providerImpl.GetLatestNotificationsAsync(None) |> Job.fromAsync
latest >>= Ch.give p.getCh // error here
}
Job.start server
In this case, the compiler complains: Expecting a type supporting the operator '>>=' but given a function type. You may be missing an argument to a function.
Similarly, if I use a slightly different syntax:
// ...
let rec server =
let latest = providerImpl.GetLatestNotificationsAsync(None) |> Job.fromAsync
latest >>= fun l -> Ch.give p.getCh l // error here
// ...
In this case, the error is: This function takes too many arguments, or is used in a context where a function is not expected.
I asked haf about his in slack, and his suggestion was to check for alternative definitions of >>=. The tooling doesn't really do much to help me figure that one out, but the only namespace/module I have opened that defines >>= is Hopac (the BitThicket ones are just trivially simple namespaces with some type definitions in them).
What am I doing wrong here?
I'm looking at the source code, and I see that the bind operator is actually defined in Hopac.Infixes, not in Hopac.

Is it possible to attach to the AppDomain.UnhandledException event?

I'm trying to create an AppDomain and attach to its UnhandledException event from F#, and I'm not having much luck. In my event handler I need a reference to the domain that fired the event, as well as the event arguments.
To reproduce this issue, the following code must be run in a compiled application. Running it in F# Interactive produces a different, probably-not-related error.
open System
open System.Reflection
let domainError (domain:AppDomain) (args:UnhandledExceptionEventArgs) =
()
let domain = AppDomain.CreateDomain("test")
domain.UnhandledException
|> Event.add (domainError domain)
This compiles just fine, but at runtime I get the following error:
SerializationException: Type 'Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers+h#720' in assembly 'FSharp.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' is not marked as serializable.
The stack trace that goes along with this error is as follows:
at System.AppDomain.add_UnhandledException(UnhandledExceptionEventHandler value)
at Program.clo#9.Invoke(UnhandledExceptionEventHandler eventDelegate) in c:\users\joel\documents\visual studio 2010\Projects\EventTest\Program.fs:line 9
at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.CreateEvent#716.Subscribe(IObserver`1 observer)
at Microsoft.FSharp.Control.CommonExtensions.SubscribeToObservable[T](IObservable`1 x, FSharpFunc`2 callback)
at Microsoft.FSharp.Control.EventModule.Add[T,TDel](FSharpFunc`2 callback, IEvent`2 sourceEvent)
at <StartupCode$EventTest>.$Program.main#() in c:\users\joel\documents\visual studio 2010\Projects\EventTest\Program.fs:line 9
Now I obviously can't make Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers+h#720 serializable, so can anyone suggest a workable way of attaching to this particular event? Thanks for any suggestions.
Update
Thanks to a suggestion from ChaosPandion, this version works:
open System
open System.Reflection
let domainError (sender:obj) (args:UnhandledExceptionEventArgs) =
let problemDomain = sender :?> AppDomain
printfn "Unhandled exception in app domain: %s" problemDomain.FriendlyName
()
let domain = AppDomain.CreateDomain("test")
domain.UnhandledException.AddHandler(UnhandledExceptionEventHandler(domainError))
After a bit more tweaking I found this example will fail:
let domainError (domain:AppDomain) =
let handler (sender:obj) (e:UnhandledExceptionEventArgs) =
let msg = (e.ExceptionObject :?> Exception).Message
printfn "An exception was unhandled in %s\nMessage:%s" domain.FriendlyName msg
new UnhandledExceptionEventHandler(handler)
let main() =
let domain = AppDomain.CreateDomain("test")
let handler = domainError domain
domain.UnhandledException.AddHandler handler
I broke open Reflector and found the root cause. (Which is actually quite obvious now that I think about it.)
internal class handler#28 : OptimizedClosures.FSharpFunc<object, UnhandledExceptionEventArgs, Unit>
{
// Fields
public AppDomain domain;
// Methods
internal handler#28(AppDomain domain);
public override Unit Invoke(object sender, UnhandledExceptionEventArgs e);
}
This pretty much means that the only way you can get this to work is to not create the closure around the domain object. I am not sure if this will work for you but you may want to try using the following from within the handler.
AppDomain.CurrentDomain

Resources