I was wondering what is the easiest way to stream F# seq in Giraffe. Not much, but here's what I have:
module HttpHandler =
let handlerGuids : HttpHandler =
handleContext(
fun ctx ->
task {
let collection =
seq {
let mutable i = 0
while (not ctx.RequestAborted.IsCancellationRequested) && i <10 do
i <- i + 1
Async.Sleep(2000) |> Async.RunSynchronously
yield Guid.NewGuid()
}
return! ctx.WriteJsonChunkedAsync collection
})
let router: HttpFunc -> HttpContext -> HttpFuncResult =
choose [ route "/" >=> handlerGuids ]
I laso have this test in C#
[Fact]
public async void Test1()
{
using var httpClient = new HttpClient();
httpClient.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);
var requestUri = "http://localhost:8080/";
var stream = await httpClient.GetStreamAsync(requestUri);
using var reader = new StreamReader(stream);
while (!reader.EndOfStream) {
var currentLine = reader.ReadLine();
}
}
But it waits until all guids are generated on the server. Can someone give me some hints? Giraffe documentation says sth about streaming but it is related to files.
Related
I want to return an HttpResponseMessage, which implements IDisposable, within a Result (or any other discriminated union). However, since Result itself is not IDisposable, I can't use! it like I would for the disposable object itself. What do I do? Can I implement my own DU called DisposableResult that implements IDisposable itself?
Below is an example of what I mean. crawlStuff asynchronously returns Result<HttpResponseMessage, string>. I can't use! the result, leading to a memory leak unless I manually release it.
open System.Net.Http
let crawlStuff (client : HttpClient) : Async<Result<HttpResponseMessage, string>> =
async {
// res is HttpResponseMessage which implements IDisposable
let! res = client.GetAsync("https://ifconfig.me") |> Async.AwaitTask
return
if res.IsSuccessStatusCode then
Ok res
else
Error "Something wrong"
}
async {
use client = new HttpClient()
// Memory leak because result it could carry HttpResponseMessage.
// I want to use! here, but can't because Result<> is not IDisposable
let! response = crawlStuff client
printfn "%A" response
} |> Async.RunSynchronously
Yes, you can implement your own disposable result type, like this:
type DisposableResult<'t, 'terr when 't :> IDisposable> =
| DOk of 't
| DError of 'terr
interface IDisposable with
member this.Dispose() =
match this with
| DOk value -> value.Dispose()
| _ -> ()
Usage would then be:
open System.Net.Http
let crawlStuff (client : HttpClient) : Async<Result<HttpResponseMessage, string>> =
async {
// res is HttpResponseMessage which implements IDisposable
let! res = client.GetAsync("https://ifconfig.me") |> Async.AwaitTask
return
if res.IsSuccessStatusCode then
DOk res
else
DError "Something wrong"
}
async {
use client = new HttpClient()
use! response = crawlStuff client
printfn "%A" response
} |> Async.RunSynchronously
I would've create wrapper around Result, that will dispose underlying values:
let (|AsDisposable|) x =
match box x with
| :? IDisposable as dis -> ValueSome dis
| _ -> ValueNone
type ResultDisposer<'v, 'e> =
struct
val Result : Result<'v, 'e>
new res = { Result = res }
interface IDisposable with
member r.Dispose() =
match r.Result with
// | Ok (:? IDisposable as dis) // causes FS0008, so we have to hack
| Ok (AsDisposable (ValueSome dis))
| Error (AsDisposable (ValueSome dis)) -> dis.Dispose()
| _ -> ()
end
type Result<'a, 'b> with
member r.AsDisposable = new ResultDisposer<'a, 'b>(r)
And use it this way
async {
use client = new HttpClient()
let! response = crawlStuff client
use _ = response.AsDisposable
printfn "%A" response
} |> Async.RunSynchronously
This solution avoids need to rewrite existing code to DisposableResult and avoids allocations when disposable value is reference type, like in case of HttpResponseMessage. But decompilation shows that F# boxes ResultDisposer, even though it shouldn't :(
I am trying to subscribe to a RabbitMq topic using the PubSub.Subscribe method in EasyNetq with F#. The function subscribeToAppQueueWithoutTopic compiles and works but the subscribeToAppQueueWithTopic function will not compile at all.
let subscribeToAppQueueWithTopic (callback : Subscription<AppEnvelope>) =
bus.PubSub.Subscribe<AppEnvelope>(String.Empty, callback.OnMessageReceived,
(fun (x:ISubscriptionConfiguration) -> x.WithTopic("app.queue")), cts.Token)
Error FS0041 No overloads match for method 'Subscribe'.
Known types of arguments: string * (AppEnvelope -> unit) * (ISubscriptionConfiguration -> ISubscriptionConfiguration) * CancellationToken
Available overloads:
- (extension) IPubSub.Subscribe<'T>(subscriptionId: string, onMessage: Action<'T>, configure: Action<ISubscriptionConfiguration>, ?cancellationToken: CancellationToken) : ISubscriptionResult // Argument 'configure' doesn't match
- (extension) IPubSub.Subscribe<'T>(subscriptionId: string, onMessage: Func<'T,CancellationToken,Tasks.Task>, configure: Action<ISubscriptionConfiguration>, ?cancellationToken: CancellationToken) : ISubscriptionResult // Argument 'onMessage' doesn't match
I found a c# example of subscribing with a topic here EasyNetQ subscription tests which looks like this
bus.PubSub.SubscribeAsync<Message>(
Guid.NewGuid().ToString(),
firstTopicMessagesSink.Receive,
x => x.WithTopic("first"),
cts.Token
and thought I could use fun (x:ISubscriptionConfiguration) -> x.WithTopic("app.queue") as equivalent in F#. Alas this will not compile.
Here is an example app showing the problem
open System
open EasyNetQ
open System.Threading
type Subscription<'T> = { OnMessageReceived: 'T -> unit }
[<Queue("appqueue", ExchangeName = "demopappexchange")>]
type AppEnvelope = { Message : obj }
[<EntryPoint>]
let main argv =
let bus = RabbitHutch.CreateBus("host=localhost")
let cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
let printMessage message =
printfn "%s" message
let subscription = {
OnMessageReceived = fun (envelope: AppEnvelope) -> (envelope.Message.ToString() |> printMessage )
}
let sendToAppWithTopic message =
async {
do! bus.PubSub.PublishAsync({AppEnvelope.Message = message}, "app.queue") |> Async.AwaitTask
// bus.Dispose()
} |> Async.Start
let subscribeToAppQueueWithoutTopic (callback : Subscription<AppEnvelope>) =
printfn "subscribe called"
bus.PubSub.Subscribe<AppEnvelope>(String.Empty, callback.OnMessageReceived)
(* ** Will not compile **
let subscribeAsyncToAppQueueWithTopic =
async {
do! bus.PubSub.SubscribeAsync<AppEnvelope>(String.Empty, callback.OnMessageReceived,
fun (x: ISubscriptionConfiguration) -> x.WithTopic "scanservice.queue")
|> Async.AwaitTask
} |> Async.Start
*)
// Will not compile
let subscribeToAppQueueWithTopic (callback : Subscription<AppEnvelope>) =
bus.PubSub.Subscribe<AppEnvelope>(String.Empty, callback.OnMessageReceived, (fun (x:ISubscriptionConfiguration) -> x.WithTopic("app.queue")), cts.Token)
subscribeToAppQueueWithoutTopic subscription |> ignore
sendToAppWithTopic "Testing"
Console.ReadKey() |> ignore
0
I don't know anything about EasyNetQ, but I think the problem here is that WithTopic returns a reference to the mutated configuration, which you need to explicitly ignore in F# in order to produce an Action<_>, like this:
let subscribeToAppQueueWithTopic (callback : Subscription<AppEnvelope>) =
bus.PubSub.Subscribe<AppEnvelope>(
String.Empty,
callback.OnMessageReceived,
(fun (x:ISubscriptionConfiguration) -> x.WithTopic("app.queue") |> ignore),
cts.Token)
Apparently, the API does this in order to provide a fluent C# interface:
/// <summary>
/// Allows publish configuration to be fluently extended without adding overloads
///
/// e.g.
/// x => x.WithTopic("*.brighton").WithPriority(2)
/// </summary>
public interface IPublishConfiguration
{
/// <summary>
/// Sets a priority of the message
/// </summary>
/// <param name="priority">The priority to set</param>
/// <returns>Returns a reference to itself</returns>
IPublishConfiguration WithPriority(byte priority);
/// <summary>
/// Sets a topic for the message
/// </summary>
/// <param name="topic">The topic to set</param>
/// <returns>Returns a reference to itself</returns>
IPublishConfiguration WithTopic(string topic);
From a functional programming perspective, this is a confusing way to do things, but such is life in the C# world, I suppose.
Suppose I have a handler that computes some data from a request (e.g. the user ID) and that data is used by subsequent code in the application.
I can think of two methods for passing this data along:
1. Mutate the ctx.Items collection
This would look like this:
let authenticate : HttpHandler =
fun (next : HttpFunc) (ctx : HttpContext) ->
task {
printfn "Authenticating... "
let userID = Guid.NewGuid () // Dummy
printfn "Authenticated as %A" userID
ctx.Items.["userID"] <- userID
return! next ctx
}
Then to get the data, just do:
let showUser : HttpHandler =
fun (next : HttpFunc) (ctx : HttpContext) ->
task {
let userID = ctx.Items.["userID"] :?> Guid
return! text (sprintf "You are authenticated as %A. " userID) next ctx
}
2. Build a "wrapper" function (IoC)
This would look like this:
let authenticated (innerHandler : Guid -> HttpHandler) : HttpHandler =
fun (next : HttpFunc) (ctx : HttpContext) ->
task {
printfn "Authenticating... "
let userID = Guid.NewGuid () // Dummy
printfn "Authenticated as %A" userID
return! (innerHandler userID) next ctx
}
Getting the data is even easier and extra type-safe!
let showUser (userID : Guid) : HttpHandler =
fun (next : HttpFunc) (ctx : HttpContext) ->
task {
return! text (sprintf "You are authenticated as %A. " userID) next ctx
}
However you lose some composition with >=>.
Questions
What approach did the Giraffe designers intend?
What are the trade-offs with each approach?
I'm trying to create an asyncEventBasicConsumer in F#.
In order to do that I need to use add_Received that has the follow signature
member add_Received:
value: AsyncEventHandler<BasicDeliverEventArgs>
-> unit
But I don't have idea how to create an AsyncEventHandler from
async {}
This is the concrete function
let subscribe : Subscribe =
fun factory queueName handleMessage ->
let connection = factory.CreateConnection()
let model = connection.CreateModel()
let consumer = AsyncEventingBasicConsumer(model)
consumer.add_Received //?? how create a AsyncEventHandler
model.BasicConsume(queueName, false, consumer) |> ignore
(fun () ->
model.Close()
connection.Close())
And here the c# code
static void Main(string[] args)
{
var factory = new ConnectionFactory() { DispatchConsumersAsync = true };
const string queueName = "myqueue";
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queueName, true, false, false, null);
// consumer
var consumer = new AsyncEventingBasicConsumer(channel);
consumer.Received += Consumer_Received;
channel.BasicConsume(queueName, true, consumer);
// publisher
var props = channel.CreateBasicProperties();
int i = 0;
while (true)
{
var messageBody = Encoding.UTF8.GetBytes($"Message {++i}");
channel.BasicPublish("", queueName, props, messageBody);
Thread.Sleep(50);
}
}
}
private static async Task Consumer_Received(object sender, BasicDeliverEventArgs #event)
{
var message = Encoding.UTF8.GetString(#event.Body);
Console.WriteLine($"Begin processing {message}");
await Task.Delay(250);
Console.WriteLine($"End processing {message}");
}
This should do the trick
consumer.add_Received(fun sender event -> Consumer_Received sender event |> Async.StartAsTask :> Task)
Notable points are casting async to Task and upcasting Task<'a> to Task.
Given the HOCON below for a consitent-hashing-poolrouter, how do I specify the hashMapping.
akka {
actor {
serializers {
wire = "Akka.Serialization.WireSerializer, Akka.Serialization.Wire"
}
serialization-bindings {
"System.Object" = wire
}
deployment {
/data-collector {
router = consistent-hashing-pool
nr-of-instances = 10
}
}
}
loggers = ["Akka.Logger.NLog.NLogLogger,Akka.Logger.NLog"]
}
Given
let config = Configuration.load()
let systemName = config.GetString("app-config.actorsystem", "my-system")
let system = System.create systemName config
let collectorRouter =
let hashMapping (msg:obj) =
match msg with
| :? Message as msg ->
match msg with
| Parse (_, req) -> req.uniqueId |> box
| _ -> "-" |> box
ConsistentHashingPool (10, ConsistentHashMapping hashMapping)
let dataCollectorProps =
{ props (dataCollector settings.collector) with
Router = Some (collectorRouter :> RouterConfig)} //(FromConfig.Instance.WithFallback collectorRouter)
let test = spawn system "data-collector" <| dataCollectorProps
Router = Some (collectorRouter :> RouterConfig) work
Router = Some (FromConfig.Instance.WithFallback collectorRouter) doesn't
What is the correct way to specify the hashMapping function?
Edit 1
The warning from the console is
Akka.Routing.ConsistentHashingRoutingLogic|Message [Message] must be handled by hashMapping, or implement [IConsistentHashable] or be wrapped in [ConsistentHashableEnvelope]
There are three ways of specifying the hash key. My favourite is to implement IConsistentHashable and return the key (based on some properties of the message) from the ConsistentHashKey property.
Example excerpt from my Consistent Hashing article (in C#; sorry I don't know F#):
public class CurrencyPriceChangeMessage : IConsistentHashable
{
public string CurrencyPair { get; }
public decimal Price { get; }
public object ConsistentHashKey
{
get
{
return this.CurrencyPair;
}
}
The hash key will be used to route messages with the same key always to the same routee.