I'm seeking for best practices in modeling request-response based system using functional approach in F#.
Generalized case requirements for example:
there are many request/response pair types: A, B, C, etc
each response must represent two states: succeed and failed
each request consists of common data piece and specific A/B/... data piece
while processing, request may be rejected both for common and specific reasons
As for now, I end up with following design:
type CommonData = {...}
type RequestMessage =
| ARequestMessage of CommonData * ARequestData
| BRequestMessage of CommonData * BRequestData
...
type Response<'S, 'F> =
| Success of 'S
| Failure of 'F
type ResponseMessage =
| AResponseMessage of Response<AResponseSuccess, AResponseFailure>
| BResponseMessage of Response<BResponseSuccess, BResponseFailure>
...
where AResponseSuccess and BResponseSuccess contains same discriminated union cases or ResponseMessage type is covered by GeneralResponseMessage type like
type GeneralResponseMessage =
| CommonFailure of CommonResponseFailure
| SpecificResponse of ResponseMessage
both cases look terrible and I can't figure out more clear and elegant solution.
Edit
System processes requests in statemachine-manner over single evolving state, so it isn't possible to separate different types of request processing
This may be better suited for https://codereview.stackexchange.com/, but let me try to answer this here.
My take on this would be to group the operations vertically, not horizontally. What I mean by that is that instead of grouping all requests in one type and all responses in another, I would pair up the corresponding request and response types, and build a generic wrapper for the processing itself.
In your case, the processing fundamentally is a function of type RequestMessage -> GeneralResponseMessage. I propose generalizing this into the following:
type ProcessRequest<'requestData, 'responseMessage, 'responseFailure> =
CommonData -> 'requestData ->
Result<'responseMessage, ErrorResponse<'responseFailure>>
and ErrorResponse<'responseFailure> =
| CommonFailure of CommonResponseFailure
| SpecificResponse of 'responseFailure
Just a note here, I use Result from FSharp.Core instead of your Response type, as they model the exact same thing.
Then you can create a specific instances, for example
type AProcessRequest =
ProcessRequest<ARequestMessage, AResponseMessage, AResponseFailure>
And this can then be used to process specific messages. In your case, the processing logic would have to match on RequestMessage case and proceed based on that, in my case you have to this match before calling the specific processor, but if you need it grouped in one type, you can always do something this to supply all the implementations as a single piece of data
type ProcessMessage =
{ A : AProcessRequest
B : BProcessRequest
C : ... }
and you can supply the implementations in composition root.
If you don't like so many specific types, you don't have to give names to the concrete instances of the generic processor, C : ProcessMessage<CRequestMessage, CResponseMessage, CResponseFailure> will do just fine, but may be less readable if there are many instances of similar types.
Related
I have written several helper functions in F# that enable me to deal with the dynamic nature of Excel over the COM/PIA interface. However when I go to use these functions in an Excel-DNA UDF they do not work as expected as Excel-DNA is pre-processing the values in the array from excel.
e.g. null is turned into ExcelDna.Integration.ExcelEmpty
This interferes with my own validation code that was anticipating a null. I am able to work around this by adding an additional case to my pattern matching:
let (|XlEmpty|_|) (x: obj) =
match x with
| null -> Some XlEmpty
| :? ExcelDna.Integration.ExcelEmpty -> Some XlEmpty
| _ -> None
However it feels like a waste to convert and then convert again. Is there a way to tell Excel-DNA not to do additional processing of the range values in a UDF and supply them equivalent to the COM/PIA interface? i.e. Range.Value XlRangeValueDataType.xlRangeValueDefault
EDIT:
I declare my arguments as obj like this:
[<ExcelFunction(Description = "Validates a Test Table Row")>]
let isTestRow (headings: obj) (row: obj) =
let validator = TestTable.validator
let headingsList = TestTable.testHeadings
validateRow validator headingsList headings row
I have done some more digging and #Jim Foye's suggested question also confirms this. For UDF's, Excel-DNA works over the C API rather than COM and therefore has to do its own marshaling. The possible values are shown in this file:
https://github.com/Excel-DNA/ExcelDna/blob/2aa1bd9afaf76084c1d59e2330584edddb888eb1/Distribution/Reference.txt
The reason to use ExcelEmpty (the user supplied an empty cell) is that for a UDF, the argument can also be ExcelMissing (the user supplied no argument) which might both be reasonably null and there is a need to disambiguate.
I will adjust my pattern matching to be compatible with both the COM marshaling and the ExcelDNA marshaling.
Firstly, obtain a schema and parse:
type desc = JsonProvider< """[{"name": "", "age": 1}]""", InferTypesFromValues=true >
let json = """[{"name": "Kitten", "age": 322}]"""
let typedJson = desc.Parse(json)
Now we can access typedJson.[0] .Age and .Name properties, however, I'd like to pattern match on them at compile-time to get an error if the schema is changed.
Since those properties are erased and we cannot obtain them at run-time:
let ``returns false``() =
typedJson.[0].GetType()
.FindMembers(MemberTypes.All, BindingFlags.Public ||| BindingFlags.Instance,
MemberFilter(fun _ _ -> true), null)
|> Array.exists (fun m -> m.ToString().Contains("Age"))
...I've made a runtime-check version using active patterns:
let (|Name|Age|) k =
let toID = NameUtils.uniqueGenerator NameUtils.nicePascalName
let idk = toID k
match idk with
| _ when idk.Equals("Age") -> Age
| _ when idk.Equals("Name") -> Name
| ex_val -> failwith (sprintf "\"%s\" shouldn't even compile!" ex_val)
typedJson.[0].JsonValue.Properties()
|> Array.map (fun (k, v) ->
match k with
| Age -> v.AsInteger().ToString() // ...
| Name -> v.AsString()) // ...
|> Array.iter (printfn "%A")
In theory, if FSharp.Data wasn't OS I wouldn't be able to implement toID. Generally, the whole approach seems wrong and redoing the work.
I know that discriminated unions can't be generated using type providers, but maybe there's a better way to do all this checking at compile-time?
As far as I know it cannot be possible to find out if "Json schema has changed" at compile-time using the given TP.
That's why:
JsonProvider<sample> is exactly what kicks in at compile-time providing a type for manipulating Json contents at run-time. This provided erased type has couple of run-time static methods common for any sample and type Root
extending IJsonDocument with few instance properties including ones based on compile-time provided sample (in your case - properties Name and Age).There is exactly one very relaxed implicit
Json "schema" behind JsonProvider-provided type, no another such entity to compare with for change at compile-time;
at run-time only provided type desc with its static methods and its Root type with correspondent instance methods
are at your service for manipulating arbitrary Json contents. All this jazz is pretty much agnostic with regard to
"Json schema", in your given case as long as run-time Json contents represent an array its elements may be pretty much any.
For example,
type desc = JsonProvider<"""[{"name": "", "age": 1}]"""> // # compile-time
let ``kinda "typed" json`` = desc.Parse("""[]""") // # run-time
let ``another kinda "typed" json`` =
desc.Parse("""[{"contents":"whatever", "text":"blah-blah-blah"},[{"extra":42}]]""")
both will be happily parsed at run-time as "typed Json" conforming to "schema" derived by TP from the given sample, although apparently Name and Age are missing and exceptions will be raised if accessed.
It is doable building another Json TP that relies upon a formal Json schema.
It may consume reference to the given schema from a schema repository upon type creation and allow manipulating elements
of the parsed Json payload only via provided accessors derived from the schema at compile-time.
In this case change of the
referred schema may break compilation if provided accessors being used in the code are incompatible with the change.
Such arrangement accompanied by run-time Json payload validator or validating parser may provide reliable enterprise-quality
Json schema change management.
JsonProvider TP from Fsharp.Data lacks such Json schema handling abilities, so payload validations are to be done in run-time only.
Quoting your comments which explain a little better what you are trying to achieve:
Thank you! But what I'm trying to achieve is to get a compiler error
if I add a new field e.g. Color to the json schema and then ignore it
while later processing. In case of unions it would be instant FS0025.
and:
yes, I must process all fields, so I can't rely on _. I want it so
when the schema changes, my F# program won't compile without adding
necessary handling functionality(and not just ignoring new field or
crashing at runtime).
The simplest solution for your purpose is to construct a "test" object.
The provided type comes with two constructors: one takes a JSonValue and parses it - effectively the same as JsonValue.Parse - while the other requires every field to be filled in.
That's the one that interests us.
We're also going to invoke it using named parameters, so that we'll be safe not only if fields are added or removed, but also if they are renamed or changed.
type desc = JsonProvider< """[{"name": "SomeName", "age": 1}]""", InferTypesFromValues=true >
let TestObjectPleaseIgnore = new desc.Root (name = "Kitten", age = 322)
// compiles
(Note that I changed the value of name in the sample to "SomeName", because "" was being inferred as a generic JsonValue.)
Now if more fields suddenly appear in the sample used by the type provider, the constructor will become incomplete and fail to compile.
type desc = JsonProvider< """[{"name": "SomeName", "age": 1, "color" : "Red"}]""", InferTypesFromValues=true >
let TestObjectPleaseIgnore = new desc.Root (name = "Kitten", age = 322)
// compilation error: The member or object constructor 'Root' taking 1 arguments are not accessible from this code location. All accessible versions of method 'Root' take 1 arguments.
Obviously, the error refers to the 1-argument constructor because that's the one it tried to fit, but you'll see that now the provided type has a 3-parameter constructor replacing the 2-parameter one.
If you use InferTypesFromValues=false, you get a strong type back:
type desc = JsonProvider< """[{"name": "", "age": 1}]""", InferTypesFromValues=false >
You can use the desc type to define active patterns over the properties you care about:
let (|Name|_|) target (candidate : desc.Root) =
if candidate.Name = target then Some target else None
let (|Age|_|) target (candidate : desc.Root) =
if candidate.Age = target then Some target else None
These active patterns can be used like this:
let json = """[{"name": "Kitten", "age": 322}]"""
let typedJson = desc.Parse(json)
match typedJson.[0] with
| Name "Kitten" n -> printfn "Name is %s" n
| Age 322m a -> printfn "Age is %M" a
| _ -> printfn "Nothing matched"
Given the typedJson value here, that match is going to print out "Name is Kitten".
<tldr>
Build some parser to handle your issues. http://www.quanttec.com/fparsec/
</tldr>
So...
You want something that can read something and do something with it. Without knowing apriori what any of those somethings is.
Good luck with that one.
You do not want type provider to do this for you. Type providers are made with the full purpose of being "at compile time this is what I saw, and thats what I will use".
With that said:
You want some other type of parser, where you are able to check the "schema" (some definition of what you know is going to come or you saw last time vs. what actually came). In effect some dynamic parser getting the data to some dynamic structure, with dynamic types.
And mind you, dynamic is not static. F# has static types and a lot is based on that. And type providers more so. Fighting that will make your head ache. It is of course possible, and it might even be possible by fighting the type providers to actually work with such an approach, but then again its not really a type provider nor its purpose.
While working through Expert F# again, I decided to implement the application for manipulating algebraic expressions. This went well and now I've decided as a next exercise to expand on that by building a more advanced application.
My first idea was to have a setup that allows for a more extendible way of creating functions without having to recompile. To that end I have something like:
type IFunction =
member x.Name : string with get
/// additional members omitted
type Expr =
| Num of decimal
| Var of string
///... omitting some types here that don't matter
| FunctionApplication of IFunction * Expr list
So that say a Sin(x) could be represented a:
let sin = { new IFunction() with member x.Name = "SIN" }
let sinExpr = FunctionApplication(sin,Var("x"))
So far all good, but the next idea that I would like to implement is having additional interfaces to represent function of properties. E.g.
type IDifferentiable =
member Derivative : int -> IFunction // Get the derivative w.r.t a variable index
One of the ideas the things I'm trying to achieve here is that I implement some functions and all the logic for them and then move on to the next part of the logic I would like to implement. However, as it currently stands, that means that with every interface I add, I have to revisit all the IFunctions that I've implemented. Instead, I'd rather have a function:
let makeDifferentiable (f : IFunction) (deriv : int -> IFunction) =
{ f with
interface IDifferentiable with
member x.Derivative = deriv }
but as discussed in this question, that is not possible. The alternative that is possible, doesn't meet my extensibility requirement. My question is what alternatives would work well?
[EDIT] I was asked to expand on the "doesn't meet my extenibility requirement" comment. The way this function would work is by doing something like:
let makeDifferentiable (deriv : int -> IFunction) (f : IFunction)=
{ new IFunction with
member x.Name = f.Name
interface IDifferentiable with
member x.Derivative = deriv }
However, ideally I would keep on adding additional interfaces to an object as I add them. So if I now wanted to add an interface that tell whether on function is even:
type IsEven =
abstract member IsEven : bool with get
then I would like to be able to (but not obliged, as in, if I don't make this change everything should still compile) to change my definition of a sine from
let sin = { new IFunction with ... } >> (makeDifferentiable ...)
to
let sin = { new IFunction with ... } >> (makeDifferentiable ...) >> (makeEven false)
The result of which would be that I could create an object that implements the IFunction interface as well as potentially, but not necessarily a lot of different other interfaces as well; the operations I'd then define on them, would potentially be able to optimize what they are doing based on whether or not a certain function implements an interface. This will also allow me to add additional features/interfaces/operations first without having to change the functions I've defined (though they wouldn't take advantage of the additional features, things wouldn't be broken either.[/EDIT]
The only thing I can think of right now is to create a dictionary for each feature that I'd like to implement, with function names as keys and the details to build an interface on the fly, e.g. along the lines:
let derivative (f : IFunction) =
match derivativeDictionary.TryGetValue(f.Name) with
| false, _ -> None
| true, d -> d.Derivative
This would require me to create one such function per feature that I add in addition to one dictionary per feature. Especially if implemented asynchronously with agents, this might be not that slow, but it still feels a little clunky.
I think the problem that you're trying to solve here is what is called The Expression Problem. You're essentially trying to write code that would be extensible in two directions. Discriminated unions and object-oriented model give you one or the other:
Discriminated union makes it easy to add new operations (just write a function with pattern matching), but it is hard to add a new kind of expression (you have to extend the DU and modify all code
that uses it).
Interfaces make it easy to add new kinds of expressions (just implement the interface), but it is hard to add new operations (you have to modify the interface and change all code that creates it.
In general, I don't think it is all that useful to try to come up with solutions that let you do both (they end up being terribly complicated), so my advice is to pick the one that you'll need more often.
Going back to your problem, I'd probably represent the function just as a function name together with the parameters:
type Expr =
| Num of decimal
| Var of string
| Application of string * Expr list
Really - an expression is just this. The fact that you can take derivatives is another part of the problem you're solving. Now, to make the derivative extensible, you can just keep a dictionary of the derivatives:
let derrivatives =
dict [ "sin", (fun [arg] -> Application("cos", [arg]))
... ]
This way, you have an Expr type that really models just what an expression is and you can write differentiation function that will look for the derivatives in the dictionary.
What are the pros and cons of each of the following two ways to create a tree node?
type TreeNode = | TreeNode of int * (TreeNode option) * (TreeNode option) * (TreeNode option)
type Node = | Node of int * Node * Node
| None
They are very close, but not entirely equivalent. You can actually reason about functional types quite nicely using mathematics (see for example my recent article), but you can see that even informally.
Given any TreeNode(num, optLeft, optRight), you can always create a Node containing the same information (None of the option type will be mapped to Node.None).
Interestingly, this does not work the other way round. If someone gives you Node.None, you cannot turn it into TreeNode, because TreeNode needs at least one integer in the root.
So, they are equivalent with the exception that the second one allows empty trees. From the practical perspective, the second type seems easier to use (because you only have to handle empty tree in one place).
So, by a hilarious series of events, I downloaded the FParsec source and tried to build it. Unfortunately, it's not compatible with the new 1.9.9.9. I fixed the easy problems, but there are a couple of discriminated unions that still don't work.
Specifically, Don Syme's post explains that discriminated unions containing items of type obj or -> don't automatically get equality or comparison constraints, since objects don't support comparison and functions don't support equality either. (It's not clear whether the automatically generated equality/comparison was buggy before, but the code won't even compile now that they're no longer generated.)
Here are some examples of the problematic DUs:
type PrecedenceParserOp<'a,'u'> =
| PrefixOp of string * Parser<unit,'u> * int * bool * ('a -> 'a)
| others ...
type ErrorMessage =
| ...
| OtherError of obj
| ...
Here are the offending uses:
member t.RemoveOperator (op: PrecedenceParserOp<'a, 'u>) =
// some code ...
if top.OriginalOp <> op then false // requires equality constraint
// etc etc ...
or, for the comparison constraint
let rec printMessages (pos: Pos) (msgs: ErrorMessage list) ind =
// other code ...
for msg in Set.ofList msgs do // iterate over ordered unique messages
// etc etc ...
As far I can tell, Don's solution of tagging each instance with a unique int is the Right Way to implement a custom equality/comparison constraint (or a maybe a unique int tuple so that individual branches of the DU can be ordered). But this is inconvenient for the user of the DU. Now, construction of the DU requires calling a function to get the next stamp.
Is there some way to hide the tag-getting and present the same constructors to users of the library? That is, to change the implementation without changing the interface? This is especially important because it appears (from what I understand of the code) that PrecedenceParserOp is a public type.
What source did you download for FParsec? I grabbed the latest from the FParsec BitBucket repository, and I didn't have to make any changes at all to the FParsec source to get it to compile in VS 2010 RC.
Edit: I take that back. I did get build errors from the InterpLexYacc and InterpFParsec sample projects, but the core FParsec and FParsecCS projects build just fine.
One thing you could do is add [<CustomEquality>] and [<CustomComparison>] attributes and define your own .Equals override and IComparable implementation. Of course, this would require you to handle the obj and _ -> _ components yourself in an appropriate way, which may or may not be possible. If you can control what's being passed into the OtherError constructor, you ought to be able to make this work for the ErrorMessage type by downcasting the obj to a type which is itself structurally comparable. However, the PrecendenceParserOp case is a bit trickier - you might be able to get by with using reference equality on the function components as long as you don't need comparison as well.