I have a data pipeline where at each step more data fields are required. I would like to do this in a functional way by respecting immutability. I could achieve this with a class by I am wondering if there is an F# way of doing it?
// code that loads initial field information and returns record A
type recordA = {
A: int
}
// code that loads additional field information and returns record AB
type recordAB = {
A: int
B: int
}
// code that loads additional field information and returns record ABC
type recordABC = {
A: int
B: int
C: int
}
As records are sealed I can't just inherit them. How can I avoid having to define a new record with the exact same fields as the previous step and adding the required fields? Preferably I would like to have something like one record that has all required fields and the fields get assigned to their values in each step.
Note that the number of fields added in each step could be more than 1.
I think this can be a good use case for the anonymous records recently introduced in F#.
let a = {| X = 3 |}
let b = {| a with Y = "1"; Z = 4.0|}
let c = {| b with W = 1 |}
printfn "%d, %s, %f, %d" c.X c.Y c.Z c.W
One way to do it in a very FP-style would be to use a DU with a case for each step of the workflow, and the appropriate data for each step in each case:
type WorfklowState =
| StepOne of int
| StepTwo of int * int
| StepThree of int * int * int
Then your entire workflow state, both what step you're currently on and the data produced/consumed by that step, would be modeled in the data type. Of course, you would probably create record types for the data of each case, rather than using progressively larger tuples.
Depending on the application, this may be a (mis-)use case for a dynamic data container.
F# might help by providing user-defined dynamic lookup operators, for which a special syntactic translation occurs.
let (?) (m : Map<_,_>) k = m.Item k
// val ( ? ) : m:Map<'a,'b> -> k:'a -> 'b when 'a : comparison
let (?<-) (m : Map<_,_>) k v = m.Add(k, v)
// val ( ?<- ) : m:Map<'a,'b> -> k:'a -> v:'b -> Map<'a,'b> when 'a : comparison
let m = Map.empty<_,_>
let ma = m?A <- "0"
let mabc = (ma?B <- "1")?C <- "2"
ma?A // val it : string = "0"
mabc?C // val it : string = "2"
You can "inherit" records:
type RecordA =
{
a : int
}
type RecordAB =
{
a : RecordA
b : int
}
type RecordABC =
{
ab : RecordAB
c : int
}
Then you can access all of the elements, though with longer and longer chain as you go deeper and deeper.
However, why don't you just use a list of elements to store the result?
First, I would create a type to handle all possible types that you may have on each step, e.g.:
type Step =
| Int of int
| String of string
// ...
Then you can represent the workflow simply as:
type WorkflowState = list<Step>
and if you want to ensure that you always have at least one element then you can use:
type WorkflowState = Step * list<Step>
However, the records have labels and the structure above does not have them! So, if you do need labels, then you can represent them by a map using either a strong type:
type Label =
| A
| B
// ...
type WorkflowMappedState = Map<Label, Step>
or just a string based one, e.g.
type WorkflowMappedState = Map<string, Step>
The benefits of list or map based approach in comparison to the answers above is that you don't have to know the maximum number of possible steps. What if the number of steps is over 100? Would you want to create a record with 100+ labels? Most likely not! The anonymous records are great, but what if you want to use them outside of module where they were created? I think that that would cause some troubles.
Having said all that, I think that I would go with a list based approach: type WorkflowState = list<Step>. It is very F# way and it is very easy to transform further.
Related
See I have a single case discriminated union
type R = R of string * int * sting option * .....
And I got a value of R.
let r: R = getAValue ()
Now I need to replace the first item of r to an empty string and keep all other value. How to do it? Record type has the with construct
let r' = { r with Item1 = "" }
I know it can use 'pattern match' to extract all the items and create a new one. But it seems very cumbersome.
I assume you do not want to involve reflection, do you?
Then, I believe your only option would be using pattern matching. The (quite limited) burden would be defining the r-ity of your type Ras a pattern for matching.
Let's assume, for example, that your R wraps a tuple of 3 elements, i.e. has r-ity 3:
type R = R of string * int * string option
In this case all you need is to do define the following function:
let modR = function
| R(x,y,z) -> R("",y,z)
The signature of modR is R -> R, a quick check of your scenario:
let r = R("abc",1,None)
modR r
in fsi brings back
>
val it : R = R ("",1,None)
All you would need for applying the above to your specific R is set the actual r-ity of your type into the pattern.
UPDATE: As Fyodor Soikin pointed, a matching function isn't needed at all for unwrapping a single-case DU (see the docs). The sought convertion function definition may be defined as simple as
let modR (R(_,y,z)) = R("",y,z)
UPDATE2: While considering the comment from ca9163d9 I recalled just another flavor of pattern matching, namely as Pattern. Using it while implementing the sought conversion in the form of DU member gives:
type R = R of string * int * string option with
member self.modR() = let R(_,b,c) as x = self in R("",b,c)
Also #FyodorSoikin and #kaefer have pointed out in the comments that as x form isn't required for the simple DU unwrapping, similarly to terser modR function definition above:
member self.modR() = let (R(_,b,c)) = self in R("",b,c)
In OPL (Optimization Programming Language), we have a data structure name tuple. OPL tuple correspond to Record in F#. Here is how it is defined:
tuple Point {
int x;
int y;
};
Like in F#, we can access field by using dot notation
int x = p.x;
We can group tuples in a Set:
{Point} points = {<1,2>, <2,3>};
A difference is that like in database systems, tuple structures can be associated with keys. Tuple keys enable to access data organized in tuples using a set of unique identifiers. In the following example, the nurse tuple is declared with the key name of type string.
tuple nurse {
key string name;
int seniority;
int qualification;
int payRate;
}
{ nurse } nurses = …;
The nice thing about key, is that we can initialize an array this way
int NumberOfChild [n in nurses] = 0;
and accessing a value by using only the key:
NumberOfChild[<"Isabelle">]=20;
leaving out the fields with no keys. This is equivalent to:
NumberOfChild[<"Isabelle",3,1,16>]=20;
Also, using key means that there will be no two tuples with the same key. Like primary key in database.
Question is: Does some type like this exist in F#? Record with key?
My goal: I would like to define a node structure with many attribute. And load a graph structure by only giving the key of the node and not the entire Record since I would load the graph from a database.
type Node = {
nodeKey : int;
nodeName : string;
nodeAttribute1 : string;
nodeAttribute2 : string }
let Graph = [
(1, 2);
(1, 3);
(2, 4);
(3, 4) ]
Where the int in the graph tuple represent nodeKey.
I would like to do operation using the graph but accessing the node information using the key only.
OPL Grammar
No, there's no such language-level concept. All record fields are created equal, so to speak.
That doesn't preclude you from:
synthesizing a key for a record based on one or more field values,
using such a key as a key in a Map that would hold your records or any additional values.
So you can have something like this:
type Nurse = { name: string; seniority: int; qualification: int; payRate: int }
let nurses = [ { name = "Isabelle"; seniority = 3; qualification = 1; payRate = 16 } ]
let numberOfChildren =
[ "Isabelle", 20 ]
|> Map.ofSeq
let nursesWithNumberOfChildren =
[ for nurse in nurses do
match numberOfChildren |> Map.tryFind nurse.name with
| Some children -> yield nurse, children
| None -> yield nurse, 0 ]
Using similar approach you can separate your graph and node data - store only keys in the graph and maintain a mapping that goes from keys to full node records.
//If I read data from a database, I would receive the data in the following form:
type XYZ = {X:int;
Y:string;
Z:float}
let recordsXYZ = [{X=1;Y="A";Z=1.0};{X=2;Y="b";Z=1.0};{X=3;Y="A";Z=1.0}]
//I can create a map this way
let mapXYZ1=recordsXYZ|>Seq.groupBy (fun a ->a.X)|>Map.ofSeq
//But I don't want a Map<int,seq<XYZ>>
//This is what I want
let mapXYZ2=recordsXYZ|>Seq.map (fun a -> (a.X,{X=a.X;Y=a.Y;Z=a.Z}))|>Map.ofSeq
//Or maybe this is cleaner but this need to define another type
type YZ = {Y:string;
Z:float}
let mapXYZ3=recordsXYZ|>Seq.map (fun a -> (a.X,{Y=a.Y;Z=a.Z}))|>Map.ofSeq
If I understand correctly, your best bet is simply a cleaner alternative to Seq.groupBy for your purposes. Here is the core of it, in one line:
let inline project projection value = projection value, value
Given a simple helper function, not specific to XYZ
let projectToMap projection values = values |> Seq.map (project projection) |> Map.ofSeq
it becomes trivial to cleanly create maps of XYZ from any "key":
let mappedByX = xyzs |> projectToMap (fun { X=x } -> x) // Map<int, XYZ>
let mappedByY = xyzs |> projectToMap (fun { Y=y } -> y) // Map<string, XYZ>
let mappedByZY = xyzs |> projectToMap (fun { Y=y; Z=z } -> z, y) // Map<float*string, XYZ>
Online Demo
I need a data structure for the following:
In a device that has memory slots, each of the slots has a set of parameters. These parameters have different types. The list of possible parameters is fixed, so there is no need for generic flexibility à la »Support of arbitrary parameters without change«. Also, for each parameter, the structure of the contents is known. Typical use cases are the retrieval and modification of one specific parameter as well as a transformation of the complete parameter set into a different (but already defined) data structure.
The natural choice of F# data structure would be a sum type like this:
type SomeParameterContentType = { Field1 : string, Field2 : int }
type SomeOtherParameterContentType = { Other1 : bool option, Other2 : double }
type Parameter =
| SomeParameter of SomeParameterContentType
| SomeOtherParameter of SomeOtherParameterContentType
This way I could create a set and store the parameters there with a very nice data structure. The question here is: Given this idea, how would looking for a specific parameter look like? I don't know of any way to specify a predicate for a find-function for sets. It would be possible to define another sum type listing just the Parameter Types without their contents using this as key for a Dictionary but I don't like this idea too much. Using strings instead of the second sum type doesn't make things better as it still would require providing the list of possible parameters twice.
Does anyone have a better idea?
Thx
--Mathias.
Sounds like all you want is a tryFind for a Set:
module Set =
let tryFind p =
Set.toList >> List.tryFind p
Usage:
let mySet = Set.ofList [1;2;3;4;5]
let m = mySet |> Set.tryFind (fun t -> t = 2)
val m : int option = Some 2
Usage with your Types:
let yourSet = Set.ofList [SomeParameter {Field1="hello";Field2=3}]
let mYours = yourSet |> Set.tryFind (fun t -> match t with
|SomeParameter p -> true
|SomeOtherParameter p -> false)
val mYours : Parameter option = Some (SomeParameter {Field1 = "hello";
Field2 = 3;})
The msdn page documenting Records (F#) details record expressions for record construction and record patterns for deconstruction, the latter without naming them as such.
Here's an example which uses both techniques for an arithmetic operator:
// Simple two-dimensional generic vector defintion
type 'a UV =
{ U : 'a; V : 'a }
static member inline (+) ({ U = au; V = av }, { U = bu; V = bv }) =
{ U = au + bu; V = av + bv }
This appears unwieldy and not very readable. For deconstruction, there are dot-notation or functions as alternatives. Since the dot-notation operator has a special dispensation in section 8.4.2 Name Resolution and Record Field Labels of the spec (an expression’s type may be inferred from a record label), there's normally no need to annotate. Accessor functions like let u { U = u } = u wouldn't give us any advantages then.
For construction, I think a case can be made for a function as record constructor. Access to the original constructor might even be restricted:
type 'a UV =
internal { U : 'a; V : 'a }
let uv u v = { U = u; V = v }
type 'a UV with
static member inline (+) (a, b) =
uv (a.U + b.U) (a.V + b.V)
Is this an idiomatic thing to do? How to package such functions in modules and handle namespace issues?
Short answer: I don't think there is a general convention here at the moment so it will be a personal decision in the end.
To summarise what you get for free with records in F# is:
Construct: { U = u; V = v } (bracket-notation)
Deconstruct: let u = record.u (dot-notation) and let {U = u} = record (pattern matching)
Update: {record with U = u} (bracket-notation)
But you don't get first class functions for free, if you want you can code them by hand.
The following is what I would personally use as convention:
A static member New with curried arguments for record construction.
For update and deconstruction I would use some kind of Lenses abstraction.
Here's an example of the code I would have to add by hand:
// Somewhere as a top level definition or in a base library
type Lens<'T,'U> = {Get: 'T -> 'U; Set: 'U -> 'T -> 'T } with
member l.Update f a = l.Set (f (l.Get a)) a
type UV<'a> = {U : 'a; V : 'a } with
// add these static members to your records
static member New u v : UV<'a> = {U = u; V = v}
static member u = {Get = (fun (x: UV<'a>) -> x.U); Set = fun t x -> {x with U = t}}
static member v = {Get = (fun (x: UV<'a>) -> x.V); Set = fun t x -> {x with V = t}}
let uvRecord = UV.New 10 20
let u = UV.u.Get uvRecord
let uvRecord1 = UV.u.Set (u+1) uvRecord
let uvRecord2 = UV.u.Update ((+)1) uvRecord
This way I would have first class functions for construction, deconstruction but also for updates plus other very interesting Lenses properties as you can read in this post.
UPDATE (in response to your comments)
Of course they can be defined later, what does it change?
The same applies for the New constructor, it can be defined later but that's actually a good thing.
The accessor functions you defined can also be defined later, indeed any first-class getter, setter or updater value can be defined later.
Anyway the answer to your question is "no, there are no conventions" the rest it's a personal decision, which would be my decision and also many Haskellers are pushing to get some kind of automatic Lenses for Haskell records.
Why would I decide to go this way? Because in terms of lines of code the effort of adding a simple accessor function is almost the same as adding a get-Lens, so for the same price I get more functionality.
If you are not happy with the Lenses discussion please tell me, I can delete it and leave the short answer, or I can delete the whole answer too if it's confusing instead of clarifying.
Or may be I misunderstood your question, for me your question was about which convention is generally used to add first-class constructors, getters and setters values for records.
Composition is not the only advantage of Lenses, you can do many things, keep reading about them, they provide a very interesting abstraction and not only restricted to records.
I am writing a compiler of mini-pascal in Ocaml. I would like my compiler to accept the following code for instance:
program test;
var
a,b : boolean;
n : integer;
begin
...
end.
I have difficulties in dealing with the declaration of variables (the part following var). At the moment, the type of variables is defined like this in sib_syntax.ml:
type s_var =
{ s_var_name: string;
s_var_type: s_type;
s_var_uniqueId: s_uniqueId (* key *) }
Where s_var_uniqueId (instead of s_var_name) is the unique key of the variables. My first question is, where and how I could implement the mechanism of generating a new id (actually by increasing the biggest id by 1) every time I have got a new variable. I am wondering if I should implement it in sib_parser.mly, which probably involves a static variable cur_id and the modification of the part of binding, again don't know how to realize them in .mly. Or should I implement the mechanism at the next stage - the interpreter.ml? but in this case, the question is how to make the .mly consistent with the type s_var, what s_var_uniqueId should I provide in the part of binding?
Another question is about this part of statement in .mly:
id = IDENT COLONEQ e = expression
{ Sc_assign (Sle_var {s_var_name = id; s_var_type = St_void}, e) }
Here, I also need to provide the next level (the interpreter.ml) a variable of which I only know the s_var_name, so what could I do regarding its s_var_type and s_var_uniqueId here?
Could anyone help? Thank you very much!
The first question to ask yourself is whether you actually need an unique id. From my experience, they're almost never necessary or even useful. If what you're trying to do is making variables unique through alpha-equivalence, then this should happen after parsing is complete, and will probably involve some form of DeBruijn indices instead of unique identifiers.
Either way, a function which returns a new integer identifier every time it is called is:
let unique =
let last = ref 0 in
fun () -> incr last ; !last
let one = unique () (* 1 *)
let two = unique () (* 2 *)
So, you can simply assign { ... ; s_var_uniqueId = unique () } in your Menhir rules.
The more important problem you're trying to solve here is that of variable binding. Variable x is defined in one location and used in another, and you need to determine that it happens to be the same variable in both places. There are many ways of doing this, one of them being to delay the binding until the interpreter. I'm going to show you how to deal with this during parsing.
First, I'm going to define a context: it's a set of variables that allows you to easily retrieve a variable based on its name. You might want to create it with hash tables or maps, but to keep things simple I will be using List.assoc here.
type s_context = {
s_ctx_parent : s_context option ;
s_ctx_bindings : (string * (int * s_type)) list ;
s_ctx_size : int ;
}
let empty_context parent = {
s_ctx_parent = parent ;
s_ctx_bindings = [] ;
s_ctx_size = 0
}
let bind v_name v_type ctx =
try let _ = List.assoc ctx.s_ctx_bindings v_name in
failwith "Variable is already defined"
with Not_found ->
{ ctx with
s_ctx_bindings = (v_name, (ctx.s_ctx_size, v_type))
:: ctx.s_ctx_bindings ;
s_ctx_size = ctx.s_ctx_size + 1 }
let rec find v_name ctx =
try 0, List.assoc ctx.s_ctx_bindings v_name
with Not_found ->
match ctx.s_ctx_parent with
| Some parent -> let depth, found = find v_name parent in
depth + 1, found
| None -> failwith "Variable is not defined"
So, bind adds a new variable to the current context, find looks for a variable in the current context and its parents, and returns both the bound data and the depth at which it was found. So, you could have all global variables in one context, then all parameters of a function in another context that has the global context as its parent, then all local variables in a function (when you'll have them) in a third context that has the function's main context as the parent, and so on.
So, for instance, find 'x' ctx will return something like 0, (3, St_int) where 0 is the DeBruijn index of the variable, 3 is the position of the variable in the context identified by the DeBruijn index, and St_int is the type.
type s_var = {
s_var_deBruijn: int;
s_var_type: s_type;
s_var_pos: int
}
let find v_name ctx =
let deBruijn, (pos, typ) = find v_name ctx in
{ s_var_deBruijn = deBruijn ;
s_var_type = typ ;
s_var_pos = pos }
Of course, you need your functions to store their context, and make sure that the first argument is the variable at position 0 within the context:
type s_fun =
{ s_fun_name: string;
s_fun_type: s_type;
s_fun_params: context;
s_fun_body: s_block; }
let context_of_paramlist parent paramlist =
List.fold_left
(fun ctx (v_name,v_type) -> bind v_name v_type ctx)
(empty_context parent)
paramlist
Then, you can change your parser to take into account the context. The trick is that instead of returning an object representing part of your AST, most of your rules will return a function that takes a context as an argument and returns an AST node.
For instance:
int_expression:
(* Constant : ignore the context *)
| c = INT { fun _ -> Se_const (Sc_int c) }
(* Variable : look for the variable inside the contex *)
| id = IDENT { fun ctx -> Se_var (find id ctx) }
(* Subexpressions : pass the context to both *)
| e1 = int_expression o = operator e2 = int_expression
{ fun ctx -> Se_binary (o, e1 ctx, e2 ctx) }
;
So, you simply propagate the context "down" recursively through the expressions. The only clever parts are those when new contexts are created (you don't have this syntax yet, so I'm just adding a placeholder):
| function_definition_expression (args, body)
{ fun ctx -> let ctx = context_of_paramlist (Some ctx) args in
{ s_fun_params = ctx ;
s_fun_body = body ctx } }
As well as the global context (the program rule itself does not return a function, but the block rule does, and so a context is created from the globals and provided).
prog:
PROGRAM IDENT SEMICOLON
globals = variables
main = block
DOT
{ let ctx = context_of_paramlist None globals in
{ globals = ctx;
main = main ctx } }
All of this makes the implementation of your interpreter much easier due to the DeBruijn indices: you can have a "stack" which holds your values (of type value) defined as:
type stack = value array list
Then, reading and writing variable x is as simple as:
let read stack x =
(List.nth stack x.s_var_deBruijn).(x.s_var_pos)
let write stack x value =
(List.nth stack x.s_var_deBruijn).(x.s_var_pos) <- value
Also, since we made sure that function parameters are in the same order as their position in the function context, if you want to call function f and its arguments are stored in the array args, then constructing the stack is as simple as:
let inner_stack = args :: stack in
(* Evaluate f.s_fun_body with inner_stack here *)
But I'm sure you'll have a lot more questions to ask when you start working on your interpeter ;)
How to create a global id generator:
let unique =
let counter = ref (-1) in
fun () -> incr counter; !counter
Test:
# unique ();;
- : int = 0
# unique ();;
- : int = 1
Regarding your more general design question: it seems that your data representation does not faithfully represent the compiler phases. If you must return a type-aware data-type (with this field s_var_type) after the parsing phase, something is wrong. You have two choices:
devise a more precise data representation for the post-parsing AST, that would be different from the post-typing AST, and not have those s_var_type fields. Typing would then be a conversion from the untyped to the typed AST. This is a clean solution that I would recommend.
admit that you must break the data representation semantics because you don't have enough information at this stage, and try to be at peace with the idea of returning garbage such as St_void after the parsing phase, to reconstruct the correct information later. This is less typed (as you have an implicit assumption on your data which is not apparent in the type), more pragmatic, ugly but sometimes necessary. I don't think it's the right decision in this case, but you will encounter situation where it's better to be a bit less typed.
I think the specific choice of unique id handling design depends on your position on this more general question, and your concrete decisions about types. If you choose a finer-typed representation of post-parsing AST, it's your choice to decide whether to include unique ids or not (I would, because generating a unique ID is dead simple and doesn't need a separate pass, and I would rather slightly complexify the grammar productions than the typing phase). If you choose to hack the type field with a dummy value, it's also reasonable to do that for variable ids if you wish to, putting 0 as a dummy value and defining it later; but still I personally would do that in the parsing phase.