Pipelining functions with different parameters in F# - f#

I have the following code:
type CustomerStatus = |Valid |Invalid
type Customer = {Id:int; FirstName: string; LastName:string; Status: CustomerStatus}
let checkCustomerFirstName (customers: Customer seq) =
customers
let checkCustomerLastName (customers: Customer seq) =
customers
let checkCustomerInBlockedList (customers: Customer seq) (blockedList: string seq) =
customers
The Customer record type has 4 attributes: Id, FirstName, LastName, and Status. CustomerStatus is defined in the line above as a DU with Valid and Invalid. I then have three functions to validate a sequence of customers, I omitted the function body for the sake of brevity. Notice the 3rd function takes in a second parameter - a list of strings that the customer's name is compared to see if they are invalid.
Here is some test data:
let customers =
[
{Id=0;LastName="Customer0";FirstName="Test";CustomerClass=Valid}
{Id=1;LastName="Customer1";FirstName="Test";CustomerClass=Valid}
{Id=2;LastName="Customer2";FirstName="Test";CustomerClass=Valid}
{Id=3;LastName="Customer3";FirstName="Test";CustomerClass=Valid}
{Id=4;LastName="Customer4";FirstName="Test";CustomerClass=Valid}
]
let blockedList = ["Customer3"]
When I pipeline the first two functions like this:
customers
|> checkCustomerFirstName
|> checkCustomerLastName
Everything works as expected.
However, when I add in the 3rd function,
customers
|> checkCustomerFirstName
|> checkCustomerLastName
|> checkCustomerInBlockedList blockedList
I get a error like this
How do I pipeline functions that have additional parameters?

You can try changing the order of your last method parameters.
let checkCustomerInBlockedList (blockedList: string seq) (customers: Customer seq) =
customers
Forward pipe operator sends the parameter that you have on left side to the function on the right side. In case of your last call you have or the left side a sequence of Customers, but on right side you have a method with parameter, so your sequence will be fed as last parameter.
Your last statement looks like checkCustomerInBlockedList blockedList customers that's why changing the order of parameters will make compiler happy.

Related

Merging data from different sources with different types

I have four data sources that are related by a common ID (incrementing integer).
One data source P is always available and the three others L's may or may not be available for a given ID (i.e. ID's are not guaranteed to exist) and are received over the internet and are distinguishable by their IP address.
I gather the data sources over some interval and then I would like to join them using their ID's.
My mental model is something like this
The data from source P in the interval P: List<'T>
The data from all L sources: L: List<Dictionary<string, 'U>>
Get the ID from source P a: 'T -> int
Get the ID from L source b: 'U -> int
The resulting type where data is combined 'V = {Id: int; P: 'T; L: Dictionary<string, 'U> }
The result R: List<'V>
Before I run amok using hash sets and whatnot, it would be nice to get some ideas on how to do this. Maybe there are some cool F# features that makes this really easy.
I think you can translate your domain model into types in F# a bit more directly.
One data source P is always available and the three others L's may or
may not be available for a given ID (i.e. ID's are not guaranteed to
exist) and are received over the internet and are distinguishable by
their IP address.
This describes a record type in F# pretty neatly.
type MyDataSources =
{Id: int;
P : MyDataSource;
L1: MyOtherDataSource option;
L2: MyOtherDataSource option;
L3: MyOtherDataSource option;}
I gather the data sources over some interval and then I would like to
join them using their ID's.
So, we need another record type for the data.
type JoinedData<'T, 'U> =
{PData : 'T list;
L1Data : 'U list option;
L2Data : 'U list option;
L3Data : 'U list option;}
We then just need a function that takes the data sources and populates the data.
let populateDataFromSources dataSources =
{PData = getDataFrom dataSources.P // replace with whatever logic you want here
....
}
If needed, your getDataFrom function could take the Id from your data source as an argument and filter for data returned which relates to that Id (for example).

Extending Query Expressions

Are there any documents or examples out there on how one can extend/add new keywords to query expressions? Is this even possible?
For example, I'd like to add a lead/lag operator.
In addition to the query builder for the Rx Framework mentioned by #pad, there is also a talk by Wonseok Chae from the F# team about Computation Expressions that includes query expressions. I'm not sure if the meeting was recorded, but there are very detailed slides with a cool example on query syntax for generating .NET IL code.
The source code of the standard F# query builder is probably the best resource for finding out what types of operations are supported and how to annotate them with attributes.
The key attributes that you'll probably need are demonstrated by the where clause:
[<CustomOperation("where",MaintainsVariableSpace=true,AllowIntoPattern=true)>]
member Where :
: source:QuerySource<'T,'Q> *
[<ProjectionParameter>] predicate:('T -> bool) -> QuerySource<'T,'Q>
The CustomOperation attribute defines the name of the operation. The (quite important) parameter MaintainsVariableSpace allows you to say that the operation returns the same type of values as it takes as the input. In that case, the variables defined earlier are still available after the operation. For example:
query { for p in db.Products do
let name = p.ProductName
where (p.UnitPrice.Value > 100.0M)
select name }
Here, the variables p and name are still accessible after where because where only filters the input, but it does not transform the values in the list.
Finally, the ProjectionParameter allows you to say that p.UnitValue > 100.0M should actually be turned into a function that takes the context (available variables) and evaluates this expression. If you do not specify this attribute, then the operation just gets the value of the argument as in:
query { for p in .. do
take 10 }
Here, the argument 10 is just a simple expression that cannot use values in p.
Pretty cool feature for the language. Just implemented the reverse to query QuerySource.
Simple example, but just a demonstration.
module QueryExtensions
type ExtendedQueryBuilder() =
inherit Linq.QueryBuilder()
/// Defines an operation 'reverse' that reverses the sequence
[<CustomOperation("reverse", MaintainsVariableSpace = true)>]
member __.Reverse (source : Linq.QuerySource<'T,System.Collections.IEnumerable>) =
let reversed = source.Source |> List.ofSeq |> List.rev
new Linq.QuerySource<'T,System.Collections.IEnumerable>(reversed)
let query = ExtendedQueryBuilder()
And now it being used.
let a = [1 .. 100]
let specialReverse =
query {
for i in a do
select i
reverse
}

F# Type providers printing

I'm currently trying out the SqlDataConnection type provider, and was wondering how to display these types.
Is there some way I can have my calls to printfn "%A" display something more meaningful than the type name?
I do not think there is a way to intercept the behaviour of printfn "%A" for existing types generated by a type provider (that you cannot modify). If you could modify the type provider, then you could change it to generate the StructuredFormatDisplay attribute for the generated types, but that's not possible for SqlDataConnection.
If you're using it in F# Interactive, then you can use fsi.AddPrintTransformer to define how individual values are printed when they are a result of some computation. For example:
// Using Northwind database as a sample
type DB = SqlDataConnection<"Data Source=.\\SQLExpress;Initial Catalog=Northwind;...">
let db = DB.GetDataContext()
// A simple formatter that creates a list with property names and values
let formatAny (o:obj) =
[ for p in o.GetType().GetProperties() ->
p.Name, p.GetValue(o) ]
// Print all Northwind products using the formatter
fsi.AddPrintTransformer(fun (p:DB.ServiceTypes.Products) ->
formatAny p |> box)
// Take the first product - will be printed using custom formatter
query { for p in db.Products do head }
The specified PrintTransformer is only used when you get the value as a result in F# interactive. It will also work when you write query { .. } |> List.ofSeq for queries that return multiple objects. But for printfn "%A", you'll have to call the conversion function (like formatAny) explicitly...

How to extract data from F# list

Following up my previous question, I'm slowly getting the hang of FParsec (though I do find it particularly hard to grok).
My next newbie F# question is, how do I extract data from the list the parser creates?
For example, I loaded the sample code from the previous question into a module called Parser.fs, and added a very simple unit test in a separate module (with the appropriate references). I'm using XUnit:
open Xunit
[<Fact>]
let Parse_1_ShouldReturnListContaining1 () =
let interim = Parser.parse("1")
Assert.False(List.isEmpty(interim))
let head = interim.Head // I realise that I have only one item in the list this time
Assert.Equal("1", ???)
Interactively, when I execute parse "1" the response is:
val it : Element list = [Number "1"]
and by tweaking the list of valid operators, I can run parse "1+1" to get:
val it : Element list = [Number "1"; Operator "+"; Number "1"]
What do I need to put in place of my ??? in the snippet above? And how do I check that it is a Number, rather than an Operator, etc.?
F# types (including lists) implement structural equality. This means that if you compare two lists that contain some F# types using =, it will return true when the types have the same length and contain elements with the same properties.
Assuming that the Element type is a discriminated union defined in F# (and is not an object type), you should be able to write just:
Assert.Equal(interim, [Number "1"; Operator "+"; Number "1"])
If you wanted to implement the equality yourself, then you could use pattern matching;
let expected = [Number "1"]
match interim, expected with
| Number a, Number b when a = b -> true
| _ -> false

Best way to determine variable type and treat each one differently in F#

I have a function that will create a select where clause, but right now everything has to be a string.
I would like to look at the variable passed in and determine what type it is and then treat it properly.
For example, numeric values don't have single quotes around them, option type will either be null or have some value and boolean will actually be zero or one.
member self.BuildSelectWhereQuery (oldUser:'a) = //'
let properties = List.zip oldUser.ToSqlValuesList sqlColumnList
let init = false, new StringBuilder()
let anyChange, (formatted:StringBuilder) =
properties |> Seq.fold (fun (anyChange, sb) (oldVal, name) ->
match(anyChange) with
| true -> true, sb.AppendFormat(" AND {0} = '{1}'", name, oldVal)
| _ -> true, sb.AppendFormat("{0} = '{1}'", name, oldVal)
) init
formatted.ToString()
Here is one entity:
type CityType() =
inherit BaseType()
let mutable name = ""
let mutable stateId = 0
member this.Name with get() = name and set restnameval=name <- restnameval
member this.StateId with get() = stateId and set stateidval=stateId <- stateidval
override this.ToSqlValuesList = [this.Name; this.StateId.ToString()]
So, if name was some other value besides a string, or stateId can be optional, then I have two changes to make:
How do I modify ToSqlValuesList to
have the variable so I can tell the
variable type?
How do I change my select function
to handle this?
I am thinking that I need a new function does the processing, but what is the best FP way to do this, rather than using something like typeof?
You can use a type test pattern in a match. Would this meet your needs?
let f (x : obj) =
match x with
| :? int -> "int"
| :? string -> "string"
| :? bool -> "bool"
| _ -> "who knows?"
I think that one clear functional approach would be to define a data type that represents the various (more complicated situations) that you need to handle. You mentioned that a value may be optional and that you need to distinguish numeric and textual values (for the encoding to SQL).
You could define a discriminated union (if there are other cases that you'd like to handle, the definition may be a bit more complicated):
type SqlValue =
| Missing
| Numeric of string
| Textual of string
Note that the Textual case also carries string, because I assume that the client who produces the value takes care of converting it to string - this is only information for your SQL query generator (so that it knows whether it needs to add quotes).
Your ToSqlValuesList member would return a list of values string & SqlValue, so for example, a sample product could be represented using the following list:
columns = [ "Name"; "Price"; "Description" ]
values = [ Textual("Tea"); Numeric(10); Missing ]
In the code that generates the SQL query, you'd use pattern matching to handle all the different cases (most importantly, encode string to avoid SQL injection in case the value is Textual :-)).
EDIT You'd need to implement the conversion from the specific data types to the SqlValue representation in every client. However, this can be simplified by writing a utility type (using the fact that members can be overloaded):
type SqlValue with
static member From(a:int) = Numeric(a.ToString())
static member From(a:int option) =
match a with None -> Missing | Some(n) -> SqlValue.From(n)
// ... similarly for other types
In the implementation of ToSqlValuesList, you would write SqlValue.From(description) and it would deal with the details autoamtically.
A more sophisticated approach would be to annotate public members of the types representing your data entities with .NET attributes and use Reflection to extract the values (and their types) at runtime. This is more advanced, but quite elegant (there is a nice exmaple of this technique in Don Syme's Expert F# book)

Resources