I want to sort items of a class and collect them in Collection-Classes that beside a List-Member also contain further information that are necessary for the sorting process.
The following example is a a very simplified example for my problem. Although it doesn't make sense, I hope it still can help to understand my Question.
type ItemType = Odd|Even //realworld: more than two types possible
type Item(number) =
member this.number = number
member this.Type = if (this.number % 2) = 0 then Even else Odd
type NumberTypeCollection(numberType:ItemType , ?items:List<Item>) =
member this.ItemType = numberType
member val items:List<Item> = defaultArg items List.empty<Item> with get,set
member this.append(item:Item) = this.items <- item::this.items
let addToCollection (collections:List<NumberTypeCollection>) (item:Item) =
let possibleItem =
collections
|> Seq.where (fun c -> c.ItemType = item.Type) //in my realworld code, several groups may be returned
|> Seq.tryFind(fun _ -> true)
match possibleItem with
|Some(f) -> f.append item
collections
|None -> NumberTypeCollection(item.Type, [item]) :: collections
let rec findTypes (collections:List<NumberTypeCollection>) (items:List<Item>) =
match items with
| [] -> collections
| h::t -> let newCollections = ( h|> addToCollection collections)
findTypes newCollections t
let items = [Item(1);Item(2);Item(3);Item(4)]
let finalCollections = findTypes List.empty<NumberTypeCollection> items
I'm unsatisfied with the addToCollection method, since it requires the items in NumberTypeCollection to be mutual. Maybe there are further issues.
What can be a proper functional solution to solve this issue?
Edit: I'm sorry. May code was too simplified. Here is a little more complex example that should hopefully illustrate why I chose the mutual class-member (although this could still be the wrong decision):
open System
type Origin = Afrika|Asia|Australia|Europa|NorthAmerika|SouthAmerica
type Person(income, taxrate, origin:Origin) =
member this.income = income
member this.taxrate = taxrate
member this.origin = origin
type PersonGroup(origin:Origin , ?persons:List<Person>) =
member this.origin = origin
member val persons:List<Person> = defaultArg persons List.empty<Person> with get,set
member this.append(person:Person) = this.persons <- person::this.persons
//just some calculations to group people into some subgroups
let isInGroup (person:Person) (personGroup:PersonGroup) =
let avgIncome =
personGroup.persons
|> Seq.map (fun p -> float(p.income * p.taxrate) / 100.0)
|> Seq.average
Math.Abs ( (avgIncome / float person.income) - 1.0 ) < 0.5
let addToGroup (personGroups:List<PersonGroup>) (person:Person) =
let possibleItem =
personGroups
|> Seq.where (fun p -> p.origin = person.origin)
|> Seq.where (isInGroup person)
|> Seq.tryFind(fun _ -> true)
match possibleItem with
|Some(f) -> f.append person
personGroups
|None -> PersonGroup(person.origin, [person]) :: personGroups
let rec findPersonGroups (persons:List<Person>) (personGroups:List<PersonGroup>) =
match persons with
| [] -> personGroups
| h::t -> let newGroup = ( h|> addToGroup personGroups)
findPersonGroups t newGroup
let persons = [Person(1000,20, Afrika);Person(1300,22,Afrika);Person(500,21,Afrika);Person(400,20,Afrika)]
let c = findPersonGroups persons List.empty<PersonGroup>
What I may need to emphasize: There can be several different groups with the same origin.
Tomas' solution using groupby is the optimal approach if you want to generate your collections only once, it's a simple and concise.
If you want to be able to add/remove items in a functional, referentially transparent style for this type of problem, I suggest you move away from seq and start using Map.
You have a setup which is fundamentally dictionary-like. You have a unique key and a value. The functional F# equivalent to a dictionary is a Map, it is an immutable data structure based on an AVL tree. You can insert, remove and search in O(log n) time. When you append/remove from the Map, the old Map is maintained and you receive a new Map.
Here is your code expressed in this style
type ItemType =
|Odd
|Even
type Item (number) =
member this.Number = number
member this.Type = if (this.Number % 2) = 0 then Even else Odd
type NumTypeCollection = {Items : Map<ItemType, Item list>}
/// Functions on NumTypeCollection
module NumberTypeCollection =
/// Create empty collection
let empty = {Items = Map.empty}
/// Append one item to the collection
let append (item : Item) numTypeCollection =
let key = item.Type
match Map.containsKey key numTypeCollection.Items with
|true ->
let value = numTypeCollection.Items |> Map.find key
let newItems =
numTypeCollection.Items
|> Map.remove key
|> Map.add key (item :: value) // append item
{Items = newItems }
|false -> {Items = numTypeCollection.Items |> Map.add key [item]}
/// Append a list of items to the collections
let appendList (item : Item list) numTypeCollection =
item |> List.fold (fun acc it -> append it acc) numTypeCollection
Then call it using:
let items = [Item(1);Item(2);Item(3);Item(4)]
let finalCollections = NumberTypeCollection.appendList items (NumberTypeCollection.empty)
If I understand your problem correctly, you're trying to group the items by their type. The easiest way to do that is to use the standard library function Seq.groupBy. The following should implement the same logic as your code:
items
|> Seq.groupBy (fun item -> item.Type)
|> Seq.map (fun (key, values) ->
NumberTypeCollection(key, List.ofSeq values))
Maybe there are further issues.
Probably. It's difficult to tell, since it's hard to detect the purpose of the OP code... still:
Why do you even need an Item class? Instead, you could simply have a itemType function:
let itemType i = if i % 2 = 0 then Even else Odd
This function is referentially transparent, which means that you can replace it with its value if you wish. That makes it as good as a property getter method, but now you've already saved yourself from introducing a new type.
Why define a NumberTypeCollection class? Why not a simple record?
type NumberTypeList = { ItemType : ItemType; Numbers : int list }
You can implement addToCollection like something like this:
let addToCollection collections i =
let candidate =
collections
|> Seq.filter (fun c -> c.ItemType = (itemType i))
|> Seq.tryHead
match candidate with
| Some x ->
let x' = { x with Numbers = i :: x.Numbers }
collections |> Seq.filter ((<>) x) |> Seq.append [x']
| None ->
collections |> Seq.append [{ ItemType = (itemType i); Numbers = [i] }]
Being immutable, it doesn't mutate the input collections, but instead returns a new sequence of NumberTypeList.
Also notice the use of Seq.tryHead instead of Seq.tryFind(fun _ -> true).
Still, if you're attempting to group items, then Tomas' suggestion of using Seq.groupBy is more appropriate.
Related
I have a list of history entries that we retrieve from 3rd party software. This list can either be null or non empty.
In C# I would have written it like this, since list can be null:
List<HistoryEntry>
However I'm struggling in writing it in F#. I've already tried:
* Nullable<HistoryEntry list>
* HistoryEntry list option
* HistoryEntry list?
* HistoryEntry list | null
However none of this works. We use a ListConverter which is pretty much used everywhere, and I dare not change it (since that breaks everything):
type ListConverter() =
inherit JsonConverter()
override __.CanConvert(t : Type) = (t.IsGenericType && t.GetGenericTypeDefinition() = typedefof<list<_>>)
override __.WriteJson(writer, value, serializer) =
let list = value :?> System.Collections.IEnumerable |> Seq.cast
serializer.Serialize(writer, list)
override __.ReadJson(reader, t, _, serializer) =
let itemType = t.GetGenericArguments().[0]
let collectionType = typedefof<IEnumerable<_>>.MakeGenericType(itemType)
let collection = serializer.Deserialize(reader, collectionType) :?> IEnumerable<_>
let listType = typedefof<list<_>>.MakeGenericType(itemType)
let cases = FSharpType.GetUnionCases(listType)
let rec make =
function
| [] -> FSharpValue.MakeUnion(cases.[0], [||])
| head :: tail ->
FSharpValue.MakeUnion(cases.[1],
[| head
(make tail) |])
make (collection |> Seq.toList)
My question is: how to create nullable list that will be understood by this serializer?
I actually think the problem is with the ListConverter, not the type of list you're trying to use. The ListConverter does not account for the entire collection being null, which is perfectly possible in JSON. I think the simplest change would be to use a custom version of Seq.toList that checks for nulls and converts them to an empty list.
let toJsonList s =
if s |> box |> isNull
then []
else s |> Seq.toList
Then just change the last line of the ListConverter to be:
make (collection |> toJsonList)
I don't want to use this for loop for iterating the JArray. Is there any other method which can replace this for loop?
let tablesInJson = jsonModel.["tables"] :?> JArray //Converting JOject into JArray
for table in tablesInJson do
let TableName = table.["name"] :?> JValue
let columns = table.["columns"] :?> JArray
for col in columns do
let name = col.["name"] :?> JValue
let types = col.["type"] :?> JValue
let length = col.["length"] :?> JValue
let Result_ = sqlTableInfos
|> List.tryFind (fun s -> s.TableName = TableName.ToString() && s.ColumnName = name.ToString())
if Result_ = Unchecked.defaultof<_> then
printfn "is null"
else
printfn "not null"
If you want to iterate over a collection and perform an imperative operation than using for loop is the idiomatic way of doing this in F# and you should just use that. After all, for is an F# language construct! There is a reason why it exists and the reason is that it lets you easily write code that iterates over a collection and does something for each element!
There are cases where for loop is not a good fit. For example, if you wanted to turn a collection of columns into a new collection with information about the tables. Then you could use Seq.map:
let tableInfos = columns |> Seq.map (fun col ->
let name = col.["name"] :?> JValue
let types = col.["type"] :?> JValue
let length = col.["length"] :?> JValue
let result = sqlTableInfos |> List.tryFind (fun s ->
s.TableName = TableName.ToString() && s.ColumnName = name.ToString())
if result = Unchecked.defaultof<_> then None
else Some result)
This looks like something you might be trying to do - but it is difficult to say. Your question does not say what is the problem that you are actually trying to solve.
Your example with printfn is probably misleading, because if you actually just want to print, then for loop is the best way of doing that.
You can use the Seq module to perform sequence-processing operations over the JArray. In your case, I think I would probably do this for the second for loop (over the columns), but not for the outer loop. The reason being, if you factor the code in the inner-loop out to a function, then you can use pipelining and partial application to clean up the code a bit:
open Newtonsoft.Json
open Newtonsoft.Json.Linq
type SqlTableInfo = {TableName: string; ColumnName: string}
let tablesInJson = JArray()
let sqlTableInfo = []
let tryFindColumn (tableName: JValue) (column: JToken) =
let columnName = column.["name"] |> unbox<JValue>
if sqlTableInfo |> List.exists (fun s -> s.TableName = tableName.ToString() && s.ColumnName = columnName.ToString())
then printfn "Table %A, Column %A Found" tableName columnName
else printfn "Table %A, Column %A Found" tableName columnName
for table in tablesInJson do
let tableName = table.["name"] |> unbox<JValue>
table.["columns"]
|> unbox<JArray>
|> Seq.iter (tryFindColumn tableName)
I have a function that should get two actual params for testing.
Both values shall be created by Arbitrary instances as they need to be of some well formdness that cant be totally arbitrary.
So I create the following code
let updating (x:SomeType) (y:SomeOtherType) =
let result = update x y
result.someProp = x.someProp
&& result.otherProp = y.otherProp
let arbSomeType =
Arb.generate<SomeType>
|> Gen.filter fun x -> x.checkSomeStuff
|> Arb.fromGen
let arbSomeType =
Arb.generate<SomeOtherType>
|> Gen.filter fun x -> x.checkPropertiesOfThis
|> Arb.fromGen
But how do I now combine those 2 Arbitrary instances so that they match up with the signature of test method?
//let prop = Prop.forAll arbSomeType + arbSomeType updating
Check.QuickThrowOnFailure prop
Given two types, SomeTypeA and SomeTypeB:
type SomeTypeA =
{ A : obj }
type SomeTypeB =
{ B : obj }
You can create a Property, where the input is those two types, like so:
let prop =
gen { let! a = Arb.generate<SomeTypeA>
let! b = Arb.generate<SomeTypeB>
return a, b }
|> Arb.fromGen
|> Prop.forAll
<| fun (a, b) ->
// 'a' is SomeTypeA
// 'b' is SomeTypeB
true // Dummy - replace with whatever you want to do with 'a' and 'b'.
You also need to take care, that the signature of the testing method now reflects the created Arbitrary - becoming a (uncurried) function on pairs.
// instead of
let updating (x:SomeType) (y:SomeOtherType) = ...
// do this
let updating (x:SomeType, y:SomeOtherType) = ...
How the example works:
The gen computation expression creates a generator of type Gen<SomeTypeA * SomeTypeB>
An Arbitrary<SomeTypeA * SomeTypeB> instance is created from that generator
Finally, a (QuickCheck/FsCheck) property is created from the arbitrary via Prop.forAll
It's always the same path:
Generator[/optional Shrinker] -> Arbitrary -> Property -> <your_code>
Hope that helps.
I am new to F#. I am trying to use List.fold to help me generate a list of categories and sub-categories based on their Id and ParentId fields. It seems I probably made this code more complex than need be, as I'm getting the stackoverflow error. What am I doing wrong or missing? All related feedback is appreciated.
// types
type CategoryStructure = {
Id: ValidString;
ParentId: ValidString;
Name: ValidString;
Abbreviation: ValidString;
Description: ValidString;
SapId: ValidString;
Section: ValidString;
SectionPosition: ValidString
}
type DynamicCategories = {
Category: CategoryStructure;
SubCategories: seq<DynamicCategories>
}
// this is the function that produces the stack overflow error
let rec private structureCategories (fullList: CategoryStructure list)
(list: CategoryStructure list) =
List.fold (fun acc elem ->
// get all categories and details
let categories = fullList
let mainAcc =
[
for row in categories do
if row = elem
then
let subs =
List.fold (fun acc' elem' ->
if row.Id = elem'.ParentStructureId
then
let foundSubCategory =
{
Category = elem';
SubCategories = structureCategories fullList list |> Seq.ofList
}
foundSubCategory :: acc'
else acc'
) List.empty<DynamicCategories> categories
|> Seq.ofList
yield{
Category = elem;
SubCategories = subs
}
]
mainAcc # acc
) List.empty<DynamicCategories> list
// this function gets the initial parent categories and calls the above function
let getStructuredCategories () =
let categories = allCategoriesAndDetails () |> List.ofSeq
[
for row in categories do
if row.ParentStructureId = NotValid
then yield row
] |> structureCategories categories |> Seq.ofList
You keep calling structureCategories with the same arguments - fullList and list. Since arguments are same, it proceeds to do exactly the same thing as on the previous pass, and ends up calling itself again, with the same arguments. And so on.
This is unbounded recursion ("unbounded" here means "doesn't know when to stop recurring"), and it is also not "tail recursion", so quite naturally, it causes stack overflow.
If you want to turn the flat list into a tree-like structure, you could do a bit simpler than this:
let getChildren fullList parentId = fullList |> List.filter (fun c -> c.ParentId = parentId)
let rec toTree fullList root =
{ Category = root;
SubCategories =
getChildren fullList root.Id
|> List.map (toTree fullList) }
With this, you'll be left with two problems, which I don't know how to solve without knowing more about your requirements:
This will still cause stack overflow if the original list happens to have cycles.
You need to decide who the root(s) of the tree is (or are). Intuitively, this would be indicated via "empty" ParentId, but it is unclear from your data structure what "empty" means.
And finally, this naive solution, while better than your original one, is still a bit slower than it needs to be. It iterates over the whole list once, and for every node does another pass to determine its children, resulting in overall complexity of O(N^2). This may be fine if you expect relatively small list, but not so fine for larger lists. In that case, I would first turn the list into a hashtable (keyed by ParentId) and then use that to find children instead of List.filter.
Thanks to Fyodor, I saw my mistake. He was dead on about calling the same arguments. I added this bit of code right before the foundSubCategory value:
let modifiedList = elem' :: List.empty<CategoryStructure>
and then called that value in the subsequent code:
let foundSubCategory =
{
Category = elem';
SubCategories = structureCategories fullList modifiedList |> Seq.ofList
}
This solved my issue, but now as Fyodor alluded to, I now have to refactor this into something more efficient.
UPDATE
With the insight that Fyodor pointed out this is the current state of my code, which replaces the original code:
let getStructuredCategories () =
let fullList = allCategoriesAndDetails ()
let parentList () =
allCategoriesAndDetails ()
|> Seq.filter (fun p -> p.ParentStructureId = NotValid)
let rec toTree (fullList': seq<CategoryStructure>) (parent: CategoryStructure) =
fullList'
|> Seq.filter (fun x -> x.ParentStructureId = parent.Id)
|> Seq.map (fun x ->
{
Category = x;
SubCategories =
toTree fullList' x
})
seq {
for row in parentList () do
yield {
Category = row;
SubCategories = toTree fullList row
}
}
I got the following error:
Error 2 Value restriction. The value 'gbmLikelihood' has been inferred to have generic type val gbmLikelihood : (float -> '_a -> float [] -> float) when '_a :> seq<float> Either make the arguments to 'gbmLikelihood' explicit or, if you do not intend for it to be generic, add a type annotation.
and this type is exactly what I want. What do I have to do to make it work, and why doesn't it just work without intervention?
EDIT:
The error comes from this file (its short, so I paste the whole lot):
module Likelihood
open System
let likelihood getDrift getVol dt data parameters =
let m = getDrift data parameters
let s = getVol data parameters
let N = float (Seq.length data)
let sqrt_dt = Math.Sqrt dt
let constant = -0.5*Math.Log(2.0*Math.PI*dt)*N
let normalizedResidue observation = (observation - (m - 0.5*s*s)*dt)/(s*sqrt_dt)
let residueSquared observation =
let r = normalizedResidue observation in r*r
let logStdDev = Math.Log s
constant - logStdDev*N - 0.5* (data |> Seq.sumBy residueSquared)
let gbmLikelihood = likelihood (fun data p -> Array.get p 0) (fun datac p -> Array.get p 1)
This error can happen when you declare a value that has a generic type. See for example this past SO question. In your case, the type suggests that you are trying to define a function, but the compiler does not see it as a syntactic function. This can happen if you perform some effects and then return function using the lambda syntax:
let wrong =
printfn "test"
(fun x -> x)
To avoid the problem, you need to write the function using the function syntax:
printfn "test"
let wrong x = x
EDIT: In your concrete example, the function gbmLikelihood is created as a result of a partial function application. To make it compile, you need to turn it into an explicit function:
let gbmLikelihood parameters =
likelihood (fun data p -> Array.get p 0) (fun datac p -> Array.get p 1) parameters
For more information why this is the case & how it works, see also this great article on value restriction in F#.
Instead of making the parameters of gbmLikelihood explicit you might also just add a generic type annotation to the function:
let gbmLikelihood<'a> =
likelihood (fun data p -> Array.get p 0) (fun datac p -> Array.get p 1)