Let's say there is a provider like this:
type ColorProvider = JsonProvider<"""
{
"id": "b35b5bcf-761a-4e50-9ff0-4c7de7dd0e5d",
"color": "Red"
}
""">
Trying to print colors from a collection will fail if one of these object doesn't have the color property at all:
dataAccess.QueryAsEnumerable<string>("SELECT Data FROM Objects")
|> Seq.map ColorProvider.Parse
|> Seq.iter (fun item -> printfn "%A" item.Color)
There is a JsonValue.Null to compare to but in this case it's not null, the property is just missing.
How to filter out items without the color property?
Your solution with TryGetProperty works, but there is much nicer way - you can use a more representative sample with two records where the color property is missing for one of them:
type ColorProvider = JsonProvider<"""[
{ "id": "b35b5bcf", "color": "Red" },
{ "id": "b2542345" } ]""", SampleIsList=true>
Then, the Color property is inferred as option<string> and you can handle it nicely using either pattern matching on options, or using defaultArg:
dataAccess.QueryAsEnumerable<string>("SELECT Data FROM Objects")
|> Seq.map ColorProvider.Parse
|> Seq.iter (fun item -> printfn "%s" (defaultArg item.Color " - "))
Okay, found it here:
dataAccess.QueryAsEnumerable<string>("SELECT Data FROM Objects")
|> Seq.map ColorProvider.Parse
|> Seq.iter (fun item ->
match item.JsonValue.TryGetProperty("color") with
| Some color -> printfn "%A" color
| None -> printfn "%s" " - "
)
Related
Edit:
it is solved. The answer is:
|> Seq.sortBy (fun (KeyValue(k,v)) -> k)
|> Seq.map (fun (KeyValue(k,v)) -> (sprintf "%s:%s" k v))
|> String.concat ","
Original question:
I'm trying to do the F# version of this:
var postData = "{"
+ string.Join(
",",
sortedParameters.Select(item => $"\"{item.Key}\":\"{item.Value}\"").ToList())
+ "}";
I have the following dictionary, coming from C#:
{
{ "a", "1" },
{ "c", "3" },
{ "b", "2" },
}
I would like to sort it, then output all the values in a string like this:
"a":"1", "b":"2", "c":"3"
so, I am starting like this:
let Test (values : Dictionary<string, string>) : string =
|> Seq.sortBy (fun (KeyValue(k,v)) -> k)
|> Seq.iter (fun (KeyValue(k,v)) -> (sprintf "%s:%s" k v))
but the last part doesn't compile:
Compilation error (line 13, col 50): Type mismatch. Expecting a string -> string -> unit but given a string -> string -> string The type 'unit' does not match the type 'string'
Compilation error (line 13, col 60): This expression was expected to have type string but here has type int
I do not understand the error message. My assumption is that this would return me a list of strings like:
{
"a:1", "b:2", "c:3"
}
that I would have to join in a string, in the next step
obviously, it's not done like that :)
what did I miss?
let showDict (x : Dictionary<string, string>) : string =
x
|> Seq.sortBy (fun (KeyValue (k, v)) -> k)
|> Seq.map (fun (KeyValue (k, v)) -> $"%s{k}:%s{v}"))
|> String.concat ", "
I have the following function that convert csv files to a specific txt schema (expected by CNTKTextFormat Reader):
open System.IO
open FSharp.Data;
open Deedle;
let convert (inFileName : string) =
let data = Frame.ReadCsv(inFileName)
let outFileName = inFileName.Substring(0, (inFileName.Length - 4)) + ".txt"
use outFile = new StreamWriter(outFileName, false)
data.Rows.Observations
|> Seq.map(fun kvp ->
let row = kvp.Value |> Series.observations |> Seq.map(fun (k,v) -> v) |> Seq.toList
match row with
| label::data ->
let body = data |> List.map string |> String.concat " "
outFile.WriteLine(sprintf "|labels %A |features %s" label body)
printf "%A" label
| _ ->
failwith "Bad data."
)
|> ignore
Strangely, the output file is empty after running in the F# interactive panel and that printf yields no printing at all.
If I remove the ignore to make sure that there are actual rows being processed (evidenced by returning a seq of nulls), instead of an empty file I get:
val it : seq<unit> = Error: Cannot write to a closed TextWriter.
Before, I was declaring the StreamWriter using let and disposing it manually, but I also generated empty files or just a few lines (say 5 out of thousands).
What is happening here? Also, how to fix the file writing?
Seq.map returns a lazy sequence which is not evaluated until it is iterated over. You are not currently iterating over it within convert so no rows are processed. If you return a Seq<unit> and iterate over it outside convert, outFile will already be closed which is why you see the exception.
You should use Seq.iter instead:
data.Rows.Observations
|> Seq.iter (fun kvp -> ...)
Apart from the solutions already mentioned, you could also avoid the StreamWriter altogether, and use one of the standard .Net functions, File.WriteAllLines. You would prepare a sequence of converted lines, and then write that to the file:
let convert (inFileName : string) =
let lines =
Frame.ReadCsv(inFileName).Rows.Observations
|> Seq.map(fun kvp ->
let row = kvp.Value |> Series.observations |> Seq.map snd |> Seq.toList
match row with
| label::data ->
let body = data |> List.map string |> String.concat " "
printf "%A" label
sprintf "|labels %A |features %s" label body
| _ ->
failwith "Bad data."
)
let outFileName = inFileName.Substring(0, (inFileName.Length - 4)) + ".txt"
File.WriteAllLines(outFileName, lines)
Update based on the discussion in the comments: Here's a solution that avoids Deedle altogether. I'm making some assumptions about your input file format here, based on another question you posted today: Label is in column 1, features follow.
let lines =
File.ReadLines inFileName
|> Seq.map (fun line ->
match Seq.toList(line.Split ',') with
| label::data ->
let body = data |> List.map string |> String.concat " "
printf "%A" label
sprintf "|labels %A |features %s" label body
| _ ->
failwith "Bad data."
)
As Lee already mentioned, Seq.map is lazy. And that's also why you were getting "Cannot write to a closed TextWriter": the use keyword disposes of its IDisposable when it goes out of scope. In this case, that's at the end of your function. Since Seq.map is lazy, your function was returning an unevaluated sequence object, which had closed over the StreamWriter in your use statement -- but by the time you evaluated that sequence (in whatever part of your code checked for the Seq of nulls, or in the F# Interactive window), the StreamWriter had already been disposed by going out of scope.
Change Seq.map to Seq.iter and both of your problems will be solved.
I have a map reduce code for which I group in each of the threads by some key and then in the reduce part merge the results. My current approach is to search for an specific key index in the accumulator and then mapi to retrieve the combined result only for this key, leaving the rest unmodified:
let rec groupFolder sequence acc =
match sequence with
| (by:string, what) :: rest ->
let index = acc |> Seq.tryFindIndex( fun (byInAcc, _) -> byInAcc.Equals(by) )
match index with
| Some (idx) ->
acc |> Seq.mapi( fun i (byInAcc, whatInAcc) -> if i = idx then (by, (what |> Array.append whatInAcc) ) else byInAcc, whatInAcc )
|> groupFolder rest
| None -> acc |> Seq.append( seq{ yield (by, what) } )
|> groupFolder rest
My question is, is it a more functional way to achieve just this?
As an example input to this reducer
let GroupsCommingFromMap = [| seq { yield! [|("key1", [|1;2;3|] ); ("key2", [|1;2;3|] ); ("key3", [|1;2;3|]) |] }, seq { yield! [|("key1", [|4;5;6|] ); ("key2", [|4;5;6|] ); ("key3", [|4;5;6|]) |] } |];;
GroupsCommingFromMap |> Seq.reduce( fun acc i ->
acc |> groupFolder (i |> Seq.toList))
the expected result should contain all key1..key3 each with the array 1..6
From the code you posted, it is not very clear what you're trying to do. Could you include some sample inputs (together with the output that you would like to get)? And does your code actually work on any of the inputs (it has incomplete pattern match, so I doubt that...)
Anyway, you can implement key-based map reduce using Seq.groupBy. For example:
let mapReduce mapper reducer input =
input
|> Seq.map mapper
|> Seq.groupBy fst
|> Seq.map (fun (k, vs) ->
k, vs |> Seq.map snd |> Seq.reduce reducer)
Here:
The mapper takes a value from the input sequence and turns it into key value pair. The mapReduce function then groups the values using the key
The reducer is then used to reduce all values associated with each key
This lets you create a word count function like this (using simple mapper that returns the word as the key with 1 as a value and reducer that just adds all the numbers):
"hello world hello people hello world".Split(' ')
|> mapReduce (fun w -> w, 1) (+)
EDIT: The example you mentioned does not really have "mapper" part, but instead it has array of arrays as an input - so perhaps it is easier to write this directly using Seq.groupBy like this:
let GroupsCommingFromMap =
[| [|("key1", [|1;2;3|] ); ("key2", [|1;2;3|] ); ("key3", [|1;2;3|]) |]
[|("key1", [|4;5;6|] ); ("key2", [|4;5;6|] ); ("key3", [|4;5;6|]) |] |]
GroupsCommingFromMap
|> Seq.concat
|> Seq.groupBy fst
|> Seq.map (fun (k, vs) -> k, vs |> Seq.map snd |> Array.concat)
When I use Console.WriteLine to print a list, it defaults to only showing the first three elements. How do I get it to print the entire contents of the list?
You can use the %A format specifier along with printf to get a 'beautified' list printout, but like Console.WriteLine (which calls .ToString()) on the object, it will not necessarily show all the elements. To get them all, iterate over the whole list. The code below shows a few different alternatives.
let smallList = [1; 2; 3; 4]
printfn "%A" smallList // often useful
let bigList = [1..200]
printfn "%A" bigList // pretty, but not all
printfn "Another way"
for x in bigList do
printf "%d " x
printfn ""
printfn "Yet another way"
bigList |> List.iter (printf "%d ")
printfn ""
You can iterate over the it, using the List.iter function, and print each element:
let list = [1;2;3;4]
list |> List.iter (fun x -> printf "%d " x)
More info:
Lists in F# (MSDN)
Here's simple alternative that uses String.Join:
open System
let xs = [1; 2; 3; 4]
let s = "[" + String.Join("; ", xs) + "]"
printfn "%A" s
I'm trying to print the output of function only when it is true but so far all attempts have been unsuccsessful.
Something on the lines of:
let printFactor a b = if b then print_any((a,b))
Where b is a boolean and a is an integer.
When I try it I get:
val printFactor : 'a -> bool -> unit
Any suggestions?
EDIT:
To put things in context im trying to use this with a pipe operator. Lets say I have a function xyz that outputs a list of (int, bool). Id like to do something on these lines:
xyz |> printFactor
to print the true values only.
You could do e.g. this
let xyz() = [ (1,true); (2,false) ]
let printFactor (i,b) =
if b then
printfn "%A" i
xyz() |> List.iter printFactor
but it would probably be more idiomatic to do, e.g. this
xyz()
|> List.filter (fun (i,b) -> b)
|> List.iter (fun (i,b) -> printfn "%d" i)
that is, first filter, and then print.