I am using SqlDataConnection data provider in F# to migrate some rows, part of that migration is to make a join between 3 tables like this, think of it as an inheritance of tables A, B, C where B and C inherit from A so the thing is I need to get is (in Linq-like):
Bs.Join(As, b.PK, a.FK).Select(new {...})
.Concat(Cs.Join(As, c.PK, a.FK).Select(new {...})
In F#, the closest I got to this was:
let result = seq {
yield! query { ... }
yield! query { ... }
}
but I've been told this will produce 2 SQL queries and the overall result will be on-memory. The question being: is there a way to make this "concatenation" as a query computation expression without using seq so that all happens in a single SQL query?
Here's some example code that combines two queries using SQL UNION or UNION ALL.
First, the setup. Note that I've added logging to dbContext so you can see what happens behind the scenes.
#r "System.Data.dll"
#r "System.Data.Linq.dll"
#r "FSharp.Data.TypeProviders.dll"
open System
open System.Linq
open Microsoft.FSharp.Data.TypeProviders
type sql = SqlDataConnection<connStr>
let createDbContext() =
let dbContext = sql.GetDataContext()
// add logging to console
dbContext.DataContext.Log <- System.Console.Out
dbContext
let db = createDbContext()
let products = db.Product
let q1 = query { for x in products do select x }
let q2 = query { for y in products do select y }
The Union extension method combines queries as one query using UNION
let qUnion = q1.Union(q2)
qUnion.ToList() |> Seq.toList
Here's the logged output:
SELECT [t2].[Id], [t2].[Name]
FROM (
SELECT [t0].[Id], [t0].[Name]
FROM [dbo].[Product] AS [t0]
UNION
SELECT [t1].[Id], [t1].[Name]
FROM [dbo].[Product] AS [t1]
) AS [t2]
The Concat extension method combines queries as one query using UNION ALL
let qConcat = q1.Concat(q2)
qConcat.ToList() |> Seq.toList
Here's the logged output:
SELECT [t2].[Id], [t2].[Name]
FROM (
SELECT [t0].[Id], [t0].[Name]
FROM [dbo].[Product] AS [t0]
UNION ALL
SELECT [t1].[Id], [t1].[Name]
FROM [dbo].[Product] AS [t1]
) AS [t2]
There's no special syntax for unions in query expressions, AFAIK.
Related
open System
open FSharp.Linq
open FSharp.Data.TypeProviders
type NorthwindDb =
SqlDataConnection<"Data Source=localhost\SQLEXPRESS;Initial Catalog=Northwind;Integrated Security=SSPI;">
let db = NorthwindDb.GetDataContext()
db.DataContext.Log <- System.Console.Out
I am following along Expert F# 4.0 Chapter 13 on queries. According to the book, an inner join query like this:
let innerJoinQuery =
query {
for c in db.Categories do
join p in db.Products on (c.CategoryID =? p.CategoryID)
select (p.ProductName, c.CategoryName)
}
|> Seq.toList
can be replaced with
let innerJoinAlternative =
query {
for p in db.Products do
select (p.ProductName, p.Category.CategoryName)
}
|> Seq.toList
I get the following error:
error FS0039: The field, constructor or member 'Category' is not defined
Note: Northwind database schema is here
Does the order of query expression operators matter? Idk, but sometimes (in some selections) it does, but sometimes it doesn't (or maybe it does but implicitly handles some particular occasions).
Is is mandatory that select operator must go last? in pretty much every combination it complains if you don't write it as a last statement, but in case of take n, this operator can go after select
I'm just interested in how the process of execution acts?
This brings me to another question. If it iterates over Iterable collection, and thus on the first iteration it selects some one(first) value, how order works on that one(first) value? it would be clear if first it returned sequence, and then executed order on that sequence.. but seems like it executes sortBy at every iteration (?). I'm interested in what the design of the executed algorithm is.
Here is my example of Query Expression.
let sq = query {
for p in datasource do
where p.age>20
sortBy p.age
select p
}
Explanations would be greatly appreciated.
Thanks
We don't need to guess, we can find out.
let sample = Seq.init 10 (fun i -> i * 10) |> Seq.map (fun i -> { age = i })
let sq = query {
for p in sample do
where (p.age > 20)
sortBy p.age
select p
}
sq |> Seq.toList |> ignore
The generated IL (cleaned up) looks like
IL_004e: newobj instance void Program/sq#16::.ctor(class [FSharp.Core]Microsoft.FSharp.Linq.QueryBuilder)
IL_0053: callvirt instance [...] For<class Program/Person,
IL_005d: callvirt instance [...] Where<class Program/Person,
IL_0067: callvirt instance [...] SortBy<class Program/Person,
IL_0071: callvirt instance [...] Select<class Program/Person,
Suppose we change the order of sortBy
let sq = query {
for p in sample do
sortBy p.age
where (p.age > 20)
select p
}
The new IL will be:
IL_006c: callvirt instance [...] For<class Program/Person,
IL_0076: callvirt instance [...] SortBy<class Program/Person,
IL_0080: callvirt instance [...] Where<class Program/Person,
IL_008a: callvirt instance [...] Select<class Program/Person,
You can clearly see that it follows the exact order you define the query in.
This wouldn't matter for T-SQL comprehensions because the query will be translated by an Expression visitor, but for object queries, query expressions are pretty much just syntactic sugar for you.
Method #2:
You can extend the query expression module to include an operator for side-effects. This is simply a port of Interactive Extensions' DoAction method.
module QueryExtensions =
type QueryBuilderEx() =
inherit Linq.QueryBuilder()
[<CustomOperation("doAction", MaintainsVariableSpace = true)>]
member __.Do(source : Linq.QuerySource<'T,System.Collections.IEnumerable>, action) =
new Linq.QuerySource<'T,System.Collections.IEnumerable>(source.Source |> Seq.map (fun v -> action(v); v))
let query = QueryExtensions.QueryBuilderEx()
Now you can debug the order like so
let sq = query {
for p in sample do
sortBy p.age
where (p.age > 20)
doAction (printfn "Next -> %A")
select p
}
If you move it above the where, you'll see that it reflects those records before filtering.
I am new to F# and having trouble translating some C# code.
I have a class similar to this:
type car () =
member val Model = "" with get,set
member val Year = "" with get,set
I have this query expression that pulls car data from the database:
query{
for row in db do
select // <-- what is the correct syntax to create a sequence of new car
// classes here
}
it's a lot easier if you don't translate 1:1 or at least if you add an constructor.
For example using a primary-constructor this should work:
type Car (model, year) =
member __.Model with get() = model
member __.Year with get() = year
query {
for row in db do
select (Car (row.Model, row.Year))
}
now of course I don't know how the rows in your db looks like and this will give you immutable data - but for what I see it should be fine
I just realised that this might be a problem (just as in C#) as the ctor probably cannot be translated into a SQL-statement - you can still try but I guess you really need to do
query {
for row in db do
select
} |> Seq.map (fun row -> Car (row.Model, row.Year))
instead (sorry - cannot really try right now)
One of the benefits of the type system on F# is avoid a null exception... or that was something I believe... because I'm getting a null problem:
[<CLIMutable>]
type Customer = {
[<AutoIncrement>] id:option<int64>
code:string
name:string
}
I'm running a SQL code:
let SqlFTS<'T>(table:string, searchTable:string, query:string) =
use db = openDb()
let sql = sprintf "SELECT * FROM %s WHERE id in (SELECT docid FROM %s WHERE data MATCH %A)" table searchTable query
printfn "%A" sql
db.SqlList<'T>(sql) |> Seq.toArray
testCase "Customers" <|
fun _ ->
let rows = GenData.genCustomers(50)
Customers.insert(rows)
isEqual "Failed to insert" 50L (DB.SqlCount<Customers.Customer>())
//Until here, it works
let c = Customers.byId(1L)
printfn "%A" c
//Customers.byId return me a record with all the properties as NULLS!
//Then c.name is null, and the code above fail.
let rows = Customers.searchCustomers(c.name)
This was very unexpected. Why I can get a record with all values to null?
Let's start with line 1:
[<CLIMutable>]
The docs for CLIMutable state
Adding this attribute to a record type causes it to be compiled to a Common Language Infrastructure (CLI) representation with a default constructor with property getters and setters.
That default constructor means the fields will be initialized to default values. The default value for string is null. That's valid to the CLR and it's valid to C# and VB.NET. You may not be able to call the default ctor from F#, but pretty much anyone else can.
Welcome to interop; it can be a pain.
There's a query to search for when an item is contained in a list, but there isn't one for when an item isn't.
This query finds customer objects which don't have ContactNum in the given list cdiffnums. What could I do to return just the customers that dont have a ContactNum in this list?
let q =
query {
for c in dc.Customers do
where (query { for n in cdiffnums do contains c.ContactNum })
select c
}
My F# is rusty, but have you tried:
let q =
query {
for c in dc.Customers do
where (not (query { for n in cdiffnums do contains c.ContactNum }))
select c
}
I think smth like this should work:
open System.Linq
let cdiffnums = [|1;2;3|]
let q =
query {
for c in dc.Customers do
where (not (cdiffnums.Contains c.ContactNum))
select c
}