I am using SQLProvider in a project and I would like to run a query with a left join and return all records that are missing from the joined table.
I suspect the answer to this question will involve one or both of the packages FSharp.Data.TypeProviders and FSharpComposableQuery, although, to be honest, I can't tell where one ends and the other begins.
The common example of a left join in the above links is given as
query {
for student in db.Student do
leftOuterJoin selection in db.CourseSelection
on (student.StudentID = selection.StudentID) into result
for selection in result.DefaultIfEmpty() do
select (student, selection)
}
And from what I can tell, this is equivalent to the sql:
select *
from Student s
left outer join CourseSelection cs on s.StudentID = cs.StudentID
But what I am looking for is the F# equivalent of the sql:
select *
from Student s
left outer join CourseSelection cs on s.StudentID = cs.StudentID
where s.StudentID is null
I realize that I can just return all records and then filter them in F#, but I want the filtering to happen on the database side where things are indexed and because, in my case especially, the number of non null records is huge, and I am only interested in the null ones.
I think this should do the trick:
query {
for student in db.Student do
leftOuterJoin selection in db.CourseSelection
on (student.StudentID = selection.StudentID) into result
where (not (result.Any()))
select student
}
or a nested query:
query {
for student in db.Student do
where (query {
for selection in db.CourseSelection do
all (student.StudentID <> selection.StudentID)
})
select student
}
Edit: since you're using FSharp.Data.TypeProviders, if you have a foreign key between these two tables then you should also have a property that gives the associated CourseSelections, something like this:
query {
for student in db.Student do
where (not (student.CourseSelections.Any()))
select student
}
I need to process the query at the bottom of this post in C#. The query works, but I don't know how to use it in EF6. I used a method and a viewmodel for it (variable query = the query below). But when it encounters null values in the OUTER JOIN, int32 cant accept this value when calling .toList(). What's the best way to deal with it?
var result = context.Database.SqlQuery<TourQueryViewModel>(query);
var reslist = result.ToList();
I tried my first steps with LINQ, but I dont get it how to translate into LINQ itself or the query-methods, that are equivalent to it. So I am hoping for your help.
SELECT toursdata.TourId AS TourId, toursdata.Tourname AS Tourname,toursdata.Tourdate Tourdate,
toursdata.VehicleId AS VehicleId, toursdata.VehicleName AS VehicleName, toursdata.LicenseNumber AS LicenseNumber,
Employees.EmployeeId AS EmployeeId, Employees.Gender AS Gender, Employees.Forename AS Forename, Employees.Surname AS Surname
FROM (
SELECT te.TourId, te.Tourname, te.Tourdate,
Vehicles.VehicleId, Vehicles.VehicleName, Vehicles.LicenseNumber,
TourEmployees.EmployeeId
FROM TourEmployees RIGHT OUTER JOIN Tours AS te ON TourEmployees.TourId = te.TourId,
Tours AS tv INNER JOIN Vehicles ON tv.VehicleId = Vehicles.VehicleId
WHERE tv.TourId = te.TourId
) toursdata
LEFT OUTER JOIN Employees ON toursdata.EmployeeId = Employees.EmployeeId
To eliminate the null problem, I changed the data type of the corresponding entity-attribute to a nullable-type
int turned to int?.
Didnt know about that language feature.
Lets say we have 3 tables, Users, Products, Purchases.
There is a view that needs to display the purchases made by a user.
I could lookup the data required by doing:
from p in DBSet<Purchases>.Include("User").Include("Product") select p;
However, I am concern that this may have a performance impact because it will retrieve the full objects.
Alternatively, I could select only the fields i need:
from p in DBSet<Purchases>.Include("User").Include("Product") select new SimplePurchaseInfo() { UserName = p.User.name, Userid = p.User.Id, ProductName = p.Product.Name ... etc };
So my question is:
Whats the best practice in doing this?
== EDIT
Thanks for all the replies.
[QUESTION 1]: I want to know whether all views should work with flat ViewModels with very specific data for that view, or should the ViewModels contain the entity objects.
Real example: User reviews Products
var query = from dr in productRepository.FindAllReviews()
where dr.User.UserId = 'userid'
select dr;
string sql = ((ObjectQuery)query).ToTraceString();
SELECT [Extent1].[ProductId] AS [ProductId],
[Extent1].[Comment] AS [Comment],
[Extent1].[CreatedTime] AS [CreatedTime],
[Extent1].[Id] AS [Id],
[Extent1].[Rating] AS [Rating],
[Extent1].[UserId] AS [UserId],
[Extent3].[CreatedTime] AS [CreatedTime1],
[Extent3].[CreatorId] AS [CreatorId],
[Extent3].[Description] AS [Description],
[Extent3].[Id] AS [Id1],
[Extent3].[Name] AS [Name],
[Extent3].[Price] AS [Price],
[Extent3].[Rating] AS [Rating1],
[Extent3].[ShopId] AS [ShopId],
[Extent3].[Thumbnail] AS [Thumbnail],
[Extent3].[Creator_UserId] AS [Creator_UserId],
[Extent4].[Comment] AS [Comment1],
[Extent4].[DateCreated] AS [DateCreated],
[Extent4].[DateLastActivity] AS [DateLastActivity],
[Extent4].[DateLastLogin] AS [DateLastLogin],
[Extent4].[DateLastPasswordChange] AS [DateLastPasswordChange],
[Extent4].[Email] AS [Email],
[Extent4].[Enabled] AS [Enabled],
[Extent4].[PasswordHash] AS [PasswordHash],
[Extent4].[PasswordSalt] AS [PasswordSalt],
[Extent4].[ScreenName] AS [ScreenName],
[Extent4].[Thumbnail] AS [Thumbnail1],
[Extent4].[UserId] AS [UserId1],
[Extent4].[UserName] AS [UserName]
FROM [ProductReviews] AS [Extent1]
INNER JOIN [Users] AS [Extent2] ON [Extent1].[UserId] = [Extent2].[UserId]
LEFT OUTER JOIN [Products] AS [Extent3] ON [Extent1].[ProductId] = [Extent3].[Id]
LEFT OUTER JOIN [Users] AS [Extent4] ON [Extent1].[UserId] = [Extent4].[UserId]
WHERE N'615005822' = [Extent2].[UserId]
or
from d in productRepository.FindAllProducts()
from dr in d.ProductReviews
where dr.User.UserId == 'userid'
orderby dr.CreatedTime
select new ProductReviewInfo()
{
product = new SimpleProductInfo() { Id = d.Id, Name = d.Name, Thumbnail = d.Thumbnail, Rating = d.Rating },
Rating = dr.Rating,
Comment = dr.Comment,
UserId = dr.UserId,
UserScreenName = dr.User.ScreenName,
UserThumbnail = dr.User.Thumbnail,
CreateTime = dr.CreatedTime
};
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name],
[Extent1].[Thumbnail] AS [Thumbnail],
[Extent1].[Rating] AS [Rating],
[Extent2].[Rating] AS [Rating1],
[Extent2].[Comment] AS [Comment],
[Extent2].[UserId] AS [UserId],
[Extent4].[ScreenName] AS [ScreenName],
[Extent4].[Thumbnail] AS [Thumbnail1],
[Extent2].[CreatedTime] AS [CreatedTime]
FROM [Products] AS [Extent1]
INNER JOIN [ProductReviews] AS [Extent2] ON [Extent1].[Id] = [Extent2].[ProductId]
INNER JOIN [Users] AS [Extent3] ON [Extent2].[UserId] = [Extent3].[UserId]
LEFT OUTER JOIN [Users] AS [Extent4] ON [Extent2].[UserId] = [Extent4].[UserId]
WHERE N'userid' = [Extent3].[UserId]
ORDER BY [Extent2].[CreatedTime] ASC
[QUESTION 2]: Whats with the ugly outer joins?
In general, only retrieve what you need, but keep in mind to retrieve enough information so your application is not too chatty, so if you can batch a bunch of things together, do so, otherwise you'll pay network traffic cost everytime you need to go back to the database and retrieve some more stuffs.
In this case, assuming you will only need those info, I would go with the second approach (if that's what you really need).
Eager loading with .Include doesn't really play nice when you want filtering (or ordering for that matter).
That first query is basically this:
select p.*, u.*, p2.*
from products p
left outer join users u on p.userid = u.userid
left outer join purchases p2 on p.productid = p2.productid
where u.userid == #p1
Is that really what you want?
There is a view that needs to display the purchases made by a user.
Well then why are you including "Product"?
Shouldn't it just be:
from p in DBSet<Purchases>.Include("User") select p;
Your second query will error. You must project to an entity on the model, or an anonymous type - not a random class/DTO.
To be honest, the easiest and most well performing option in your current scenario is to query on the FK itself:
var purchasesForUser = DBSet<Purchases>.Where(x => x.UserId == userId);
That should produce:
select p.*
from products p
where p.UserId == #p1
The above query of course requires you to include the foreign keys in the model.
If you don't have the FK's in your model, then you'll need more LINQ-Entities trickery in the form of anonymous type projection.
Overall, don't go out looking to optimize. Create queries which align with the scenario/business requirement, then optimize if necessary - or look for alternatives to LINQ-Entities, such as stored procedures, views or compiled queries.
Remember: premature optimization is the root of all evil.
*EDIT - In response to Question Update *
[QUESTION 1]: I want to know whether all views should work with flat ViewModels with very specific data for that view, or should the ViewModels contain the entity objects.
Yes - ViewModel's should only contain what is required for that View. Otherwise why have the ViewModel? You may as well bind straight to the EF model. So, setup the ViewModel which only the fields it needs for the view.
[QUESTION 2]: What's with the ugly outer joins?
That is default behaviour for .Include. .Include always produces a left outer join.
I think the second query will throw exception because you can't map result to unmapped .NET type in Linq-to-entities. You have to return annonymous type and map it to your object in Linq-to-objects or you have to use some advanced concepts for projections - QueryView (projections in ESQL) or DefiningQuery (custom SQL query mapped to new readonly entity).
Generally it is more about design of your entities. If you select single small entity it is not a big difference to load it all instead of projection. If you are selecting list of entities you should consider projections - expecially if tables contains columns like nvarchar(max) or varbinar(max) which are not needed in your result!
Both create almost the same query: select from one table, with two inner joins. The only thing that changes from a database perspective is the amount of fields returned, but that shouldn't really matter that much.
I think here DRY wins from a performance hit (if it even exists): so my call is go for the first option.
The generated SQL does a cross join but as the ID's are matched it acts like an inner join but is there a better performing way to do this by actually using the join keyword in C#? Is there a way where you don't have to specify how the properties join because they are all heirarchicly related anyway
Item is a Page class
PageRoles is an IQueryable property in a Page class
aspnet_Roles is an IQueryable property in a PageRole class
var item = _repository.GetByKey(999);
var f = (from fd in item.PageRoles
from k in fd.aspnet_Roles
where Roles.GetRolesForUser().Contains(k.RoleName)
&& k.RoleId == fd.RoleId
select k.RoleName)
.Count();
EDIT:
Here is an example of an IQueryable property in my classes. The below example comes from the PageRole class
public IQueryable<aspnet_Role> aspnet_Roles
{
get
{
var repo=NorthCadburyWebsite.Models.aspnet_Role.GetRepo();
return from items in repo.GetAll()
where items.RoleId == _RoleId
select items;
}
}
Does this compile or do you get an error?
In this case, as you mentioned, you don't need a join.
I haven't used this syntax, but it looks like you use k as a refererence (like an alias in sql syntax)
This should deliver the same result (without even using k):
var f = (from fd in item.PageRoles
where Roles.GetRolesForUser().Contains(fd.aspnet_Roles.RoleName)
&& fd.aspnet_Roles.RoleId == fd.RoleId
select fd.aspnet_Roles.RoleName)
.Count();
Note: Not tested, just a guess.
I don't have my Intellisense with me but I think you can:
var f = (from fd in item.PageRoles
join k in fd.aspnet_Roles on k.RoleId equals fd.RoleId
where Roles.GetRolesForUser().Contains(k.RoleName)
select k.RoleName)
.Count();
I think I'm confused, because I'm seeing the following hierarchy in your query which isn't making any sense:
[PageRoles]
|--RoleId <~~~~~~~~~.
|--[aspnet_Roles] |
|--RoleId ~~~~~~~*
|--RoleName
Is the hierarchy determined by the RoleId and you only mean for PageRoles.RoleId to reference aspnet_Roles.RoleId? So you might have something like:
[PageRoles] [aspnet_Roles]
|--RoleId |--RoleId
|--RoleName
And you're trying to make it into the following:
[PageRoles]
|--RoleId
|--[aspnet_Roles] (where RoleId == PageRoles.RoleId)
|--RoleName
...?
In which case if you can't do the following, then something is wrong with your IQueryable property definitions and not the query you have posted. Namely the method to get PageRoles's aspnet_Roles child rows is not set up properly.
var f = (from fd in item.PageRoles
from k in fd.aspnet_Roles
where Roles.GetRolesForUser().Contains(k.RoleName)
select k.RoleName)
.Count();
(fd.aspnet_Roles should already be constrained by RoleId and generate an inner join).
You can check your hierarchy by seeing what SQL is generated from:
var f = from fd in item.PageRoles
from k in fd.aspnet_Roles
select k;
which should vaguely resemble (with something having to do with item thrown in):
SELECT k.RoleName, k.SomethingElse
FROM PageRoles fd
INNER JOIN aspnetRoles k ON fd.RoleId = k.RoleId;
I'm sure I'm missing something, so please help me fill in the blanks...
HI,
I have 3 tables: Clips, Books and relationships between ClipBook
Problem is:
i need get book that has bookID=4 with some clips i mean many-to-many
in simple text sql it will be something like this:
select * from Clips where clipID in (select clipID from ClipBook where bookID=4)
Question is:
How can i do this with Linq without operator Join of course
this could be a solution;
from cb in ClipBooks
where cb.BookID == 4
select cb.Clip;
or
ClipBooks.Where(cb => cb.BookId == 4).Select(cb => cb.Clip);
the Contains method is in Linq converted into the IN operator, example:
String[] cities = {"London", "Paris", "Madrid"};
from p in context.Person
where cities.Contains(p.Adress.City)
select p
is translated into a sql clause like: .. WHERE a.City in ('London', 'Paris', 'Madrid') where a is the Address table alias after the join with the Person table.
edit:
you could write a single query without a subquery, but this will be converted to a join most probably:
var clips = (from c in context.ClipBooks
where c.BookID == 4
select c.Clip).Distinct();
or
var clips = from c in context.Clip
where c.ClicBooks.Any( cb => cb.BookID == 4)
select c
Any is translated in Exists()