I'm upgrading an application to EF 2.2, but using EFCore 2.2.6 keeps throwing "Must be a reducible node" ArgumentException when I try to do a simple join.
like
var list = (from a in db.TableA().Include("TableC")
join b in inMemoryList on a.Id equals b.AId
select a).ToList();
If I change to
var list = (from a in db.TableA().ToList()
join b in inMemoryList on a.Id equals b.AId
select a).ToList();
It works, but slow down the process.
Any ideas ? Thanks
This happens from time to time between EF Core versions.
In general, avoid joins to in-memory collections if you can - they've never been supported well.
Use Contains for single field filtering where possible, for instance
var ids = inMemoryList.Select(x => x.AId); // has to be outside the query expression tree
var list = db.TableA.Where(a => ids.Contains(a.Id)).ToList();
I'm trying to convert this SQL code to linq sql. But I don't understand even with the doc... someone can help me please ?
select prcleunique, LibelleProjet, from projet a
where eqcleunique in (select EqCleunique from Compo where uscleunique = '{0}')
and (a.socleunique in (select socleunique from utilisat where uscleunique = '{0}') or a.socleunique is null)
and a.archive = 2 order by LibelleProjet", idUtilisateur);
Those nested sql queries can be broken down nicely in Linq. Every time you have a select have a seperate linq query:
var clause1 = from row in _db.Compo where uscleunique == '{0}' select EqCleunique;
Then use the clauses in the last query
var result = from row in _db.project where clause1.Contains(row.eqcleunique) select row.whatever;
I hope this example is enough to get you started.
I have an issue in EF Core where I am trying to get a related entity and all it's dependent structures, but am not having much success with it.
Currently, I have a query like this:
var user = new Guid(id);
var userCustAffs = _data.UserCustomerAffiliation.Include(x => x.Customer)
.ThenInclude(x => x.Brand).Where(x => x.UserId.Equals(user)).ToList();
var result = userCustAffs.Select(p => p.Customer).ToList();
When I should be able to do something like this to simplify it (and remove unneccesary things being evaluated locally vs the db)
var user = new Guid(id);
var userCustAffs = _data.UserCustomerAffiliation.Include(x => x.Customer)
.ThenInclude(x => x.Brand).Where(x => x.UserId.Equals(user))
.Select(y => y.Customer).ToList();
However, when I do the latter query, I get an error that
The Include operation for navigation '[x].Customer.Brand' is unnecessary and was ignored
because the navigation is not reachable in the final query results
However, Brand is very important, as it drives some of the properties off of the Customer model. What is the proper way to restructure this query so that I get the results I want (e.g. Customer with its relevant Brand, limited by the userId affiliated on the UserCustomerAffiliation table).
I have seen a recommendation before to "start" the query from the Customer instead of UserCustomerAffiliation, but that seems contrary to every instinct I have from a DB optimization standpoint (and Customer does not have a navigation property back to UserCustomerAffiliation atm).
The answer to why this happens this (after some research) is quite interesting and a good example of why knowing how EF Core works is important to using it.
Linq in general works on the idea of deferred execution. To put it very simply, if I make a Linq statement on a particular line, it may not get evaluated or executed until the data is "needed." Most of the time we shortcut this with .ToList() which forces immediate execution. The general idea here is that sometimes datasets are not needed (say, if an exception occurs before it gets evaluated but after it would be 'loaded').
EF Core takes this one step further and ties the idea of deferred execution with database optimization. If, for example, I get a subset of data from the database:
var result = _context.SomeTable.Where(x => x.name == "SomeValue");
But later all I care about is the size of the dataset:
return result.Count;
The DB call can be optimized to
select count(*) from SomeTable where name = "SomeValue";
instead of
select * from SomeTable where name = "SomeValue";
Similarly, the query I have above was being optimized away. Because I chained the whole thing before it was evaluated, the EF Core optimizer threw away a table I needed.
The reason this works:
var user = new Guid(id);
var userCustAffs = _data.UserCustomerAffiliation.Include(x => x.Customer)
.ThenInclude(x => x.Brand).Where(x =>
x.UserId.Equals(user)).ToList();
var result = userCustAffs.Select(p => p.Customer).ToList();
Is because I force execution of the query that is something like
Select u.*, c.*, b.* from usercustomeraffiliation u,
inner join Customer c on u.customerid = c.id
inner join Brand b on c.brandid = c.id
where u.userid = 'userId';
And then strip out the customer object (and the brand object underneath it) in memory. It would be more efficient to be able to generate a query like:
Select c.*, b.* from Customer c on u.customerid = c.id
inner join Brand b on c.brandid = c.id
where c.id in (select u.customerid from usercustomeraffiliation u
where u.userid = 'userId');
But, that gets optimized away.
I am trying to combine SharePoint data with a legacy database data and I can get the data, but I need to do it in two queries. Here are the two Linq queries:
var query =
(from dtEai in result.AsEnumerable()
join allAP in dtAllAirports.AsEnumerable()
on dtEai.Field<int>("AirportID") equals allAP.Field<int>("ID")
select new
{
Region = allAP.Field<string>("region")
}
);
and the second which gets me my result:
var join =
(
from table in query
group table by table["Region"] into groupedTable
select new
{
Key = groupedTable.Key,
Count = groupedTable.Count()
}
);
not being an expert in Linq I converted the SharePoint lists into datatables so I could do the join. Can I combine this into a single query?
I had to make two linq methods due to the fact that I did not want to try to method chain and do a groupby afterwards. It just would of been too confusing.
Lets say we have 3 tables, Users, Products, Purchases.
There is a view that needs to display the purchases made by a user.
I could lookup the data required by doing:
from p in DBSet<Purchases>.Include("User").Include("Product") select p;
However, I am concern that this may have a performance impact because it will retrieve the full objects.
Alternatively, I could select only the fields i need:
from p in DBSet<Purchases>.Include("User").Include("Product") select new SimplePurchaseInfo() { UserName = p.User.name, Userid = p.User.Id, ProductName = p.Product.Name ... etc };
So my question is:
Whats the best practice in doing this?
== EDIT
Thanks for all the replies.
[QUESTION 1]: I want to know whether all views should work with flat ViewModels with very specific data for that view, or should the ViewModels contain the entity objects.
Real example: User reviews Products
var query = from dr in productRepository.FindAllReviews()
where dr.User.UserId = 'userid'
select dr;
string sql = ((ObjectQuery)query).ToTraceString();
SELECT [Extent1].[ProductId] AS [ProductId],
[Extent1].[Comment] AS [Comment],
[Extent1].[CreatedTime] AS [CreatedTime],
[Extent1].[Id] AS [Id],
[Extent1].[Rating] AS [Rating],
[Extent1].[UserId] AS [UserId],
[Extent3].[CreatedTime] AS [CreatedTime1],
[Extent3].[CreatorId] AS [CreatorId],
[Extent3].[Description] AS [Description],
[Extent3].[Id] AS [Id1],
[Extent3].[Name] AS [Name],
[Extent3].[Price] AS [Price],
[Extent3].[Rating] AS [Rating1],
[Extent3].[ShopId] AS [ShopId],
[Extent3].[Thumbnail] AS [Thumbnail],
[Extent3].[Creator_UserId] AS [Creator_UserId],
[Extent4].[Comment] AS [Comment1],
[Extent4].[DateCreated] AS [DateCreated],
[Extent4].[DateLastActivity] AS [DateLastActivity],
[Extent4].[DateLastLogin] AS [DateLastLogin],
[Extent4].[DateLastPasswordChange] AS [DateLastPasswordChange],
[Extent4].[Email] AS [Email],
[Extent4].[Enabled] AS [Enabled],
[Extent4].[PasswordHash] AS [PasswordHash],
[Extent4].[PasswordSalt] AS [PasswordSalt],
[Extent4].[ScreenName] AS [ScreenName],
[Extent4].[Thumbnail] AS [Thumbnail1],
[Extent4].[UserId] AS [UserId1],
[Extent4].[UserName] AS [UserName]
FROM [ProductReviews] AS [Extent1]
INNER JOIN [Users] AS [Extent2] ON [Extent1].[UserId] = [Extent2].[UserId]
LEFT OUTER JOIN [Products] AS [Extent3] ON [Extent1].[ProductId] = [Extent3].[Id]
LEFT OUTER JOIN [Users] AS [Extent4] ON [Extent1].[UserId] = [Extent4].[UserId]
WHERE N'615005822' = [Extent2].[UserId]
or
from d in productRepository.FindAllProducts()
from dr in d.ProductReviews
where dr.User.UserId == 'userid'
orderby dr.CreatedTime
select new ProductReviewInfo()
{
product = new SimpleProductInfo() { Id = d.Id, Name = d.Name, Thumbnail = d.Thumbnail, Rating = d.Rating },
Rating = dr.Rating,
Comment = dr.Comment,
UserId = dr.UserId,
UserScreenName = dr.User.ScreenName,
UserThumbnail = dr.User.Thumbnail,
CreateTime = dr.CreatedTime
};
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name],
[Extent1].[Thumbnail] AS [Thumbnail],
[Extent1].[Rating] AS [Rating],
[Extent2].[Rating] AS [Rating1],
[Extent2].[Comment] AS [Comment],
[Extent2].[UserId] AS [UserId],
[Extent4].[ScreenName] AS [ScreenName],
[Extent4].[Thumbnail] AS [Thumbnail1],
[Extent2].[CreatedTime] AS [CreatedTime]
FROM [Products] AS [Extent1]
INNER JOIN [ProductReviews] AS [Extent2] ON [Extent1].[Id] = [Extent2].[ProductId]
INNER JOIN [Users] AS [Extent3] ON [Extent2].[UserId] = [Extent3].[UserId]
LEFT OUTER JOIN [Users] AS [Extent4] ON [Extent2].[UserId] = [Extent4].[UserId]
WHERE N'userid' = [Extent3].[UserId]
ORDER BY [Extent2].[CreatedTime] ASC
[QUESTION 2]: Whats with the ugly outer joins?
In general, only retrieve what you need, but keep in mind to retrieve enough information so your application is not too chatty, so if you can batch a bunch of things together, do so, otherwise you'll pay network traffic cost everytime you need to go back to the database and retrieve some more stuffs.
In this case, assuming you will only need those info, I would go with the second approach (if that's what you really need).
Eager loading with .Include doesn't really play nice when you want filtering (or ordering for that matter).
That first query is basically this:
select p.*, u.*, p2.*
from products p
left outer join users u on p.userid = u.userid
left outer join purchases p2 on p.productid = p2.productid
where u.userid == #p1
Is that really what you want?
There is a view that needs to display the purchases made by a user.
Well then why are you including "Product"?
Shouldn't it just be:
from p in DBSet<Purchases>.Include("User") select p;
Your second query will error. You must project to an entity on the model, or an anonymous type - not a random class/DTO.
To be honest, the easiest and most well performing option in your current scenario is to query on the FK itself:
var purchasesForUser = DBSet<Purchases>.Where(x => x.UserId == userId);
That should produce:
select p.*
from products p
where p.UserId == #p1
The above query of course requires you to include the foreign keys in the model.
If you don't have the FK's in your model, then you'll need more LINQ-Entities trickery in the form of anonymous type projection.
Overall, don't go out looking to optimize. Create queries which align with the scenario/business requirement, then optimize if necessary - or look for alternatives to LINQ-Entities, such as stored procedures, views or compiled queries.
Remember: premature optimization is the root of all evil.
*EDIT - In response to Question Update *
[QUESTION 1]: I want to know whether all views should work with flat ViewModels with very specific data for that view, or should the ViewModels contain the entity objects.
Yes - ViewModel's should only contain what is required for that View. Otherwise why have the ViewModel? You may as well bind straight to the EF model. So, setup the ViewModel which only the fields it needs for the view.
[QUESTION 2]: What's with the ugly outer joins?
That is default behaviour for .Include. .Include always produces a left outer join.
I think the second query will throw exception because you can't map result to unmapped .NET type in Linq-to-entities. You have to return annonymous type and map it to your object in Linq-to-objects or you have to use some advanced concepts for projections - QueryView (projections in ESQL) or DefiningQuery (custom SQL query mapped to new readonly entity).
Generally it is more about design of your entities. If you select single small entity it is not a big difference to load it all instead of projection. If you are selecting list of entities you should consider projections - expecially if tables contains columns like nvarchar(max) or varbinar(max) which are not needed in your result!
Both create almost the same query: select from one table, with two inner joins. The only thing that changes from a database perspective is the amount of fields returned, but that shouldn't really matter that much.
I think here DRY wins from a performance hit (if it even exists): so my call is go for the first option.