ASP.NET MVC & EF4 Entity Framework - Are there any performance concerns in using the entities vs retrieving only the fields i need? - asp.net-mvc

Lets say we have 3 tables, Users, Products, Purchases.
There is a view that needs to display the purchases made by a user.
I could lookup the data required by doing:
from p in DBSet<Purchases>.Include("User").Include("Product") select p;
However, I am concern that this may have a performance impact because it will retrieve the full objects.
Alternatively, I could select only the fields i need:
from p in DBSet<Purchases>.Include("User").Include("Product") select new SimplePurchaseInfo() { UserName = p.User.name, Userid = p.User.Id, ProductName = p.Product.Name ... etc };
So my question is:
Whats the best practice in doing this?
== EDIT
Thanks for all the replies.
[QUESTION 1]: I want to know whether all views should work with flat ViewModels with very specific data for that view, or should the ViewModels contain the entity objects.
Real example: User reviews Products
var query = from dr in productRepository.FindAllReviews()
where dr.User.UserId = 'userid'
select dr;
string sql = ((ObjectQuery)query).ToTraceString();
SELECT [Extent1].[ProductId] AS [ProductId],
[Extent1].[Comment] AS [Comment],
[Extent1].[CreatedTime] AS [CreatedTime],
[Extent1].[Id] AS [Id],
[Extent1].[Rating] AS [Rating],
[Extent1].[UserId] AS [UserId],
[Extent3].[CreatedTime] AS [CreatedTime1],
[Extent3].[CreatorId] AS [CreatorId],
[Extent3].[Description] AS [Description],
[Extent3].[Id] AS [Id1],
[Extent3].[Name] AS [Name],
[Extent3].[Price] AS [Price],
[Extent3].[Rating] AS [Rating1],
[Extent3].[ShopId] AS [ShopId],
[Extent3].[Thumbnail] AS [Thumbnail],
[Extent3].[Creator_UserId] AS [Creator_UserId],
[Extent4].[Comment] AS [Comment1],
[Extent4].[DateCreated] AS [DateCreated],
[Extent4].[DateLastActivity] AS [DateLastActivity],
[Extent4].[DateLastLogin] AS [DateLastLogin],
[Extent4].[DateLastPasswordChange] AS [DateLastPasswordChange],
[Extent4].[Email] AS [Email],
[Extent4].[Enabled] AS [Enabled],
[Extent4].[PasswordHash] AS [PasswordHash],
[Extent4].[PasswordSalt] AS [PasswordSalt],
[Extent4].[ScreenName] AS [ScreenName],
[Extent4].[Thumbnail] AS [Thumbnail1],
[Extent4].[UserId] AS [UserId1],
[Extent4].[UserName] AS [UserName]
FROM [ProductReviews] AS [Extent1]
INNER JOIN [Users] AS [Extent2] ON [Extent1].[UserId] = [Extent2].[UserId]
LEFT OUTER JOIN [Products] AS [Extent3] ON [Extent1].[ProductId] = [Extent3].[Id]
LEFT OUTER JOIN [Users] AS [Extent4] ON [Extent1].[UserId] = [Extent4].[UserId]
WHERE N'615005822' = [Extent2].[UserId]
or
from d in productRepository.FindAllProducts()
from dr in d.ProductReviews
where dr.User.UserId == 'userid'
orderby dr.CreatedTime
select new ProductReviewInfo()
{
product = new SimpleProductInfo() { Id = d.Id, Name = d.Name, Thumbnail = d.Thumbnail, Rating = d.Rating },
Rating = dr.Rating,
Comment = dr.Comment,
UserId = dr.UserId,
UserScreenName = dr.User.ScreenName,
UserThumbnail = dr.User.Thumbnail,
CreateTime = dr.CreatedTime
};
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name],
[Extent1].[Thumbnail] AS [Thumbnail],
[Extent1].[Rating] AS [Rating],
[Extent2].[Rating] AS [Rating1],
[Extent2].[Comment] AS [Comment],
[Extent2].[UserId] AS [UserId],
[Extent4].[ScreenName] AS [ScreenName],
[Extent4].[Thumbnail] AS [Thumbnail1],
[Extent2].[CreatedTime] AS [CreatedTime]
FROM [Products] AS [Extent1]
INNER JOIN [ProductReviews] AS [Extent2] ON [Extent1].[Id] = [Extent2].[ProductId]
INNER JOIN [Users] AS [Extent3] ON [Extent2].[UserId] = [Extent3].[UserId]
LEFT OUTER JOIN [Users] AS [Extent4] ON [Extent2].[UserId] = [Extent4].[UserId]
WHERE N'userid' = [Extent3].[UserId]
ORDER BY [Extent2].[CreatedTime] ASC
[QUESTION 2]: Whats with the ugly outer joins?

In general, only retrieve what you need, but keep in mind to retrieve enough information so your application is not too chatty, so if you can batch a bunch of things together, do so, otherwise you'll pay network traffic cost everytime you need to go back to the database and retrieve some more stuffs.
In this case, assuming you will only need those info, I would go with the second approach (if that's what you really need).

Eager loading with .Include doesn't really play nice when you want filtering (or ordering for that matter).
That first query is basically this:
select p.*, u.*, p2.*
from products p
left outer join users u on p.userid = u.userid
left outer join purchases p2 on p.productid = p2.productid
where u.userid == #p1
Is that really what you want?
There is a view that needs to display the purchases made by a user.
Well then why are you including "Product"?
Shouldn't it just be:
from p in DBSet<Purchases>.Include("User") select p;
Your second query will error. You must project to an entity on the model, or an anonymous type - not a random class/DTO.
To be honest, the easiest and most well performing option in your current scenario is to query on the FK itself:
var purchasesForUser = DBSet<Purchases>.Where(x => x.UserId == userId);
That should produce:
select p.*
from products p
where p.UserId == #p1
The above query of course requires you to include the foreign keys in the model.
If you don't have the FK's in your model, then you'll need more LINQ-Entities trickery in the form of anonymous type projection.
Overall, don't go out looking to optimize. Create queries which align with the scenario/business requirement, then optimize if necessary - or look for alternatives to LINQ-Entities, such as stored procedures, views or compiled queries.
Remember: premature optimization is the root of all evil.
*EDIT - In response to Question Update *
[QUESTION 1]: I want to know whether all views should work with flat ViewModels with very specific data for that view, or should the ViewModels contain the entity objects.
Yes - ViewModel's should only contain what is required for that View. Otherwise why have the ViewModel? You may as well bind straight to the EF model. So, setup the ViewModel which only the fields it needs for the view.
[QUESTION 2]: What's with the ugly outer joins?
That is default behaviour for .Include. .Include always produces a left outer join.

I think the second query will throw exception because you can't map result to unmapped .NET type in Linq-to-entities. You have to return annonymous type and map it to your object in Linq-to-objects or you have to use some advanced concepts for projections - QueryView (projections in ESQL) or DefiningQuery (custom SQL query mapped to new readonly entity).
Generally it is more about design of your entities. If you select single small entity it is not a big difference to load it all instead of projection. If you are selecting list of entities you should consider projections - expecially if tables contains columns like nvarchar(max) or varbinar(max) which are not needed in your result!

Both create almost the same query: select from one table, with two inner joins. The only thing that changes from a database perspective is the amount of fields returned, but that shouldn't really matter that much.
I think here DRY wins from a performance hit (if it even exists): so my call is go for the first option.

Related

F# query expression: How to do a left join and return on those records where the joined table is null?

I am using SQLProvider in a project and I would like to run a query with a left join and return all records that are missing from the joined table.
I suspect the answer to this question will involve one or both of the packages FSharp.Data.TypeProviders and FSharpComposableQuery, although, to be honest, I can't tell where one ends and the other begins.
The common example of a left join in the above links is given as
query {
for student in db.Student do
leftOuterJoin selection in db.CourseSelection
on (student.StudentID = selection.StudentID) into result
for selection in result.DefaultIfEmpty() do
select (student, selection)
}
And from what I can tell, this is equivalent to the sql:
select *
from Student s
left outer join CourseSelection cs on s.StudentID = cs.StudentID
But what I am looking for is the F# equivalent of the sql:
select *
from Student s
left outer join CourseSelection cs on s.StudentID = cs.StudentID
where s.StudentID is null
I realize that I can just return all records and then filter them in F#, but I want the filtering to happen on the database side where things are indexed and because, in my case especially, the number of non null records is huge, and I am only interested in the null ones.
I think this should do the trick:
query {
for student in db.Student do
leftOuterJoin selection in db.CourseSelection
on (student.StudentID = selection.StudentID) into result
where (not (result.Any()))
select student
}
or a nested query:
query {
for student in db.Student do
where (query {
for selection in db.CourseSelection do
all (student.StudentID <> selection.StudentID)
})
select student
}
Edit: since you're using FSharp.Data.TypeProviders, if you have a foreign key between these two tables then you should also have a property that gives the associated CourseSelections, something like this:
query {
for student in db.Student do
where (not (student.CourseSelections.Any()))
select student
}

How to use the exceptjoin in Cognos-11?

I don't get an except join to work in Cognos-11. Where or what am I missing?
Some understanding for a beginner in this branch would be nice ;-)
What I've tried so far is making two queries. The first one holds data items like "customer", "BeginningDate" and "Purpose". The second query holds data items like "customer", "Adress" and "Community".
What I'd like to accomplish is to get in query3: the "customers" from query1 that are not available in query2. To me it sounds like an except-join.
I went to the query work area, created a query3 and dragged an "except-join" icon on it. Then I dragged query1 into the upper space and query2 into the lower. What I'm used to getting with other joins, is a possibility to set a new link, cardinality and so on. Now double clicking the join isn't opening any pop-up. The properties of the except-join show "Set operation = Except", "Duplicates = remove", "Projection list = Manual".
How do I get query3 filled with the data item "customer" that only holds a list of customers which are solely appearing in query1?
In SQL terms, you want
select T2.C1
from T1
left outer join T2 on T1.C1 = T2.C1
where T2.C1 is null
So, in the query pane of a Cognos report...
Use a regular join.
Join using customer from both queries.
Change the cardinality to 1..1 on the query1 side and 0..1 on the query2 side.
In the filters for query3, add a filter for query2.customer is null.
EXCEPT is not a join. It is used to compare two data sets.
https://learn.microsoft.com/en-us/sql/t-sql/language-elements/set-operators-except-and-intersect-transact-sql?view=sql-server-2017
What you need is an INNER JOIN. That would be the join tool in the Toolbox in Cognos.

How to deal with Null Values in OUTER JOINS using Entity Framework 6

I need to process the query at the bottom of this post in C#. The query works, but I don't know how to use it in EF6. I used a method and a viewmodel for it (variable query = the query below). But when it encounters null values in the OUTER JOIN, int32 cant accept this value when calling .toList(). What's the best way to deal with it?
var result = context.Database.SqlQuery<TourQueryViewModel>(query);
var reslist = result.ToList();
I tried my first steps with LINQ, but I dont get it how to translate into LINQ itself or the query-methods, that are equivalent to it. So I am hoping for your help.
SELECT toursdata.TourId AS TourId, toursdata.Tourname AS Tourname,toursdata.Tourdate Tourdate,
toursdata.VehicleId AS VehicleId, toursdata.VehicleName AS VehicleName, toursdata.LicenseNumber AS LicenseNumber,
Employees.EmployeeId AS EmployeeId, Employees.Gender AS Gender, Employees.Forename AS Forename, Employees.Surname AS Surname
FROM (
SELECT te.TourId, te.Tourname, te.Tourdate,
Vehicles.VehicleId, Vehicles.VehicleName, Vehicles.LicenseNumber,
TourEmployees.EmployeeId
FROM TourEmployees RIGHT OUTER JOIN Tours AS te ON TourEmployees.TourId = te.TourId,
Tours AS tv INNER JOIN Vehicles ON tv.VehicleId = Vehicles.VehicleId
WHERE tv.TourId = te.TourId
) toursdata
LEFT OUTER JOIN Employees ON toursdata.EmployeeId = Employees.EmployeeId
To eliminate the null problem, I changed the data type of the corresponding entity-attribute to a nullable-type
int turned to int?.
Didnt know about that language feature.

Unusual Joins SQL

I am having to convert code written by a former employee to work in a new database. In doing so I came across some joins I have never seen and do not fully understand how they work or if there is a need for them to be done in this fashion.
The joins look like this:
From Table A
Join(Table B
Join Table C
on B.Field1 = C.Field1)
On A.Field1 = B.Field1
Does this code function differently from something like this:
From Table A
Join Table B
On A.Field1 = B.Field1
Join Table C
On B.Field1 = C.Field1
If there is a difference please explain the purpose of the first set of code.
All of this is done in SQL Server 2012. Thanks in advance for any help you can provide.
I could create a temp table and then join that. But why use up the cycles\RAM on additional storage and indexes if I can just do it on the fly?
I ran across this scenario today in SSRS - a user wanted to see all the Individuals granted access through an AD group. The user was using a cursor and some temp tables to get the users out of AD and then joining the user to each SSRS object (Folders, reports, linked reports) associated with the AD group. I simplified the whole thing with Cross Apply and a sub query.
GroupMembers table
GroupName
UserID
UserName
AccountType
AccountTypeDesc
SSRSOjbects_Permissions table
Path
PathType
RoleName
RoleDesc
Name (AD group name)
The query needs to return each individual in an AD group associated with each report. Basically a Cartesian product of users to reports within a subset of data. The easiest way to do this looks like this:
select
G.GroupName, G.UserID, G.Name, G.AccountType, G.AccountTypeDesc,
[Path], PathType, RoleName, RoleDesc
from
GroupMembers G
cross apply
(select
[Path], PathType, RoleName, RoleDesc
from
SSRSOjbects_Permissions
where
Name = G.GroupName) S;
You could achieve this with a temp table and some outer joins, but why waste system resources?
I saw this kind of joins - it's MS Access style for handling multi-table joins. In MS Access you need to nest each subsequent join statement into its level brackets. So, for example this T-SQL join:
SELECT a.columna, b.columnb, c.columnc
FROM tablea AS a
LEFT JOIN tableb AS b ON a.id = b.id
LEFT JOIN tablec AS c ON a.id = c.id
you should convert to this:
SELECT a.columna, b.columnb, c.columnc
FROM ((tablea AS a) LEFT JOIN tableb AS b ON a.id = b.id) LEFT JOIN tablec AS c ON a.id = c.id
So, yes, I believe you are right in your assumption

Using RoR with a legacy table that uses E-A-V

I'm needing to connect to a legacy database and pull a subset of data from a table that uses the entity-attribute-value model to store a contact's information. The table looks like the following:
subscriberid fieldid data
1 2 Jack
1 3 Sparrow
2 2 Dan
2 3 Smith
where fieldid is a foreign key to a fields table that lists custom fields a given customer can have (e.g. first name, last name, phone). The SQL involved is rather hairy as I have to join the table to itself for every field I want back (currently I need 6 fields) as well as joining to a master contact list that's based on the current user.
The SQL is something like this:
select t0.data as FirstName, t1.data as LastName, t2.data as SmsOnly
from subscribers_data t0 inner join subscribers_data t1
on t0.subscriberid = t1.subscriberid
inner join subscribers_data t2
on t2.subscriberid = t1.subscriberid
inner join list_subscribers ls
on (t0.subscriberid = ls.subscriberid and t1.subscriberid = ls.subscriberid)
inner join lists l
on ls.listid = l.listid
where l.name = 'My Contacts'
and t0.fieldid = 2
and t1.fieldid = 3;
How should I go about handling this with my RoR application? I would like to abstracat this away and still be able to use the normal "dot notation" for pulling the attributes out. Luckily the data is read-only for the foreseeable future.
This is exactly what #find_by_sql was designed for. I would reimplement #find to do what you need to do, something like this:
class Contact < ActiveRecord::Base
set_table_table "subscribers_data"
def self.find(options={})
find_by_sql <<EOS
select t0.data as FirstName, t1.data as LastName, t2.data as SmsOnly
from subscribers_data t0 inner join subscribers_data t1
on t0.subscriberid = t1.subscriberid
inner join subscribers_data t2
on t2.subscriberid = t1.subscriberid
inner join list_subscribers ls
on (t0.subscriberid = ls.subscriberid and t1.subscriberid = ls.subscriberid)
inner join lists l
on ls.listid = l.listid
where l.name = 'My Contacts'
and t0.fieldid = 2
and t1.fieldid = 3;
EOS
end
end
The Contact instances will have #FirstName and #LastName as attributes. You could rename them as AR expects too, such that #first_name and #last_name would work. Simply change the AS clauses of your SELECT.
I am not sure it is totally germane to your question, but you might want to take a look at MagicModel. It can generate models for you based on a legacy database. Might lower the amount of work you need to do.

Resources