We have controllers that read Entities with certain criteria and return a set of view models containing the data to the view. The view uses a Kendo Grid, but I don't think that makes a particular difference.
In each case, we have a Linq Query that gets the overall collection of entity rows and then a foreach loop that creates a model from each row.
Each entity has certain look ups as follows:
those with a 1:1 relationship, e.g. Assigned to (via a foreign key to a single person)
those with a 1:many relationship e.g. copy parties (to 0:many people - there are not many of these)
counts of other relationships (e.g. the number of linked orders)
any (e.g. whether any history exists)
If we do these in the model creation, the performance is not good as the queries must be run separately for each and every row.
We have also tried using includes to eager load the related entities but once you get more than two, this starts to deteriorate too.
I have seen that compiled queries and LoadProperty may be an option and I am particularly interested in the latter.
It would be great to understand best practice in these situations, so I can direct my investigations.
Thanks,
Chris.
Edit - sample code added. However, I'm looking for best practice.
public JsonResult ReadUsersEvents([DataSourceRequest]DataSourceRequest request, Guid userID)
{
var diaryEventModels = new List<DiaryEventModel>();
var events = UnitOfWork.EventRepository.All().Where(e => e.UserID == userID);
foreach (var eventItem in events)
{
var diaryModel = new DiaryEventModel(eventItem);
diaryEventModels.Add(diaryModel);
}
var result = diaryEventModels.ToDataSourceResult(request);
return Json(result, JsonRequestBehavior.AllowGet);
}
public DiaryEventModel(Event eventItem) {
// Regular property from Entity examples - no issue here as data retrived simply in original query
ID = eventItem.ID;
Start = eventItem.StartDateTime;
End = eventItem.EndDateTime;
EventDescription = eventItem.Description;
// One to one looked up Property example
eventModel.Creator = eventItem.Location.FullName;
// Calculation example based on 0 to many looked up properties, also use .Any in some cases
// This is a simplified example
eventModel.AttendeeCount = eventItem.EventAttendees.Count();
// 0 to Many looked up properties
EventAttendees = eventItem.EventAttendees.Select(e => new SelectListItem
{
Text = e.Person.FullName,
Value = e.Person.ID.ToString()
}).ToList();
}
Related
I have the following layout for my solution.
Data Layer
Entities Layer (POCO's to create code-first db)
Services Layer (Web API)
Web Layer (MVC Layer for presentation)
I have created models in the Web API Layer that mimic the entities in the Entities layer so I can more easily reference properties. I have those called Models. I would like to have the API layer do the data work in case I want to upgrade the Web Layer later or go a different direction. I have used the VS 2019 Controller creation tool and I have referenced my Services Layer Model as the model and the Data Context from my Data Layer. I get the following error:
"Error CS0012 The type 'Customer' is defined in an assembly that is not referenced. You must add a reference to assembly 'XX.Entities, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.XX.Services"
I can see why I get that, the Model in the Service layer (though matches property wise) does not match the namespace. How does one reference the Data Layer without using something like AutoMapper? It seems like I am missing something obvious, but maybe not.
If your Services layer is going to contain your logic and interact with your entities, and your MVC Controllers are largely anemic, passing through to the Services, it sounds like you want your Services to return Models (DTOs or ViewModels) which decouple the data returned from your entities.
To do this, Services will need a reference to your Entities because that is what your data layer is. Ideally the data layer should be returning IQueryable<TEntity> as opposed to IEnumerable<TEntity> or even TEntity so that the services can refine queries for efficiency without bringing back more data than they need, or adding lots of complexity or a large number of similar single-purpose methods in the data layer.
I'm not sure what your aversion to Automapper is, but it is perfectly suited to handling the conversion and copying of data between Entity and ViewModel. You can certainly do it without Automapper using Linq's Select method.
For instance, to get a list of OrderModels out of your service:
public IEnumerable<OrderModel> GetOrdersForCustomer(int customerId, int pageNumber, int pageSize)
{
var orders = OrderRepository.GetByCustomerId(customerId)
.OrderByDescending(x => x.CreatedAt)
.Select(x => new OrderModel
{
OrderId = x.OrderId,
OrderNumber = x.OrderNumber,
CreatedAt = x.CreatedAt,
// ... et al,
OrderItems = x.OrderItems.Select( oi => new OrderItemModel
{
OrderItemId = oi.OrderItemId,
ProductId = oi.Product.ProductId,
ProductName = oi.Product.Name,
Quantity = oi.Quantity,
UnitPrice = oi.Product.Price,
// ...
}).ToList()
}).Skip(pageNumber*pageSize)
.Take(pageSize)
.ToList();
return orders;
}
Clumsy, but flexible and gets the job done. With Automapper mapping configured:
public IEnumerable<OrderModel> GetOrdersForCustomer(int customerId, int pageNumber, int pageSize)
{
var orders = OrderRepository.GetByCustomerId(customerId)
.OrderByDescending(x => x.CreatedAt)
.ProjectTo<OrderModel>()
.Skip(pageNumber*pageSize)
.Take(pageSize)
.ToList();
return orders;
}
The key to something like this working efficiently in terms of performance and memory use is that OrderRepository.GetCustomerById(int) returns IQueryable<Order> not IEnumerable<Order> This allows the Select/ProjectTo and Skip & Take to be compiled down to the SQL which returns just the columns needed to populate your models.
ViewModels/DTOs do not need to match their Entity counterparts 1:1. You only need to include fields you know the consumers will use. This helps protect your schema but also can streamline the amount of data that gets sent over the wire, boosting performance and reducing server memory usage. You can define as many view models as you need for an entity, and use inheritance to extend them if desired.
When going the other way, such as performing an insert or update, Automapper can help simplify data transitions, and even cover rules to ensure that data that should not be changed, never gets changed. You can load entity graphs in your service similar to above, but using Include to pre-fetch related data that might be updated and selecting the particular entity(ies) to update. With Automapper you can handle both insert scenarios and updates:
Inserts:
var newOrder = _mapper.Map<Order>(orderViewModel);
_context.Orders.Add(newOrder);
_context.SaveChanges();
Updates:
var existingOrder = _context.Orders.Single(x => x.OrderId = orderViewModel.OrderId);
_mapper.Map(orderViewModel, existingOrder);
_context.SaveChanges();
... and for singular entities you're pretty much done. A little more work may be needed for updating related entities with the Order.
I'm trying to declare DbSet in my Databases Context class.
Because I don't which class (Entity Framwork Class) user calls.
That's why I want to declare the DbSet with dynamic.
I would try to have everything explicitly defined in your context. There's a lot of functionality baked into that property declaration, not the least of which is the design-time benefit of having types.
There are many techniques for dynamic querying, but I'll post one that I've used in the past for dynamically querying lookup tables.
public List<LookupItemDTO> LoadItems(int lookupId)
{
//nothing special here, just map the lookup id to a name
var lookupRepo = new LookupRepository<LookupDefinition>(uow);
var lookup = lookupRepo.Find(id);
//map the lookup's name from the table to a type. This is important DO NOT ACCEPT A TABLE NAME FROM THE USER.
var targetType = Type.GetType("MyApp.DomainClasses.Lookup." + lookup.Name + ",MyApp.DomainClasses");
// this is an optional extra step but it allows us to instantiate an instance of our repository to ensure that business rules are enforced
var lookupType = typeof(LookupRepository<>).MakeGenericType(targetType);
dynamic LookupTable = Activator.CreateInstance(lookupType);
//just some pattern nonsense to wire up the unit of work
LookupTable.SetupUOW(uow);
//make sure to cast it into something you can work with ASAP
var data = ((IQueryable<LookupTableBase>)LookupTable.All).OrderByDescending(l => l.IsVisible).ThenBy(l => l.SortOrder);
var list = from li in data
select new LookupItemDTO
{
Id = li.Id,
Description = li.Description,
Display = li.Display,
SortOrder = li.SortOrder,
IsVisible = li.IsVisible
};
return list.ToList();
}
The key here is that you can dynamically query tables, but it's better to do it at a higher level. Also, have some kind of indirection between you and your user's input. Let them select a table from a list, use the ID from that table to look up the name. My solution here is the have a lookup table definition table. Any dynamic queries start there first and then values from that definition table are used to build the necessary types.
This is using BreezeJS and a Breeze controller through to an EF provider. I have a couple of related Entities, lets call them Customer, which has a Navigation Property called Orders which links to a set of Order Entities for that customer.
What I'd like to display on the UI is a summary of Order Counts for a set of customers who match a partial name search. I can do this through returning all the Order objects, but they're quite large objects and I don't really want to return 100's of them when I don't have to. The inlineCount() method seems to always give the count of the top-level entity (Customer) rather than of the sub-Entities, no matter where I place it in the statement.
var predicate = breeze.Predicate.create('displayName', 'contains', partialName);
return this.entityQuery.from('Customers')
.where(predicate)
.orderBy('displayName')
.using(this.manager)
.expand('Orders')
.execute();
The documentation suggests that you can chain the expand in some way, but I have yet to find a syntax which is valid.
Ideally, I'd like to apply a where to the Orders by a property on Order called Status of say 0 (incomplete) and then give me just the count of those matching Orders. ie, return me all the Customer entities, but have a matching order count for each (rather than the whole list of Order objects and filter client-side).
Would appreciate any pointers in the right direction if it's possible to achieve. My current thinking is that I'll have to create a custom method on the server-side controller and do the work there, but before I make assumptions about what OData can support, I thought I'd check here for some confirmation.
So far, this is my best approach (maybe someone can correct me if there's a better way).
On the server, add this method:
public IQueryable<object> CustomerSummaries()
{
return Context.Customers.Select(p => new
{
Customer = p,
ActiveOrderCount = p.Orders.Count(o => o.Status == 1)
});
}
Then on the client end:
var predicate = breeze.Predicate.create('customer.displayName', 'contains', partialName);
return this.entityQuery.from('CustomerSummaries')
.where(predicate)
.using(this.manager)
.execute();
I'm trying to limit the amount of data coming across when implementing Lookup Lists in Breeze.JS. The Lookup Lists sample uses queries that results in full objects but I would like to project the entities to fewer properties (e.g. primary key, foreign keys, and a descriptor property) and still have Breeze.JS recognize the entity type on the client. I know how to do the projections from the client to get partials but how would I do that with Lookup Lists (either from the client or the server Web API)?
You might satisfy your intentions with a custom JsonResultsAdapter.
You're probably wondering "What is a JsonResultsAdapter?"
That's what breeze uses to interpret the JSON arriving from the server. You can read about here and here.
Perhaps more helpful is to look at the adapter in the Web API dataservice and at the example adapter from the "Edmumds" sample.
The Edmunds sample demonstrates translating a JSON source that you don't control into breeze entities.
In this case, your JsonResultsAdapter would look at each node of JSON and say "this is a Foo, this is a Bar, and that one is a Baz". Accordingly, for each of these nodes it would return { entityType: "Foo" }, return { entityType: "Bar" }, and return { entityType: "Baz" }
Now breeze knows what to do and creates corresponding entities out of the Lookups payload.
Remember to mark these entities as partial, in the same way you would if you had made a projection query that targeted a single entity type.
Fortunately, the Lookups query returns the container object that holds the Foo, Bar, and Baz collections. So you can iterate over these and mark them partial right there in the query success callback.
Once you wrap your head around THAT ... you'll want to know how to put your custom JsonResultsAdapter to work in the Lookups query ... and ONLY in the Lookups query.
You can enlist that JsonResultsAdapter exclusively for your Lookups query with a using clause.
Here's an example:
var jsa = new breeze.JsonResultsAdapter({
name: 'myLookupsJsa',
visitNode: function() {...}
});
query = query.using(jsa);
Is this overkill? Would you be better off making three trips?
Only you will know. I would like to hear from you when you try it ... and give us your suggestions on how we might make this easier in a general way.
In the lookup lists example the controller action looks like this:
[HttpGet]
public object Lookups() // returns an object, not an IQueryable
{
var regions = _contextProvider.Context.Regions;
var territories = _contextProvider.Context.Territories;
var categories = _contextProvider.Context.Categories;
return new {regions, territories, categories};
}
You could reduce the footprint using a server-side projection like this:
[HttpGet]
public object Lookups() // returns an object, not an IQueryable
{
var regions = _contextProvider.Context.Regions
.Select(x => new { id = x.RegionId, name = x.RegionName });
var territories = _contextProvider.Context.Territories
.Select(x => new { id = x.TerritoryId, name = x.TerritoryName });
var categories = _contextProvider.Context.Categories
.Select(x => new { id = x.CategoryId, name = x.CategoryName });
return new {regions, territories, categories};
}
This approach does not answer this part of your question:
and still have Breeze.JS recognize the entity type on the client
Not sure what the solution or use case is for this piece.
I am refactoring an MVC project to make it testable. Currently the Controller uses the Entity Framework's context objects directly to ask for the required data. I started abstract this and it just doesn't work. Eventually I have an IService and an IRepository abstraction, but to describe the problem let's just look at the IRepository. Many people advise an interface with functions which return some of these: IQueriable<...>, IEnumerable<...>, IList<...>, SomeEntityObject, SomeDTO. Then when one wants to test the service layer they can implement the interface with a class which doesn't go to the database to return these.
Problem: Using linq to entities I have lazy (deferred) loading in my toolset. This is actually very useful, because my controller action functions know which data they need for the view and I didn't ask for more than required. However linq to anythingelse doesn't have lazy loading. So when my IRepository functions return any of the above mentioned things I lose lazy loading. I extended my interface with functions like "GetAnything" and "GetAnythingDeep" but it's not enough: it has to be much more fine-grained. Which would result about 5-6 functions for the same type of object, depending on the properties I want to get in the result. Maybe could be a general function with some "include properties" parameter, but I don't like that too.
Eventually atm I think if I want to make it testable that will result either much less efficient or much more complicated code. Sounds not right.
Btw I was thinking about to change the data source behind the entity model to either xml or some object data soruce, and so I could keep the linq to entities. I found that it's not supported out of the box... which is also sad: this means that entity framework means database source - not a really useful abstraction.
Specific example:
Entity objects:
Article, Language, Person. Relations: Article can have 1-N languages, and one Person (publisher).
ViewModel object:
ArticleDeepViewModel: Contains all the properties of the article, including the languages and the Name of the Person (it's for view the article, so no need for the other properties of the person).
Controller action which will return this view should get the data from somewhere.
Code before modifications:
using (var context = new Entities.Articles())
{
var article = (from a in context.Articles.Include("Languages")
where a.ID == ID
select new ViewArticleViewModel()
{
ID = a.ID,
Headline = a.Headline,
Summary = a.Summary,
Body = a.Body,
CreatedBy = a.CreatedByEntity.Name,
CreatedDate = a.CreatedDate,
Languages = (from l in context.Languages select new ViewLanguagesViewModel() { ID = l.ID, Name = l.Name, Selected = a.Languages.Contains(l) })}).Single();
this.ViewData.Model = article;
}
return View();
Code after modifications could be something like:
var article = ArticleService.GetArticleDeep(ID);
var viewModel = /* mapping */
this.ViewData.Model = viewModel;
return View();
Problem is that GetArticleDeep should return an Article object with Languages included and the entire Person object included (it shouldn't know that the viewmodel needs just the Name of the Person). Also I have so far 3 different viewmodels for an article. For example if someone wants to see the list of articles, then it's unnecessary to get the languages, the body and some other properties, however it might be useful to get the Name of the publisher (which is in the deep). Before "testable" code the controller actions could just contain the linq to entities query and get whichever data they need using lazy loading, Include function, using subqueries, referencing foreign properties (Publisher.Name) ... So there is no unnecessary query to the database and no unnecessary data transferred from the database.
What should be the IService or IRepository interface provide to get the 3-4 different level of Article objects or sometimes list of these objects?
Not sure if you are planning to stick with lazy loading, but if you want a flexible way to integrate eager loading into your repository and service layers first check out this article:
http://blogs.msdn.com/b/alexj/archive/2009/07/25/tip-28-how-to-implement-include-strategies.aspx
He basically gives you a way to build a strongly-typed include strategy like this:
var strategy = new IncludeStrategy<Article>();
strategy.Include(a => a.Author);
Which can then be passed into a general method on your repository or service layers. This way you don't have to have a separate method for each circumstance (i.e. your GetArticleDeep method).
Here is an example repository method using the above include strategy:
public IQueryable<Article> Find(Expression<Func<Article, bool>> criteria, IncludeStrategy<Article> includes)
{
var query = includes.ApplyTo(context.Articles).Where(criteria);
return query;
}