I am using 10+ (depending of use case) calls to database, just to populate dropdown lists in my view.
Like in the code shown here:
ViewBag.FK_Grupe_Predmeta_ID = new SelectList(db.Get_Grupe_Predmeta(userLanguageID).ToList(), "ID", "Naziv");
ViewBag.FK_Klasifikacione_Oznake_ID = new SelectList(db.Get_Klasifikacione_Oznake(userLanguageID).ToList(), "ID", "Naziv");
ViewBag.FK_Stranke_ID = new SelectList(db.Strankes.Where(s => s.Povjerljiv == true), "ID", "Naziv");
ViewBag.FK_Prioriteti_ID = new SelectList(db.Get_Prioriteti(userLanguageID).ToList(), "ID", "Naziv");
ViewBag.odjeljenja = new SelectList(db.Get_Odjeljenja(userLanguageID).OrderBy(x => x.Naziv).ToList(), "ID", "Naziv");
ViewBag.VrstePostupka = new SelectList(db.Vrsta_Postupkas.ToList(), "PK_Vrsta_Postupka_ID", "Naziv_Vrste_Postupka");
I call stored procedures and tables. But when I test with multiple users in same time, I get long loading time, sometimes in the minutes.
Can't find any way to make less calls and populate ViewBag in same time.
Does anyone know what is the best approach in this situation?
Thanks
One suggestion rather than passing entire entity sets to the SelectList constructor would be to just send a projection of the fields needed. I'd probably avoid calling Stored Procedures unless absolutely necessary and let EF interact with the tables to build optimal queries.
var strankes = db.Stranks
.Where(x => x.Povjerljiv)
.Select(x => new { x.ID, x.Naziv });
ViewBag.FK_Stranke_ID = new SelectList(stranks, "ID", "Naziv");
For common lookups, you can project to a view model for the ID and Name(Naziv) and then consider caching those to session to avoid re-loading them every time the page loads. Your SelectList gets built from the cached view models rather than hitting the DB again.
Make sure your DbContext lifetime scope is set no longer than per-request. DbContexts that have been alive longer and tracking far more entities over time become a lot slower for retrieving and updating entities. The more it is tracking, the more cached entities it will be sifting through when you ask for something, then going to the DB to read more entities, then stitching those resulting entities against it's local cache. (for populating any related entities, etc.) For maximum performance when loading sets fast, consider a bounded context with a minimum of navigation properties between lookup entities (mapping just FK columns) and read via a locally scoped bounded DbContext.
Other performance pitfalls to watch out for are any places where entities might be serialized and sent to the view. Entities that have navigation properties to lookups etc. that get "touched" by a serializer can end up triggering lazy loading to load each related entity one level at a time. My recommendation is always to project entities down to ViewModels to send to the view to avoid traps like this. Due to the queries lazy load traps can generate they can look a lot like cases where lookup queries are being triggered. An SQL Profiler can be very helpful for spotting situations where a lot of queries are being generated unexpectedly as a page loads.
Related
I am building an application that will interface with my database backend using EF 6. I have my database built so I will be going the database first approach.
One of the goals will be to utilize the grid control from DevExpress and allow the user to view a Master/Detail style of information for each Facility in my Facility table.
I need the ability to update/insert so I would assume using a view would not be as good as actually supplying the context to the grid's data source.
My question is trying to understand the overhead involved with Entity Framework. If my table contains 20 columns but my Grid only needs 10 of these for viewing/updating/inserting does the application still load ALL of the information into memory? Am I better off using a view for the grid and some form of custom routines for an update?
It doesn't necessarily load all of the columns. If you're smart about it, you can only have it load those columns that you need using LINQ to Entities.
For example:
var records = db.MyTable.Where(...Some condition...).ToList(); will return all the columns.
But doing it this way
`var records = db.MyTable.Where(...Some condition...).Select(x => new MyModel { ...some fields here... }
will only load those fields that are specified.
I've worked with DevExpress and Entity Framework combined before, and I can tell you that it's relatively painless as long as you understand what you're doing with Entity Framework, and don't just blindly have your grids make queries that can get expensive.
Having an entity in cache, that entity has related entities in the form of a top 20.
Now a user action can update the top 20 on the server, and I thus would like to redownload the entire entity. Server sends the correct data with top 20, but in Breeze, I end up with a top 40... And I can't figure out how to avoid this behavior.
Thanks for the tip
Update: I do not use odata webapi and iqueryable, as it offers too much power to clients for my app. So I don't want to use EntityQuery.fromEntityKey, which seems to do what I want. I'd prefer to keep using a "normal" query, to which I add a parameter.
Update 2: To add more clarity as to why I want to prevent merge, when I recompute the top 20, I delete all related entries in the db and recreate them, so they have new Id's. So I am now considering an update, which might actually solve my issue BUT I would still like to know if merge can be prevented.
The Breeze EntityManager caches entities by primary key. So presumably your 2nd query is returning a completely new set of entities with each query. If this is the case, and you really only want the "latest" 20, the simplest fix would be to simply empty the EntityManager cache for this entity type before each query. Something like:
var entities = myEntityManager.getEntities(myEntityType);
entities.forEach(function(e) {
myEntityManager.detachEntity(e);
// or
// e.entityAspect.setDetached();
});
I am not sure what is exactly your qustion, but if you want to have different results on entity queries you may do it by creating 2 different controllers with different action names, or just with different parameters - overloaded methods. Then in your datacontext you can have 2 different queries.
You use .withParameters() propriety on the query to call the controller method with parameters.
Then in the controller method you can query and filter with LINQ, in any way you want. This way you can have different results based on the query/controller you chose to call.
Documentation: http://www.breezejs.com/documentation/querying-depth
I am constanstly having problems with model design in MVC. Mostly all of my projects require some entities that are to be created, edited and deleted. Most of these entities have some common properties like creation date, modified date, author, modified by and in some cases even more descriptive properties are common. Because of that I have one single DB table to store them. For instance, Documents table and we store Invoices, Quotations and other business documents in it. I am using Entity Framework v4 for ORM, where we eventually end up with the Document entity.
How do I modify this entity or do I create a separate DocumentViewModel class for it to support multiple document types with common properties (so some form of inheritance or interface implementation should be implemented)? Besides identifying different document types I also need to have some types to have different Data Annotation rules (attributes).
For instance, Document table has PayDate column. Document type Invoice requires PayDate to be provided but document type Quotation does not.
This is the one single problem I am facing over and over with MVC and so far I've been handling it different every time but cannot decide how to properly handle it to achieve the maximum application maintainability and ease of development.
Have you considered making Documents entity abstract?
From the DB side, you will have Documents table containing just the fields shared by all Invoices/Quoations/etc. This field will have an IDENTITY PK - e.g DocId.
In the other tables, additional meta-data specific to that document can be stored, and the PK is a (non-IDENTITY) field DocId, which is also a FK to the Documents table.
On the EF side, Documents becomes an abstract entity, and the other entities inherit from this entity. This allows a nice OO paradigm to exist, and makes your code more robust.
We are currently using this scheme (EF4/SQL Server).
Your scenario sounds very similar to ours - consider using Abstract Classes.
EDIT
Thought i'd add a bit more info to how i've actually implemented this scenario, to put you on the right track.
As the comments to your Q state, we have little knowledge of your domain, therefore it's hard to make informed opinions. Personally, i chose to make my entity abstract, because certain functionality required a "mixed-bag" of items to be returned in one hit. There are other ways to do this of course (such as a stored procedure), but this allows a nice fluent interface between my UI (which is MVC by the way) and my service layer.
Works like this - here's how i get a single Post:
// var is strongly-typed to a "Post"
var somePost = repository.FindSingle(10);
Here's how i get a mixed-bag of Posts:
// var is strongly-typed to a "ICollection<Post>".
// "Title" is a property on my "Post" abstract POCO
var mixedBagOfPosts = repository.FindAll<Post>(p => p.Title = "Some Title");
Here's how i get a collection of "Reviews" (child of Post):
// var is strongly-typed to a "ICollection<Review>"
// "Rating" is a property on my "Review" POCO (derived from Post)
var reviews = repository.FindAll<Review>(r => r.Rating == 5.00);
The kicker is my repository is implemented with generics, and the type parameter ensures type-safety:
ICollection<T> FindAll<T>(Expression<Func<T,bool>> predicate) where T : Post
And it's implemented like this:
return myContext.Posts.OfType<T>.Where(predicate).ToList();
The OfType causes an inner join to the T (which is the child table), so only those records are returned.
Of course, i also have a service layer mediating between my UI and repository, but this should get you on the right track.
Also, you don't have to go with the whole Expression-predicate thing, i like this because it minimizes the number of methods on my interface, and gives full querying power to my controllers, whilst ensuring the queries are deferred to the point of the service layer, but not further.
If you don't like this, you could of course have regular parameters (string title, etc).
As i said, this architecture suited my domain requirements, so it may not necessarily suit yours, but hopefully it gives you some insight.
You can put a table in your EF model many times and just rename them to the entities you need. Just delete the columns you dont need from those.
For example, put in a Documents entity... now rename it to Invoice. Now add another one, and name it Quotation... on the Quotation, click on PayDate and hit the delete key. You can customize these all you want! You can do the same with other ORMs, just takes a little more work. With NHibernate you would manually create the Models, then map them to the same table, but only map what you need.
Let's start with this basic scenario:
I have a bunch of Tables that are essentially rarely changed Enums (e.g. GeoLocations, Category, etc.) I want to load these into my EF ObjectContext so that I can assign them to entities that reference them as FK. These objects are also used to populate all sorts of dropdown controls. Pretty standard scenarios so far.
Since a new controller is created for each page request in MVC, a new entity context is created and these "enum" objects are loaded repeatedly. I thought about using a static context object across all instances of controllers (or repository object).
But will this require too much locking and therefore actually worsen perf?
Alternatively, I'm thinking of using a static context only for read-only tables. But since entities that reference them must be in the same context anyway, this isn't any different from the above.
I also don't want to get into the business of attaching/detaching these enum objects. Since I believe once I attach a static enum object to an entity, I can't attach it again to another entity??
Please help, I'm quite new to EF + MVC, so am wondering what is the best approach.
Personally, I never have any static Context stuff, etc. For me, when i call the database (CRUD) I use that context for that single transaction/unit of work.
So in this case, what you're suggesting is that you wish to retrieve some data from the databse .. and this data is .. more or less .. read only and doesn't change / static.
Lookup data is a great example of this.
So your Categories never change. Your GeoLocations never change, also.
I would not worry about this concept on the database/persistence level, but on the application level. So, just forget that this data is static/readonly etc.. and just get it. Then, when you're in your application (ie. ASP.NET web MVC controller method or in the global.asax code) THEN you should cache this ... on the UI layer.
If you're doing a nice n-tiered MVC app, which contains
UI layer
Services / Business Logic Layer
Persistence / Database data layer
Then I would cache this in the Middle Tier .. which is called by the UI Layer (ie. the MVC Controller Action .. eg. public void Index())
I think it's important to know how to seperate your concerns .. and the database stuff is should just be that -> CRUD'ish stuff and some unique stored procs when required. Don't worry about caching data, etc. Keep this layer as light as possible and as simple as possible.
Then, your middle Tier (if it exists) or your top tier should worry about what to do with this data -> in this case, cache it because it's very static.
I've implemented something similar using Linq2SQL by retrieving these 'lookup tables' as lists on app startup and storing them in ASP's caching mechanism. By using the ASP cache, I don't have to worry about threading/locking etc. Not sure why you'd need to attach them to a context, something like that could easily be retrieved if necessary via the table PK id.
I believe this is as much a question of what to cache as how. When your are dealing with EF, you can quickly run into problems when you try to persist EF objects across different contexts and attempt to detach/attach those objects. If you are using your own POCO objects with custom t4 templates then this isn't an issue, but if you are using vanilla EF then you will want to create POCO objects for your cache.
For most simple lookup items (i.e numeric primary key and string text description), you can use Dictionary. If you have multiple fields you need to pass and back with the UI then you can build a more complete object model. Since these will be POCO objects they can then be persisted pretty much anywhere and any way you like. I recommend using caching logic outside of your MVC application such that you can easily mock the caching activity for testing. If you have multiple lists you need to cache, you can put them all in one container class that looks something like this:
public class MyCacheContainer
{
public Dictionary<int, string> GeoLocations { get; set; }
public List<Category> Categories { get; set; }
}
The next question is do you really need these objects in your entity model at all. Chances are all you really need are the primary keys (i.e. you create a dropdown list using the keys and values from the dictionary and just post the ID). Therefore you could potentially handle all of the lookups to the textual description in the construction of your view models. That could look something like this:
MyEntityObject item = Context.MyEntityObjects.FirstOrDefault(i => i.Id == id);
MyCacheContainer cache = CacheFactory.GetCache();
MyViewModel model = new MyViewModel { Item = item, GeoLocationDescription = GeoLocations[item.GeoLocationId] };
If you absolutely must have those objects in your context (i.e. if there are referential entities that tie 2 or more other tables together), you can pass that cache container into your data access layer so it can do the proper lookups.
As for assigning "valid" entities, in .Net 4 you can just set the foreign key properties and don't have to actually attach an object (technically you can do this in 3.5, but it requires magic strings to set the keys). If you are using 3.5, you might just try something like this:
myItem.Category = Context.Categories.FirstOrDefault(c => c.id == id);
While this isn't the most elegant solution and does require an extra roundtrip to the DB to get a category you don't really need, it works. Doing a single record lookup based on a primary key should not really be that big of a hit especially if the table is small like the type of lookup data you are talking about.
If you are stuck with 3.5 and don't want to make that extra round trip and you want to go the magic string route, just make sure you use some type of static resource and/or code generator for your magic strings so you don't fat finger them. There are many examples here that show how do assign a new EntityKey to a reference without going to the DB so I won't go into that on this question.
I've got a page which will have about ten drop down lists which are generated from my SQL database. What's the best way to populate them? I was going to use a stored procedure with LINQ to return multiple result sets, but it seems a bit overkill. Is there another way of doing it? Using HtmlHelpers or something like that?
Seeing as though everyone seems to be confused by this, I will elaborate.
This isn't to do with caching, that's not what the problem states
This isn't to do with ASP.NET controls such as DropDownList, I tagged it asp.net-mvc
This isn't to do with code-behind models, I thought that was implicitly obvious by how I tagged this question originally as ASP.NET MVC
The problem is that multiple results sets are required on the page for drop down lists (think HTML!). So I have a drop down list for your favourite breed of badger, a drop down list for how many birthdays you've had, a drop down list for how many clouds are in the sky today. All of these are dynamically populated (please note, I am joking, this is a finance system I work on). I need all of them to be on my view page, but I'd rather NOT use the IMultipleResult return type in a LINQ stored procedure to bring back multiple result sets. It just gets messy.
So in basic, I want about 10 drop down lists on my view page, all of which are populated with data from a database (which constantly change). What is the best way to get them on the view?
I would just pass the required data to the view, either in multiple ViewData dictionaries or as a special view model if you want strongly typed access. Then use HtmlHelper.DropDownList() to display the actual drop downs.
Weakly typed solution
Controller
ViewData["Data1"] = SomeRepository.GetList();
ViewData["Data2"] = SomeRepositoty.GetList();
return View();
View
<%= Html.DropDownList("Data1") %>
<%= Html.DropDownList("Data2") %>
Strongly typed solution
View model
public class DataViewModel
{
public IEnumerable<string> Data1 { get; set; }
public IEnumerable<string> Data2 { get; set;}
}
Controller
var model = SomeRepository.GetModel(); // returns an instance of DataViewModel
return View(model);
View
<%= Html.DropDownList("Data1", new SelectList(Model.Data1)) %>
<%= Html.DropDownList("Data2", new SelectList(Model.Data2)) %>
It depends on the data, if the data in the database is not updated that often, then you could have a process that creates XML files once a day. Then use the XML files as the source to the dropdowns, this would speed up the application and limit the calls to the database server.
This is really a question of data management, not anything to do with HtmlHelpers until you get to the very pointy end of things. Anyhow, the first thing I would ask myself is "how is this data updated and what kind of filtering do I need?"
If this list is pretty much constant, then you could go for a bit of caching.
If this list is pretty fluid, then just pull it out of the database as needed and be done with it. Worry about caching if your DB box starts melting.
Either way you probably want to wrap things up in some sort of LookupService class, but there isn't enough to go on to make any more specific recommendations.
I second Wyatt's recommendation about grabbing from the DB each time if the list is fluid.
However, I'm doing a similar thing (with 10+ dropdowns on each page, all very static... the values will rarely change).
The first version of the application had a helper method for each dropdown list (about 20 total by the end of the project) that grabbed from the respective tables and cached via another helper. Before I started caching, there was no db context available for the view (i created it in the controller and didn't pass it), each dropdown had to create a new connection. This got noticeably slow. Plus, I had some problems with my caching routines, and saving them with 20 magic strings in the cache, etc. Also, I had a separate querying object where I had to manually build the relationships, and having to create the 20 relationships for inner joins was a pain.
So... my new version:
I'm using a single "selectables" table. There's a PK, and a "selectable type" (which I have to admit is a string). There's a selectables enumerable, which makes things a bit cleaner. There's a main getAllSelectables() method that looks for the entire result set in the cach (and gets all rows from the db if its not in the cache) and returns it. Then there's a getSelectables(enum) that grabs out only the relevant values, and a third function getSelectListItems(enum) that calls getSelectables(enum) and returns a ienumerable for the mvc helper function.
Hope that helps,
James