I work with a senior developer who is a guru .NET architect. We have had many constructive arguments over the last 6+ months and generally I concede defeat in most of our discussions. I have learned stacks from working with him. There is one design problem we are currently in disagreement with, however, and I would like some opinions/suggestions because he hasn't managed to convince me of his position, and I will stick to my guns until someone can give me evidence that I am wrong.
We use Entity Framework 4.0 and we use BOTH persistence aware AND Self Tracked Entities in different models. We started using Self Tracked Entities for tracking changes to Entity Graphs that we serialised/deserialised over the WCF wire to our Silverlight application. It works fantastically. We have also started using Self Tracked Entities for models that we are NOT taking across WCF, but many remain as persistent aware Entities/Models.
My co-worker believes that Entity Frameworks ObjectContext should be kept around for the shortest time possible. He insists that it should only be around for the length of time needed to do a query and the length of time needed to persist something. Any business work done with Entities should be done detached. For each Entity Model we have, we have a query class and a persistent class that are both IDisposable and have the ObjectContext at instance scope and have the query/persistence logic in methods. We use these query/persistence classes in our business logic rather than using the the ObjectContext directly in business logic. When these class instances are constructed so is the ObjectContext and when Disposed, so is the ObjectContext Disposed. These classes are like wrappers around our LINQ operations separating EF from the business logic and assisting with LINQ query re-use.
Now first consider non-Self Tracked Entities:
I understand why he wants this and the desire to not have a long running ObjectContext, but my problem is that he always wants even trivial business logic to be detached from the ObjectContext and the fact that we have a separate query and persistence context under our design means that there is no ObjectContext state tracking of Entities being used in our business logic. For non-Self Tracked Entities this means that if we modify Entities in our business logic we must also set the Modified state of the entities manually prior to persisting. This is a REAL pain when persisting complex graphs with multiple changes. Also I doubt we can do it manually as well as EF does it automatically.
For our Self Tracked Entities this situation is the same because tracking is only turned on when graphs are de-serialised and so when working WITHIN a service where the query was performed, detached from the Context, there is still no tracking with Self-Tracked Entities.
My question is, what is the best practice for Entity Framework use and how should an Entity Framework DAL be designed? How long should the ObjectContext be around? Should business logic always be done detached from the ObjectContext? If so how do we do state tracking when working detached from the ObjectContext? One option I am thinking about is converting ALL of our entities to Self-Tracked Entities and using some graph traversal code to traverse queried graphs and turn tracking on for all of the Entities in the graph so Self Tracking is turned on even when working service side (basically mimicking what happens when the Self-Tracked Entities graph is de-serialised)...
I am not proposing that we keep the ObjectContext around for long periods of time, but working detached between query and persistence and losing the benefits of the ObjectContext state tracking to me seems silly...we are losing one of the great benefits of EntityFramework...
Sorry for long post...any help appreciated.
Microsoft recommends that the ObjectContext be short lived, and my experience has led me to believe that the life-span of an ObjectContext should ideally correlate to the duration of a web-request / user operation.
I made the mistake of trying to use a global, long-lived ObjectContext in a desktop application, and it was a total disaster. If you're going to implement things like caching, under/redo edit semantics, etc. then you should consider a design that uses ViewModel classes which are distinctly seperate from the underlying data entities. Use the EntityFramework to help you construct your Model object graph, then build the ViewModel from that. When you want to save changes, allow your ViewModel Repository to reconnect to the underlying entities, alter their properties and save changes. This allows for a more "cachable", flexible, ORM independant, unit-test-friendly design.
In terms of change tracking, try not to think of it as a cheap "user-made-a-change" mechanism for the UI, but rather as an expensive mechanism for tracking which recently-created entities need to be considered when saving changes.
Related
I am in the process of making project decisions on development patterns for a solution, which involves the use of Entity Framework 6 as the ORM choice,
and ASP.NET MVC 5.
I need insight on how transactions and business logic will be implemented. In respect to layers, I came to an initial assumption for the design where
Entity Framework on top of SQL Server can be considered the Data Access Layer (DAL). On top of Entity Framework, there will be a Service Layer, where business logic and validation will be implemented. On top of the Service Layer, I will have ASP.NET MVC Controllers consuming what the service layer offers.
Let me ellaborate on this initial conclusion drawn as a starting point for defining the architecture:
I want to follow principles to achieve the minimum complexity scenario possible, in respect to layers, abstractions and all the solution components responsibilities. As an excercise, with this "simplicity" in mind, I could just embrace the template "proposed" by Microsoft as when you just create a new Visual Studio ASP.NET MVC Web application, but I believe that does not fit the minimum design scenario needed for an enterprise application, since in Microsoft's template, the controller directly makes use of Entity Framework DbContext and consumes the Data Access Layer, besides the fact that no service layer is present. This leads to several issues, such as extremely tight coupling
between the presentation and data access layer, as well as the so called "fat controller" problem, where controllers become the bloated piece of the software with all the added responsibilities of business logic, transactions, and so on, making it truly a mess to software maintainability with, for example, the most basic principle of DRY (don't repeat yourself) being violated since you would get duplicated code and logic all over your fat controllers.
Ramping up the next stage on the path from simplicity to complexity, I assume it is fair to add a Service Layer to the design, because this way ASP.NET MVC controllers would talk only to this service layer, who would be responsible for all CRUD and validation of CRUD operations, and all other more complex business logic operations. This Service Layer then would talk to the data access layer being represented by Entity Framework.
I would stop there and say the design with these proposed layers is enough, but that's where I need more insight on how to proceed. I need to resolve the question on how would transactions be implemented, if you think of them as a wrapper for a series of individual operations performed by methods responsible for validation and business logic residing in classes inside the service layer. In terms of implementation using Entity Framework, if I get every individual operation performed by a service layer method to issue a .SaveChanges(), I would lose the ability of having DBContext to behave like a Unit of Work, wrapping up a single .SaveChanges() for many individual DBSet operations. In this case, the DBSets may behave like repositories. Many people argue that Entity Framework's DBContext and DBSet are implementations if Unit of Work and Repository pattern, respectively.
In a more objective question then, how can I implement these patterns using directly DBContext and DBSet, without any further abstraction into new generic or specific entity repository classes and unit of work generic class? The implementation needs to rely on the consumption of a service layer for the reasons I have already stated.
I think an answer to that would be just the last complexity leap I feel necessary to get my "least complex viable design".
Let me put a more concrete example to illustrate:
In my service layer, I have 2 methods to implement validation logic for 2 insert operations, with a programmer defined method Insert such as:
EntityOneService.Insert
EntityTwoService.Insert
Each of these methods in their corresponding service layer classes would have access to a DBContext and use DBSet.Add to signal they should be persisted,
in case all validation and/or business logic passes. The desired scenario is that I can use each service layer method call in an isolated way, and/or in groups, such as in a new different service layer class method, such as:
OperationOnePlusTwoService.Insert
This new method would implement calls to EntityOneService.Insert and EntityTwoService.Insert, IN A TRANSACTION-LIKE FASHION.
By transaction-like I mean that all calls must succeed, not violating any validation or business rule, in order to have the persistence layer to commit the operations.
DBContext.SaveChanges() apparently would have to be called only once for this to happen, OUTSIDE of any service layer Insert method implementation. In the
context of an ASP.NET Controller consuming service layer methods, how could that be achieved, without actual implementation of a Unit of Work and Repostory abstraction over DBContext and DBSet?
Any advice please would be very much appreciated. I am not posting this to argue the value of a real abstraction and implementation of Repository and Unit of Work patterns, or if Entity Framework's DBContext and DBSet are or are not equivalent to proper Repository and Unit of Work patterns, that's not the point. My project requirements do not involve in any way the need to decouple the application from Entity Framework, or to ideally and fully promote testability.
These are not concerns and I am well aware of consequences and future maintainability impacts on not adopting full fledged implementations of half a dozen layers and all design patterns possible to make a big world-class enterprise solution.
desired scenario is that I can use each service layer method call in an isolated way ... but that behave IN A TRANSACTION-LIKE FASHION.
This is rather simple with EF, assuming your services are not remote (in which case transactions are not advisable in the first place).
In the simplest implementation, each service instance requires a DbContext to be passed in its constructor, and contributes its changes to that DbContext. The orchestrating controller code can control the lifetime of the DbContext and control its use of transactions.
In practice interfaces, rather than concrete service types are typically used, and Dependency Injection is often used instead of constructors. But the pattern is the same.
eg
using (var db = new MyDbContext())
using (var s1 = new SomeService(db))
using (var s2 = new SomeOtherService(db))
using (var tran = db.Database.BeginTransaction())
{
s1.DoStuff();
s2.DoStuff();
tran.Commit();
}
David
Well, not sure if that's exactly the right title, but basically I have been having a lot of problems using repositories in MVC applications in such a way that you can substitute one set of repositories, implementing a different data storage technology, for another.
For example, suppose I want to use Entity Framework for my application. However, I also want to have a set of test data implemented in hard-coded Lists. I would like to have a set of interfaces (IUserRepository, IProductRepository, etc. -- let's not talk about a more generic IRepository<T> for now) that both approaches can instantiate. Then, using (say) a Dependency Injection tool such as Ninject or Castle Windsor, I can switch back and forth between the entity framework provider (accessing the actual database) and the test provider (accessing the lists).
In a nutshell, here's the problem:
-- If you are going to use Entity Framework, you want your repositories returning IQueryable<SomeType>.
-- If you are going to use hard-coded lists, you do NOT want your repositories returning IQueryable, because it adds hugely to the overhead, and plus, Linq to Entities is significantly different from Linq to Objects, causing many headaches in the code that is common to both providers.
In other words, I have found that the best approach isolates all the EF-dependent code within the repositories, so that the repositories themselves return IEnumerable or IList or some such -- then both EF and some other technology can use the same repositories. Thus, all the IQueryable's would be contained WITHIN the EF repositories. That way, you can use Linq to Entities with the EF repositories, and Linq to Objects with the Test repositories.
Yet this approach puts an enormous amount of the business logic into the repositories, and results in much duplicated code -- the logic has to be duplicated in each of the repositories, even if the implementations are somewhat different.
The whole idea of the repositories as this layer that is very thin and just connects to the database is then lost -- the repositories are "repositories" of business logic as well as of data store connectivity. You can't just have Find, Save, Update, etc.
I've been unable to resolve this discrepancy between needing to isolate provider-dependent code, and having business logic in a centralized location.
Any ideas? If anyone could point me to an example of an implementation that addresses this concern, I would be most appreciative. (I've read a lot, but can't find anything that specifically talks about these issues.)
UPDATE:
I guess I'm starting to feel that it's probably not possible to have repositories that can be swapped out for different providers -- that if you are going to use Entity Framework, for example, you just have to devote your whole application to Entity Framework. Unit tests? I'm struggling with that. My practice to this point has been to set up a separate repository with hard-coded data and use that for unit testing, as well as to test the application itself before the database is set up. I think I will have to look to a different solution, perhaps some mocking tool.
But then that raises the question of why use repositories, and especially why use repository interfaces. I'm working on this. I think determining the best practice is going to take a bit of research.
What I can say? Welcome to the club ...
What you found is problem reached by many developers who followed "repository boom" with EFv4. Yes it is the problem and the problem is really complex. I discussed this several times:
ASP.NET MVC 3 and Entity Framework code first architecture
Organizationally, where should I put common queries when using Entity framework
Separate topic is why to use repositories:
Generic repository, what is the point
Basically your proposed way is a solution but do you really want it? In my opinion the result is not repository but the Data Access Object (DAO) exposing plenty of access methods. Repository definition by Martin Fowler is:
A Repository mediates between the
domain and data mapping layers, acting
like an in-memory domain object
collection. Client objects construct
query specifications declaratively and
submit them to Repository for
satisfaction. Objects can be added to
and removed from the Repository, as
they can from a simple collection of
objects, and the mapping code
encapsulated by the Repository will
carry out the appropriate operations
behind the scenes. Conceptually, a
Repository encapsulates the set of
objects persisted in a data store and
the operations performed over them,
providing a more object-oriented view
of the persistence layer. Repository
also supports the objective of
achieving a clean separation and
one-way dependency between the domain
and data mapping layers.
I believe exposing IQueryable fulfils this 100 times better then creating a public interface similar to repositories from Stored procedures era - one access method per stored procedure (fixed query).
The problem can be summarized by the rule of leaky abstraction. IQueryable is an abstraction of the database query but the features provided by IQueryable are dependent on the provider. Different provider = different feature set.
What is a conclusion? Do you want such architecture because of testing? In such case start using integration tests as proposed in first two linked answers because in my opinion it is the lest painful way. If you go with your proposed approach you should still use integration tests to verify your repositories hiding all EF related logic and queries.
MVC total noobe here but long time web developer - 10 + years. The tutorials for MVC 3 (and earlier versions) are great but as usual they lack a ton for real on the job type scenarios.
For example, how often do you find yourself in a situation where you are going to create a new database from scratch with no stored procs so that you could actually use EF Code-First. I don't know about you but in my career it has been NEVER.
The usual story is that you are creating a new app or enhancing an existing app with new functionality that will connect to an existing very mature database with tons of stored procs, user defined functions and views and you are required either by management or time restrictions to use it all. And of course you may get to create some new tables but they usually will have joins to existing tables or in the least your app will have to query existing tables for some of the data.
To see a tutorial based on that scenario would be WORTH IT'S WEIGHT IN GOLD. Especially the stored procedure scenario.
Thank you for any advice
Most of the earlier examples (e.g. the original NerdDinner) were based on either Linq to Sql or Entity Framework (without CodeFirst). Since CodeFirst is the 'new hotness' most of the latest examples use it.
The interesting part of the question, though, is that it highlights an important point: "it doesn't matter". Your data access strategy (EF, EF code first, NHibernate, L2S, raw SQL) is totally irrelevant to MVC. By that I don't mean it's unimportant, I mean that MVC doesn't place any constraints on you at all in that regard.
You will generally (in a well-designed MVC app) pass your controllers interfaces which let them call data access methods of various sorts (or perhaps another layer of indirection with services that do other things before hitting the storage layer). The implementation of the data access, if it is using an ORM like EF or Nhibernate, will then have mechanisms for you to either use a query syntax of some sort (e.g. LINQ) or to call stored procedures (possible in all major ORMs that Iv'e used) or to push raw SQL if hte situation calls for it.
So as I understand it with good loose coupling I should be able to swap out my DAL with a couple lines of code at the application root.
I have 2 DAL written, Linq-to-sql and a JSon file repository (for testing and because I wanted to try out the System.Web.Scripting.JavascriptSerializer).
linq to sql will create entities instead of my business models. and feed them upwards through an IRepository which is using constructor injection at the application root.
my JSon layer doesn't have any autogenerated classes from which to deserialize so I'm lost as to a simple way to have it depend on an interface or abstract class and still function.
This question is based on the following assumptions/understandings:
I believe I would need the linq to sql layer to implement an interface so the application domain at compile time can dictate that the entity classes are going to have a place to read/write all the current model's fields
Any Business logic dictates a need for another set of classes with almost the same names and same properties in the model layer
Then conversion methods that take the DALs objects and translate them to business objects and back would be needed. (even if both sides are implementing the same interface this seems very inefficient)
This code is yet another place that would have to make a change if the model class or interface changed (interface, business class, view, dal entity)
Deserialization of any alternative DALs requires I create 'entities' with the same properties and fields in that layer(more duplication)
So to meet all of the flexibility/agility goals it appears I need an interface for each application domain/business object, a concrete class where business logic can live, and DAL objects that implement the interface (this means layers that don't autogenerate entities would have to be hand coded pure duplication).
How would I use loose coupling without a ton of duplication and loss of DRY?
Welcome to the beautiful and exciting world of loosely coupled code :)
You understand the problem correctly, but let me first reiterate what you are already implying: The Domain Model (that is, all Domain classes) must be defined independently of any specific Data Access technology, so you can't use auto-generated LINQ to SQL (L2S) classes as a basis for your Domain classes for the simple reason that you can't really reuse those together with other technologies (as you have found out with your JSON-based Repository).
Interfaces for each Domain object is not even going to help you, because to avoid Anemic Domain Models you will need to implement behavior in the Domain classes (and you can't put behavior into interfaces).
This means that to hydrate and dehydrate Domain objects you must have some mapping code. It has always been like this: in the old days we had to map from IDataReader instances to Domain classes, while now we need to map from Data (L2S) classes to Domain classes.
Could we wish for something better? Yes. Can we get something better? Probably. The next version of the Entity Framework will support Persistence Ignorance for exactly this reason: you should be able to define your Domain model as POCOs and if you provide a map and a database schema, EF will take care of the rest.
Until that arrives Microsoft doesn't have anything that offers this kind of functionality, but NHibernate does (caveat: I have zero experience with NHibernate, but lots of smart people say that this is true, and I trust them on that). This is a major reason that so many people prefer NHibernate over EF.
Loose coupling requires lots of mapping, so I can only second queen3's suggestion of employing AutoMapper for this kind of tedious work.
As a closing note I do want to point out a related issue: Mapping doesn't necessarily imply a violation of DRY. The best example is when it comes to strongly typed ViewModels that correspond to a given Domain object. Don't be fooled by the semantic similarity. They may have more or less the same properties with the same values, but their responsibilities differ vastly. As an application grows, you will likely experience that little divergences sneak in here and there, and you will be glad you have that separation of concerns - even if it initially looked like a lot of repetitious work.
In any case: Loose coupling is more work at the beginning, but it will enable you to keep on evolving an application where a tightly coupled application would freeze in maintenance hell long before. You are in for the long haul, but instant gratification it ain't.
Not that I understand the problem correctly, but to solve duplicated classes you may use AutoMapper.
Note that you may apply mapping declaratively or using reflection, i.e. semi-automatically. For example see here - this is not about data layer, but shows how simple attributes can help to automate mapping. In that case MVC applies attributes, but you may invent your own engine that looks for [Entity("Order")] attribute and applies AutoMapper.
Also you cannot have 100% persistence independency with just "couple of lines". ORM selection plays big role here. For example Linq-To-SQL cannot use plain classes (POCOs) so it's not as easy to re-use them as with NHibernate, for example. And with Repository you're going to have many queries in the data layer; different ORMs usually have different query syntax or implementation (even Linq not always compatible between ORMs) so switching data access can be a matter of replacing data layer completely, which is not couple of lines (unless your app is "Hello, world!").
The solution with AutoMapper above is actually a kind of self-baked ORM... so maybe you need to consider a better ORM that suites your requirements? Why don't you use EF4, especially given that it supports POCOs now, and is very similar to Linq-to-SQL, at least with query language?
I am looking for some guidance on how to incorporate business rules into an asp.net mvc application and how they relate to the model.
First a little background so we know what kind of solutions are relative for this question. At work we use WinForms, MVP, BusinessObjects, DataAccessObjects, and DataTransferObjects. The boundaries of the layers use DTOs to send parameters to methods and as return types, or return List types.
Right now we are adding a facade layer to translate the DTOs into Domain Objects for the UI to work with, since the architect does not like how using DTOs in the PresentationLayer is working currently. I am comfortable about all of this in theory aside from it being practical or not.
I am making a website for fun, but for considerations lets say it serves the same amount of traffic as SO, something like 60,000 hits a month last I heard. I am comfortable with the mechanics of the controllers and the views, and how the model integrates with the two.
I am using NerdDinner as a sample for building the site and I follow the Repository pattern implementation in the examples. What I don't get is how to incorporate business objects into the mix.
I hear people talk about LINQ as the DataAccessLayer/DataAccessObjects. If I force all of my requests though the business objects as I am used to I have introduced some weird dependencies. Both my UI and my BO have to know about my DAO.
What would kind of make sense is to use the LINQ classes as a true DAO layer, hide it behind the BO, and have the BO transform between POCO and LINQ objects.
My only concern there is I am fine with binding my UI to LINQ classes, and don't really need all the extra work, I am happy with a thin lightweight approach as in NerdDinner.
So what I have essentially is the Repository that is instantiated in the controllers that takes and return LINQ objects. My business objects have static methods that just take LINQ classes and perform some calculation, say apply a certain states tax %, or w/e.
Since a lot of these calculations have to be done across the results of the repository I am thinking of combining them into one central area, like a facade layer, but one that just does transforms against the data and not translating to other objects sets (DomainObjects <-> DTOs).
Should I do that, or should I say that those business methods really are part of my model and that they should be in the repository methods that return the objects?
From a design standpoint I would design it like this. Of course naming is just for the purpose of this post you don't have to name your DAL and BLL ..Repository and ..Service.
Have repositories (or one) where your data access/queries should be happening. It should ideally just contain queries (compiled or not). I personally have a repository for each data type to help keep queries separated.
The next layer should be your business layer which I like to call services. These classes are responsible for all logic regarding validation, prep steps and anything else needed to be done to get the consumer of the service the information it needs. As with an ASP.NET MVC app I have my services return view models which are then directly passed into strongly-typed views. With my services I usually group them logically together instead of one for each data type.
This is a great design because it keeps your data access code and presentation code nice and thin and most of the logic where things can go wrong is in your service (or business) layer.