Question
What is the proper way to share an EF DbContext across multiple repositories in an MVC web app? Is it prudent/necessary to do so, what are the pitfalls of doing or not doing this?
Background
Assume:
App.MvcSite (Several dozen controllers, multiple areas, etc)
App.Services (service layer, many services)
App.Data (many repositories, etc)
... etc excluded for simplicity (EF Code First latest)
Research To Date
I seem to find at least two/three schools of thought on SO and the interwebs.
Share/scope your DbContext to the Request so that a single request has a single DbContext shared by all repositories.
Share/scope your DbContext at the service layer -- the service maintains a single DbContext and passes it to each repository as needed.
Do not share a DbContext, since they are cheap let each Repo have its own.
In a small website this is a non-issue which is why most MS and community examples simply don't even address this.
In my experience thus far I have not used finite repositories. I have always had services use a DbContext and directly change it so I didn't need to worry about this. I'm told there is a great benefit to finite repositories from a unit testing perspective... we'll see if it makes the rest of this worthwhile.
My Thoughts
(1) Share/scope your DbContext to the Request
This is interesting as it smartly avoids the pitfall of a singleton context which some developers think is the answer but find DbContext doesn't work that way. But it seems to have a downside in that it assumes all repositories, services, etc are going to be in coordination across an entire request... this is often not the case, right? What if changes are saved by one repo before another completes its work. (outer(inner(inner)))
(2) Share/scope your DbContext at the service layer
This makes more sense to me because each service should be coordinating a specific unit of work (lower case intentional). So if multiple services were used in one request it would be proper (if not required) that each had its own context to the database.
(3) Do not share a DbContext, since they are cheap
This is the way I've always done it... well actually I almost always only had one DbContext per request because only one service was being called. Sometimes it might be two because two services were called by a controller who was coordinating the work. But given my current application, with many finite repositories, each repository having its own context would mean a given request might have 3-10 instances of DbContext. I assume (perhaps incorrectly) that this is problematic.
Repeating the question:
What is the proper way to share an EF DbContext across multiple repositories in an MVC web app? Is it prudent/necessary to do so, what are the pitfalls of doing or not doing this?
DbContext are cheap, but distributed transactions are not.
Objects attached to one context can't be used in another context (if you have object relations)
The easiest way to share a context is to start using an inversion of control container: http://www.codeproject.com/Articles/386164/Get-injected-into-the-world-of-inverted-dependenci
I would go for a combination between the first two options and regarding your take on the first option, don't let repositories save any changes (that's not the recommended way of doing things, Martin Fowler says that him self) and for that he introduced the Unit of Work pattern.
Related
Background
I am about to start the process of creating a new application with MVC 5 and EF6 and building it out with TDD. This is my first MVC application so i have decided to use it as a bit of a learning platform to better understand a whole range of patterns and methodologies that i have been exposed to but have only used in passing up until this point.
I started with this in my head:
EF - Model
Repositories
Services
UI (controllers views)
Removing the Repositories
I shifted this thinking to remove one layer, repositories simply as my understanding has grown i can see the EF (specifically IDbSet) implements a repository pattern or sorts and the context itself is a unit of work, so wrapping it around a further abstraction, for this application at least seems pointless, at that level anyway.
EF will be abstracted at the Service Layer
Removing the Repo's doesn't mean EF will be directly exposed to the controllers as in most cases i will use the services to expose certain methods and business logic to the controllers, but not exclusively exclude EF as i can use it outside of services to do things like building specific queries which could be used at a service level and a controller level, the service layer will simply be a simpler way of mapping specifics from the controller to the EF and data concerns.
This is where it gets a bit ropey for me
Service Layer
My services feel a little bit like repositories in the way they will map certain functions (getById etc), which i am not sure is just naturally the way they are or if my understanding of them is way off and there is more information that i can't find to better my knowledge.
TDD & EF
I have read a ton of stuff about the EF and how you can go about testing with unit wise, how you shouldn't bother as the leakyness of IQueryable and the fact that Linq-to-entities and Linq-to-objects means that you won't get the results that you intend all of the time, but this has led to simply confusing the hell out of me to the point where i have an empty test file and my head is completely blank because i am now over thinking the process.
Update on TDD the reason the TDD tag was included as i thought maybe someone would have an idea on how they approach something like this without a repository because that is an abstraction for abstractions sake. Would they not unit test against it and use other tests to test the query-able behavior like a integration test or end to end test? but from my limited understanding that wouldn't be TDD as the tests would not be driving my design in this instance?
Finally, To The Point
Is the:
EF
Service
UI
architecture a good way to go, initially at least?
Are there any good examples of a well defined service layer out there so i can learn, and are they in the main a way to map certain business operations that have data connotations to some for of persistence mechanic (in this case an ORM and EF) without having the persistence requirements of say a repository?
With the TDD stuff, is it ok to forgo unit tests for service methods that are basically just calling EF and returning data and just opting for slower integration tests (probably in a seperate project so they are not part of the main test flow and can be run on a more ad-hoc basis?
Having one of those weeks and my head feels like it is about to explode.
Lol I've had one of those weeks myself for sure. ;)
I've had the same kind of internal discussions over how to structure MVC projects, and my conclusion is find what's most comfortable to you.
What I usually do is create the following projects:
Core/Domain - here I have my entities/domain model, and any
other thing that may be shared among layers: interfaces, for
example, configuration, settings, and so on.
Data/EF - here
I have all my EF-dependent code: DataContext and Mappings
(EntityTypeConfiguration). Ideally I could create another
version of this using, say NHibernate and MySQL, and the rest of the
solution will stay the same.
Service - this depends on Core
and Data. I agree in the beginning it will look like a simple facade
to your Data, but as soon as you start adding features, you'll find
this is the place to add your "servicemodels". I'm not saying
ViewModel as this is quite Web-ui related. What i mean with
"ServiceModel" is creating a simpler version of your domain objects.
Real example: hide your CreatedOn, CreatedBy properties, for
example. Also, whenever one of your controller's actions grow to
anything over quite simplistic, you should refactor and move that
logic to the service and return to the controller what you really
need.
Web/UI This will be your webApp. It will depend on Core and Service.
You didn't mention dependency injection but you have to definitely look at it.
For testing, you can test your Data using a SqlCompact provider that re-creates the database for each test instead of using a full SqlExpress. This means your DataContext should accept a connectionString parameter. ;)
I've learned a lot seeing big projects source code, like http://www.nopcommerce.com. You could also have a look at http://sharparchitecture.net/ although I bet you already saw that.
Be prepared to have some nightmares with complex object graphs in EntityFramework. ;)
My final advice is: find something specific to do and dive in. Too much abstraction will keep you from starting, and starting is key to practice and understanding.
I'm building an MVC3 app, trying to use IoC and constructor injection. My database has (so far) about 50 tables. I am using EF4 (w/ POCO T4 template) for my DAC code. I am using the repository pattern, and each table has its own repository. My service classes in my service layer are injected w/ these repositories.
Problem: My service classes are growing in the number of repositories they need. In some cases, I am approaching 10 repositories, and it's starting to smell.
Is there a common approach for designing repositories and service classes such that the services don't require so many repositories?
Here are my thoughts, I'm just not sure which one is right:
1) This is a sign I should consider combining/grouping my repositories into related sections of tables, reducing the number or dependent repositories per service class. The problem with this approach, though, is that it will bloat and complicate my repositories, and will keep me from being able to use a common interface for all repositories (standard methods for data retrieval/update).
2) This is a sign I should consider breaking my services into groups based on my repositories (tables). Problem with this is that some of my service methods share common implementation, and breaking these across classes may complicate my dependencies.
3) This is a sign that I don't know what I'm doing, and have something fundamentally wrong that I'm not even able to see.
UPDATE: For an idea of how I'm implementing EF4 and repositories, check out this sample app on codeplex (I used version 1). However, looking at some of the comments there (and here), looks like I need to do a bit more reading to make sure this is the route I want to take -- sounds like it may not be.
Chandermani is right that some of your tables might not be core domain classes. This means you would never search for that data except in terms of a single type of parent entity. In those cases you can reference them as "complex types" rather than full-blown entities, and EF will still take care of you.
I am using the repository pattern, and each table has its own repository
I hope you're not writing these yourself from scratch.
The EF 4.1 already implements the Repository Pattern (DbSet), and the Unit of Work pattern (DbContext). The older versions do too, though the DbContext template can easily be tweaked to provide a clean mockable implementation by changing those properties to an IDbSet.
I've seen several tutorial articles where people still write their own, though. It is strange to me, because they usually don't provide a justification, other than the fact that they are "implementing the Repository Pattern".
Writing wrappers for these repositories for access methods like FindById make it slightly easier to access, but as you've seen is a big amount of effort potentially little payback. Personally, unless I find that there is interesting domain logic or complex queries to be encapsulated, I don't even bother and just use Linq directly against the IDbSet.
My service classes in my service layer are injected w/ these repositories.
Even if you choose to use custom query wrappers, you might choose to simply inject the DbContext, and let the service code instantiate the wrappers it needs. You'd still be able to mock your data access layer, you just wouldn't be able to mock up the wrapper code. I'd still recommend you inject less generic ones though, because complex implementation is exactly the type of thing you'd like to be able to factor out in maintenance, or replace with mocks.
If you look at DDD Aggregate Root pattern and try to see you data in this perspective you would realize that many of the table do not have a independent existence at all. Their data is only valid in context of their parent. Most of the operations on them require you to get the parent as well. If you can group such tables and find the parent entity\repository all other child repository can be removed. The complexity of associating the parent child which till now you would be doing in your business layer (assuming you are retrieving parent and child using independent repo) not would be shifted to the DAL
Refactoring the Service interface is also a viable option, and any common functionality can be moved into a base class and\or can be itself defined as a service which is consumed by all your existing services (Is A vs Has A)
#Chandermani has a good point about aggregate roots. Repositories should not, necessary have a 1:1 mapping to tables.
Getting large numbers of dependencies injected in is a good sign your services are doing too much. Follow the Single Responsibility Principle, and refactor them into more manageable pieces.
are your services writing to all of the repositories? i find that my services line up pretty closely with repositories, that they provide the business logic around the CRUD operations that the repository expose.
Well, not sure if that's exactly the right title, but basically I have been having a lot of problems using repositories in MVC applications in such a way that you can substitute one set of repositories, implementing a different data storage technology, for another.
For example, suppose I want to use Entity Framework for my application. However, I also want to have a set of test data implemented in hard-coded Lists. I would like to have a set of interfaces (IUserRepository, IProductRepository, etc. -- let's not talk about a more generic IRepository<T> for now) that both approaches can instantiate. Then, using (say) a Dependency Injection tool such as Ninject or Castle Windsor, I can switch back and forth between the entity framework provider (accessing the actual database) and the test provider (accessing the lists).
In a nutshell, here's the problem:
-- If you are going to use Entity Framework, you want your repositories returning IQueryable<SomeType>.
-- If you are going to use hard-coded lists, you do NOT want your repositories returning IQueryable, because it adds hugely to the overhead, and plus, Linq to Entities is significantly different from Linq to Objects, causing many headaches in the code that is common to both providers.
In other words, I have found that the best approach isolates all the EF-dependent code within the repositories, so that the repositories themselves return IEnumerable or IList or some such -- then both EF and some other technology can use the same repositories. Thus, all the IQueryable's would be contained WITHIN the EF repositories. That way, you can use Linq to Entities with the EF repositories, and Linq to Objects with the Test repositories.
Yet this approach puts an enormous amount of the business logic into the repositories, and results in much duplicated code -- the logic has to be duplicated in each of the repositories, even if the implementations are somewhat different.
The whole idea of the repositories as this layer that is very thin and just connects to the database is then lost -- the repositories are "repositories" of business logic as well as of data store connectivity. You can't just have Find, Save, Update, etc.
I've been unable to resolve this discrepancy between needing to isolate provider-dependent code, and having business logic in a centralized location.
Any ideas? If anyone could point me to an example of an implementation that addresses this concern, I would be most appreciative. (I've read a lot, but can't find anything that specifically talks about these issues.)
UPDATE:
I guess I'm starting to feel that it's probably not possible to have repositories that can be swapped out for different providers -- that if you are going to use Entity Framework, for example, you just have to devote your whole application to Entity Framework. Unit tests? I'm struggling with that. My practice to this point has been to set up a separate repository with hard-coded data and use that for unit testing, as well as to test the application itself before the database is set up. I think I will have to look to a different solution, perhaps some mocking tool.
But then that raises the question of why use repositories, and especially why use repository interfaces. I'm working on this. I think determining the best practice is going to take a bit of research.
What I can say? Welcome to the club ...
What you found is problem reached by many developers who followed "repository boom" with EFv4. Yes it is the problem and the problem is really complex. I discussed this several times:
ASP.NET MVC 3 and Entity Framework code first architecture
Organizationally, where should I put common queries when using Entity framework
Separate topic is why to use repositories:
Generic repository, what is the point
Basically your proposed way is a solution but do you really want it? In my opinion the result is not repository but the Data Access Object (DAO) exposing plenty of access methods. Repository definition by Martin Fowler is:
A Repository mediates between the
domain and data mapping layers, acting
like an in-memory domain object
collection. Client objects construct
query specifications declaratively and
submit them to Repository for
satisfaction. Objects can be added to
and removed from the Repository, as
they can from a simple collection of
objects, and the mapping code
encapsulated by the Repository will
carry out the appropriate operations
behind the scenes. Conceptually, a
Repository encapsulates the set of
objects persisted in a data store and
the operations performed over them,
providing a more object-oriented view
of the persistence layer. Repository
also supports the objective of
achieving a clean separation and
one-way dependency between the domain
and data mapping layers.
I believe exposing IQueryable fulfils this 100 times better then creating a public interface similar to repositories from Stored procedures era - one access method per stored procedure (fixed query).
The problem can be summarized by the rule of leaky abstraction. IQueryable is an abstraction of the database query but the features provided by IQueryable are dependent on the provider. Different provider = different feature set.
What is a conclusion? Do you want such architecture because of testing? In such case start using integration tests as proposed in first two linked answers because in my opinion it is the lest painful way. If you go with your proposed approach you should still use integration tests to verify your repositories hiding all EF related logic and queries.
I am building an MVC app using the Service Layer/Repository/Unit of Work pattern with EF4.
I am a bit confused on the logic. I know the point is to decouple the system, but I am a little confused.
So the MVC Controllers call on the Services to fill the View Models. So is it safe to say the MVC App is coupled to the Service Layer?
Then the Service Layer calls on the Repository to get and persist objects. Is then safe to say the Service Layer is dependent to the Repository?
The Repository the utilizes EF4 to get and persist data to SQL server, so I would assume the Repository depends on EF4 which in turn depends on SQL Server.
Where does the unit of work all fit in.
Any examples please?
Thanks!!
I started with hiding Unit of work somewhere in lower layer but it is wrong way to do that. After some experience my opinion is:
In case of monolitic application UnitOfWork should be accessible by Controller and lower layers.
In case of distributed application (UI and BL are on different servers) UnitOfWork should be accessible by business layer facade (service layer for remote calls) and lower layers.
The reason is that mentioned layers define what is the "business transaction" = what is current unit of work. Only this layer knows when it wants to commit changes to data store. Doing it this way allows service composition (code reuse). I discussed similar question here and here.
Sam,
Julie Lerman did a good screencast on DNR tv, talking about this, there is also another screen cast on Channel 9, around creating and testing repositories just EF here.
The general thing abut these is create the abstraction of the Unit of Work in Nhibernate it would be Session, in EF would be you context and passing that session or context into your repositories, as part of you test you can fake the connections to use a list of dictinary.
Hope these help.
Iain
You are correct in your assumptions on the layering. Your EF Context is the Unit Of Work. Generally you'll abstract this away through an interface and then constructor inject into each Repository for CRUD operations. Another approach is to expose your Repositories on the UoW interface (I prefer the former). Either way allows for easier unit testing of each layer. A single call to Save on the UnitOfWork from within the service layer will then persist all changes across all Repositories.
Here's a nice article on MSDN that looks at UoW from a unit testing perspective but covers repositories also. Where it references Repositories from the MVC Controller you'll have another intermediary service layer.
I have looked at simpler applications like Nerddinner and ContactManager as well as more complicated ones like Kigg. I understand the simpler ones and now I would like to understand the more complex ones.
Usually the simpler applications have repository classes and interfaces (as loosely coupled as they can get) on top of either LINQtoSQL or the Entity Framework. The repositories are called from the controllers to do the necessary data operations.
One common pattern I see when I examine more complicated applications like Kigg or Oxite is the introduction of (I am only scratching the surface here but I have to start somewhere):
IOC DI (in Kigg's case Unity)
Web Request Lifetime manager
Unit of Work
Here are my questions:
I understand that in order to truly have a loosely coupled application you have to use something like Unity. But it also seems like the moment you introduce Unity to the mix you also have to introduce a Web Request Lifetime Manager. Why is that? Why is it that sample applications like Nerddinner do not have a Web Request Lifetime Manager? What exactly does it do? Is it a Unity specific thing?
A second pattern I notice is the introduction of Unit of Work. Again, same question: Why does Nerddinner or ContactManager not use Unit of Work? Instead these applications use the repository classes on top of Linq2Sql or Entity Framework to do the data manipulation. No sign of any Unit of Work. What exactly is it and why should it be used?
Thanks
Below is a example of DI in Nerddiner at the DinnersController level:
public DinnersController()
: this(new DinnerRepository()) {
}
public DinnersController(IDinnerRepository repository) {
dinnerRepository = repository;
}
So am I right to assume that because of the first constructor the controller "owns" the DinnerRepository and it will therefore depend on the lifetime of the controller since it is declared there?
With Linq-to-SQL is used directly, your controller owns the reference to the data context. It's usually a private reference inside the controller, and so is created as part of its construction. There's no need in lifetime management, since it's in one place.
However, when you use IoC container, your data repository are created outside your controller. Since IoC container that creates it for you doesn't know how and how long you're going to use the created object, a lifetime strategy is introduced.
For example, data context (repository) is usually created at the beginning of the web request and destroyed at the end. However, for components that work with external web service, or some static mapper (e.g. logger) there's no need to create them each time. So you may want to say to create them once (i.e. singletone lifestyle).
All this happen because IoC container (like Unity) are designed to handle many situations, and they don't know your specific needs. For example, some applications use "conversation" transactions where NHibernate (or Entity Framework maybe) may last during several pages / web requests. IoC containers allow you to tweak objects lifetime to suit your needs. But as said this comes at price - since there's no single predefined strategy, you have to select one yourself.
Why NerdDinner and other applications do not use more advanced techniques is simply because they are intended to demonstrate MVC features, not advanced usages of some other libraries. I remember an article written to demonstrate one IoC container advanced functionality - this article broke some approved design patterns like separation of concerns - but this wasn't that important because design patterns were not the goal of the article. Same with simple MVC-demonstration-applications - they do not want you, the MVC newcomer, to be lost in IoC labyrinths.
And I would not recommend to look at Oxite as a design reference example:
http://codebetter.com/blogs/karlseguin/archive/2008/12/15/oxite-oh-dear-lord-why.aspx
http://ayende.com/Blog/archive/2008/12/19/oxite-open-exchangable-informative-troubled-engine.aspx
Most if not all of the DI containers touch the concept of life times, I believe. Depending on the scenario involved, you may want the container to always return the same instance of a registered component, while for another component, you may want it to always return a new instance. Most containers also allow you to specify that within a particular context, you want it to return the same instance, etc..
I don't know Unity very well (so far I have used Windsor and Autofac), but I suspect the web request lifetime manager to be an implementation of lifetime strategies where the same instance is provided by the container during the lifetime of a single web request. You will find similar strategies in containers like Windsor.
Finally, I suppose you are referring to Unit of Work. A Unit of Work is in essence a group of actions that you want to succeed or fail as one atomic business transaction. For a more formal description, look at Martin Fowler's definition. It is a concept that has gained more popularity in the context of Domain Driven Design. A unit of work keeps track of the changes you apply in such a transaction, and when the time is right, it commits these changes in one ACID transaction. In NHibernate e.g., the session supports the notion of unit of work and more specifically the change tracking, while in Linq2SQL it is the Context ...