I'm writing automation to black box test applications. The applications will not work with mock data, so I'll need to grab or create real data from the database using pre-written stored procedures that I have no control over. There are a lot of stored procedures.
I'm also using onion architecture and DI to abstract away the CRUD-ing of my entities, be it through stored procedures or API's. Right now I have 5 layers: entities, repository interfaces, repositories, service interfaces, and services.
Lets say in my test I want to create an entity. I would use a call to an IEntityService, service.Create(entity). This service would then make use of some of the stored procedures, such as is_entity_valid, and create_entity to validate and create the entity. Again since I am merely testing, I do not control the business logic or how it is implemented, I just need to be able access the things that do implement it.
My question is this: How do I fit those stored procedures into my layers? Should they go in a repo? A service? Somewhere else?
Right now I'm thinking of dumping related sp's into classes in the repo layer, then calling them from whatever repo implementations need them. Am I on the right track?
Related
As generic repositories can be use for Insert, Update, Delete etc. but it is limited only for simple Crud operations, if I want to use my generic repositories for some more complex insertion and searching like joining of tables and retrieve by stored procedures. And insert data to multiple tables using stored procedures so how will I handle it from generic repositories?
1. Can I call stored procedures from my generic repository?
2. Is it a good idea to use generic repositories + stored procedures?
3. Will it compatible for criteria base complex search?
Your question should be more specific in order to give you a right answer, but here's my two cents:
Yes it is possible to call stored procedures from your repository. If you are using Entity Framework, it is possible to map stored procedures to a CLR object, which you can call from your repository. Depending on the framework (or lack of it) you use, you'll need to figure out the right syntax to do so.
Personally I wouldn't use this approach as I'd rather put business logic in the business layer. However in some cases it will definitely be useful, for instance when performance is ultra important or where you want complete control over your queries, or some other complex operations.
Do keep in mind that you do can use LINQ to build your complex queries (joining, ordering, filtering, aggregating, selecting,, ..). Some parts can even be automated, like the WHERE clause using PredicateBuilder, enabling you to dynamically create and/or queries based on user input, search criteria,..
In general, YES it is possible but it all depends on the context where you're working in.
In situations where the client consuming a web service is looking for data that at present matches a database entity one to one (ie. GetAccount, GetTransactions); we still want to use a data transfer object (dto) to decouple the two to allow the database entity to change if needed, without the changes rippling through the entire system? If so, from a consuming standpoint, we don't want a client's model directly based on the data transfer object. We would want to cast whatever was being returned from the web service to a local model, for the same reasons we used a DTO in the first place? Do I have that right?
The DTO matches the shape of the data that you want to transfer. If that matches the underlying data model then so be it but that is by the by. You might at some point change the shape of the underlying data model but the DTOs would not change.
Also, a DTO is what would get passed by a web service to a client. If you have your data model defined on a server then you don't want a distributed client to have to know about that data model.
Finally, your entity classes might contain additional logic whereas a DTO is simply properties exposing data and nothing else.
Your application can have many conceptual layers. Data Layer, Business Layer, Service Layer, Presentation Layer to name a few. Each has a specific function with an input and output. The goal is how to design a system that is loosely coupled (i.e. things don't rely heavily on one another so any changes are isolated and have minimum impact). You want to maximise maintainability, improve re-use and maximise speed. The trick is to keep complexity down but build in some basic separation that allows you to move your design forward easily as you need it. Keep in mind YAGNI! YOU AINT GONNA NEED IT! People tend to look for and design the most complicated architectures for problems they will never encounter! The number of times I have heard "We have a separate data tier so we can swap out the database in the future"...yet I have never seen it happen! A simple Repository Pattern works.
As per previous poster, you have data in the database (in a physical model) that you want to get into your application transformed into a conceptual model. Any clients will then consume your conceptual model. I would do this immediately at or after your application code has retrieved the data from the database from your data layer. This ensures any changes made to your database are isolated from the rest of your application. Therefore you don't have to start changing your User Interface because you have changed the name of a database field. You just have to update your data layer.
If you choose Entity Framework as your data access strategy it will map your database to a conceptual model for you. If you are converting other data objects from one type to another look at technologies like AutoMapper. You can map your Entity Framework POCOs to a service contract (XSD). From the client perspective 'you could' write a client proxy that calls your web service, maps the returned service contract to a nice shared conceptual object model.
Keep it simple. Keep re-use and simplicity at the heart of what you choose. If you start seeing database column names in your client objects, you need to start thinking about putting in some abstractions in much earlier in your solution to insulate your app from changes.
We are doing a new project, for all devices and browsers compatibility we have decided to use asp.net mvc 4, Html5, css 3, for communicating with Database Entity Framework we want to use.
Our senior members(Manager, DBA(they are also new to mvc 4, EF)) in the team asking us to write every thing will be in the stored procedures while communicating Database so that maintenance becomes easy.
Is it the correct match if we go like that(MVC4+ EF + stored procedures)? Will i not get maintenance and performance if i go with Code first reverse engineering(because database tables are ready i want to do like that), Please reply.
Below is the flow we want to do, please correct me
As Database is already ready, so first we will write the stored procedures for communication with DB.
New Mvc 4 project and will add .edmx file(EF) and select tables and Stored procedures
in mvc controller or web api we write the consuming stored procedures
There is nothing technically wrong with ASP.NET MVC + EF + Stored Procedures approach, from the first sight.
But my experience show, is that typically it's huge overkill. The common problem I see is the conflicting interests between developers and DBA's. In most worst scenarios all DB releated stuff are controlled by DBA, so if developer what to add/change some feature he needs to wait for implementation of it by DBA (or wait for approve, which could also take long).
So, I personally see that as more bureaucratic way of development.
My own perpective is to be more agile on development and tools like Code First matches that. Stored Procedures could still play major role, while code/performance optimization, but not something to start with.
I agree that using stored procedures in the database is a good approach. Centralizing data validation and calculations in the database ensures data integrity. Client-side validation is important for the user experience but you must also ensure that you test the data validity in the database.
Using Entity Framework, you can generate entities which relate directly to tables in your database, or else you can design entities which use procedures for insert/update/delete operations rather than simple table updates.
In MVC you will use the entities as models to manage your data interactions.
Good luck
This is my personal view. I am sure others might have different ones. Since you are asking this question I am hoping you are open for discussions, otherwise I wouldn’t have bothered as this topic is like a religious discussion as lots of people have very strong opinions and are not likely to change them.
Personally I don't think stored procedures are meant to write business logic. They should be used for writing data access logic. I would only use a stored procedure if I want to optimize an expensive query such as a dynamic search but nothing else. You will get slightly less performance if you have your logic in the domain model, but its not even noticeable in most situations.
One of the strong arguments for writing business logic in stored procedures is because you can easily change some logic by changing your stored procedure. But should we really go and change the business logic of a deployed application without doing proper testing. What will happen if you accidently do a mistake? Doing a deployment is not such a big deal now with continuous builds and I don’t think as a professional developer you should take that risk.
When you decide to write your logic in stored procedures, you give up all the object oriented concepts and you end up writing some procedural code that we wrote maybe 10 years ago. C# language has come a long way now and you will not be able to use those new language features in heart of your application which is business logic. You also loose the visual studio features to refactor code, advanced and easy debugging features etc.
I also don’t like the idea of having triggers as it’s not visible in source code. Imagine someone new in your team trying to add a new feature some time later and if he doesn’t know that a trigger exists, he might write some incorrect logic.
If your application contains some complex business logic, (I am sure most applications do) you should have a domain model that contains not only just properties of your entities, but also your logic. Otherwise you will be falling in to the anti-pattern called anemic data model.
You will not be able to test your business logic by writing unit testing if you have your logic in stored procedures.
You will also not be able to deploy your business logic to multiple servers if you have them in stored procedures if your site becomes really successful.
You will also not be using all powerful capabilities of Entity framework and LINQ if you have all your logic in your stored procedures. You actually don’t need an ORM Mapper if that is the approach you are going to take.
This is what I would recommend for your project.
Even though you already have the database, you can still use code first approach of Entity framework. You can download the EF code first reverse engineer power tool and have the code first code auto generated for you. This is going to be a one off thing and after than if you have any more changes, you can directly do to the database and update the code first code accordingly. Fluent API is bit confusing at first, but you can easily learn that from the generated code.
Do not access your data context from the controller. Have a repository layer that will contain all your data access logic. You can access the repository from your controller. (This allows you to unit test your code by mocking the repository). There are lots of video tutorials on how to use the repository pattern on asp.net site.
Your domain model is going to be the entities that got generated from the Entity framework. Try to have your business logic in those models. It takes a little while to get use to the domain model pattern. But one you get used to it you will start to appreciate its benefits.
Hope this helps.
I'm building an MVC3 app, trying to use IoC and constructor injection. My database has (so far) about 50 tables. I am using EF4 (w/ POCO T4 template) for my DAC code. I am using the repository pattern, and each table has its own repository. My service classes in my service layer are injected w/ these repositories.
Problem: My service classes are growing in the number of repositories they need. In some cases, I am approaching 10 repositories, and it's starting to smell.
Is there a common approach for designing repositories and service classes such that the services don't require so many repositories?
Here are my thoughts, I'm just not sure which one is right:
1) This is a sign I should consider combining/grouping my repositories into related sections of tables, reducing the number or dependent repositories per service class. The problem with this approach, though, is that it will bloat and complicate my repositories, and will keep me from being able to use a common interface for all repositories (standard methods for data retrieval/update).
2) This is a sign I should consider breaking my services into groups based on my repositories (tables). Problem with this is that some of my service methods share common implementation, and breaking these across classes may complicate my dependencies.
3) This is a sign that I don't know what I'm doing, and have something fundamentally wrong that I'm not even able to see.
UPDATE: For an idea of how I'm implementing EF4 and repositories, check out this sample app on codeplex (I used version 1). However, looking at some of the comments there (and here), looks like I need to do a bit more reading to make sure this is the route I want to take -- sounds like it may not be.
Chandermani is right that some of your tables might not be core domain classes. This means you would never search for that data except in terms of a single type of parent entity. In those cases you can reference them as "complex types" rather than full-blown entities, and EF will still take care of you.
I am using the repository pattern, and each table has its own repository
I hope you're not writing these yourself from scratch.
The EF 4.1 already implements the Repository Pattern (DbSet), and the Unit of Work pattern (DbContext). The older versions do too, though the DbContext template can easily be tweaked to provide a clean mockable implementation by changing those properties to an IDbSet.
I've seen several tutorial articles where people still write their own, though. It is strange to me, because they usually don't provide a justification, other than the fact that they are "implementing the Repository Pattern".
Writing wrappers for these repositories for access methods like FindById make it slightly easier to access, but as you've seen is a big amount of effort potentially little payback. Personally, unless I find that there is interesting domain logic or complex queries to be encapsulated, I don't even bother and just use Linq directly against the IDbSet.
My service classes in my service layer are injected w/ these repositories.
Even if you choose to use custom query wrappers, you might choose to simply inject the DbContext, and let the service code instantiate the wrappers it needs. You'd still be able to mock your data access layer, you just wouldn't be able to mock up the wrapper code. I'd still recommend you inject less generic ones though, because complex implementation is exactly the type of thing you'd like to be able to factor out in maintenance, or replace with mocks.
If you look at DDD Aggregate Root pattern and try to see you data in this perspective you would realize that many of the table do not have a independent existence at all. Their data is only valid in context of their parent. Most of the operations on them require you to get the parent as well. If you can group such tables and find the parent entity\repository all other child repository can be removed. The complexity of associating the parent child which till now you would be doing in your business layer (assuming you are retrieving parent and child using independent repo) not would be shifted to the DAL
Refactoring the Service interface is also a viable option, and any common functionality can be moved into a base class and\or can be itself defined as a service which is consumed by all your existing services (Is A vs Has A)
#Chandermani has a good point about aggregate roots. Repositories should not, necessary have a 1:1 mapping to tables.
Getting large numbers of dependencies injected in is a good sign your services are doing too much. Follow the Single Responsibility Principle, and refactor them into more manageable pieces.
are your services writing to all of the repositories? i find that my services line up pretty closely with repositories, that they provide the business logic around the CRUD operations that the repository expose.
Well, not sure if that's exactly the right title, but basically I have been having a lot of problems using repositories in MVC applications in such a way that you can substitute one set of repositories, implementing a different data storage technology, for another.
For example, suppose I want to use Entity Framework for my application. However, I also want to have a set of test data implemented in hard-coded Lists. I would like to have a set of interfaces (IUserRepository, IProductRepository, etc. -- let's not talk about a more generic IRepository<T> for now) that both approaches can instantiate. Then, using (say) a Dependency Injection tool such as Ninject or Castle Windsor, I can switch back and forth between the entity framework provider (accessing the actual database) and the test provider (accessing the lists).
In a nutshell, here's the problem:
-- If you are going to use Entity Framework, you want your repositories returning IQueryable<SomeType>.
-- If you are going to use hard-coded lists, you do NOT want your repositories returning IQueryable, because it adds hugely to the overhead, and plus, Linq to Entities is significantly different from Linq to Objects, causing many headaches in the code that is common to both providers.
In other words, I have found that the best approach isolates all the EF-dependent code within the repositories, so that the repositories themselves return IEnumerable or IList or some such -- then both EF and some other technology can use the same repositories. Thus, all the IQueryable's would be contained WITHIN the EF repositories. That way, you can use Linq to Entities with the EF repositories, and Linq to Objects with the Test repositories.
Yet this approach puts an enormous amount of the business logic into the repositories, and results in much duplicated code -- the logic has to be duplicated in each of the repositories, even if the implementations are somewhat different.
The whole idea of the repositories as this layer that is very thin and just connects to the database is then lost -- the repositories are "repositories" of business logic as well as of data store connectivity. You can't just have Find, Save, Update, etc.
I've been unable to resolve this discrepancy between needing to isolate provider-dependent code, and having business logic in a centralized location.
Any ideas? If anyone could point me to an example of an implementation that addresses this concern, I would be most appreciative. (I've read a lot, but can't find anything that specifically talks about these issues.)
UPDATE:
I guess I'm starting to feel that it's probably not possible to have repositories that can be swapped out for different providers -- that if you are going to use Entity Framework, for example, you just have to devote your whole application to Entity Framework. Unit tests? I'm struggling with that. My practice to this point has been to set up a separate repository with hard-coded data and use that for unit testing, as well as to test the application itself before the database is set up. I think I will have to look to a different solution, perhaps some mocking tool.
But then that raises the question of why use repositories, and especially why use repository interfaces. I'm working on this. I think determining the best practice is going to take a bit of research.
What I can say? Welcome to the club ...
What you found is problem reached by many developers who followed "repository boom" with EFv4. Yes it is the problem and the problem is really complex. I discussed this several times:
ASP.NET MVC 3 and Entity Framework code first architecture
Organizationally, where should I put common queries when using Entity framework
Separate topic is why to use repositories:
Generic repository, what is the point
Basically your proposed way is a solution but do you really want it? In my opinion the result is not repository but the Data Access Object (DAO) exposing plenty of access methods. Repository definition by Martin Fowler is:
A Repository mediates between the
domain and data mapping layers, acting
like an in-memory domain object
collection. Client objects construct
query specifications declaratively and
submit them to Repository for
satisfaction. Objects can be added to
and removed from the Repository, as
they can from a simple collection of
objects, and the mapping code
encapsulated by the Repository will
carry out the appropriate operations
behind the scenes. Conceptually, a
Repository encapsulates the set of
objects persisted in a data store and
the operations performed over them,
providing a more object-oriented view
of the persistence layer. Repository
also supports the objective of
achieving a clean separation and
one-way dependency between the domain
and data mapping layers.
I believe exposing IQueryable fulfils this 100 times better then creating a public interface similar to repositories from Stored procedures era - one access method per stored procedure (fixed query).
The problem can be summarized by the rule of leaky abstraction. IQueryable is an abstraction of the database query but the features provided by IQueryable are dependent on the provider. Different provider = different feature set.
What is a conclusion? Do you want such architecture because of testing? In such case start using integration tests as proposed in first two linked answers because in my opinion it is the lest painful way. If you go with your proposed approach you should still use integration tests to verify your repositories hiding all EF related logic and queries.