I'm building a Repository layer for my MVC application with methods like GetObject, UpdateObject, DeleteObject, etc.
This is what I have now:
public List<Object> GetObjects()
{
return _db.Objects.Where(o => o.IsArchived == false).ToList();
}
But I'm wondering if it would be better to return IQueryables for lists so that the least amount of data gets sent to the client when filters are applied in the UoW or Service layers. Would it be best to do something like this?
public IQueryable<Object> GetObjects()
{
return _db.Objects.Where(o => o.IsArchived == false);
}
The not nice thing about returning IQueryable, is that if you ever have a different implementation of repository, say using different ORM, storing data in non-SQL database, cloud or XML file, it would be hard to implement same interface. It would be much easier to implement if you return more generic colections of domain objects. For example IEnumerable. You can always pass filtering criteria in.
The other drawback of returning IQueryable, is that it may happen, that when you actually run the query your object context may be already disposed (Depending on your implementation) or may be kept in memory longer than required.
A leaky abstraction such as IQueryable could cause problems, for example imagine you want to get some data from database and order it by Guid. If you enumerate the query by calling ToList() prior to sorting, you'll get different results if you do it after. The reason is that in first case the sorting will happen in .NET, but in other case it will happen in SQL which uses completely different order.
The nice thing about returning IQueryable here is that you can continue to build up your query further without hitting the db. Once you call ToList it will hit the db and you can't customize your query further without hitting the database a second time.
Related
I watched this video and read this blog post. There is something in this post confused me; The last part of the post. In the last part Mosh emphasized, Repository should never return IQueryable, because it results in performance issue. But I read something that sounds contradictory.
This is the confusing part:
IEnumerable: While querying data from database, IEnumerable executes select query on server side, load data in-memory on client side and then filter data. Hence does more work and becomes slow.
IQueryable: While querying data from database, IQueryable executes select query on server side with all filters. Hence does less work and becomes fast.
this is another answer about IQueryable vs IEnumerable in Repository pattern.
These are opposite of Mosh's advice. If these are true, why we should not use IQueryable instead of IEnumerable.
And something else, What about situations that we want to use OData; As you know it’s better to use IQueryable instead of IEnumerable when querying by OData.
one more thing, is it good or bad to use OData for querying e-Commerce website APIs.
please let me know your opinion.
Thank you
A repository should never return a IQueryable. But not due to performance. It's due to complexity. A repository is about reducing the complexity in the business layer.
Buy exposing an IQueryable you increase the complexity in two ways:
You leak persistence knowledge to the business domain. There is things that you must know about the underlying Linq to Sql provider to write effective queries.
You must design the business entities so that querying them is possible (i.e. not pure business entities).
Examples:
var blockedUsers = _repository.GetBlockedUsers();
//vs
var blockUsers = _dbContext.Users.Where(x => x.State == 1);
var user = _repos.GetById(1);
//and an enum is used internally in the user class
user.Block();
_repos.Update(user);
// vs
var user = _dbContext.Users.FirstOrDefault(x => x.Id == 1);
user.State = 1;
_dbContext.SaveChanges();
By wrapping everything behind your repository, you design your business entities in a way that make it easy to work with them (child entites, enums, date management etc). And you design the repository so that those entities can be stored in an efficient way. No compromises and code that is more easily maintained.
Regarding OData: Do not use the repository pattern. It doesn't add any value in that case.
If you insist on using IQueryable in your business domain, do not use the repository pattern. It would only complicate things without adding any value.
Finally:
Business logic that uses properly designed repositories is so much easier to test (unit tests). Code where LINQ and business logic is mixed must ALWAYS be integration tests (against a DB) since Linq to Sql differs from Linq to Objects.
One of our contractors implemented a repository pattern with code first approach. We use Service Locator as DI pattern. what we do when we retrieve data from DB, we pass interface to GetQueryable function and get the data. However, I see serious performance issues on our application. I implemented MiniProfiler and MiniProfiler.EF to see where the bottleneck is.
We have a case table which has quite a few fields(around 25) and some of those fields are associated to other tables as one to one and one to many(only one field has many relation to other table). when I try to see the case detail, it runs around 400 SQL queries and SQL takes around 40 percent of the load time as far as the miniprofiler concerned. Here our GetQueryable and Find methods
public IQueryable<T> GetQueryable<T>(params string[] includes)
{
Type type = _impls.Value[typeof (T).Name].GetType();
DbSet dbSet = Db.Set(type);
foreach (var include in includes)
{
dbSet.Include(include);
}
return ((IQueryable<T>) dbSet);
}
I added included to this method to attach other related tables, but it did not make any difference. and here is the Find Method
public T Find<T>(long? id)
{
Type type = _impls.Value[typeof(T).Name].GetType();
return (T) Db.Set(type).Find(id);
}
I pretty much tried to apply all the performance improvements, but the number of the SQL queries has not gone down. I tried to disable lazy loading, but it caused many problems in other parts of the application.
Just some additional information, in case table, there are 70000 rows and in out dialogs table, there are 500000 rows. Case and Dialog are associates as one-to-many. and each case has 20-40 dialog entries.
My questions are;
Why does include not make any difference when I use?
Is there any other way to crop number of the queries run?
Do you think the implementation is the problem?
Thanks
Include returns a new IQueryable and does not modify the source query. In addition you can use the generic version of Set which simplifies the code a bit:
public IQueryable<T> GetQueryable<T>(params string[] includes)
{
IQueryable<T> query = Db.Set<T>();
foreach (var include in includes)
{
query = query.Include(include);
}
return query;
}
Step 1: Fire your contractor. Seriously. Like right now. That is some awful code. Not only did they miss something as simple and basic as using the generic version of Set, but they've successfully only made working with Entity Framework more complex, because all the repository does is proxy Entity Framework methods with its own unique and bastardized API.
That said, there's really not enough here to diagnose what your problem is. The use of Include may give you larger queries, but it should actually serve to reduce the overall number of queries issued. It's possible, you're just not using includes where you should be.
Now, the fact that you "tried to disable lazy loading, but it caused many problems in other parts of the application", means that you're relying too heavily on lazy-loading. Basically, you're loading in stuff you don't even know about, which is the antithesis of optimization. Ironically, you'd actually be best served by going ahead and disabling lazy-loading, and then tracking down where your code fails because of that. If you want to actually lazy-load that thing, you can use .Load (see: Explicit Loading). But, if you want to eager-load to reduce queries, then you know what includes you need to add.
I found some lines of code online and I understand the first two lines. Data of a particular
type is cached and stored in the two properties of the models below.
model.payment = (List<CompInfor>)HttpRuntime.Cache[cacheKey + "_received"];
model.FilteredPayment = (List<CompInfor>)HttpRuntime.Cache[cacheKey + "_received"];
However I don't understand the line below as I have never written code like this below.
Please what does this line do? What does it mean? I know you can save a lot of resources by using IQueryable.
IQueryable<CompInfor> payment = model.FilteredPayment.AsQueryable<CompInfor>();
It simply returns an instance of the IQueryable<T> interface which will utilize a query provider to act upon the object in question (in your case, the model.FilteredPayment list). It doesn't seem to make much sense when you're acting against a List locally, but (as an example) in the case of entity framework where you build query statement to be executed against a database via SQL, the Linq to Entities query provider processes the IQueryable into the appropriate SQL statement for execution against the database and processes the results.
I've got a multi-layer application that starts with the repository pattern for all data access and it returns IQueryable to the Services layer. The Services layer, which includes all of the business logic, returns IList to the Controllers (note: I'm using ASP.NET MVC for the UI layer). The benefit of returning IQueryable in the data access layer is that it allows my repositories to be extremely simple and the database queries to be deferred. However, I'm triggering the database queries in my services layer so that my unit tests is more reliable and I don't give flexibility to the Controllers to reshape my queries. However, I've recently encountered several situations where deferring the execution of queries down to the Controllers would have been significantly more performant because the Controllers had to do some projections on the data that was UI specific. Additionally, with the emergence of things like oData, I was starting to wonder if end points (e.g. web UI or web apis) should be working directly with IQueryable. What are your thoughts? Is it time to start returning IQueryable from the services layer to the UI layer? Or stick with IList?
This thread here: To return IQueryable<T> or not return IQueryable<T>
seems to vouch for returning IList to the UI layers, but I was wondering if things are changing because of new emerging technologies and techniques.
I like to stick with the IQueryable Interface when possible, the only problem is when you end up doing complex filtering or re-query on demand at the Controller level, if you have something like:
//DATA ACCESS
public IQueryable<T> GetStudents()
{
return db.Students;
}
And in your controller you do some re-sharping because your client want to filter some data of that result, surely you will be tempted to do it at the controller level:
var result = obj.GetStudents().Where(d=>d...);
and for me its ok, but just imaging if any other module need to use that same filter, you cant call it because its on the controller level.
So for me its a thing of balance between DRY, flexibility, and how scalable is the system.
If you need a fully scalable system you will need to do some or several overloads to GetStudents() method and get rid of any re-sharping at the controller level.
Reviewing Conery's storefront, and I dont understand why he used Linqs auto-generated classes (ie Order class) and then he has another Order class defined that is not a partial class. WHen using repository pattern should one manually create the classes, and disregard Datacontext altogether?
If you don't decouple your front end from the linq classes using an intermediary class, you can't control with the data context gets garbage collected. Typically with data context types of instances you want to rid of them as soon as you're done using them. Here's how you might want to do this with the linq to sql context:
using (MyDataContext data = new MyDataContext())
{
SomeThing thing = data.Things(t => t.ID == 1);
return thing;
}
... the MyDataContext instance is gone
With the "using" block, you're disposing of the instance of MYDataContext at the last "}". However, if you do this you'll get an error then trying to use "thing" because the data context instance is gone. If you don't dispose of the data context, it's left hanging around until it's eventually garbage collected.
If you introduce an intermediary class to decouple the linq to sql code from the calling app you can still get rid of your data context instance and return the same data (just in a different object):
using (MyDataContext data = new MyDataContext())
{
SomeThing thing = data.Things(t => t.ID == 1);
SometThingElse otherThing = ConvertSomethingToSomethingElse(thing);
return otherThing;
}
... the MyDataContext instance is gone
Hope that helps.
Rob has answered on this question in one of his show.
He Using POCO classes to be aware from all dataaccess classes. For example when he change LINQ-to-SQL to NHibernate all he will need to do i change his "mappings" in his filters, and he will not have to make any changes in bussiness logic.
He said in one of his recent videos he doesn't like the way LINQ to SQL does mapping. I agree though I think it is complete overkill.
I'd say you're not breaking any major design patterns as long as you're sticking to the repository pattern itself. I think it's a matter of choice to have 2 sets of classesa, allbeit a bad one, still a choice.