EF Core 3.1.x:
I would not like to load all products in memory, which below queries do!
Guess what happen if i do have millions of products in table?
var products = context.Products.ToList();
products = products.Where(p => p.Name.Contains("xxx")).ToList();
And below query throws
The LINQ expression 'DbSet-Product-
.Where(b => b.Name.Contains(
value: "xxx",
comparisonType: InvariantCultureIgnoreCase))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync().
var products = context.Products.Where(p => p.Name.Contains("xxx", StringComparison.InvariantCultureIgnoreCase)).ToList();
Related Issue on github: #19087
Can anyone help me out. how to filter data with server side evaluation with ef core 3.1.x?
EF Core does translate Contains for server-side evaluation - but not the overload that accepts StringComparison.InvariantCultureIgnoreCase (or any other StringComparison).
Closed issue here
Related
I have had to move queries off the main database in order to meet requirements for complex authorization - for example a user with a given authorization role can only view data for individuals in the same institution.
I am using the Breeze .net DocCode sample for guidance, and have copied the premise for the mapping of domain models to DTOs.
get { return ForCurrentUser(Context.Orders).Select(o => new Order {
OrderID = o.OrderID,
....
OrderDetails = o.OrderDetails.Select(od => new OrderDetail
{
ProductID = od.ProductID,
UnitPrice = od.UnitPrice
...
})
The problem is that which mapped properties to .include(entity framework method)/.expand (breeze method) is now a concern of the mapping function (for example, the above code will always return the OrderDetails collection, whether I want them or not). I would like to still only eagerly load/expand properties if the javascript client generated predicate has a .expand directive for that property.
Is this at all possible, or am I stuck with manually defining different mapping functions on the server, depending on what properties I want expanded? (I am happy to use tools such as automapper if that would solve or simplify the problem)
Thank you
You will need to use the ODataQueryOptions as a parameter to your controller method. This gives you the details of the query predicates in your server method, so that you can apply them as needed rather that having them applied automatically. This will let you expand, or not, based upon the query.
See this answer and this answer to see how it works.
I am working on an ASP.NET MVC 4 web application. I am using Entity Framework as the data access layer, using database first approach (.edmx file).
Currently I have a problem in join tables that are defined inside two different databases (i.e. I have two .edmx files).
For example if I want to join tables I am performing the following query:-
public ActionResult AutoComplete(string term)
{
var tech = repository.AllFindTechnolog(term).Take(100);//Call to the first database
var resources = repository.GetResources(tech.Select(a => a.IT360ID.Value).ToArray(), false);//call to the second database
var query = from techItems in tech
join resourcesItems in resources
on techItems.IT360ID.Value equals resourcesItems.RESOURCEID // join based on db2ID
orderby techItems.PartialTag
select new //code goes here
return Json(query, JsonRequestBehavior.AllowGet);
}
I will have two separate calls to the database, and a join inside the application server, which is not the best performance-oriented solution. Ideally the joins will happen completely inside the database engine.
I know that a stored procedure will allow me to join tables from different databases purely on the server, but I do not want to use SP because it will make my code less maintainable and less testable.
So I am searching for a solution where I can do the join using entity framework and to result in a single database join?
If you want to do it with a single database call you will have to create a View in the database that joins the 2 tables from separate db's. Once the view is created you can add it to EF as a single object, which you can manipulate further and Query off of. The view will basically be a table and it will be easily maintable and easy to bind to a strongly typed model
Another way ,similiar like you have posted, you can query separate .edmx files and then join them.
Yes, there is 2 calls to the database but it shouldn't be that expensive and probably won't notice a difference.
using(var db = new MyEntities())
using (var db2 = new MyEntities2())
{
var one = db.Table1.AsEnumerable();
var two = db2.Table2.AsEnumerable();
var result = from o in one
join t in two on o.Id equals t.Id
// blah blah
}
#CSharper's answer is close. As #Oliver mentioned in the comments, IEnumerable loads the table into application memory, leading to crashes if you have a large database.
The solution is to use IQueryable, which can be called with LINQ - this produces SQL which is much faster.
// This is a generic method, modify to your needs
public ActionResult Details(int? id)
var one = db.Table1.AsQueryable();
var two = db2.Table2.AsQueryable();
// since you're using MVC EF, I assume you want to put this in a viewmodel
// (in this case ObjectCombined)
// assume "id" is passed as parameter
Object1 result1 = (from o in one where one.id == id select o).Single();
Object2 result2 = (from t in two where t.id == o.id select t).Single();
ObjectCombined result = new ObjectCombined(result1, result2);
return View(result);
}
Might I suggest that you look into using a synonym in your database. For instance, you can create a synonym to the resources table in the database that your tech table is located. This will ensure that you will not need to maintain 2 EDMX files. Instead you can have a single EDMX file and you can simply join your tech table to the synonym of the resource table and voila - you are on your way.
UPDATE: Please note that if you are using synonyms there is an extra bit of work you will need to do to the EDMX file to get it working in Entity Framework. Here is a blog post that was put out by a programmer who got it to work. Here is the original stackoverflow question she asked.
HAPPY CODING!!! :)
you can create a view or a stored procedure, your sql statement can then make cross db query just make sure your credentials can DML or DDL on both db. otherwise try the nested using entities that will make sure you will not get the linq bug when you dont declare the db entity inside a using statement.
I have a Users table of 76 users and UserGroups table.
Using MVC, OData, a generic repository and EF, I am trying to optimize data retrieval when filtering based on the user group:
/api/Users?$filter=USERGROUPS/any(usergroup: usergroup/ID eq 'Group1')
On the client side, I get the right number of users - 71 (as OData is filtering based on the result), however I want to limit the number of records being returned form the actual query - ie. I do not want to return all records then filter (not optimal for very large data sets).
My API controller method is as follows:
[Queryable(AllowedQueryOptions = AllowedQueryOptions.All)]
public IQueryable<USER> Get()
{
var unitOfWork = new ATMS.Repository.UnitOfWork(_dbContext);
var users = unitOfWork.Repository<USER>()
.Query()
.Include(u => u.USERGROUPS)
.Get()
.OrderBy(order => order.USERNAME);
unitOfWork.Save(); // includes Dispose()
return users.AsQueryable();
}
I read in this post that:
Entity framework takes care of building dynamic query based on the
request.
However, using a SQL Server profiler, the query executed is requesting all the records, rather than a filtered query.
Adding a .Take() to the query does not accomplish the desired result, as we also need the actual number of records returned for paging purposes.
I was thinking of using the grabbing some properties through ODataQueryOptions, but that doesn't seem quite right either.
Is my implementation of Unit of Work and Repository incorrect, in relation to what I am trying to accomplish, and if so, how can this be corrected?
Simple - Just set the Page size for the Queryable atrribute [Queryable(PageSize=10)]
http://www.asp.net/web-api/overview/odata-support-in-aspnet-web-api/supporting-odata-query-options#server-paging
If You'd tell the EF where to apply the options, it would work.
Like this :
//[Queryable(AllowedQueryOptions = AllowedQueryOptions.All)]
public IQueryable<USER> Get(ODataQueryOptions<USER> options)
{
var users = options.ApplyTo(_dbContext.Set<USER>()
.Query()
.Include(u => u.USERGROUPS)
.Get()
.OrderBy(order => order.USERNAME));
return users;
}
Your code didn't work, because it tried to apply the options onto the last line "users.AsQueryable()", so what actually happened, is that EF pull the FULL dataset, and then applied the query onto the last line (that being a in memory collection). And that's why You didn't see that "filter" not being passed to the SQL.
The mechanics are such, that EF tries to apply the Query, to the IQueryable collection that it finds in the code (there's still a question how does it find the correct line).
I have the following model:
Products (DbSet<Product>)
ProductHasWidgets (DbSet<ProductHasWidget> many-to-many w/ payload)
Widgets (DbSet<Widget>)
I am getting stuck with querying across and/or from the many-to-many table and have two questions:
How do I write a breeze query to return all Products where:
Widget.IsActive == true
ProductHasWidgets.WidgetId == 1
You can't do that from a Breeze client yet because Breeze does not support the any keyword at this time.
You can write a service method to do the query on the server. The client can pass query parameters to that method using the BreezeJS withParameters query verb.
As of Breeze 1.4.6, Breeze now supports "any" and "all" queries. See: http://www.breezejs.com/documentation/query-examples
Depending on your model, this means that your query would look something like:
var predicate = breeze.Predicate.create("WidgetId", "==" 1)
.and("Widget.IsActive", "==", true);
var query = EntityQuery.from("Products")
.where("ProductHasWidgets", "any", predicate);
I have looked here :
LINQ To SQL exception: Local sequence cannot be used in LINQ to SQL implementation of query operators except the Contains operator
and here :
LINQ to Populate a range
but I can't figure this one out. I have based my Task Manager on the NerdDinner project. Instead of FindUpcommingDinners I have a method below :
Public Function FindAllTeamTasks(ByVal TeamRole As String) As IQueryable(Of Task) Implements ITaskRepository.FindAllTeamTasks
Return From task In db.Tasks
Join usrs In System.Web.Security.Roles.GetUsersInRole(TeamRole)
On task.TaskAssignedToID Equals usrs
Order By task.InsertDateTime _
Select task
End Function
When the result is passed into the Paginatedlist as source it errors with the "Local sequence cannot ..." error. The answer might be here as well but I can't figure it out. Please help.
ASP.NET MVC2 LINQ - Repository pattern, where should the pagination code go?
You can't use the result of GetUsersInRole() in a join to a database table via LINQ to SQL. You can try this instead:
Dim usrs = System.Web.Security.Roles.GetUsersInRole(TeamRole)
Return From task In db.Tasks _
Where usrs.Contains(task.TaskAssignedToID) _
Order By task.InsertDateTime _
Select task
Whenever your are using Linq to SQL, your query has to be able to be translated into SQL to be run against your database. The Linq provider for Linq to SQL only knows how to do this for very certain operations that would occur outside the database. So, for instance, Linq to SQL has no idea what to do with
Join usrs In System.Web.Security.Roles.GetUsersInRole(TeamRole)
because GetUsersInRole is not (as far as Linq is concerned) a table in the database, but rather a collection in your application.
As the error message mentions, there is an "exception" in that, in some cases, Linq to SQL does know what to do with Contains (by turning it into IN in SQL, if I recall) for a collection that does not reside in the database. With that in mind, you might turn your query into something like (sorry it's in C#, but I hope you can translate):
string[] users = System.Web.Security.Roles.GetUsersInRole(TeamRole);
return db.Tasks
.Where<Task>(t => users.Contains<string>(t.TaskAssignedToID))
.OrderBy<Task, DateTime>(t => t.InsertDateTime)
.AsQueryable<Task>();
Now, if your join was there because TaskAssignedToID was a collection and not merely a string (I'm not familiar with the model you're using, sorry), you might instead need to do something like
string[] users = System.Web.Security.Roles.GetUsersInRole(TeamRole);
return db.Tasks
.Where<Task>(t => t.TasksAssignedToID.Any<string>(i => users.Contains<string>(i)))
.OrderBy<Task, DateTime>(t => t.InsertDateTime)
.AsQueryable<Task>();
I'm not 100% sure that Linq to SQL is fine with Any -- I work mostly with Linq to Entities -- but I hope this makes some sense.