Inefficient MVC ViewModel Making Multiple Calls to the Database? - asp.net-mvc

I clearly don't know what I'm doing. This MVC stuff is really blowing my mind in trying to keep with the pattern. I've been following the MVC tutorials as well as mega-googling and this is the corner I've painted myself into.
I have multiple similar pieces of data I'm trying to get to a view. I'm able to get my code to work, but to me it just looks like it's going to be highly inefficient as we start pulling large recordsets from the db due to multiple calls to the db. So, I have a OrderSummary class, inside the class is this:
public IEnumerable<Order> GetOrders()
{
var orders = (from s in db.Orders
where s.UserId == uId
select s);
return orders.ToList();
}
Then this:
public decimal GetGrossProfitTotal()
{
var orders = (from s in db.Orders
where s.UserId == uId
select s);
decimal? grossprofittotal = orders.Sum(s => s.Profit);
return grossprofittotal ?? decimal.Zero;
}
So, if we take that last chunk of code and copy it for totalcommission and netprofittotal that's basically how I have things layed out. I would guess four calls to the db?
Then in the controller:
var ordersummary = new OrdersSummary();
var viewModel = new OrderSummary
{
Orders = ordersummary.GetOrders(),
GrossProfitTotal = ordersummary.GetGrossProfitTotal(),
CommissionTotal = ordersummary.GetCommissionTotal(),
NetProfitTotal = ordersummary.GetNetProfitTotal(),
};
return View(viewModel);
This gets me all the data I need in the view so I can work with it. To me, it just seems unnecessarily redundant and I'm guessing inefficient? If you throw in that I'm also doing sort and search parms, it's a lot of duplicate linq code as well. It seems like I should be able to do something to consolidate the data like this:
var orders = (from s in db.Orders
where s.UserId == uId
select s).ToList();
decimal grossprofittotal = orders.Sum(s => s.Profit);
decimal commissiontotal = orders.Sum(s => s.Commission);
decimal netprofittotal = orders.Sum(s => s.Profit + s.Commission);
and then wrap those four pieces of data (orders list, and three decimal values) up nicely in an array (or whatever) and send them to the controller/view. In the view I need to be able to loop through the orders list. Am I way off here? Or, what is standard procedure here with MVC? Thanks.

Yes, fetching the same data four times is indeed inefficient, and completely unneccesary. You can very well fetch it only once and then do the other operations on the data that you have.
You can keep the GetOrders method as it is if you like, but that's all the data that you need to fetch. If you fetch the data in the controller or in the model constructor is mostly a matter of taste. Personally I tend to put more logic in the model than the controller.
As long as you use ToList to make sure that you actually fetch the data (or any other method that realises the result as a collection), you can calculate the sums from what you have in memory. (Without it, you would still be doing four queries to the database.)
Instead of summing up the profit and commision from all items to get the net profit total, you can just calculate it from the other sums:
decimal netprofittotal = grossprofittotal + netprofittotal;

LinqToEntities tranlates all query into SQL. If you don't want to make more than one transaction, you can fetch the result into a variable by .ToList(),querying this object make the calculation by linqToObject in the memory.
Backward: It fetchs all orders from database first.
var ordersInMemory = orders.ToList();
decimal grossprofittotal = ordersInMemory.Sum(s => s.Profit);
decimal commissiontotal = ordersInMemory.Sum(s => s.Commission);
decimal netprofittotal = grossprofittotal + commissiontotal ;

Related

Search a collection of objects

I'm trying to search for a collection of potential nodes but unable to do it...
I have a product that has a relationship with many instances. I would like to query the DB and get all the instances that are in a list that i get from the user.
Cypher:
var query = _context
.Cypher
.Start(new
{
instance = startBitsList,
product = productNode.Reference,
})
.Match("(product)-[:HasInstanceRel]->(instance)")
.Return(instance => instance.Node<ProductInstance>());
The problem is startBitsList... I use StringBuilder to generate a query that contains all the instances I'm looking for:
private static string CreateStartBits(IEnumerable<string> instanceNames)
{
var sb = new StringBuilder();
sb.AppendFormat("node:'entity_Name_Index'(");
foreach (var id in productIds)
{
sb.AppendFormat("Name={0} OR ", id);
}
sb.Remove(sb.Length - 4, 4);
sb.Append(")");
var startBitsList = sb.ToString();
return startBitsList;
}
I get exceptions when trying to run this cypher...
Is there a better way to search for multiple items that are stored in the collection I get from the user?
OK, I think there are a couple of issues at play here, first I'm presuming you are using Neo4j 1.9 and not 2.0 - hence using the .Start.
Have you tried taking your query and running it in Neo4j? This should be your first port of call, typically it's easy to add a breakpoint on the .Results call and add a 'watch' for query.Query.DebugText.
However, I don't think you need to use the StartBits the way you are, I think you'd be better off filtering with a .Where as you already have the start point:
private static ICypherFluentQuery CreateWhereClause(ICypherFluentQuery query, ICollection<string> instanceNames)
{
query = query.Where((Instance instance) => instance.Name == instanceNames.First());
query = instanceNames.Skip(1).Aggregate(query, (current, localInstanceName) => current.OrWhere((Instance instance) => instance.Name == localInstanceName));
return query;
}
and your query becomes something like:
var prodReference = new NodeReference<Product>(2);
var query =
Client.Cypher
.ParserVersion(1, 9)
.Start(new {product = prodReference})
.Match("(product)-[:HasInstanceRel]->(instance)");
query = CreateWhereClause(query, new[] {"Inst2", "Inst1"});
var resultsQuery = query.Return(instance => instance.As<Node<Instance>>());
2 things of note
We're not using the indexes - there is no benefit to using them as you have the start point and traversing to the 'instances' is a simple process for Neo4j.
The 'CreateWhereClause' method will probably go wrong if you pass in an empty list :)
The nice thing about not using the indexes is that - because they are legacy - you are set up better for Neo4j 2.0

LINQ query with omitted user input

so I have a form with several fields which are criteria for searching in a database.
I want to formulate a query using LINQ like so:
var Coll = (from obj in table where value1 = criteria1 && value2 = criteria2...)
and so on.
My problem is, I don't want to write it using If statements to check if every field has been filled in, nor do I want to make separate methods for the various search cases (criteria 1 and criteria 5 input; criteria 2 and criteria 3 input ... etc.)
So my question is: How can I achieve this without writing an excessive amount of code? If I just write in the query with comparison, will it screw up the return values if the user inputs only SOME values?
Thanks for your help.
Yes, it will screw up.
I would go with the ifs, I don't see what's wrong with them:
var query = table;
if(criteria1 != null)
query = query.Where(x => x.Value1 == criteria1);
if(criteria2 != null)
query = query.Where(x => x.Value2 == criteria2);
If you have a lot of criteria you could use expressions, a dictionary and a loop to cut down on the repetitive code.
In an ASP.NET MVC app, chances are your user input is coming from a form which is being POSTed to your server. In that case, you can make use of strongly-typed views, using a viewmodel with [Required] on the criteria that MUST be provided. Then you wrap your method in if (ModelState.IsValid) { ... } and you've excluded all the cases where the user hasn't given you something they need.
Beyond that, if you can collect your criteria into a list, you can filter it. So, you could do something like this:
filterBy = userValues.Where(v => v != null);
var Coll = (from obj in table where filterBy.Contains(value1) select obj);
You can make this more complex by having a Dictionary (or Lookup for non-unique keys) that contains a user-entered value along with some label (an enum, perhaps) that tells you which field they're filtering by, and then you can group them by that label to separate out the filters for each field, and then filter as above. You could even have a custom SearchFilter object that contains other info, so you can have filters with AND, NOT and OR conditions...
Failing that, you can remember that until you trigger evaluation of an IQueryable, it doesn't hit the database, so you can just do this:
var Coll = (from obj in table where value1 == requiredCriteria select obj);
if(criteria1 != null)
{
query = query.Where(x => x.Value1 == criteria1);
}
//etc...
if(criteria5 != null)
{
query = query.Where(x => x.Value5 == criteria5);
}
return query.ToList();
That first line applies any criteria that MUST be there; if there aren't any mandatory ones then it could just be var Coll = table;.
That will add any criteria that are provided will be applied, any that aren't will be ignored, you catch all the possible combinations, and only one query is made at the end when you .ToList() it.
As I understand of your question you want to centralize multiple if for the sake of readability; if I were right the following would be one of some possible solutions
Func<object, object, bool> CheckValueWithAnd = (x, y) => x == null ? true : x==y;
var query = from obj in table
where CheckValue(obj.value1, criteria1) &&
CheckValue(obj.value2, criteria2) &&
...
select obj;
It ls flexible because in different situations or scenarios you can change the function in the way that fulfill your expectation and you do not need to have multiple if.
If you want to use OR operand in your expression you need to have second function
Func<object, object, bool> CheckValueWithOr = (x, y) => x == null ? false : x==y;

JQGrid Loading lots of data

SITUATION
I am using Trirand JQGrid for MVC[server side] in my proj.
I've got more than 5 hundred thousand records in a single table.
I load the data by calling this piece of code. this is what gives 500000 records collection.
IEnumerable<myIndexViewModel> myviewmodel= _allincidents.Select(x => new myIndexViewModel
{
IncidentRequestStatus = x.RequestStatus,
RequestByUserName = x.RequestByUserName,
Subject = x.Subject
});
gridModel.JqGrid.DataBind(myviewmodel.AsQueryable());
JQgrid handles the json based ajax requests very nicely for every next page i click.
PROBLEM
I dont want to load 5 hundred thousand records all together on the page load event as it kills jqgrid.
If i write a stored procedure in the DB for requesting a specific page to be displayed then its gonna load only that page in the myviewmodel collection.
How do i get pages on the fly from the DB when the next page is clicked. is this even possible in jqgrid?
SITUATION 2
Based on the answers from VIJAY and MARK the approach they have shown is absolutely correct but over here the JQGRID for MVC sets up the DATAURL property for making the method call. In this case its the IncidentGridRequest.
How do i send in the page number when the grid next page or previous page is clicked?
incidentModel.IncidentGrid.DataUrl = Url.Action("IncidentGridRequest")
public JsonResult IncidentGridRequest()
{
}
Your controller action that will provide your grid with results can accept some extra information from jqGrid.
public ActionResult GetGridData(string sidx, string sord, int page, int rows, bool _search, string filters)
The main parts you are interested in is the page, rows (sidx is for column sorting, sord for the sorting order, _search if there was a search done on the grid, and if so filters contains the search information)
When you generate your results you should be able to then
IEnumerable<myIndexViewModel> myviewmodel = allincidents.Select(x => new myIndexViewModel
{
IncidentRequestStatus = x.RequestStatus,
RequestByUserName = x.RequestByUserName,
Subject = x.Subject
}).Skip((page - 1) * rows).Take(rows)
PS. I'm not sure if you using IEnumberable will be moving a large amount of data from your DB but you might want to use IQueryable when you generate this subset of data for the jqGrid.
Edit: To deal with your paging issues, You should be calculating the number of total records in your query and passing that value to the grid, Ex
int totalRecords = myviewmodel.Count();
and then later you would pass that to your grid as a jSon value. Ex
var jsonData = new
{
total = (totalRecords + rows - 1) / rows,
page = page,
records = totalRecords,
userdata = new {SearchResultsFound = searchResultsFound},
rows = (
......
Yes, for example if you are accepting the page number you want to turn to in a variable named page and the have the size of page in a variable pageSize then:
IEnumerable<myIndexViewModel> myviewmodel = allincidents.Select(x => new myIndexViewModel
{
IncidentRequestStatus = x.RequestStatus,
RequestByUserName = x.RequestByUserName,
Subject = x.Subject
}).Skip((page-1)*pageSize).Take(pageSize));
will give you the records of size pageSize to you.
The Trirand jqGrid for ASP.NET MVC is using IQueryable interface inside the JqGrid.DataBind() method to implement pagin, sorting and filtering.
So the key here is to use datasource, which handle these types of operations at the database level (by crafting SQL queries to the database in such a way that only the data required is fetched). All major ORMs have this support, this includes: LINQ-2-SQL, Entity Framework, NHbiernate, LLBLGen.
You just need to use one of this technologies, and past the required context directly to JqGrid.DataBind() method (without extracting the data manually like you do it in your sample).
An easier approach by using PagedList library (from Nuget). There is a useful blog by Joseph Schrag
public JsonResult Users(int PageNo, int Rows)
{
var UserList = db.Users.Select(t => new
{
t.UserId,
t.Username,
t.Firstname,
t.Lastname,
t.Designation,
t.Country,
t.Email
}).OrderBy(t => t.UserId);
var pagedUserList = UserList.ToPagedList(PageNo, Rows);
var results = new
{
total = pagedUserList.PageCount, //number of pages
page = pagedUserList.PageNumber, //current page
records = UserList.Count(), //total items
rows = pagedUserList
};
return new JsonResult() { Data = results, JsonRequestBehavior = JsonRequestBehavior.AllowGet };
}

Getting a list of distinct entities projected into a new type with extra field for the count

I'm designing an interface where the user can join a publicaiton to a keyword, and when they do, I want to suggest other keywords that commonly occur in tandem with the selected keyword. The trick is getting the frequency of correlation alongside the properties of the suggested keywords.
The Keyword type (EF) has these fields:
int Id
string Text
string UrlString
...and a many-to-many relation to a Publications entity-set.
I'm almost there. With :
var overlappedKeywords =
selectedKeyword.Publications.SelectMany(p => p.Keywords).ToList();
Here I get something very useful: a flattened list of keywords, each duplicated in the list however many times it appears in tandem with selectedKeyword.
The remaining Challenge:
So I want to get a count of the number of times each keyword appears in this list, and project the distinct keyword entities onto a new type, called KeywordCounts, having the same fields as Keyword but with one extra field: int PublicationsCount, into which I will populate the count of each Keyword within overlappedKeywords. How can I do this??
So far I've tried 2 approaches:
var keywordCounts = overlappingKeywords
.Select(oc => new KeywordCount
{
KeywordId = oc.Id,
Text = oc.Text,
UrlString = oc.UrlString,
PublicationsCount = overlappingKeywords.Count(ok2 => ok2.Id == oc.Id)
})
.Distinct();
...PublicationsCount is getting populated correctly, but Distinct isn't working here. (must I create an EqualityComarer for this? Why doesn't the default EqualityComarer work?)
var keywordCounts = overlappingKeywords
.GroupBy(o => o.Id)
.Select(c => new KeywordCount
{
Id = ???
Text = ???
UrlString = ???
PublicationsCount = ???
})
I'm not very clear on GroupBy. I don't seem to have any access to 'o' in the Select, and c isn't comping up with any properties of Keyword
UPDATE
My first approach would work with a simple EqualityComparer passed into .Distinct() :
class KeywordEqualityComparer : IEqualityComparer<KeywordCount>
{
public bool Equals(KeywordCount k1, KeywordCount k2)
{
return k1.KeywordId== k2.KeywordId;
}
public int GetHashCode(KeywordCount k)
{
return k.KeywordId.GetHashCode();
}
}
...but Slauma's answer is preferable (and accepted) because it does not require this. I'm still stumped as to what the default EqualityComparer would be for an EF entity instance -- wouldn't it just compare based on primary ids, as I did above here?
You second try is the better approach. I think the complete code would be:
var keywordCounts = overlappingKeywords
.GroupBy(o => o.Id)
.Select(c => new KeywordCount
{
Id = c.Key,
Text = c.Select(x => x.Text).FirstOrDefault(),
UrlString = c.Select(x => x.UrlString).FirstOrDefault(),
PublicationsCount = c.Count()
})
.ToList();
This is LINQ to Objects, I guess, because there doesn't seem to be a EF context involved but an object overlappingKeywords, so the grouping happens in memory, not in the database.

How to join multiple tables using LINQ-to-SQL?

I'm quite new to linq, so please bear with me.
I'm working on a asp.net webpage and I want to add a "search function" (textbox where user inputs name or surname or both or just parts of it and gets back all related information). I have two tables ("Person" and "Application") and I want to display some columns from Person (name and surname) and some from Application (score, position,...). I know how I could do it using sql, but I want to learn more about linq and thus I want to do it using linq.
For now I got two main ideas:
1.)
var person = dataContext.GetTable<Person>();
var application = dataContext.GetTable<Application>();
var p1 = from p in Person
where(p.Name.Contains(tokens[0]) || p.Surname.Contains(tokens[1]))
select new {Id = p.Id, Name = p.Name, Surname = p.Surname}; //or maybe without this line
//I don't know how to do the following properly
var result = from a in Application
where a.FK_Application.Equals(index) //just to get the "right" type of application
//this is not right, but I don't know how to do it better
join p1
on p1.Id == a.FK_Person
2.) The other idea is just to go through "Application" and instead of "join p1 ..." to use
var result = from a in Application
where a.FK_Application.Equals(index) //just to get the "right" type of application
join p from Person
on p.Id == a.FK_Person
where p.Name.Contains(tokens[0]) || p.Surname.Contains(tokens[1])
I think that first idea is better for queries without the first "where" condition, which I also intended to use. Regardless of what is better (faster), I still don't know how to do it using linq. Also in the end I wanted to display / select just some parts (columns) of the result (joined tables + filtering conditions).
I really want to know how to do such things using linq as I'll be dealing also with some similar problems with local data, where I can use only linq.
Could somebody please explain me how to do it, I spent days trying to figure it out and searching on the Internet for answers.
var result = from a in dataContext.Applications
join p in dataContext.Persons
on p.Id equals a.FK_Person
where (p.Name.Contains("blah") || p.Surname.Contains("foo")) && a.FK_Application == index
select new { Id = p.Id, Name = p.Name, Surname = p.Surname, a.Score, a.Position };
Well as Odrahn pointed out, this will give you flat results, with possibly many rows for a single person, since a person could join on multiple applications that all have the same FK. Here's a way to search all the right people, and then add on the relevant application to the results:
var p1 = from p in dataContext.Persons
where(p.Name.Contains(tokens[0]) || p.Surname.Contains(tokens[1]))
select new {
Id = p.Id, Name = p.Name, Surname = p.Surname,
BestApplication = dataContext.Applications.FirstOrDefault(a => a.FK_Application == index /* && ???? */);
};
Sorry - it looks like this second query will result in a roundtrip per person, so it clearly won't be scalable. I assumed L2S would handle it better.
In order to answer this properly, I need to know if Application and Person are directly related (i.e. does Person have many Applications)? From reading your post, I'm assuming that they are because Application seems to have a foreign key to person.
If so, then you could create a custom PersonModel which will be populated by the fields you need from the different entities like this:
class PersonModel
{
string Name { get; set; }
string Surname { get; set; }
List<int> Scores { get; set; }
List<int> Positions { get; set; }
}
Then to populate it, you'd do the following:
// Select the correct person based on Name and Surname inputs
var person = dataContext.Persons.Where(p => p.Name.Contains("firstname") || p.Name.Contains("surname")).FirstOrDefault();
// Get the first person we find (note, there may be many - do you need to account for this?)
if (person != null)
{
var scores = new List<int>();
var positions = new List<int>();
scores.AddRange(person.Applications.Select(i => i.Score);
positions.AddRange(person.Applications.Select(i => i.Position);
var personModel = new PersonModel
{
Name = person.Name,
Surname = person.Surname,
Scores = scores,
Positions = positions
};
}
Because of your relationship between Person and Application, where a person can have many applications, I've had to account for the possibility of there being many scores and positions (hence the List).
Also note that I've used lambda expressions instead of plain linqToSql for simple selecting so that you can visualise easily what's going on.

Resources