Hey everyone, I'm getting killed on database queries in my MVC3 app when I'm mapping my repository to a View Model. The View is fairly complicated, the main object is Assignment but the View returns data from a few of it's relationships too. The View Mode looks like this
public int Id { get; set; }
public DateTime? RelevantDate { get; set; }
public String Name { get; set; }
public String ProcessName { get; set; }
public String Status { get; set; }
public String Assignee { get; set; }
public int AssigneeId { get; set; }
public bool HasAttachment { get; set; }
public bool IsGroupAssignee { get; set; }
public bool Overdue { get; set; }
public List<String> AvailableActions { get; set; }
public Dictionary<String, String> AssignmentData { get; set; }
public Dictionary<String, String> CompletionData { get; set; }
public List<Transactions> Comments { get; set; }
public List<Transactions> History { get; set; }
public List<File> Files { get; set; }
There's a lot going on there, but all of the data is relevant to the View. In my repository I explicitly load all of the required relationships with .Include (I'm using Entity Framework) but the data doesn't actually get loaded until I start iterating over the list.
var _assignments = (from ctx.Assignments
.Include("Process").Include("Files")
.Include("AssignmentDataSet")
.Include("Transactions")
.where w.Tenant.Id == _tenantId
select w);
In my controller I call a method on the repository that uses a query similar to this to get my data. A few variations but nothing too different from what's above.
Now, here's where I'm chewing up Database Transactions. I have to get this data into a ViewModel so I can display it.
private IList<AssignmentViewModel> CreateViewModel(IEnumerable<Assignment> aList)
{
var vList = new List<AssignmentViewModel>();
foreach (var a in aList)
{
var assigneeId = a.Assignee;
vList.Add(new AssignmentViewModel()
{
Id = a.Id,
AssigneeId = (int) a.Assignee,
HasAttachment = (a.Files.Count > 0),
Name = a.Name,
IsGroupAssignee = a.AssignedToGroup,
ProcessName = a.Process.Name,
RelevantDate = a.RelevantDate,
Status = a.Status,
AvailableActions = _assignmentRepository.GetAvailableActions(_user, a),
Assignee =
_users.Where(i => i.Id == assigneeId).Select(v => v.FullName).
FirstOrDefault(),
AssignmentData =
a.AssignmentDataSet.Where(st => st.State == "Assign").ToDictionary(
d => d.Name, d => d.Value),
CompletionData = a.AssignmentDataSet.Where(st => st.State == "Complete").ToDictionary(
d => d.Name, d => d.Value),
Comments = a.Transactions.Where(t => t.Action == "New Comment").ToList(),
History = a.Transactions.Where(t => t.Action != "New Comment").ToList(),
Overdue =
a.RelevantDate >= DateTime.UtcNow.AddHours(-5) || a.RelevantDate == null ||
a.Status == "Complete" || a.Status == "Canceled"
? false
: true
});
}
return vList;
}
This is resulting in approx 2.5 db queries per Assignment. This View will return a max of 30 results. That's a lot of hits to my database. Needless to say, the page is dead slow. Response times of 5-7 seconds. I'm embarrassed by it! Am I just trying to do too much here, or am I just doing it wrong?
How can I optimize this?
Two things I can see:
HasAttachment = (a.Files.Count > 0),
Just use Any() instead so you don't have to iterate over all files assuming this is still an IEnumerable:
HasAttachment = a.Files.Any(),
The other thing is comments and history:
Comments = a.Transactions.Where(t => t.Action == "New Comment").ToList(),
History = a.Transactions.Where(t => t.Action != "New Comment").ToList(),
You can combine these calls by materializing the full list before you create the AssignmentViewModel and then just take the appropriate parts:
var transactions = a.Transactions.ToList();
vList.Add(new AssignmentViewModel()
{
..
Comments = transactions.Where(t => t.Action == "New Comment").ToList(),
History = transactions.Where(t => t.Action != "New Comment").ToList(),
}
If you have to support this way of displaying your data you should consider a view and a corresponding entity in the database that selects all or most of the appropriate data for you using joins, so the effort you have to spend on bringing the data to the UI is much less.
Related
I have a Quote object, with a collection of QuoteAnswer objects. I also want a shortcut to the latest QuoteAnswer. So I modeled (irrelevant code ommitted for brevity):
public class Quote
{
public int Id { get; set; }
public ICollection<QuoteAnswer> Answers { get; set; }
public QuoteAnswer LatestAnswer { get; set; }
public int LatestAnswerId { get; set; }
}
public class QuoteAnswer
{
public int Id { get; set; }
public Quote Quote { get; set; }
public int QuoteId { get; set; }
/* Did not map this, not interested/not needed
* public Quote LastAnswerFor { get; set; }
* public int LastAnswerForId { get; set; }
*/
}
That's beacuse I want to be able to do this:
var quotes = context.Quotes
.Include(x => x.LatestAnswer)
.ToList();
Instead of this:
var quotes = context.Quotes
.Include(x => x.Answers)
.ToList();
foreach (var q in quotes)
{
var latestAnswer = q.Answers.OrderByDescending(x => x.Date).FirstOrDefault();
}
Which would obviously force me to load unecessary data.
The Problem
When I try to map this do database code (add a migration), I get a new property I don't know where it's comming from.
Generated migration code (parts ommitted for brevity):
CreateTable(
"dbo.QuoteAnswer",
c => new
{
Id = c.Int(nullable: false),
QuoteId = c.Int(nullable: false),
QuoteId1 = c.Int(),
})
.PrimaryKey(t => t.Id)
.ForeignKey("dbo.Quote", t => t.QuoteId)
.ForeignKey("dbo.Quote", t => t.QuoteId1)
.Index(t => t.QuoteId)
.Index(t => t.QuoteId1);
AddColumn("dbo.Quote", "LatestAnswerId", c => c.Int());
CreateIndex("dbo.Quote", "LatestAnswerId");
AddForeignKey("dbo.Quote", "LatestAnswerId", "dbo.QuoteAnswer", "Id");
What's that QuoteId1 thing? I get the QuoteId, but I don't recognize QuoteId1.
How can I achive this mapping? Is this even supported in EF6?
First, it's possible. The explicit FK property should be removed:
public class Quote
{
public int Id { get; set; }
public string Data { get; set; }
public ICollection<QuoteAnswer> Answers { get; set; }
public QuoteAnswer LatestAnswer { get; set; }
}
and the new relationship should be mapped with fluent API:
modelBuilder.Entity<Quote>()
.HasOptional(e => e.LatestAnswer)
.WithOptionalDependent()
.Map(m => m.MapKey("LatestAnswerId"));
But I won't recommend you doing it because it would introduce a lot of maintenance problems - aside of the obvious need to keep it up-to-date, it would create circular FK dependency, so basically all CUD operations would be problematic (if working at all).
I think you are trying to solve the "loading unnecessary data" problem is a wrong way. You can achieve the same goal by using simple projection:
var quotesWithLatestAnswer = context.Quotes
.Select(q => new { Quote = q, LatestAnswer = q.Answers.OrderByDescending(a => a.Date).FirstOrDefault() })
.ToList();
Note that the code inside Select will be translated to SQL and executed in the database, returning only the data needed.
To return the latest answer as part of your entity, you can make mark it as unmapped:
public class Quote
{
public int Id { get; set; }
public string Data { get; set; }
public ICollection<QuoteAnswer> Answers { get; set; }
[NotMapped]
public QuoteAnswer LatestAnswer { get; set; }
}
and use a combination of LiNQ to Entities (SQL) and LINQ to Objects query:
var quotes = context.Quotes
.Select(q => new { Quote = q, LatestAnswer = q.Answers.OrderByDescending(a => a.Date).FirstOrDefault() })
.AsEnumerable() // end of db query
.Select(qi => { qi.Quote.LatestAnswer = qi.LatestAnswer; return qi.Quote; })
.ToList();
This way you'll have clean and easy to maintain relational database model as well as efficient retrieval of the data needed.
I cannot get the data from adc to update to database. I am using LINQ2SQL dbml.
I get no sql output from CAPcontext.Log, and Both tables have primary ID's
I feel it is an issue with the join in the LINQ query but couldnt find anything on the web regarding it. Can someone point out what am I doing wrong?
CAP_MDF_dataContextDataContext CAPcontext = new CAP_MDF_dataContextDataContext();
public bool LoadCAPadultdaycare()
{
CAPcontext.Log = Console.Out;
var adc = (from v in CAPcontext.AdultDayCares
join s in CAPcontext.States on v.state equals s.Name
select new CAPadultdaycare {
Avg = v.Avg,
City = v.city,
High = v.High,
Low = v.Low,
StateFullName = v.state,
StateAbbr = s.Code
}).ToList();
foreach (var item in adc)
{ item.City = "some city";
// updating more fields but omitted here
}
CAPcontext.SubmitChanges();
return true;
}
My CAPadultdaycare class is...
public class CAPadultdaycare
{
public decimal? High { get; set; }
public decimal? Low { get; set; }
public decimal? Avg { get; set; }
public string Zip { get; set; }
public string City { get; set; }
public string StateAbbr { get; set; }
public string StateFullName { get; set; }
}
You are creating new CAPadultdaycare objects, which are not attached to your data Context and hence not submitted.
The following may work
var adc = (from v in CAPcontext.AdultDayCares
join s in CAPcontext.States on v.state equals s.Name
select v).ToList();
foreach (var item in adc)
{ item.City = "some city";
// updating more fields but omitted here
}
CAPcontext.SubmitChanges();
but this means you are pulling all columns from your database.
This question already has answers here:
EF Code First: How to get random rows
(4 answers)
Closed 8 years ago.
I got the following 2 models:
public class CrawledImage
{
public CrawledImage()
{
CrawlDate = DateTime.UtcNow;
Status = ImageStatus.Unchecked;
}
public int ID { get; set; }
public DateTime CrawlDate { get; set; }
public string FileUrl { get; set; }
public long Bytes { get; set; }
public ImageStatus Status { get; set; }
public string FileName { get; set; }
public string Type { get; set; }
public virtual ICollection<ImageTag> ImageTags { get; set; }
}
public class ImageTag
{
public ImageTag()
{
}
public int ID { get; set; }
public string Name { get; set; }
public virtual ICollection<CrawledImage> CrawledImages { get; set; }
}
Now what i want is to get a random CrawledImage, which is not the current image and which has all the tags i supply in a List (or HashSet of Collection that doesn't matter). Note that this is a many to many relation. I tried with the code below but i failed so i commented the part about the tags. I prefer to do this with Entity Framework.
public static CrawledImage GetRandomImage(int currentid, List<ImageTag> listtags)
{
try
{
while (true)
{
var randomId = new Random().Next(0, DbContext.CrawledImages.Count());
if (!randomId.Equals(currentid))
{
var image =
DbContext.CrawledImages.Single(i => i.ID.Equals(randomId));
//DbContext.CrawledImages.Where(i => i.ImageTags.Any(tag => listtags.Contains(tag))).First();
if (ProcessImage(image))
return image;
}
}
}
catch (Exception ex)
{
// Failed so return image with id -1
return new CrawledImage {ID = -1};
}
}
Thanks!
Edit:
My question was marked as duplicated. It linked to a question only about retrieving a random record. I got this working. It is more about the many to many relation. Below there is an answer on that but that gives me the error:
An exception of type 'System.NotSupportedException' occurred in EntityFramework.SqlServer.dll but was not handled in user code Additional information: Unable to create a constant value of type 'ChickSpider.Logic.Models.ImageTag'. Only primitive types or enumeration types are supported in this context.
Edit 2:
With the help from Shoe's answer i figured it out. Here is the code
public static CrawledImage GetRandomImage(int currentid = 0, HashSet<string> tags = null)
{
if (tags == null)
return DbContext.CrawledImages.Where(c => c.ID != currentid).OrderBy(c => Guid.NewGuid()).First();
// ImageTags should match any given tags
return DbContext.CrawledImages.Where(c => c.ID != currentid && c.ImageTags.Any(ci => tags.Any(lt => lt == ci.Name))).OrderBy(c => Guid.NewGuid()).First();
// ImageTags should match all given tags
//return DbContext.CrawledImages.Where(c => c.ID != currentid && c.ImageTags.All(ci => tags.Any(lt => lt == ci.Name))).OrderBy(c => Guid.NewGuid()).First();
}
You might be able to accomplish this in less code...
//All crawledImages not equal to current image and contain listtags
var crawledImages = DbContext.CrawledImages.Where(ci => ci.ID != currentid &&
ci.ImageTags.All(ci => listtags.Any(lt => lt.ID == ci.ID));
var random = new Random();
var randImage = crawledImages.OrderBy(ci => random.Next()).FirstOrDefault();
On Another note (maybe personal preference?) I wouldn't return dummy data from the method.
catch (Exception ex)
{
//Log exception
//..
return null;
}
Hi i'm having a little trouble with the following
I have a table (CallRecords) with an navigation property to another table (ResultCodes)
I want to perform a GroupBy from (CallRecords) on ResultCodesId
Sum (The occurrences of ResultCodesId)
First on an included table and field ResultCodes.Name, I.e the name of the resultCode (via navigation property)
var exceptions = Db.CallRecords
.Include(x => x.ResultCode)
.Where(x => x.ClientId == id && x.ResultCodeId < 0 && x.StartTime >= startDate && x.StartTime <= finishDate)
.GroupBy(o => o.ResultCodeId)
.Select(g => new ExceptionViewModel
{
Code = g.Key ?? 0,
Total = g.Count(),
Name = g.First(x => x.ResultCode.Name)
});
This is the problem, the following line wont compile
Name = g.First(x => x.ResultCode.Name)
cannot convert expression type 'string' to return type bool
The answer to this would (seemingly) be fairly simple, however my Google and stack searches are given me back everything except examples i need, so i thought an answer to this might help other unwary travelers
Update
Additional Info
View model
public class ExceptionViewModel
{
public int Code { get; set; }
public int Total { get; set; }
public String Name { get; set; }
}
Data
public class ResultCode
{
[Required,Key]
public int Code { get; set; }
public string Name { get; set; }
public string Description { get; set; }
}
public class CallRecord
{
[Required]
public int Id { get; set; }
// other properties removed for brevity
[Required]
public int? ResultCodeId { get; set; }
public virtual ResultCode ResultCode { get; set; }
}
As you can see all properties involved with that problem expression above are of type string, im not sure whether i'm having a brain-fart or there is something i just dont understand
The expression g.First(x => x.ResultCode.Name) does not do what you think it does. When First has an argument that argument is supposed to be a predicate that filters the enumerable sequence.
In other words, .First(x => ...) is equivalent to .Where(x => ...).First(). If you look at it this way it's clear that x.ResultCode.Name is not valid in that context.
What you want is to get the first item in each group and then fish out some data from it. Do that like so:
g.First().ResultCode.Name
I know my problem is really basic. If I write /api/category/1, I wanna list all Tales with the categoryId==1. I wrote this code, but it gives an error.
[HttpGet]
public IEnumerable<Tale> GetAllTalesByCategory(int id)
{
var tales = TaleService.FindAllTale().Select(x => new Tale
{
TaleId = x.TaleId,
TaleName = x.TaleName,
Content = x.Content,
VoicePath = x.VoicePath
}).Where(x => new Tale
{
x.Categories.Select(c => c.CategoryId).First() == id
});
}
Error:
Error 1 Cannot initialize type 'MasalYuvasi.Entities.Tale' with a collection initializer because it does not implement 'System.Collections.IEnumerable' D:\MvcProject\MasalYuvasi\MasalYuvasi\Controllers\DenemeController.cs 33 13 MasalYuvasi
Models:
public class Tale
{
public int TaleId { get; set; }
public string TaleName { get; set; }
public string Content { get; set; }
public string VoicePath { get; set; }
public virtual ICollection<Category> Categories { get; set; }
public Tale()
{
this.Categories = new List<Category>();
}
}
public class Category
{
public int CategoryId { get; set; }
public string CategoryName { get; set; }
public virtual ICollection<Tale> Tales { get; set; }
public Category()
{
this.Tales = new List<Tale>();
}
}
Try this:
[HttpGet]
public IEnumerable<Tale> GetAllTalesByCategory(int id)
{
var tales = TaleService.FindAllTale().Select(x => new Tale
{
TaleId = x.TaleId,
TaleName = x.TaleName,
Content = x.Content,
VoicePath = x.VoicePath
}).Where(x => x.Categories.Select(c => c.CategoryId).First() == id).ToList();
}
Fixed the where condition, and added .ToList().
The problem is that your code is using a collection initializer here:
new Tale
{
x.Categories.Select(c => c.CategoryId).First() == id
}
I'm not sure what this code is supposed to be doing, but as x.Categories.Select(c => c.CategoryId).First() == id will return a bool, I don't think this is doing what you want it to.
Based on your comment:
I want to list in the category of tales. Forexample I have 2 tales in CategoryId is 1. If I write "/api/category/1" ı want to list this 2 tales.
I think you are looking for something simpler than what you've got. You want to select Tales (represented by x) where Any of the categories have a CategoryId of id:
.Where(x => x.Categories.Any(c => c.CategoryId == id ));
Note that you can append .ToList() to the end of the where clause, as suggested by pjobs, but this may have a subtle effect on the behavior of your application. For more detail, see LINQ and Deferred Execution.