I would like to dynamically customize some POCO classes overriding myself the virtual members to be able to compile LINQ to Entities queries. I know about the ObjectMaterialized event but that happens after the class instantiation. I would like to be able to create the proxy myself, override the virtual members I want and then pass along to the EF, is that possible?
Imagine the following POCO class:
public class Consumer {
/* That´s a virtual property with an association in EF */
public virtual ICollection <Message> Messages { get; set; }
/* That´s the business logic I would like to optimize. */
public virtual Message GetMyLatestMessage()
{
return Messages.Where(m => m.Writer != null && m.Writer.ID == ID && m.Type == "Message")
.OrderByDescending(m => m.Date)
.Take(1)
.FirstOrDefault();
}
}
When I use this code against EF 4 the expression inside GetMyLatestMessage() becomes a SQL query, but I would like to pre-compile these expressions because some of them gets pretty slow to parse everytime.
EF doesn't offer intercepting or replacing dynamic proxy generated for POCOs. Moreover what you show cannot be optimized because it is Linq-to-Objects. EF always loads all messages to memory first time you execute it. That is how navigation properties and lazy loading works and it is also probably reason for your performance problems.
If you want to make optimization exclude your GetMyLatestMessage from Consumer to separate class and use:
public Message GetLatestMessage(int consumerId)
{
return context.Messages.Where(m => m.Consumer.Id == consumerId &&
m.Writer != null &&
m.Writer.ID == ID &&
m.Type == "Message")
.OrderByDescending(m => m.Date)
.Take(1)
.FirstOrDefault();
}
Related
I am making a MSTest project of my app created with Data-driven strategy.
Now, I would like to mock the database using interface. But the DB Entity is automatically generated, so my modification for DI will be destroyed when entity was updated.
Should I give up testing with a mock and access the actual DB every time?
(Update 2022/01/28) I am thinking for compromise: rather than accessing DB entity directly on the model, I would make service facades that on the one class handles DB (for production usage), while the other just works itself.
Short crude examples here:
public interface IMemberDatabaseService
{
Member Search(string name);
void Create(MemberModel model);
}
public class MemberDatabaseService : IMemberDatabaseService, IDisposable
{
private AutomaticallyGeneratedDBContext Con = new();
public Member Search(string name)
{
return Con.Member.SingleOrDefault(mb => mb.Name == name);
}
public void Create(MemberModel model)
{
Member member = Convert(model);
Con.Member.Add(model);
Con.SaveChanges();
}
private static Member Convert(MemberModel model)
{
// convert model to Member
}
// Dispose pattern here...
}
public class MemberTestService : IMemberDatabaseService, IDisposable
{
private static List<Member> MemberList = new();
public Member Search(string name)
{
return name == "John Doe" ? new Member{ Name = name, ...} : null;
}
public void Create(MemberModel model)
{
Member member = Convert(model); // convert model to Member
MemberList.Add(model);
}
private static Member Convert(MemberModel model)
{
// convert model to Member
}
// Dispose pattern here...
}
The drawback is I cannot test the LINQ portion or conflict handling without connecting the DB.
You will need to add specific detail about what your implementation looks like.
Mocking DbContexts/DbSets is possible, but arguably a fair bit of work and ugly to work with. Unit testing is one good argument for implementing a Repository pattern to serve as a boundary for the mocks, but if you're past the point where something like that can be implemented, a simpler solution can be to point your DbContext at an in-memory database that is seeded with suitable test data.
The downside is that most test frameworks accommodate running tests in parallel and not necessarily in order so you need to ensure each test's data row dependencies are safely isolated from each other to ensure you don't get intermittent failures due to one test tampering with data that another test relies on.
I've adopted Serilog for my logging needs.
I (do my best to) follow the SOLID principles and have thus adopted Steven's adapter which is an excellent implementation.
For the most part, this is great. I have a class called LogEntryDetail which contains certain properties:
class LogEntryDetail
{
public string Message {get;set;}
public string MessageTemplate {get;set;}
public string Properties {get;set;}
// etc. etc.
}
I will log the LogEntryDetail like this:
public void Log(LogEntryDetail logEntryDetail)
{
if (ReferenceEquals(null, logEntryDetail.Layer))
{
logEntryDetail.Layer = typeof(T).Name;
}
_logger.Write(ToLevel(logEntryDetail.Severity), logEntryDetail.Exception, logEntryDetail.MessageTemplate, logEntryDetail);
}
I am using the MSSqlServer sink (Serilog.Sinks.MSSqlServer) For error logging, all is well.
I have a perf logger, which I plug into my request pipeline. For this logger, I don't want to save every property in the LogEntry object. I only want to save the Message property in the Message column of the table which I have created.
So, normally, when you call write on the serilog logger and pass in a complex object, the Message column contains the whole object, serialized as JSON.
I want to know if there is some way that I can specify the MessageTemplate to be something like {Message} or {#Message}, so that the Message column in the database only contains the string stored in the Message property of the LogEntryDetail object. Any other property is redundant and a waste of storage space.
When I specify the MessageTemplate to be {Message}, the Message property contains the full name of the LogEntryDetail type (including namespace).
I feel like I am close and just missing some little thing in my comprehension of Serilog's MessageTemplate feature.
I'll just explain what I did here to try and get the best of both worlds. It seems here we have the age-old developer conundrum of sacrificing specific features of a library in order to comply with the SOLID principles. We've seen this before with things like repository abstractions which make it impossible to leverage the granular features of some of the ORMs which they abstract.
So, my SerilogAdapter looks like this:
public class SerilogLogAdapter<T> : ILogger
{
private readonly Serilog.ILogger _logger;
public SerilogLogAdapter(Serilog.ILogger logger)
{
_logger = logger;
}
public void Log(LogEntryDetail logEntryDetail)
{
if (ReferenceEquals(null, logEntryDetail.Layer))
{
logEntryDetail.Layer = typeof(T).Name;
}
if (logEntryDetail.MessageTemplate.Equals(MessageTemplates.LogEntryDetailMessageTemplate, StringComparison.Ordinal))
{
_logger.Write(ToLevel(logEntryDetail.Severity), logEntryDetail.Exception, logEntryDetail.MessageTemplate, logEntryDetail);
}
else
{
_logger.Write(ToLevel(logEntryDetail.Severity), logEntryDetail.MessageTemplate, logEntryDetail.Message, logEntryDetail.AdditionalInfo);
}
}
private static LogEventLevel ToLevel(LoggingEventType severity) =>
severity == LoggingEventType.Debug ? LogEventLevel.Debug :
severity == LoggingEventType.Information ? LogEventLevel.Information :
severity == LoggingEventType.Warning ? LogEventLevel.Warning :
severity == LoggingEventType.Error ? LogEventLevel.Error :
LogEventLevel.Fatal;
}
If the MessageTemplate is one which represents the whole object, then that will be logged. Otherwise, a custom MessageTemplate can be used and the Message property, along with the AdditionalInfo property (a dictionary) can be logged.
We at least squeeze one more thing out of Serilog, and it is one of its strengths - the ability log using different Message templates and to search the log by Message Template.
By all means let me know if it could be better!
I'm developing a multitenant application and would like to have to option to remove a tenant. This however seems to be less trivial than one would assume.
My goal is to delete all references to the tenant everywhere in the database. I understand that Tenant is Soft-Delete, but I since I don't want my database to fill up with old meaningless data I've tried disabling the soft-delete filter.
Here is some code that I've tried:
using (_unitOfWorkManager.Current.DisableFilter(AbpDataFilters.SoftDelete))
{
await TenantRepository.DeleteAsync(x => x.Id == tenantId);
}
This did not work. The tenant is marked as "IsDeleted" but not removed.
Then I figured that maybe it has something to do with UnitOfWork so I made sure no UnitOfWork was active and then manually controlled it:
using (var unitOfWork = _unitOfWorkManager.Begin())
{
// the codeblock above went here
unitOfWork.Complete();
}
This did not work, same result. And this is just the AbpTenant table. I'm also trying to delete from all other tables. For example AbpSettings and AbpLanguages. It's very unclear to me how to do that at all - the "managers" doesn't contain any Delete functions.
I tried creating IRepository for these entities but it does not work. The error reads
The type Abo.Configuration.Setting cannot be used as a type parameter TEntity in the generic type or method IRepository. There is no implicit reference conversion from Abp.Configuration.Setting to Abo.Domain.Entities.IEntity.
That leaves me with the option to use the DataContext directly:
using (EntityFramework.MyDbContext db = new EntityFramework.MyDbContext())
{
List<PermissionSetting> perms = await db.Permissions.Where(x => x.TenantId == tenantId).ToListAsync();
for (int i=0; i<perms.Count(); i++)
{
db.Permissions.Remove(perms[i]);
}
// I also tried deleting them in bulk at first
// ((DbSet<PermissionSetting>)db.Permissions).RemoveRange(db.Permissions.Where(x => x.TenantId == tenantId));
await db.SaveChangesAsync();
}
I tried that with and without UnitOfWork.
But it simply does not get deleted from the database. I'm getting no errors or Exceptions.
Why does it not get deleted? How can I delete it? Surely it must be possible?
since I don't want my database to fill up with old meaningless data I've tried disabling the soft-delete filter.
From the question on Disable SoftDelete for AbpUserRole:
protected override void CancelDeletionForSoftDelete(EntityEntry entry)
{
if (IsSoftDeleteFilterEnabled)
{
base.CancelDeletionForSoftDelete(entry);
}
}
The type Abo.Configuration.Setting cannot be used as a type parameter TEntity in the generic type or method IRepository. There is no implicit reference conversion from Abp.Configuration.Setting to Abo.Domain.Entities.IEntity.
Inject IRepository<Setting, long> instead of IRepository<Setting>.
That leaves me with the option to use the DataContext directly
...
But it simply does not get deleted from the database. I'm getting no errors or Exceptions.
From the documentation on Data Filters:
using (_unitOfWorkManager.Current.DisableFilter(AbpDataFilters.MayHaveTenant))
{
using (var db = new ...)
{
// ...
}
}
That said, there is no way to easily delete related tenant data completely. Consider writing SQL.
Tech:
EF Core 2.0.0
Asp.Net Core Mvc 2.0.0
When i execute this method it throws "InvalidOperationException: Sequence contains no matching element" on evaluating CurrentGrade. Why does it throw and how can I fix it?
I have a search method filtering on quite a lot of properties on a large dataset (10 000 users with thousands of related entities). I'm trying to optimize the query and i don't want to execute the query until all filtering has been made. While using ToList() makes the method work i would rather work against an IQueryable and execute the query when done filtering.
I'm pretty sure this worked before updating EF Core from 1.x to 2.0.
public MemberQueryResult Search(MemberQuery filter)
{
var query = Context.Users
.Include(x => x.Honours)
.Include(x => x.Grades)
.Include(x => x.Strokes)
.Include(x => x.Posts)
.Include(x => x.Loge)
.AsNoTracking();
query = query.ApplyFiltering(filter);
return result;
}
ApplyFiltering() works well for filtering on foreign keys but when filtering on a navigation property collection using .Where() it throws on ICollection Grades on Member when just before the filtering it was included.
This is the method inside ApplyFiltering() that throws:
private static IQueryable<Member> SearchByCurrentGradeRange(MemberQuery filter, IQueryable<Member> result)
{
if (filter.GradeRange == null) return result;
var gradeRange = filter.GradeRange.Split(',');
var gradeFrom = (Grade)int.Parse(gradeRange[0]);
var gradeTo = (Grade)int.Parse(gradeRange[1]);
result = result.Where(x => x.CurrentGrade >= gradeFrom && x.CurrentGrade <= gradeTo);
return result;
}
CurrentGrade is a calculated property on a member, Grade is just an enum.:
public sealed class Member : IdentityUser
{
public Grade CurrentGrade => Grades.OrderBy(x => x.Grade).Last(x => x.ReceivedDate != null).Grade;
public ICollection<MemberGrade> Grades { get; set; } = new Collection<MemberGrade>();
}
The problem is that the unmapped ("calculated") property is causing client evaluation, but at the time EF evaluates the client part of the Where clause, the navigation properties are not loaded yet, hence your Grades collection is empty (as it has been iinitialized with new Collection<MemberGrade> - if you remove the initializer, then you'll get NullReferenceException).
Now, probably it could be treated as EF Core bug. But I strongly recommend not using unmapped properties in LINQ queries in general, and especially in query filter conditions. Even if they work, the client evaluation will cause loading a lot of data in memory just to apply the filter there, rather than at the database (SQL) side.
Also make sure to use SQL translatable constructs. For instance, Last / LastOrDefault have no natural SQL translation, while FirstOrDefault does, so the usual pattern is OrderByDescending().FirstOrDefault() rather than OrderBy().LastOrDefault().
With that being said, the working server side evaluating solution in your case would be like this:
result = result.Where(m => m.Grades
.Where(x => x.ReceivedDate != null).OrderByDescending(x => x.Grade).Take(1)
.Any(x => x.Grade >= gradeFrom && x.Grade <= gradeTo));
Psssst...!
Read on, by all means. But I can tell you here that the problem had nothing to do with the DataContext, but with Dependency Injection. I have left the question up, as it documents one possible issue with the "row not found or changed error" that has nothing to do with real world concurrency conflicts.
It seems the problems have been caused by badly written dependency injection. Or rather, I am beginning to believe, by default lifecycle management by the DI container I used.
The problem was that I used a DataContext as a constructor argument that was supplied by Ninject. It seems that the default behaviour was to cache this DataContext, leading to all manner of unexpected behaviour. I will ask a separate question about this.
Anyway, what follows is my original question, which as you will see, misses the real cause of the issue by a mile...
The Problem
I am getting a number of errors that imply that the DataContext, or rather, the way I am using the DataContext is getting out of synch.
The error occurs on db.SubmitChanges() where db is my DataContext instance. The error is:
Row not found or changed.
The problem only occurs intermitently, for example, adding a row then deleting it. If I stop the dev server and restart, the added row is there and I can delete it no problem.
Ie, it seems that the problem is related to the DataContext losing track of the rows that have been added.
IMPORTANT:
Before anyone votes to close this thread, on the basis of it being a duplicate, I have checked the sql server profiler and there is no "Where 0 = 1" in the SQL.
I have also recreated the dbml file, so am satisfied that the database schema is in synch with the schema represented by the dbml file.
Ie, no cases of mismatched nullable/not nullable columns, etc.
My Diagnosis (for what it is worth):
The problem seems (to me) related to how I am using the DataContext. I am new to MVC, Repositories and Services patterns, so suspect that I have wired things up wrong.
The Setup
Simple eLearning app in its early stages. Pupils need to be able to add and delete courses (Courses table) to their UserCourses.
To do this, I have a service that gets a specific DataContext instance Dependency Injected into its constructor.
Service Class Constructor:
public class SqlPupilBlockService : IPupilBlockService
{
DataContext db;
public SqlPupilBlockService(DataContext db)
{
this.db = db;
CoursesRepository = new SqlRepository<Course>(db);
UserCoursesRepository = new SqlRepository<UserCourse>(db);
}
// Etc, etc
}
The CoursesRepository and UserCoursesRepository are both private properties of the service class that are of type IRepository (just a simple generic repository interface).
SqlRepository Code:
public class SqlRepository<T> : IRepository<T> where T : class
{
DataContext db;
public SqlRepository(DataContext db)
{
this.db = db;
}
#region IRepository<T> Members
public IQueryable<T> Query
{
get { return db.GetTable<T>(); }
}
public List<T> FetchAll()
{
return Query.ToList();
}
public void Add(T entity)
{
db.GetTable<T>().InsertOnSubmit(entity);
}
public void Delete(T entity)
{
db.GetTable<T>().DeleteOnSubmit(entity);
}
public void Save()
{
db.SubmitChanges();
}
#endregion
}
The two methods for adding and deleting UserCourses are:
Service Methods for Adding and Deleting UserCourses:
public void AddUserCourse(int courseId)
{
UserCourse uc = new UserCourse();
uc.IdCourse = courseId;
uc.IdUser = UserId;
uc.DateCreated = DateTime.Now;
uc.DateAmended = DateTime.Now;
uc.Role = "Pupil";
uc.CourseNotes = string.Empty;
uc.ActiveStepIndex = 0;
UserCoursesRepository.Add(uc);
UserCoursesRepository.Save();
}
public void DeleteUserCourse(int courseId)
{
var uc = (UserCoursesRepository.Query.Where(x => x.IdUser == UserId && x.IdCourse == courseId)).Single();
UserCoursesRepository.Delete(uc);
UserCoursesRepository.Save();
}
Ajax
I am using Ajax via Ajax.BeginForm
I don't think that is relevant.
ASP.NET MVC 3
I am using mvc3, but don't think that is relevant: the errors are related to model code.
The problem only occurs intermitently,
for example, adding a row then
deleting it. If I stop the dev server
and restart, the added row is there
and I can delete it no problem.
Your code does not show what the link is between the Added Row and the Delete/Update. Your Add() doesn't return an object reference.
I'm thinking you are missing a Refresh (ie reload the object after Insert). Is your IdCourse also the PK in the Table?
Edit:
Further research has revealed that the problem is with the dependency injection.
The problem was related to how Dependency Injection manages the items it creates. Google for 'lifecycle management' in IoC or DI. Essentially, DI cached a DataContext constructor argument that I injected.
For a way to solve this using the Factory Pattern, see this thread:
Ninject caching an injected DataContext? Lifecycle Management?
The accepted answer solved it all.