Bottom Line Up Front
Is there a succinct way that I can constrain the number of child entities that can belong to a parent in Entity Framework. I am using 4.3.1 at the moment.
The Problem
I am developing an ASP.NET MVC3 site which accesses data via a data access layer that uses Entity Framework. I have a SearchList entity which has a many to many relationship to a Search entity. A SearchList may have many Searches, and a Search may belong to many SearchLists.
At one point in the workflow of the site, a user needs to select the searches and other items to use in a batch search. We want the page to load the entire search list.
SearchLists can get quite large, and as a test we created one with 21,000 searches. It took a few seconds, and the data returned was about 9.5 MB, which we were expecting, but jQueryUI choked when trying to table-ify that much.
What we would like
So we want to impose a limit on the number of searches any search list can have. I can go through the application and put a bunch of rules in that checks the size of the collection and if the searches that are trying to be added plus the size of the current... yada yada yada.
If however there was a better way (especially one that could easily output an error message that MVC would pick up) I would totally take that instead.
I have googled, and perused a number of EF blogs to no avail. Constrain children and max number of children in collection and similar searches have returned results that are about Linq queries and the Count and Max methods.
Any help would be appreciated.
There is no built-in way so you will have to code such validation yourselves. Some quick ideas:
You can for example use custom collection for the navigation property which will fire exception when you try to add additional search exceeding the threshold. It is simple but it demands you to have all searches loaded, it will have concurrency problems and moreover it can fire during loading search list and searches from database.
You can handle it in overriden SaveChanges. You will at least have to check how many searches are already related to search list but you will still have concurrency problem (what if other request tries to add search to the same list but only one place is remaining - both can succeed the check and insert related search)
You can handle it in database trigger - again it will have concurrency problems
Avoiding concurrency problems completely requires hand written queries with locking hints to ensure that only one request can check number of searches per search list and insert a new search in atomic transaction.
I ended up going with CustomValidationAttribute, and implemented it with a great deal of success. See below for my implementation info:
In the SearchList entity
[NotMapped]
public String ValidationMessage { get; set; }
[CustomValidation(typeof(EntityValidation.EntityValidators), "ValidateSearchCount")]
public virtual List<Search> Searches { get; set; }
public static bool Create(ProjectContext db, SearchList searchList)
{
try
{
db.SearchLists.Add(searchList);
db.SaveChanges();
return true;
}
catch (DbEntityValidationException dbEx)
{
foreach (var validationErrors in dbEx.EntityValidationErrors)
{
foreach (var validationError in validationErrors.ValidationErrors)
{
searchList.ValidationMessage += validationError.ErrorMessage;
}
}
return false;
}
catch (Exception)
{
return false;
}
}
EntityValidators Class
public static ValidationResult ValidateSearchCount(List<Search> Searches)
{
bool isValid;
int count = Searches.Count();
isValid = (count <= 5000 ? true : false);
if (isValid)
{
return ValidationResult.Success;
}
else
{
return new ValidationResult("A maximum of 5000 searches may be added to a SearchList.");
}
}
A similar exception block is on the update method. In this way, when SaveChanges gets called it attempts to validate the entity and its child collections, and when the collection count is greater than 5000 the validator will return an error message which gets caught in the exception handler and stored in a local property for my controller to check when things go wrong.
Related
I'm new using entity framework, and I'm trying to insert into the DB.But I'm having an issue, because I need to only SaveChanges from objects of other 3 scopes. Like this:These are my three Actions that Add the objects into my entities:
public void AddEndereco(entidade_endereco entEndereco)
{
db.entidade_endereco.Add(entEndereco);
}
public void addContato(entidade_contato entContato)
{
db.entidade_contato.Add(entContato);
}
public void addBanco(entidade_banco entBanco)
{
db.entidade_banco.Add(entBanco);
}
And in this action I need to insert all the objects into my DB:
[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult Create(entidade entidade, string Grupo, string Situacao)
{
if (ModelState.IsValid)
{
if (Grupo != "")
entidade.gre_codigo = Convert.ToInt32(Grupo);
if (Situacao != "")
entidade.sie_codigo = Convert.ToInt32(Situacao);
if (entidade.ver_ativo)
entidade.ent_ativo = "S";
else
entidade.ent_ativo = "N";
if (entidade.ver_cliente)
entidade.ent_cliente = "S";
else
entidade.ent_cliente = "N";
if (entidade.ver_fornecedor)
entidade.ent_fornecedor = "S";
else
entidade.ent_fornecedor = "N";
//ADDING ANOTHER OBJECT
db.entidades.Add(entidade);
//HERE IS WHERE I NEED TO SAVE ALL (entidade_endereco, entidade_contato, entidade_banco, entidade)
db.SaveChanges();
return RedirectToAction("Index");
}
return View(entidade);
}
But it is only saving the entidade object, the others don't exist anymore when db.SaveChanges() is executed.
How can I insert into the DB with objects that were added to my entity in other scopes?
If you really want to make this work as is, you would need to store either the Context (really bad idea) or Entities (slightly less bad) across requests. Session State jumps to mind, but using it can bring in a whole load of new pain.
Ideally, you should change your design to take advantage of the stateless nature of HTTP. Each action method should be a separate transaction, saving the data from it's execution when the method is done. If those separate entities only make sense when they are all saved together, then you need to create all of them within a single action and save them to the context together. Managing the boundaries of different business entities and when they are saved is a critical part of application design, I highly recommend you read about Aggregate Roots within Domain Driven Development. Even if you don't go the full DDD route, the Aggregate Root concept will be extremely helpful to you. The CQRS Journey from Microsoft gives an in-depth tutorial of these concepts (and many others)
Im not sure, if I got your question right (excuse my poor spanish). In the Action Create you only add "entidade" to your entidades collection, and so its the only one affected by SaveChanges(). If you want to add others, include in the Create-Action or try making a EF-transaction.
Without transaction the context is lost after the Create-Method ends
I am using dapper and also dapper.contrib.My question is how can I create an generic pagination class.
Here is the what I have tried so far.
public class GenericRepository<T> :IGenericRepository<T> where T : class
{
public async Task<IEnumerable<T>> GetAllPagedAsync(int limit,int offset)
{
var list = await Connection.GetAllAsync<T>();//but this return IEnumarable
return list;
}
}
What I am thinking is get the T name of the class which is the same as Table name,and write an sql string called sql_statement which is apply pagination.later apply this code.
var list = await Connection.QueryAsync<T>("sql_statement")
Does this make sense? I s there any better way to achive that.
It looks currently as though you are planning to retrieve all of the rows in the table and then select from them the page of data you actually require. It would likely be quicker to just select the page you need straight from the database, unless you do actually need all of the rows for some reason.
Assuming that your table names are always going to match exactly with their respective class/entity names, the following will give you a paged result (using postgres):
public class GenericRepository<T> : IGenericRepository<T> where T : class
{
public async Task<IEnumerable<T>> GetAllPagedAsync(int limit, int offset)
{
var tableName = typeof(T).Name;
// assuming here you want the newest rows first, and column name is "created_date"
// may also wish to specify the exact columns needed, rather than *
var query = "SELECT * FROM #TableName ORDER BY created_date DESC Limit #Limit Offset #Offset";
var results = Connection.QueryAsync<T>(query, new {Limit = limit, Offset = offset});
return results;
}
}
A note regarding this. I am obviously not familiar with the structure or size of your database, however for most general purposes the limit/offset approach to paging shown here will most probably be sufficient. There are however some potential issues you may wish to consider:
When the offset value gets very large performance may suffer.
Paging tables with a high frequency of inserts in this fashion may cause results to be duplicated/ appear on multiple pages as the offset values does not take into account new rows added to the table since the last retrieval.
Whether or not these are likely to cause issues to your particular case, these potential drawbacks, as well as some alternatives solutions are outlined here.
I've just added a fourth layer of expand to my query - ie:
.expand("..., ScanDates.Printouts.BMDSites, ...");
And I've discovered that although the data is being returned in the response, it is not populating the objects below "Printouts" (ie. patient.ScanDates.Printouts.BMDSites is an empty array, despite several elements being returned in the response).
I've altered the MaxExpansionDepth on the controller action, and there are no errors appearing on the console or server side. I've also successfully filled BMDSite objects by just querying for them individually, but that would mean ten or twenty return trips to the server... not ideal.
Edit: I've just tried several other queries, and it seems that even if I'm just doing a single expand (ie: .expand("BMDSites")), the same problem occurs - data is in response, but not materialized into entities. When I query just for the BMDSites, (say for a specific Printout) the array is filled and materialized properly.
Edit 2: It just occurred to me that the Printout class is the base class of a TPH inheritance hierarchy... Looking around a bit, I suspect that this is likely the source of the issue.
Thanks so much for any ideas!
-Brad
Looks like it had nothing to do with inheritance after all... In creating a simplified model for Jay, I of course found that it worked just fine. Adding back in the features that I thought were irrelevant, I eventually broke it, and replicated my issue by adding in some [NotMapped] properties that were providing some easy access to the list of BMDSites. For example, in a class derived from Printout:
[NotMapped]
public BMDSite _Ud = null;
[NotMapped]
public BMDSite Ud
{
get
{
if (_Ud == null)
{
_Ud = BMDSites.Find(b => b.Region == Region.Forearm_UD);
}
return _Ud;
}
}
Upon adding this back in, once again, my list of BMDSites were not populating (edit- more specifically, any BMDSite that was touched by an unmapped property was being excluded from the list of BMDSites). Turns out that the JSON.net classes that Breeze uses don't look at [NotMapped] (which makes sense, as it is serialization, not DB mapping)... By adding a reference to JSON.net in my EF model, and adding it's equivilent tag - ie: [NotMapped, JsonIgnore], it doesn't look at the properties, and everything works just fine.
Bottom line (for those that skim)... code above causes issues, code below works fine:
[NotMapped, JsonIgnore]
public BMDSite _Ud = null;
[NotMapped, JsonIgnore]
public BMDSite Ud
{
get
{
if (_Ud == null)
{
_Ud = BMDSites.Find(b => b.Region == Region.Forearm_UD);
}
return _Ud;
}
}
Cheers,
Brad
I would start by insuring that the 'expand' you are doing is in fact valid by trying the exact same query on the server using 'Include's. If this fails then the issue is likely with your model. Breeze 'expand's get turned into EF 'Include's.
If the query works in pure EF, then can you detail the relevant properties in your model and what the inheritance hierarchy looks like so that we can try to replicate your issue?
I am learning MVC4 in Visual Studio and I have many questions about it. My first statement about MVC is that MVC's Model doesnt do what I expected. I expect Model to select and return the data rows according to the needs.
But I read many tutorial and they suggest me to let Model return ALL the data from the table and then eliminate the ones I dont need in the controller, then send it to the View.
here is the code from tutorials
MODEL
public class ApartmentContext : DbContext
{
public ApartmentContext() : base("name=ApartmentContext") { }
public DbSet<Apartment> Apartments { get; set; }
}
CONTROLLER
public ActionResult Index()
{
ApartmentContext db = new ApartmentContext();
var apartments = db.Apartments.Where(a => a.no_of_rooms == 5);
return View(apartments);
}
Is this the correct way to apply "where clause" to a select statement? I dont want to select all the data and then eliminate the unwanted rows. This seems weird to me but everybody suggest this, at least the tutorials I read suggest this.
Well which ever tutorial you read that from is wrong (in my opinion). You shouldn't be returning actual entities to your view, you should be returning view models. Here's how I would re-write your example:
public class ApartmentViewModel
{
public int RoomCount { get; set; }
...
}
public ActionResult Index()
{
using (var db = new ApartmentContext())
{
var apartments = from a in db.Apartments
where a.no_of_rooms == 5
select new ApartmentViewModel()
{
RoomCount = a.no_of_rooms
...
};
return View(apartments.ToList());
}
}
Is this the correct way to apply "where clause" to a select statement?
Yes, this way is fine. However, you need to understand what's actually happening when you call Where (and various other LINQ commands) on IQueryable<T>. I assume you are using EF and as such the Where query would not execute immediately (as EF uses delayed execution). So basically you are passing your view a query which has yet to be run and only at the point of where the view attempts to render the data is when the query will run - by which time your ApartmentContext will have been disposed and as a result throw an exception.
db.Apartments.Where(...).ToList();
This causes the query to execute immediately and means your query no longer relys on the context. However, it's still not the correct thing to do in MVC, the example I have provided is considered the recommended approach.
In our project, we will add a Data Access Layer instead of accessing Domain in controller. And return view model instead of Domain.
But your code, you only select the data you need not all the data.
If you open SQL Profiler you'll see that's a select statement with a where condition.
So if it's not a big project I think it's OK.
I can't see these tutorials but are you sure it's loading all the data? It looks like your using entity framework and entity framework uses Lazy laoding. And Lazy loading states:
With lazy loading enabled, related objects are loaded when they are
accessed through a navigation property.
So it might appear that your loading all the data but the data itself is only retrieved from SQL when you access the object itself.
I'm still trying to get my feet on the ground with Silverlight and RIA Services, and of course starting with some of the more "fun" stuff like grids and intelligent paging. I can connect to RIA Services (using a home-grown ORM, not L2S or EF), get data on the grid, and connect to a DataPager. The domain service is working well with the home-grown ORM, at least for queries. (Still working on full CRUD.) However, there are still problems:
To support the user application, I need user-controlled sorting and filtering, in addition to smart paging (only run the query for the rows needed to display) and grouping.
So far, I've seen nothing in the DataGrid or DataPager to externalize these capabilities so that filtering, sorting, and paging parameters can be passed to the server to build the appropriate query.
The datasets are potentially quite large; my table I've chosen for prototyping work can have up to 35,000 entries at some customers, and I'm sure there are other tables far larger that I will have to deal with at some point. So the "smart paging" aspect is essential.
Ideas, suggestions, guidance, and nerf bricks are all welcome.
OK, I've spent a few days in the weeds with this one, and I think I've got a handle on it.
First, an important piece of magic. For paging to work properly, the pager has to know the total item count, no matter how many items were returned by the current query. If the query returns everything, the item count is obviously the number of items returned. For smart paging, the item count is still the total of available items, although the query returns only what gets displayed. With filtering, even the total of available items changes every time the filter changes.
The Silverlight Datapager control has a property called ItemCount. It is readonly and cannot be databound in XAML, or set directly in code. However, if the user control containing the pager has a DataContext that implements IPagedCollectionView, then the data context object must implement an ItemCount property with PropertyChanged notification, and the DataPager seems to pick this up automagically.
Second, I highly recommend Brad Abrams' excellent series of blog posts on RIA Services, especially this one on ViewModel. It contains most of what you need to make paging and filtering work, although it's missing the critical piece on managing the item count. His downloadable sample also contains a very good basic framework for implementing ModelViewViewModel (MVVM). Thank you, Brad!
So here's how to make the item count work. (This code refers to a custom ORM, while Brad's code uses Entity Framework; between the two you can figure you what you need in your environment.)
First, your ORM needs to support getting record counts, with and without your filter. Here's my domain service code that makes the counts available to RIA Services:
[Invoke]
public int GetExamCount()
{
return Context.Exams.Count();
}
[Invoke]
public int GetFilteredExamCount(string descriptionFilter)
{
return Context.Exams.GetFilteredCount(descriptionFilter);
}
Note the [Invoke] attribute. You need this for any DomainService method that doesn't return an Entity or an Entity collection.
Now for the ViewModel code. You need an ItemCount, of course. (This is from Brad's example.)
int itemCount;
public int ItemCount
{
get { return itemCount; }
set
{
if (itemCount != value)
{
itemCount = value;
RaisePropertyChanged(ItemCountChangedEventArgs);
}
}
}
Your LoadData method will run the query to get the current set of rows for display in the DataGrid. (This doesn't implement custom sorting yet, but that's an easy addition.)
EntityQuery<ExamEntity> query =
DomainContext.GetPagedExamsQuery(PageSize * PageIndex, PageSize, DescriptionFilterText);
DomainContext.Load(query, OnExamsLoaded, null);
The callback method then runs the query to get the counts. If no filter is being used, we get the count for all rows; if there's a filter, then we get the count for filtered rows.
private void OnExamsLoaded(LoadOperation<ExamEntity> loadOperation)
{
if (loadOperation.Error != null)
{
//raise an event...
ErrorRaising(this, new ErrorEventArgs(loadOperation.Error));
}
else
{
Exams.MoveCurrentToFirst();
if (string.IsNullOrEmpty(DescriptionFilterText))
{
DomainContext.GetExamCount(OnCountCompleted, null);
}
else
{
DomainContext.GetFilteredExamCount(DescriptionFilterText, OnCountCompleted, null);
}
IsLoading = false;
}
}
There's also a callback method for counts:
void OnCountCompleted(InvokeOperation<int> op)
{
ItemCount = op.Value;
TotalItemCount = op.Value;
}
With the ItemCount set, the Datapager control picks it up, and we have paging with filtering and a smart query that returns only the records to be displayed!
LINQ makes the query easy with .Skip() and .Take(). Doing this with raw ADO.NET is harder. I learned how to do this by taking apart a LINQ-generated query.
SELECT * FROM
(select ROW_NUMBER() OVER (ORDER BY Description) as rownum, *
FROM Exams as T0 WHERE T0.Description LIKE #description ) as T1
WHERE T1.rownum between #first AND #last ORDER BY rownum
The clause "select ROW_NUMBER() OVER (ORDER BY Description) as rownum" is the interesting part, because not many people use "OVER" yet. This clause sorts the table on Description before assigning row numbers, and the filter is also applied before row numbers are assigned. This allows the outer SELECT to filter on row numbers, after sorting and filtering.
So there it is, smart paging with filtering, in RIA Services and Silverlight!
Here's the quick and dirty solution (that I went for):
Just move your DomainDataSource to your ViewModel! Done!
May not exactly be great for testability and probably some other limitations I haven't discovered yet, but personally I don't care about that until something better comes along.
Inside your ViewModel just instantiate the data source :
// Feedback DataSource
_dsFeedback = new DomainDataSource();
_dsFeedback.DomainContext = _razorSiteDomainContext;
_dsFeedback.QueryName = "GetOrderedFeedbacks";
_dsFeedback.PageSize = 10;
_dsFeedback.Load();
and provide a bindable property :
private DomainDataSource _dsFeedback { get; set; }
public DomainDataSource Feedback
{
get
{
return _dsFeedback;
}
}
And add your DataPager to your XAML:
<data:DataPager Grid.Row="1"
HorizontalAlignment="Stretch"
Source="{Binding Feedback.Data}"
Margin="0,0,0,5" />
<data:DataGrid ItemsSource="{Binding Feedback.Data}">
PS. Thanks to 'Francois' from the above linked page. I didn't even realize I could take the DomainDataSource out of the XAML until I saw your comment!
This is an interesting article from May 2010 about the possible future support for this type of feature in the framework.
http://www.riaservicesblog.net/Blog/post/WCF-RIA-Services-Speculation-EntityCollectionView.aspx