I am having problems with the validation of navigation properties.
I do not know if it's a bug or just does not work as I expected.
When you have a navigation property required with his foreign key (in my case int type) is successfully added the validator and, as an int can not be null, is assigned the default value 0, but when the entity is being validated, as the property has value (0), is given as correct. I expected that the entity was not valid.
Is it a bug or correct behavior?
Greetings.
It is a good question. I don't think this is a bug or a feature. Nor would I consider an async validation option.
Instead, I would add a custom validation on the FK (or the property) that declared that the FK property is invalid when it is 0. Yes, that means the entity is in an invalid state at the moment of creation. Nothing can be done about that. You're using '0' as a sentinel value that means "I don't know yet". The validation means "... and the entity is invalid until I know."
Btw, Breeze cannot assume that 0 is invalid. That could be a valid FK to the related entity.
This problem isn't unique to FKs. You have it when creating a new order lineitem with 'quantity=0' and your business rules say it must be >0. The type requires you to specify SOMETHING and yet you cannot provide the correct answer a priori. Again, Breeze can't assume that '0' is a bad value. You have to specify that.
There is no bug per se in either scenario. Both require additional developer attention to "do the right thing".
It's a good question.
Breeze can't actually validate the foreign key alone without performing a roundtrip to the datastore/database, and if it did, validation would become a asynchronous operation. Currently, breeze does not support async validations, but, this might be exactly what you want in some scenarios.
There are some problems with async validations that we would need to tackle. The most important is that another operation such as a save or requery or another change to the fk field might occur before the first validation completes. These can all be handled but there is additional complexity.
If you think that this would be a useful addition, please add it to the Breeze UserVoice. We take these submissions seriously.
Related
Im trying to find out which attributes of an entity have been changed.
As far I have seen, there is a PersistenceSession with a method to check an object if an attribute isDirty. But its always true because it never registers the old object.
So if I take the demo from the QuickGuide and override the update method in the CoffeeBeanRepository:
/**
* #param \Acme\Demo\Domain\Model\CoffeeBean $coffeeBean
*/
public function update($coffeeBean) {
\TYPO3\Flow\var_dump($this->persistenceSession->isDirty($coffeeBean, 'name'), "name changed before");
parent::update($coffeeBean);
\TYPO3\Flow\var_dump($this->persistenceSession->isDirty($coffeeBean, 'name'), "name changed after");
}
... its always TRUE (both), despite I didn't change anything.
Anyone an idea/reference how this can be accomplished?
I am using it for a REST API where a user can't update several fields and on editing of some fields additional actions have to be executed.
The persistenceSession is part of the generic persistence backend of Flow and is neither maintained, nor really used unless you explicitly deactivate doctrine. Hence persistenceSession will not help you, because all entities are considered new for the persistenceSession as you noticed.
With doctrine you need to get the entity changeset from the "UnitOfWork", which you can get from an injected \Doctrine\Common\Persistence\ObjectManager. See also Is there a built-in way to get all of the changed/updated fields in a Doctrine 2 entity
However, this is a suboptimal solution and a hacky work-around at best. If you need to track changes to your entity, it should be an explicit part of your domain model. For example make your setters record a changed properties list, when the given value is different from the current.
When done, you could even optimize doctrines change tracking on the way with that: http://doctrine-orm.readthedocs.org/en/latest/reference/change-tracking-policies.html#notify
I have a case where a breeze entity loses it's originalValues for unmapped properties when that entity is fetched a second time.
I know why this is happening I just don't think it should.
The originalValues are being cleared on the subsequent fetch because breeze sees that the entity is not modified and correctly merges in the incoming values from the server. However, my entity has unmapped properties that have changed and whose original values are stored in the originalValues hash. These are being cleared.
I can understand an argument that says this should be the behaviour with a MergeStrategy set to OverwriteChanges.
However, this clearing of originalValues occurs with a MergeStrategy of PreserveChanges. That behaviour I do not think is a good idea.
With a MergeStrategy of PreserveChanges, when an entity is considered unmodifed, etc, and the incoming values are merged in I do not think originalValues should be cleared.
Now there is a possible workaround by setting using a MergeStrategy of SkipMerge, however this causes problems as the entity is then never refreshed with incoming values.
So can this be behaviour be changed please?
Thanks
Christian
A very interesting use case ... one worth some serious reflection.
Let's walk through the semantics of merging query results when the MergeStrategy is "PreserveChanges".
First, some observations:
Breeze does not change the EntityState when you change an unmapped property
Breeze DOES preserve the pre-change value of an unmapped property in OriginalValues.
Let's consider how Breeze merges query results:
when the EntityState is in a changed state, all properties (mapped and unmapped) are preserved even if a corresponding queried property value is different.
when the EntityState is "Unchanged", all properties (mapped and unmapped) are replaced by corresponding queried property values discovered in the query results payload.
Breeze does not support a partial merge. It will not overwrite some properties and preserve others.
I don't know for certain why query processing clears OriginalValues (I say that without having looked into it yet or even confirmed that it does). But I think it is doing the right thing.
Consider what it means when any property is merged into an unchanged entity. Suppose property foo was 1 before the query and the new queried value is 2.
Subsequently, we set foo to 3. What should the the "original value" be at that time? It should be 2, right? It shouldn't be 1. It should be the last value retrieved from the server.
It follows that after a merge, original values are implicitly identical to the entity's most current values. That is, if a property had a value in OriginalValues, it should be identical to the corresponding current value. And if the current value === the original value, we might as well remove that property from the OriginalValues ... which means clearing the OriginalValues for that property is harmless.
Because Breeze performs full merges (not partial merges), it follows that clearing the entire OriginalValues is consistent behavior.
According to this line of reasoning - if you believe that mapped and unmapped properties should follow the same merge rules - Breeze is doing exactly the right thing.
Perhaps what you want is to treat a change to an unmapped property as a change to the EntityState.
I think we did that at one time in the past but we were persuaded that the EntityState should only reflect changes to mapped properties which are presumed to be the persisted properties.
It appears that you disagree, at least in your specific use case. I think (please confirm) that when you change an unmapped property, you want Breeze to treat the entity as "dirty".
Fortunately, you can handle this yourself. Add a handler to the EntityManager.entityChanged event and change the EntityState when something updates an unmapped property. You can do this broadly or selectively.
You'll have an uphill climb if you really seek different treatment of mapped and unmapped properties. We'll listen of course ... but you'll have to make one heck of a great argument.
You might get further by proposing something more general purpose such as a new MergeStrategy that supports a developer-defined custom policy.
Maybe we'd extend the metadata such that you can specify a merge function that governs how Breeze merges properties for a given EntityType. That would be cool.
It would be a significant new feature, requiring a ton of testing. I think it's going to take a lot of persuading to get that one on the board.
I'm having some trouble trying to figure out how to order ActiveRecord writes to make my validations be happy, and I'm not sure what to search for this kind of problem.
The problem is that before the request would occur, everything would be valid; after the transformations would occur, everything would be valid again; but while the transformation is happening, since it's impacting more than one model instance, the database would enter an invalid state if I update each model one by one without taking into account both changes at the same time. I'd love some suggestions!
Background
I have a model called HelpRequest and another called HelperAssignments.
The rule is that a HelpRequest may have 0 or 1 active HelperAssignments. But if a Helper cannot complete the request, they may reassign it to another Helper, creating a new HelperAssignment. Since we need the history of assignments to a particular HelpRequest, there may be a number of HelperAssignments for a HelpRequest, but only one is active.
As a result, the HelperAssignment table has a few relevant attributes:
help_request_id: Refers to the HelpRequest corresponding to this assignment.
close_status: If this is set to reassigned, reassignment_id must be present.
reassignment_id: For a given help_request_id, only one may be nil (i.e. it is the current active assignment)
Problem
When a reassignment happens...
... if I create the new HelperAssignment first, it would break validations because more than one active HelperAssignment for the request would be present :(
... if I update the old HelperAssignment first to have a close_status of reassigned, the new HelperAssignment wouldn't exist yet so I couldn't get its ID, and therefore the validations would fail.
Is there an idiomatic way to do this transformation? I'd like to avoid a) disabling validations for this particular type of requests, or b) adding an extra database state for "being in the process of reassigning". Looks like enforcing referential integrity in models can get a little tricky in Rails... thanks in advance!
The Problem
I'm just trying to figure out exactly how much of my own security I need to implement on the server side when saving changes in Breeze. In particular, I'm thinking about how a malicious user could manually hack the SaveChanges request, or hack the javascript in the client, to bypass my normal business rules - for example, to maliciously alter foreign key IDs on my entities.
I want to understand exactly where I need to focus my security efforts; I don't want to waste time implementing layers of security that are not required.
I'm using Breeze with .net and Entity Framework on the server side.
Example
Here's a trivial example. ObjectA has a reference to an ObjectB, and ObjectA is owned by a particular User. So, my database looks like this:
ObjectA:
Id ObjectB_Id SomeField User_Id
1 1 Alice's ObjectA 1
2 2 Bob's ObjectA 2
ObjectB:
Id SomeOtherField
1 Foo
2 Bar
User:
Id Name
1 Alice
2 Bob
From this model, the security concerns I have are:
I don't want unauthenticated users to be changing any data
I don't want Bob to be able to make any changes to Alice's ObjectA
I don't want Alice to try to point her ObjectA at Bob's ObjectB.
I don't want Bob to try to change the User_Id on his ObjectA to be Alice.
The solution for (1) is trivial; I'll ensure that my SaveChanges method has an [Authorize] attribute.
I can easily use Fiddler to build a SaveChanges request to reproduce issues 2 to 4 - for example, I can build a request which changes Alice's ObjectA to point to Bob's ObjectB. This is what the message content might look like:
"entities":
[
{
"Id":1,
"ObjectB_Id":2,
"SomeField":"Alice's ObjectA",
"User_Id":1,
"entityAspect":
{
"entityTypeName":"ObjectA:#MyNamespace",
"defaultResourceName":"ObjectAs",
"entityState":"Modified",
"originalValuesMap":
{
"ObjectB_Id":"1"
},
"autoGeneratedKey":
{
"propertyName":"Id",
"autoGeneratedKeyType":"Identity"
}
}
}
],
As I'd expect, when no security is implemented on the server side, this persists the updated value for ObjectB_Id into the database.
However, I've also confirmed that if there is no entry for ObjectB_Id in the originalValuesMap, then even if I change the value for ObjectB_Id in the main body of the message it is NOT updated in the database.
General Rules?
So, I think this means that the general security rules I need to follow on the server are:
[Edited 4 July 2013 - rewritten for clarity]
In general:
Nothing in the message can be trusted: neither values in the originalValuesMap nor supposedly "unchanged" values
The only exception is the identity of the entity, which we can assume is correct.
Supposedly "unchanged" properties may have been tampered with even if they are not in the originalValuesMap
For "Unchanged" properties (properties which are not also on the originalValuesMap):
When "using" any "unchanged" property, we must NOT use the value from the message; we must retrieve the object from the database and use the value from that.
for example, when checking owenership of an object to ensure that the user is allowed to change it, we cannot trust a UserId on the message; we must retrieve the entity from the database and use the UserId value from that
For any other "unchanged" property, which we are not using in any way, we don't need to worry if it has been tampered with because, even if it has, the tampered value will not be persisted to the database
For changed properties (properties which are also on the originalValuesMap):
Business rules may prevent particular properties being changed. If this is the case, we should implement a check for each such rule.
If a value is allowed to be changed, and it is a foreign key, we should probably perform a security check to ensure that the new value is allowed to be used by the session identity
We must not use any of the original values in the originalValuesMap, as these may have been tampered with
[End of edit]
Implementing the Rules
Assuming that these rules are correct, I guess there are a couple of options to implement security around the changed foreign keys:
If the business rules do not allow changes to a particular field, I will reject the SaveChanges request
If the business rules DO allow changes to a particular field, I will check that the new value is allowed. In doing this, CANNOT use the originalValuesMap; I'll need to go to the database (or other trusted source, eg session Cookie)
Applying these rules to the security concerns that I gave above,
security concern (2). I'll need to check the user identity on the session against the User_ID on the ObjectA that is currently in the database. This is because I cannot trust the User_ID on the request, even if it is not in the originalValuesMap.
security concern (3). If the business rules allow a change of ObjectB, I will need to check who owns the new value of ObjectB_Id; I'll do this by retrieving the specified ObjectB from the database. If this ObjectB is not owned by ObjectA's owner, I probably want to reject the changes.
security concern (4). If the business rules allow a change of User, this is already covered by (2).
Questions
So, really, I'm looking for confirmation that I'm thinking along the right lines.
Are my general rules correct?
Does my implementation of the rules sound reasonable?
Am I missing anything?
Am I over complicating things?
Phil ... you are absolutely on the right track here. You've done a nice job of laying out the issues and the threats and the general approach to mitigating those threats. It is almost as if you had written the introduction to the Breeze security chapter ... which we haven't gotten to yet.
I do not think that you are "over complicating things"
Someone reading this might think "wow ... that's a lot of work ... that Breeze stuff must be insecure".
Well it is a lot of work. But it isn't Breeze that is making it difficult. This is the necessary thinking for every web application in existence. Authentication is only the first step ... the easiest step ... in securing an application.
You shouldn't trust any client request ... even if the client is authenticated. That means making sure the client is authorized to make the request and that the content entering and exiting the server is consistent with what the client is both claiming to do and is allowed to do. These are general principles that apply to all web applications, not just Breeze applications. Adhering to these principles is no more difficult in Breeze than in any other technology.
One Breeze technicality you may have overlooked. The EFContextProvider.Context should only hold the entities to save; don't use it to retrieve original entities.You'll need a separate DbContext to retrieve the original entities to compare with the change-set entities from the client.
We are working on samples that demonstrate ways to handle the issues you described. For example, we're recommending (and demo'ing) a "validation rules engine" that plugs into the BeforeSaveEntitiesDelegate; this "engine" approach makes it easier to write bunches of server-side rules and have them applied automatically.
Our samples and guidance aren't quite ready for publication. But they are coming along.
Meanwhile, follow your instincts as you've described them here. Blog about your progress. Tell us about it ... and we'll be thrilled to highlight your posts.
I've been looking for guidance on the same matter and I am very happy to find your brilliant analysis. In my opinion the answer to our problem is different though, assuming that we are talking about applications which are to be composed of more than a few modules and are to live longer than a year.
If rules become too complicated it means that we might be using inappropriate approach. I'm sure many brilliant developers would cope following these rules but the sad truth is that most of our peers would either get it wrong or would forget about some of them under pressure.
I'd say that we need to go back to Fowler's, Evans' and Nilssons' publications and repeat after them that in larger applications (and these have strong security requirements) the entity model is not something that should be exposed to the client at all (for other reasons than security too - e.g. maintainability).
On the other hand it is worth looking at revisions to these original ideas proposed later by Greg Young and Udi Dahan. These in essence say that model for reading does not have to and often is not the same as model for writing 'data'.
To sum this up I'd say that the base rule should be DON'T use Breeze for writing and DO use it for reading (with DTOs/Projections), provided you don't query the 'real' model but the model built specially for reading (e.g. Views not Tables).
All this quite naturally emerges if you follow your domain and use cases and above all if you follow Test-Driven approach. Would you really end up with BeforeSaveEntities solution for business rules while following Test-Driven-Development?
I have this problem of confusing when to include the entire object as a property of another object, or just its ID. It seems that if I include the entire object, the calls to load the containing object will unnecessarily also load the included object when I probably only need references. What is propert approach?
Generally always refer to another object.
Many ORM technologies have the idea of "proxies" and "lazy loading", meaning, unless you reference the object, it won't load it.
I prefer to include the object itself, since one object actually has a relationship with another actual object -- the object ID is just an implementation detail. To deal with the problem of unnecessary calls, look into "lazy loading".
Only include the other object if you need the details.
in MVC use a ViewModel ideally and not your entities. Your ViewModel contains only what it needs, so for example OrderEditViewModel would contain a customerid unless you want to display the custom name, in that case you would include the fields from customer. Some people recomend you flatten out your objects to a view model, so you dont have OrderEditViewModel.Customer.CustomerId but instead ORderEditViewModel.CustomerId. Automapper can help you do this (As well as valueinjecter - note the spelling)
If you must include an ID ensure when you save back to the database your update include a clause to say 'where id=#customerId and (logic here to ensure your user actually has access to that customerid and root object)
I have mvcsecurity.codeplex.com to help encrypt record ids on a web page to prevent against tampering as well (it helps but you should still have something in your query to prevent field tampering so an attacker cant add someone else's customer id for example_)
I go more into parameter tampering in MVC here if anyone is interested:
http://www.pluralsight-training.net/microsoft/Courses/TableOfContents?courseName=hack-proofing-dotnet-app
My suggestion would be to always think about the design and not about performance. Performance can be tweaked but design can't. So, if the two objects have that kind of a relationship where Aggregation/Composition is required, you should do that.
But, if your containing object only has to deal with the ID (for e.g. passing it to a different object which processes the ID to do something) then you can keep the ID field only. No need to expose the whole object (but make sure that your containing object does not need to know anything about the other object.).