This is from the category "unexpected behavior" - take the following query (you can paste it in Graph Explorer):
https://graph.microsoft.com/v1.0/users?$filter=idc eq 'test'
This returns status code 400 and "Property 'idc' does not exist as a declared property or extension property." Which is a sensible and understandable response.
Now, if try to $select this property:
https://graph.microsoft.com/v1.0/users?$select=idc
I get a result I totally don't expect:
{
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#users(idc)",
"#odata.nextLink": "https://graph.microsoft.com/v1.0/users?$select=idc&$skiptoken=cut",
"value": [
{},
{},
...
{}
]
}
(a list of empty objects; asking for a single user with that invalid property name returns me an emtpy response).
So my question is - why does $filter error out and $select does not? Would there be a way to force $select to error out too? (eg I am using the /beta endpoint and a property name changes - I want my code to fail to find out)
Sorry for the late answer. We had a discussion on this and would love to get some of your thoughts (and the thoughts of other developers). We don't have a clear answer on this yet.
There are 2 schools of thought here:
Make $select and $filter behave consistently when dealing with properties that don't exist.
It's OK for these to differ in behavior, since the caller’s intent when specifying $select is likely different than that of $filter. The service cannot ignore a property specified in $filter because it completely changes the set of objects that are returned. However, $select does not change the set of objects but only drops the properties that are not available. Hence $select and $filter do not need to be consistent.
Thoughts?
Hope this helps,
Related
I have an ODATA query that pulls the first record from the Dynamics AX table and ignores the $filter clause when the $top parameter is the first parameter. However, it pulls the correct record for the $filter clause when the $top parameter is at the end of the query.
Question: How do I make the OData server honor the full query no matter where the $top parameter is placed?
the following two cases return identical results:
Case 1: https://url/data/Users?$top=1
{
"#odata.context":"https://url/data/$metadata#Users","value":[
{
"#odata.etag":"W/\"J5zEx4NDg33OD2YyM4TEs5NTY6zNz7E0ND7U3Nic=\"",
"Id":"first_user_in_the_table",
"Alias":"first_user_in_the_table#domain.com",
"Name":"first_user_in_the_table"
}
]
}
Case 2: https://url/data/Users?$top=1&$filter=Alias%20eq%20%27specific_user#domain.com%27
{
"#odata.context":"https://url/data/$metadata#Users","value":[
{
"#odata.etag":"W/\"J5zEx4NDg33OD2YyM4TEs5NTY6zNz7E0ND7U3Nic=\"",
"Id":"first_user_in_the_table",
"Alias":"first_user_in_the_table#domain.com",
"Name":"first_user_in_the_table"
}
]
}
However, when I move the $top parameter to the end of the query string I get the correct result.
Case 3: https://url/data/Users?$filter=Alias%20eq%20%27specific_user#domain.com%27&$top=1
{
"#odata.context":"https://url/data/$metadata#Users","value":[
{
"#odata.etag":"W/\"Jz3kz4Nj5g2N6zcz7MCw81Nj9M3M2TU3w4NT4c54Jw==\"",
"Id":"specific_user",
"Alias":"specific_user#domain.com",
"Name":"The Specific User I Actually Want"
}
]
}
As far as I understand the OData specification, this is incorrect behavior. I base this on 11.2 Requesting Data which states that the $top query option should be evaluated last and after the $filter query option. Also section 2. Query Options of the OData ABNF construction rules seems to indicate that the query options may appear in any order in the URL.
This indicates that this particular behavior may have been incorrectly implemented. You will probably not be able to solve this short of implementing a custom service that wraps the existing one to reorder the query options as needed. I would also assume that this impacts the behavior of all Odata services.
I suggest you create a support request with Microsoft so they can adress this.
I am getting a dynamic value (_FilterDate) on the parent type that I want to use as a filter for the nested type /Trips but can't get it to work because I still get entries in the nested data that do not meet the filter. Actually, there is no difference whether I use this filter or not.
$filter=Trips/all(d:d/EndDate ge _FilterDate)
I also tried this:
$expand=Trips($filter=EndDate ge $it/_FilterDate)
but got the error: "Could not find a property named '_FilterDate' on type 'Default.Trips'."
So I'm wondering how to get the syntax right and thus kindly ask for help.
Example portion:
"value": [
{
"_FilterCompany": "YES",
"_FilterLocation": "YES",
"_FilterDate": "2020-01-08",
"Trips": [
{
"StartDate": "2019-06-24",
"EndDate": "2019-06-28",
},
{
"StartDate": "2020-02-07",
"EndDate": "2020-02-07",
}
]
}
There are two issues going on here:
this response is specifically regarding the OData v4 specification and the .Net ODataLib implementation.
You have correctly identified that when filtering results based on a nested collection you you must separately apply the filter within that collection if you want the fitler to apply to the items within that collection as well.
This is because the root level $filter criteria only affects the selection of the root items, think of it as if the $expand operator is applied after the $filter has identified the top level of row the return, $expand is simply executed as a Linq Include statement.
In your second attempt, $it references the instance of Trips, which is a known bug/by design, according to the spec it is expected to work the way you have implemented it:
5.1.1.6.4 $it
Example 82: customers along with their orders that shipped to the same city as the customer's address. The nested filter expression is evaluated in the context of Orders; $it allows referring to values in the outer context of Customers.
http://host/service/Customers?
$expand=Orders($filter=$it/Address/City eq ShipTo/City)
So knowing the $it is broken, the spec doc does specify a $root identifier that you might also be able to use, but in ODataLib 7.3 $root is still not supported OOTB either. There is an issue logged here: $it references the wrong resource #616
Workaround
If your Trips data type has a navigation property back to the Filter/root record, then you can use that navigation property as part of the $filter:
Assuming the navigation property is called Filter
$filter=Trips/all(d:d/EndDate ge _FilterDate)&$expand=Trips($filter=EndDate ge Filter/_FilterDate)
If your Trips type does not have this navigation link back to the parent record then you are stuck at this stage with these two workarounds:
Create a Function on the controller to return this filtered data specifically, as this would be simple to evaluate as a Linq query in the server-side.
Accept that the server will return extra rows in the Trips collections, and apply the filter over the results in the client-side.
How to count number of items in EntitySet of MicrosoftGraph, for example 'users' or 'groups'? I tried:
https://graph.microsoft.com/v1.0/users?$count
Returns: lists all users
https://graph.microsoft.com/v1.0/users/$count
Returns:
{ "error": { "code": "Request_BadRequest", "message": "Unexpected segment Edm.Int32.", } }
Also in Annotations of target "microsoft.graph.directoryObject" which are those EntitySets based on I see that it is Selectable=false, Countable=false ...
Will $skip be ever available on 'users' or other toplevel EntitySet items ( https://graph.microsoft.com/v1.0/groups?$skip=5 ) ? It is available on other items ( https://graph.microsoft.com/v1.0/me/contacts?$skip=5 ). I know about $skipToken, but it is not the same.
Can I find somewhere in the $metadata if property is sortable? For example user.displayName is sortable, but user.mail or user.givenName are not. This would be handy in the $metadata. Is there plan to introduce this into $metadata?
OrderBy DESC in this formula https://graph.microsoft.com/v1.0/users?$orderBy=displayName%20desc is ignored, it shows items ordered ASC, am I doing something wrong?
Not much help I know, but if you do an API call that gets a Collection you can get a count using the second form you gave in the first question. Eg:
https://graph.microsoft.com/v1.0/users/<id>/events/$count
returns the count (6 in my case, and not in JSON - the returned data is actually "\x{ef}\x{bb}\x{bf}6" (in Perl formatting)). If we use the ? as the last separator (which is what http://graph.microsoft.io/en-us/docs/overview/query_parameters seems to indicate we should) with this URL:
https://graph.microsoft.com/v1.0/users/<id>/events?$count
I just get the list of events with no count as you do.
So that seems to indicate two things to me:
a) The $count doesn't appear to work as a query parameter, despite the documentation and the OData standards saying it should,
and
b) There seems to be a bug in the API for handling EntitySets which isn't there for Collections.
Sorry I can't be of more help, but its another data point at least (I just came unstuck with the same thing which is why I noticed this StackOverflow post!)
Ad 1. https://graph.microsoft.com/v1.0/users/$count is the correct OData syntax (http://docs.oasis-open.org/odata/odata/v4.0/errata02/os/complete/part2-url-conventions/odata-v4.0-errata02-os-part2-url-conventions-complete.html#_Toc406398087), but as indicated in the metadata directoryObject collections are not currently countable. ODataV4 also allows $count in query parameters (http://docs.oasis-open.org/odata/odata/v4.0/errata02/os/complete/part2-url-conventions/odata-v4.0-errata02-os-part2-url-conventions-complete.html#_Toc406398166), but then it should be specified with true or false value (e.g. https://graph.microsoft.com/v1.0/me/events?$count=true) and the response will include both the collection and its count in the #odata.count property. This is again not supported for directoryObject collections.
Ad 2. There is no plan right now to support $skip for directoryObject collections.
Ad 3. Yes, we plan to indicate which properties are sortable by in metadata using the SortRestrictions annotation defined in the OData capabilities vocabulary (http://docs.oasis-open.org/odata/odata/v4.0/errata02/os/complete/vocabularies/Org.OData.Capabilities.V1.xml)
Ad 4. Your request is correct, but we only support sorting users by displayName in the ascending order.
I've been trying to call a entity, but i'm supposed to call the results from its associated entities. I tried to do it with the following URL:
/sap/opu/odata/XXXXXXXX/SERVICE_NAME/MatnrGetdetailCollection?$filter=IVendorId eq '1701' and ILanguage eq 'P' and IMaterial eq 'M-05'&$expand=MatnrClassGetdetail
I must use the filter because the called function has these mandatory parameters.
Am I making any mistake on the URL or the error isn't there?
In general $filter and $expand can be combined, we use it in our application. Therefore please see Layla's Comment. In addition, you should tell us what the actual error is.
If MatnrGetdetailCollection is indeed an entity set, then the corresponding entity must have a navigation property of the name MatnrClassGetdetail, otherwise $expand won't work.
There is some problem with the URL when you want to go for obligatory parameters. Please pass them as key values in segw and and go for the format I'm sending:
/sap/opu/odata/sap/SERVICE_NAME/EntitySet(keyfield='value',keyfield='value')/?$expand=navigationName
The object extraMetadata is undefined and throwing an error on line 247 of breeze.labs.dataservice.sharepoint.js
rawEntity.__metadata = { 'type': aspect.extraMetadata.type };
I suspect it is because I have not defined the type found in __metadata object on my entity definitions for breeze. Any suggestions on how to define my type correctly would be very welcome! Here is my type definition for one of the objects.
models.Project = {
name: 'Project',
defaultResourceName: 'getbytitle(\'Projects\')/items',
dataProperties: {
ID: {
type: breeze.DataType.Int32
},
Title: {
nullable: false
},
StatusId: {
type: breeze.DataType.Int32,
nullable: false
},
SelectedApproverId: {
type: breeze.DataType.Int32,
nullable: false
},
Created: {
type: breeze.DataType.DateTime
},
Modified: {
type: breeze.DataType.DateTime
}
},
navigationProperties: {
Status: {
type: "Status",
foreignKeyNames: ['StatusId'],
hasMany: false
},
SelectedApprover: {
type: "User",
foreignKeyNames: ["SelectedApproverId"]
}
}
};
UPDATE: 11/11/2013
If I run the following query:
return breeze.EntityQuery
.from(metadataStore.getEntityType('Project').defaultResourceName)
.orderBy('Created desc')
.using(manager)
.execute()
.then(function (data) {
console.log(data.results);
return data.results;
});
the results are an array of simple JavaScript objects, not Breeze Entities, that lack an __metadata properties. I'm trying to figure out why this is the case.
Update: 11/12/2014
I have confirmed that this issue presents itself when I have multiple entities defined under navigationProperties.
Please be sure you are using BreezeJS v.1.4.12 or later.
To be clear, the code to which you refer is on line 147 (not 247) of the breeze.labs.dataservice.sharepoint.js file in my possession.
It's located within the _createChangeRequest where it is preparing to save a modified entity. I'll assume that you have queried a Product entity, made changes to it, and are saving it back when the error occurs.
I don't believe the problem will be traced to how you defined the metadata for your Product type.
You should NOT define a __metadata property for your type. The __metadata property is something we expect SharePoint (any OData source in fact) to add to the JSON entity data that it sends to the client when you query that OData source for entities.
__metadata wouldn't be defined for results returned by a projection but then your issue concerns a modified entity so I'm assuming that you acquired this entity through a normal query ... one that did not have a select clause.
I'd like to know if you see the __metadata property in the JSON payload of a query that retrieved the entity you were modifying. Please examine the network traffic from the query request. If you don't see it there, we have to find out why the server didn't send it.
Background
The __metadata property on the JSON node is a crucial part of the contract with the SharePoint OData server. That's how the Breeze client learns about the entity's type and its etag.
Look at the jsonResultsAdapter.visitNode and updateEntityNode methods. You'll see how the adapter uses __metadata to determine the EntityType for that data. You'll also see that the adapter moves the __metadata to the adapter result's extraMetadata property. BreezeJS subsequently moves that "extra metadata" from this result object to the entity's entityAspect.extraMetadata property.
Does this seem tortured? It is tortured. OData requires extra information to be carried around with the entity (specifically the etag) without which the server simply will not update or delete the entity. We have to squirrel that info away somewhere, out of your hair, and then bring it back when we make save requests to the server. We put it on the entityAspect in keeping with that property's role as the keeper of the "entity-ness" that has nothing to do with your object's business purpose and everything to do with how it is persisted.
So much for the why. Where is the bug?
The bug
The underlying bug is that this __metadata from the SharePoint OData source has disappeared. We don't know how it disappeared yet. But we're in big trouble without it.
The sharepoint adapter should give a better message when extraMetadata is missing. We actually look for that problem a few lines later; see adjustUpdateDeleteRequest:
var extraMetadata = aspect.extraMetadata;
if (!extraMetadata) {
throw new Error("Missing the extra metadata for an update/delete entity");
}
That test appears too late. I'll make a note to move the test up.
But such a fix will only cause the save to fail with a better message. It won't tell you how to fix it.
So let's work on finding where the __metadata disappeared ... starting with whether it ever arrived in the first place.
I await your report.
Update 17 July 2014
I'm still waiting to hear if you are seeing the __metadata property in the payload of the response to the original entity query.
Meanwhile, I checked the OData specs (plural) for references to the __metadata property. It appears that the __metadata property has always been optional. It follows that an OData provider need not send or honor the etag ... and we know that this is possible because Web API 2 OData didn't support etags ... a defect soon to be corrected.
See the OData v.2 spec where it describes JSON format. Search for the term "__metadata".
The OData v.3 spec also calls for the __metadata property in a JSON response (at least a JSON verbose response).
But ... heavy sigh ... it appears that the __metadata property is gone from the v.4 spec and that the metadata information is supplied entirely through JSON annotations. The DataJS library (used by many but not all BreezeJS OData adapters) may map those annotations into the node's __metadata property but I can't confirm it yet. We have some work to do coping with all of these variations.
In the meanwhile, I think all BreezeJS OData dataservice adapters should take a more defensive position regarding extra metadata and should simply ignore the omission rather than throw exceptions.
We'll make these defensive changes very soon.
Of course the server will reject your update or delete request if the OData service actually requires an etag or other metadata. I don't know what we can do about that just yet.
There hasn't been a post in a while but I am going to share what I found as the problem and how I resolved it for me (because it took me a long time).
Basically the breeze.labs.dataservice.sharepoint adapter has a function serverTypeNameToClientDefault() that expects the SharePoint custom list type as returned by REST/OData in the __metadata "type" field to be in the exact format of:
SP.Data.**mylistname**sListItem** (notice the "sListItem" suffix; ; Ex. SP.Data.CustomersListItem)
This function does a string regex to extract the Breeze entity name from the SharePoint type and uses that name to look up the entity in the metadata store ("Customer" in the above example). If there is no match, Breeze will not find your entity and will return a basic object instead of a Breeze entity. Therefore your REST JSON result returned from SharePoint, even though it does have the __metadata property is not converted into a Breeze entity that contains the property entityAspect.extraMetadata, among other things. This is what leads to the error "Unable to get property 'type' of undefined or null reference"
For my solution, since in my case I don't care as much what the URL of my custom lists are, I just made sure that when my custom list was provisioned by SharePoint that it resulted in a name according to what Breeze expects. You accomplish this by setting the Url attribute of the ListInstance element like this:
<ListInstance
Title="My Customers"
OnQuickLaunch="TRUE"
TemplateType="10000"
Url="Lists/Customers" <!-- List/Customer will not work -->
Description="My List Instance">
...
The better solution would be to make the serverTypeNameToClientDefault() function more robust or fix it to my needs locally but hopefully this can be addressed in a future version of the adapter.
Note that I have tested this solution with the following configurations (not all dependencies listed):
Breeze.Client 1.4.9 with Breeze.DataService.SharePoint 0.2.3
Breeze.Client 1.5.0 with Breeze.DataService.SharePoint 0.3.2
Also note that the 0.3.2 version of the adapter now displays a better error message when this happens as mentioned above -- "Missing the extra metadata for an update/delete entity"; but it doesn't fix the problem.
Hope this helps someone.
For breeze v1.4.14 and breeze labs sharepoint 2013 v0.2.3 i am using small fix in file breeze.labs.dataservice.sharepoint.js.
At the end of function
function visitNode(node, mappingContext, nodeContext)
just before
return result;
i just set property extraMetadata like this:
result.extraMetadata = node.__metadata;
This seems to fix problem when i try to save modified entity back to sharepoint.
Sorry folks for the long overdue aspect of this, but I got the bug with the extra "s" resolved today... FINALLY. You can track the issue here: https://github.com/andrewconnell/breeze.js.labs/issues/6
This all stemmed from a very incorrect assumption I made. It's been fixed in version 0.6.2 of the data service adapter for SharePoint. Note that you MUST use the same name for your entity when creating it in the metadata store as the list where the data is coming from.
I resolved my issue with multiple navigationProperties on an entity by editing line 319 of breeze.labs.dataservice.sharepoint.js v.0.10.0
I changed:
if (entityType._mappedPropertiesCount <= Object.keys(node).length - 1)
to:
if (entityType.dataProperties.length <= Object.keys(node).length - 1)
It looks like the _mappedPropertiesCount includes the navigationProperties count too. e.g. dataProperties.length + navigationProperties.length
The query node was then thought to not contain a full set of properties for the entity (it was assumed to be the result of a partial projection).
It therefore wasn't being treated as an entity, its metadata wasn't being set, and it ultimately wasn't being added to the cache.
It worked with only one navigationProperty as there were two extra items in Object.keys(node), __Metadata and ID. So it would still pass the test with one navigationProperty, but not two or more.