We are running into an issue with updating non scalar complex type.
Our simplified metadata looks like this
Table
shortName: 'Table',
defaultResourceName: "/table",
dataProperties: {
name: { max: 50, required: true },
description: { max: 500},
columns: {complexType: 'Column', isScalar: false}
...
}
Column
shortName: 'Column',
isComplexType: true,
dataProperties: {
name: { max: 50, required: true },
description: { max: 500 },
...
}
Our backend is document based nosqldb.
In the UI, we display all the tables and its columns in a tree like structure. User can click on a Table name or column and edit it's properties in a property editor. Table and Column have separate editors, even though internally they are part of the same object.
Initially when we display the tree we only fetch limited no. of dataproperties of the json, only the ones that are required to be displayed in the tree. But when the user clicks on a particular table or a column we fetch the entire table (columns are complex types) from the server and update the cache to display them in the editor.
We are using knockout as the model library so all the properties are knockout observables. We are passing the same breeze entity objects to viewmodels of these editors as well as the tree.
If the user clicks on a table in the tree, and edits name or description in the editor, we are seeing table name in the tree is also changing as it should, since they are both the same observable. But when I click on a column name in the tree, and edit the column name in the editor, I do not see the change getting reflected in the tree.
What I think is happening here is, when I am re-querying the same object from the server to display in the editor, table name which a simple property is being updated with the new value from the server, but column name which is part of complex object, is actually getting replaced with new values from server, since each complex object in the array itself is being replaced. So it seems like tree and editor has different obsrevables.
Funny thing is, if after clicking on a particular column and having it displayed in the editor, I move to a different module (we are using spa), and then come back to the original module again, then if I click on the same column again and update the name, this time the change is getting reflected in the tree. But not the first time.
Could this be a bug, or am I missing something? Is there a workaround?
We are using breeze js 1.5.6
Thanks!
Related
My question is:
What kind of field would you use in an Appian interface to display [but not allow editing] of a field like "created by" and "created date" that you want to display but not allow user editing - such that the data from these fields is actually stored into the database.
Details of issue:
My colleague and I are fairly new to appian development and are working on a problem.
She created an interface that that allows us to add and edit the rows in a particular reference table.
The interfae [mostly] works - in that it allows you to add and edit rows in the database table.
The problem we have however is that there are some fields, which are not supposed to be directly edited by users. For example:
"Created by"
"Created date"
Rather than the users adding values for these fields, we want them to be automatically set by the system by retrieving the currently logged in user and the system date.
My colleague has been able to set the field values to their initial states using the loggedInUser() function and the now() function.
a!textField(
label: "Created by",
labelPosition: "ABOVE",
value: if(ri!RefDataTable.referenceId, ri!RefDataTable.createdby , loggedInUser()),
saveInto: a!save(ri!RefDataTable.createdby, upper(save!value)),
characterLimit: 255,
)
and
a!localVariables(
local!storedValue1:now(),
a!dateTimeField(
label: "Created Date and Time",
value: if(ri!RefDataTable.referenceId, ri!RefDataTable.createddate, local!storedValue1),
saveInto:{
a!save(ri!RefDataTable.createddate, local!storedValue1)
},
)
)
However, it seems that these values are not transferred into the database unless the lose-focus event is triggered for the field, which causes the method recorded against the "saveInto" field to be executed.
So it becomes necessary for the user to manually click on each of these fields and then out of them again for the values to get into the object that will then be stored into the database. If this is not done, then even though the form appears to save the record, it does not actually save any of the data at all.
I am thinking that perhaps some field type other than a!textField() should be used for the "created by" fields. Further, perhaps some field type other than a!dateTimeField() should be used for the "date created" fields - since we don't actually want the user to edit these - but rather we simply want to store the values extracted from the system into these fields.
However, to date, we have not been able to figure out how to capture the point at which the data is stored to the database so that we can [simply] store the values into the database [presumably via the CDT] at the point the record is stored.
You should not write to the db from an interface it is very difficult to follow data if you do.
The recommended solution is to pass the values captured in the interface upon submission and pass it along to process variables and perform the db updates in another node in the process.
avoid a!writeToDataStoreEntity() if you are not experienced.
a!localVariables(
local!currentUser: loggedInUser(),
a!formLayout(
contents: {
a!columnsLayout(
columns: {
a!columnLayout(
contents: {
a!textField(
label: "Customer",
value: ri!customer,
saveInto: ri!customer,
readOnly: false
),
a!textField(
label: "User",
value: local!currentUser,
readOnly: true
)
}
)
}
)
},
buttons: a!buttonLayout(
primaryButtons: {
a!buttonWidget(
label: "Submit",
submit: true,
saveInto: {
a!save(ri!currentUsser, local!currentUser)
}
)
}
)
)
)
regarding the date issue you can display one date in the interface and insert another into the database just create an script task right before writing to the db and write that date instead. This is one technique among others.
I am creating a #ViewEntity() with TypeORM on MySQL in which I directly select a JSON column. The view is correct and just a normal SQL view.
The column definition for the view class looks as such:
#ViewColumn() document: Estimate;
where Estimate is an interface specifying the data shape of the JSON, although I have tried Estimate | Object as well. The entities retrieved by the Repository are always having the document property as a string, evidently the ORM is not parsing the JSON. Therefore I am having to do some annoying JSON.parse() and mutating the retrieved record before responding to requests.
ViewColumnOptions only takes a property of name, so I cannot specify { type: 'json' } as I would on a conventional #Entity()'s #Column(). Are JSON columns in TypeORM views even implemented? I have been unable to find results in the docs nor on the github issues.
This is not well-covered in the documentation.
Turns out you can just use a #Column() decorator within a view, so my solution for the above is to specify my view's column definition as
#Column({ type: 'json' }) document: Estimate;
instead of as
#ViewColumn() document: Estimate; in the original question.
Furthermore, for anyone who may stumble upon this later, if you want to do relations against your view, you must decorate at least one column in your view as #PrimaryColumn() in order for TypeORM to formulate the SQL properly. Again, this is not covered in the docs; you could be forgiven after having read them for believing that only #ViewColumn() is valid within a #ViewEntity().
Your relations inside the #ViewEntity() will need to look as such for an example Customer relation:
#ManyToOne(() => Customer)
#JoinColumn({ name: 'customerId' })
customer: Customer;
#ViewColumn() customerId: string;
In the above I have selected in my view's SQL customerId alias and you can see I must specify that both as the name of the #JoinColumn() as well as a #ViewColumn() after the property declaration of the related entity.
I'm trying to understand how Relay works.
1.) Let's say I have UpdateProductMutation (FIELDS_CHANGE type) that updates fields of single product i.e. 'title', 'description', etc. I can send mutation with those fields changes and it works. (Say the product here is Banana)
2.) Now I add product_categories field of type ProductCategoryConnection to the Product type. And in the UpdateProductMutation mutation, I send "catIds" array as additional inputField of the mutation. (i.e. catIds: {type: new GraphQLList(GraphQLID)}) It also successfully mutates the Product with the product_categories field.
3.) To make it easier to follow, let's say Banana product had Fruit and Yellow categories. And I've added Healthy in step 2.)
4.) Problem is, if I've queried Healthy page before step 2. and Relay already has the cache in store, visiting Healthy page again after step 2. doesn't show newly associated product, Banana.
This is my mutation config and fat query.
getFatQuery() {
return Relay.QL`
fragment on UpdateProductPayload {
updatedProduct,
}
`;
}
getConfigs() {
console.warn('getConfigs', this.props);
return [{
type: 'FIELDS_CHANGE',
fieldIDs: {updatedProduct: this.props.id},
}];
}
I'm not sure if I understand it totally wrong. It seems like FIELDS_CHANGE is not the correct type to use here because the mutation works fine and the mutation payload gives the changed product category edge. The problem occurs only when I go to the previously fetched related product category page and don't see the updated product there. (Relay doesn't even send the query again). From my understanding, RANGE_ADD or RANGE_DELETE don't allow you to change the normal field either (i.e. 'title', 'description', etc.) Also in my case, catIds array could either cause RANGE_ADD or RANGE_DELETE, depends on the selection on the page.
Any advise on which direction I should take?
The object extraMetadata is undefined and throwing an error on line 247 of breeze.labs.dataservice.sharepoint.js
rawEntity.__metadata = { 'type': aspect.extraMetadata.type };
I suspect it is because I have not defined the type found in __metadata object on my entity definitions for breeze. Any suggestions on how to define my type correctly would be very welcome! Here is my type definition for one of the objects.
models.Project = {
name: 'Project',
defaultResourceName: 'getbytitle(\'Projects\')/items',
dataProperties: {
ID: {
type: breeze.DataType.Int32
},
Title: {
nullable: false
},
StatusId: {
type: breeze.DataType.Int32,
nullable: false
},
SelectedApproverId: {
type: breeze.DataType.Int32,
nullable: false
},
Created: {
type: breeze.DataType.DateTime
},
Modified: {
type: breeze.DataType.DateTime
}
},
navigationProperties: {
Status: {
type: "Status",
foreignKeyNames: ['StatusId'],
hasMany: false
},
SelectedApprover: {
type: "User",
foreignKeyNames: ["SelectedApproverId"]
}
}
};
UPDATE: 11/11/2013
If I run the following query:
return breeze.EntityQuery
.from(metadataStore.getEntityType('Project').defaultResourceName)
.orderBy('Created desc')
.using(manager)
.execute()
.then(function (data) {
console.log(data.results);
return data.results;
});
the results are an array of simple JavaScript objects, not Breeze Entities, that lack an __metadata properties. I'm trying to figure out why this is the case.
Update: 11/12/2014
I have confirmed that this issue presents itself when I have multiple entities defined under navigationProperties.
Please be sure you are using BreezeJS v.1.4.12 or later.
To be clear, the code to which you refer is on line 147 (not 247) of the breeze.labs.dataservice.sharepoint.js file in my possession.
It's located within the _createChangeRequest where it is preparing to save a modified entity. I'll assume that you have queried a Product entity, made changes to it, and are saving it back when the error occurs.
I don't believe the problem will be traced to how you defined the metadata for your Product type.
You should NOT define a __metadata property for your type. The __metadata property is something we expect SharePoint (any OData source in fact) to add to the JSON entity data that it sends to the client when you query that OData source for entities.
__metadata wouldn't be defined for results returned by a projection but then your issue concerns a modified entity so I'm assuming that you acquired this entity through a normal query ... one that did not have a select clause.
I'd like to know if you see the __metadata property in the JSON payload of a query that retrieved the entity you were modifying. Please examine the network traffic from the query request. If you don't see it there, we have to find out why the server didn't send it.
Background
The __metadata property on the JSON node is a crucial part of the contract with the SharePoint OData server. That's how the Breeze client learns about the entity's type and its etag.
Look at the jsonResultsAdapter.visitNode and updateEntityNode methods. You'll see how the adapter uses __metadata to determine the EntityType for that data. You'll also see that the adapter moves the __metadata to the adapter result's extraMetadata property. BreezeJS subsequently moves that "extra metadata" from this result object to the entity's entityAspect.extraMetadata property.
Does this seem tortured? It is tortured. OData requires extra information to be carried around with the entity (specifically the etag) without which the server simply will not update or delete the entity. We have to squirrel that info away somewhere, out of your hair, and then bring it back when we make save requests to the server. We put it on the entityAspect in keeping with that property's role as the keeper of the "entity-ness" that has nothing to do with your object's business purpose and everything to do with how it is persisted.
So much for the why. Where is the bug?
The bug
The underlying bug is that this __metadata from the SharePoint OData source has disappeared. We don't know how it disappeared yet. But we're in big trouble without it.
The sharepoint adapter should give a better message when extraMetadata is missing. We actually look for that problem a few lines later; see adjustUpdateDeleteRequest:
var extraMetadata = aspect.extraMetadata;
if (!extraMetadata) {
throw new Error("Missing the extra metadata for an update/delete entity");
}
That test appears too late. I'll make a note to move the test up.
But such a fix will only cause the save to fail with a better message. It won't tell you how to fix it.
So let's work on finding where the __metadata disappeared ... starting with whether it ever arrived in the first place.
I await your report.
Update 17 July 2014
I'm still waiting to hear if you are seeing the __metadata property in the payload of the response to the original entity query.
Meanwhile, I checked the OData specs (plural) for references to the __metadata property. It appears that the __metadata property has always been optional. It follows that an OData provider need not send or honor the etag ... and we know that this is possible because Web API 2 OData didn't support etags ... a defect soon to be corrected.
See the OData v.2 spec where it describes JSON format. Search for the term "__metadata".
The OData v.3 spec also calls for the __metadata property in a JSON response (at least a JSON verbose response).
But ... heavy sigh ... it appears that the __metadata property is gone from the v.4 spec and that the metadata information is supplied entirely through JSON annotations. The DataJS library (used by many but not all BreezeJS OData adapters) may map those annotations into the node's __metadata property but I can't confirm it yet. We have some work to do coping with all of these variations.
In the meanwhile, I think all BreezeJS OData dataservice adapters should take a more defensive position regarding extra metadata and should simply ignore the omission rather than throw exceptions.
We'll make these defensive changes very soon.
Of course the server will reject your update or delete request if the OData service actually requires an etag or other metadata. I don't know what we can do about that just yet.
There hasn't been a post in a while but I am going to share what I found as the problem and how I resolved it for me (because it took me a long time).
Basically the breeze.labs.dataservice.sharepoint adapter has a function serverTypeNameToClientDefault() that expects the SharePoint custom list type as returned by REST/OData in the __metadata "type" field to be in the exact format of:
SP.Data.**mylistname**sListItem** (notice the "sListItem" suffix; ; Ex. SP.Data.CustomersListItem)
This function does a string regex to extract the Breeze entity name from the SharePoint type and uses that name to look up the entity in the metadata store ("Customer" in the above example). If there is no match, Breeze will not find your entity and will return a basic object instead of a Breeze entity. Therefore your REST JSON result returned from SharePoint, even though it does have the __metadata property is not converted into a Breeze entity that contains the property entityAspect.extraMetadata, among other things. This is what leads to the error "Unable to get property 'type' of undefined or null reference"
For my solution, since in my case I don't care as much what the URL of my custom lists are, I just made sure that when my custom list was provisioned by SharePoint that it resulted in a name according to what Breeze expects. You accomplish this by setting the Url attribute of the ListInstance element like this:
<ListInstance
Title="My Customers"
OnQuickLaunch="TRUE"
TemplateType="10000"
Url="Lists/Customers" <!-- List/Customer will not work -->
Description="My List Instance">
...
The better solution would be to make the serverTypeNameToClientDefault() function more robust or fix it to my needs locally but hopefully this can be addressed in a future version of the adapter.
Note that I have tested this solution with the following configurations (not all dependencies listed):
Breeze.Client 1.4.9 with Breeze.DataService.SharePoint 0.2.3
Breeze.Client 1.5.0 with Breeze.DataService.SharePoint 0.3.2
Also note that the 0.3.2 version of the adapter now displays a better error message when this happens as mentioned above -- "Missing the extra metadata for an update/delete entity"; but it doesn't fix the problem.
Hope this helps someone.
For breeze v1.4.14 and breeze labs sharepoint 2013 v0.2.3 i am using small fix in file breeze.labs.dataservice.sharepoint.js.
At the end of function
function visitNode(node, mappingContext, nodeContext)
just before
return result;
i just set property extraMetadata like this:
result.extraMetadata = node.__metadata;
This seems to fix problem when i try to save modified entity back to sharepoint.
Sorry folks for the long overdue aspect of this, but I got the bug with the extra "s" resolved today... FINALLY. You can track the issue here: https://github.com/andrewconnell/breeze.js.labs/issues/6
This all stemmed from a very incorrect assumption I made. It's been fixed in version 0.6.2 of the data service adapter for SharePoint. Note that you MUST use the same name for your entity when creating it in the metadata store as the list where the data is coming from.
I resolved my issue with multiple navigationProperties on an entity by editing line 319 of breeze.labs.dataservice.sharepoint.js v.0.10.0
I changed:
if (entityType._mappedPropertiesCount <= Object.keys(node).length - 1)
to:
if (entityType.dataProperties.length <= Object.keys(node).length - 1)
It looks like the _mappedPropertiesCount includes the navigationProperties count too. e.g. dataProperties.length + navigationProperties.length
The query node was then thought to not contain a full set of properties for the entity (it was assumed to be the result of a partial projection).
It therefore wasn't being treated as an entity, its metadata wasn't being set, and it ultimately wasn't being added to the cache.
It worked with only one navigationProperty as there were two extra items in Object.keys(node), __Metadata and ID. So it would still pass the test with one navigationProperty, but not two or more.
I have created a data model using TAFFYDB. Some of the fields have nested records. I am facing difficulties querying and updating the nested records.
For example:
var friends = TAFFY([
{
"id":1,
"gender":"M",
"first":"John",
"last":"Smith",
"city":"Seattle, WA",
"comp":
[
{
"id":1,
"audience":"cavern"
},
{
"id":2,
"audience":"cottage"
}
]
},
{
"id":2,
"gender":"F",
"first":"Basic",
"last":"Smith",
"city":"Seattle, WA",
"comp":
[
{
"id":1,
"audience":"bush"
},
{
"id":2,
"audience":"swamp"
}
]
}
]);
Supposing I need to update any of the comp field's audience, how will I go about it?
With regards to queries:
When you have simpler nested arrays, you should be able to select specific records using the has and hasAll methods. However, there is an open issue that states neither of these methods work correctly. There are commits but since the issue has been left open, I assume they are not 100% fixed.
For for complex nested data, like your example, the only thing I found was this old mailing list conversation talking about some sort of find method. No such method seems to exist though nor is there any mention of it in the docs.
With regards to updates:
You should be able to update the "comp" data by passing in the modified JSON that goes with it (assuming you are able to get the data out of the db in the first place) into a normal update. However, there is an open bug showing that update does not work when record values are objects. So even if you were able to query the data and were able to modify it, you wouldn't be able to update a record anyway because of the bug. You can however do a remove and an insert.
Despite what I found above, I did some testing and found that you can update files by passing in objects. So this is a quick example of how to do a simple update:
// To show what TAFFYDB looks like:
console.log(friends().stringify());
"[{"id":1,"gender":"M","first":"John","last":"Smith","city":"Seattle, WA","comp":[{"id":1,"audience":"cavern"},{"id":2,"audience":"cottage"}],"___id":"T000003R000002","___s":true},{"id":2,"gender":"F","first":"Basic","last":"Smith","city":"Seattle, WA","comp":[{"id":1,"audience":"bush"},{"id":2,"audience":"swamp"}],"___id":"T000003R000003","___s":true}]"
// Get a copy of the comp file from the database for what you want to modify.
// In this example, let's get the **first** record matching people with the name "John Smith":
var johnsComp = friends({first:"John",last:"Smith"}).first().comp;
// Remember, if you want to use select("comp") instead, this will return an array of results.
// So to get the first result, you would need to do this despite there being only one matching result:
// friends({first:"John",last:"Smith"}).select("comp")[0];
// There are no nested queries in TAFFYDB so you need to work with the resulting object as if it were normal javascript.
// You should know the structure and you can either modify things directly, iterate through it, or whatever.
// In this example, I'm just going to change one of the audience values directly:
johnsComp[0].audience = "plains";
// Now let's update that record with the newly modified object.
// Note - if there are more than one "John Smith"s, then all of them will be updated.
friends({first:"John",last:"Smith"}).update({comp:johnsComp});
// To show what TAFFYDB looks like after updating:
console.log(friends().stringify());
"[{"id":1,"gender":"M","first":"John","last":"Smith","city":"Seattle, WA","comp":[{"id":1,"audience":"plains"},{"id":2,"audience":"cottage"}],"___id":"T000003R000002","___s":true},{"id":2,"gender":"F","first":"Basic","last":"Smith","city":"Seattle, WA","comp":[{"id":1,"audience":"bush"},{"id":2,"audience":"swamp"}],"___id":"T000003R000003","___s":true}]"
For a better targeted query or update (something that perhaps acts like a nested query/update), you can possibly try passing in a function. If you look at the docs, there is a simple example of this for update():
db().update(function () {this.column = "value";return this;}); // sets column to "value" for all matching records
I have an example, in this case i made an update to a nested field.
To acces the data you can do like this:
console.log( JSON.stringify(
data({'id':'489'}).get()[0].review[0][0].comments
))
This is an example how it works