Livebinding JSON objects and arrays - delphi

Good evening all.
I'm currently trying to get to grips with livebindings in Delphi as I'd like to refresh one of my current projects (complete rework from the base for the purpose of pushing to other platforms, optimizing performance and minimizing the code). I'm working with a web API which returns JSON data. The returned JSON format for one example call would look like this;
{
"response": {
"ips": [
{
"ip": "111.222.333.444",
"classification": "regular",
"hits": 134,
"latitude": 0.0000,
"longitude": 0.0000,
"zone_name": "example.com"
},
{
"ip": "555.666.777.888",
"classification": "regular",
"hits": 134,
"latitude": 50.0000,
"longitude: 50.0000,
"zone_name": "example-2.com"
},
]
},
"result": "success",
"msg": null
}
As you can see, it's a JSON object with an array and some data fields of various types (string, float, integer, etc).
Within my application, I've got the TRESTClient, TRESTRequest, TRESTResponse, TRESTResponseDataSetAdapter, TClientDataSet, and TBindSourceDB components. I've also got a TButton, a TMemo, and a TListView. I've managed to hook all the components up via livebindings and the entire data returned from the call is displayed in the memo when I click the button (which executes the request).
Where I'm struggling is with linking the data to the ListView. I've created the FieldDefs for the TClientDataSource as such (this is the literal tree view in relation to ChildDefs);
|--result (Type: ftString)
|--response (Type: ftObject)
|--|--ips (Type: ftArray, Size: 6)
|--|--|-- ip (Type: ftString)
|--|--|-- classification (Type: ftString)
|--|--|-- hits (Type: ftInteger)
|--|--|-- latitude (Type: ftFloat)
|--|--|-- longitude (Type: ftFloat)
|--|--|-- zone_name (Type: ftString)
I've then livebinded/livebound BindSourceDB1's response.ips[0] to the TListView's Item.Text field. Unfortunately, when I run the application and execute the request, I get an error;
ClientDataSet1: Field 'response.ips[0]' not found
In this instance, I'm trying to retrieve the response.ips[index].ip field of each item in the array and output it as an individual item in the TListView. Unfortunately, even livebinding the response.ips field without an index still presents a similar error. However, if I link it to the result field, then it returns the 'success' message inside the listview as expected.
I did take a look at Jim McKeeth's REST client example and that got me to the current point, but working out how to adapt it for my own data is proving a little challenging. I've noticed that the TRESTResponseDataSetAdapter also has it's own FieldDefs property, so I'm not sure whether I should define my fields there or not.
I imagine I've just got the data types setup incorrectly or missed something minor, but I'd appreciate any assistance.

I figured it out;
Set up your REST components
For the TRESTResponseDataSetAdapter, set it's RootElement property to response.ips
Then, add the fields ip, classification, hits, latitude, longitude, and zone_name as it's FieldDefs
Right-click the TRESTResponseDataSetAdapter and select 'Update DataSet'
Livebind one of the fields from the TRESTResponseDataSetAdapter to the item.text property of the TListView
The application then worked correctly and reflects the data properly.

Related

Add term to listItem in Microsoft Graph API

How do I add a term to a listItem in Microsoft Graph API?
For simple String types (ProductSegment in the example) I do the following:
PATCH https://graph.microsoft.com/v1.0/sites/{{sharepoint_site_id}}/lists/{{sharepoint_list_id}}/items/{{num}}/fields
{
"DisplayedName": "asdasfsvsvdvsdbvdfb",
"DocumentType": "FLYER",
"ProductSegment": ["SEG1"],
"TEST_x0020_2_x0020_ProductSegment": [{
"TermGuid": "c252c37d-1fa3-4860-8d3e-ff2cdde1f673"
}],
"Active": true,
"ProductSegment#odata.type": "Collection(Edm.String)",
"TEST_x0020_2_x0020_ProductSegment#odata.type": "Collection(Edm.Term)"
}
Obviously it won't work for TEST_x0020_2_x0020_ProductSegment. But I just cannot find any hints in the documentation.
I got one step closer thanks to the duplicated issue. First I found the name (not the id) of the hidden field TEST 2 ProductSegment_0 (notice the _0 suffix). Then assembled the term value to send: -1;#MyLabel|c352c37d-1fa3-4860-8d3e-ff2cdde1f673.
PATCH https://graph.microsoft.com/v1.0/sites/{{sharepoint_site_id}}/lists/{{sharepoint_list_id}}/items/{{num}}/fields
{
"DisplayedName": "asdasfsvsvdvsdbvdfb",
"DocumentType": "FLYER",
"ProductSegment": ["SEG1"],
"i9da5ea20ec548bfb2097f0aefe49df8": "-1;#MyLabel|c352c37d-1fa3-4860-8d3e-ff2cdde1f673",
"Active": true,
"ProductSegment#odata.type": "Collection(Edm.String)"
}
and so I can add one item. I would need to add multiple, so I wanted to add the values to an array and set the field type (i9da5ea20ec548bfb2097f0aefe49df8#odata.type) to Collection(Edm.String).
Now I get an error with the code generalException as opposed to an invalidRequest.
As far as I know, graph API does not support updating SharePoint taxonomy. For now, you can go with classic SharePoint REST API for example to accomplish "advanced" things like updating taxonomy terms. Probably a duplicate of: Can't Update Sharepoint Managed Meta Data Field from Microsoft Graph Explorer
Finally I got it.
Thanks #Nikolay for the linked issue.
As I also added this to the end of the question, first you need the name (not the id!) of the hidden field TEST 2 ProductSegment_0 (notice the _0 suffix). Then assemble the term values to send: -1;#MyLabel|c352c37d-1fa3-4860-8d3e-ff2cdde1f673 and -1;#SecondLabel|1ef2af46-1fa3-4860-8d3e-ff2cdde1f673, and separate them with ;# (actually the content of the label is irrelevant but some string needs to be there).
Looks utterly ridiculous but works.
PATCH https://graph.microsoft.com/v1.0/sites/{{sharepoint_site_id}}/lists/{{sharepoint_list_id}}/items/{{num}}/fields
{
"DisplayedName": "asdasfsvsvdvsdbvdfb",
"DocumentType": "FLYER",
"ProductSegment": ["SEG1"],
"i9da5ea20ec548bfb2097f0aefe49df8": "-1;#MyLabel|c352c37d-1fa3-4860-8d3e-ff2cdde1f673";#-1;#SecondLabel|1ef2af46-1fa3-4860-8d3e-ff2cdde1f673,
"Active": true,
"ProductSegment#odata.type": "Collection(Edm.String)"
}

Exclude empty fields from Log4J2 JsonTemplateLayout output

The log4j2 PatternLayout offers a %notEmpty conversion pattern that allows you to skip sections of the pattern that refer to empty variables.
Is there any way to do something similar for JsonTemplateLayout, specifically for thread context data (MDC)? It correctly (IMO) suppresses null fields, but it doesn't do the same with empty ones.
E.g., given the following in my JSON template:
"application": {
"name": { "key": "x-app", "$resolver": "mdc" },
"context": { "key": "x-app-context", "$resolver": "mdc" },
"instance": {
"name": { "key": "x-appinst", "$resolver": "mdc" },
"context": { "key": "x-appinst-context", "$resolver": "mdc" }
}
}
is there a way to prevent blocks like this from being logged, where the only data in the subtree is the empty string values for context?
"application":{"context":"","instance":{"context":""}}
(Yes, ideally I'd prevent those empty strings being put into the context in the first place, but this isn't my app, I'm just configuring it.)
JsonTemplateLayout author speaking here. Currently, JsonTemplateLayout doesn't support blank property exclusion for the following reasons:
The definition of empty/blank is ambiguous. One might have, null, {}, "\s*", [], [[]], [{}], etc. as valid JSON values. Which one of these are empty/blank? Let's assume we have agreed on a certain behavior. Will it apply to the rest of its users?
Checking if a value is empty/blank incurs an extra runtime cost.
Most of the time you don't care. You persist logs in a storage system, e.g., ELK stack, and there blank value elimination is provided out of the box by the storage engine in the most efficient way.
Would you mind sharing your use case, please? Why do you want to prevent the emission of "context": "" properties? If you deliver your logs to Elasticsearch, there you can easily exclude such fields via appropriate index mappings.
Near as I can tell, no. I would suggest you create a Jira issue to get that addressed.

Azure Data Factory get data for "For Each"component from query

The situation is as follows: I have a table in my database that recieves about 3 million rows each day. We want to archive this table on a regular base, so that only the 8 most recents weeks are in the table. The rest of the data can be archived tot AZure Data lake.
I allready found out how to do this by one day at a time. But now I want to run this pipeline each week for the first seven days in the table. I assume I should do this with the "For Each" component. It should itterate along the seven distinct dates that are present in the dataset I want to backup. This dataset is copied from the source table to an archive table on forehand.
It's not difficult to get the distinct dates with a SQL query, but how to get the result of this query into an array that is used for the "For Each" component?
The issue is solved thanks to a co-worker.
What we have to do is assign a parameter to the dataset of the sink. Does not matter how you name this and you do not have to assign a value to it. But let's assume this parameter is called "date"
After that you can use this parameter in the filename of the sink (also in dataset) with by using "#dataset().Date".
After that you go back to the copyactivity and in the sink you assign a dataset property to #item().DateSelect. (DateSelect is the field name from the array that is passed to the For Each activity)
See also the answer from Bo Xioa as part of the answer
This way it works perfectly. It's just a shame that this is not well documented
You can use lookup activity to fetch the column content, and the output will be like
{
"count": "2",
"value": [
{
"Id": "1",
"TableName" : "Table1"
},
{
"Id": "2",
"TableName" : "Table2"
}
]
}
Then you can pass the value array to the Foreach activity items field by using the pattern of #activity('MyLookupActivity').output.value
ref doc: Use the Lookup activity result in a subsequent activity
I post this as an answer, because the error does not fit into a comment :D
have seen antoher option to accomplish this. That is by executing a pipeline from another pipeline. And in that way I can define the dates that I should iterate over as a parameter in the second pipeline (learn.microsoft.com/en-us/azure/data-factory/…). But unformtunately this leads to the same rsult as when just using the foreach parameter. Because in the filename of my data lake file I have to use: #{item().columname}. I can see in the monitoring view that the right values are passed in the iteration steps, but I keep getting an error:
{
"errorCode": "2200",
"message": "Failure happened on 'Sink' side. ErrorCode=UserErrorFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The request to 'Unknown' failed and the status code is 'BadRequest', request id is ''. {\"error\":{\"code\":\"BadRequest\",\"message\":\"A potentially dangerous Request.Path value was detected from the client (:). Trace: cf3b4c3f-1681-4073-b225-17e1c07ec76d Time: 2018-08-02T05:16:13.2141897-07:00\"}} ,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The remote server returned an error: (400) Bad Request.,Source=System,'",
"failureType": "UserError",
"target": "CopyDancerDatatoADL"
}

Can Neo4j property order be controlled?

Is there any mechanism for controlling the order of properties?
I cannot reproduce this in http://www.neo4j.org/console
Using Neo4j 1.9.2 Community if I do the following:
CREATE (m1 {`$type`: {moduleTypeName}, Name: 'M1', ModelNumber: 'MN1'})
Then later I get this node back from a cypher query using the REST cypher endpoint I get back...
{
"extensions": {},
"paged_traverse": "http://localhost:7575/db/data/node/3777/paged/traverse/{returnType}{?pageSize,leaseTime}",
"outgoing_relationships": "http://localhost:7575/db/data/node/3777/relationships/out",
"traverse": "http://localhost:7575/db/data/node/3777/traverse/{returnType}",
"all_typed_relationships": "http://localhost:7575/db/data/node/3777/relationships/all/{-list|&|types}",
"property": "http://localhost:7575/db/data/node/3777/properties/{key}",
"all_relationships": "http://localhost:7575/db/data/node/3777/relationships/all",
"self": "http://localhost:7575/db/data/node/3777",
"properties": "http://localhost:7575/db/data/node/3777/properties",
"outgoing_typed_relationships": "http://localhost:7575/db/data/node/3777/relationships/out/{-list|&|types}",
"incoming_relationships": "http://localhost:7575/db/data/node/3777/relationships/in",
"incoming_typed_relationships": "http://localhost:7575/db/data/node/3777/relationships/in/{-list|&|types}",
"create_relationship": "http://localhost:7575/db/data/node/3777/relationships",
"data": {
"ModelNumber": "MN1",
"$type": "ModuleType",
"Name": "M1"
}
}
I'm using http://james.newtonking.com/pages/json-net.aspx to parse JSON and for it to automatically infer an object type, the $type property must be first. It makes sense when parsing the JSON in a stream when you don't want to load the entire thing into memory first.
It does not appear to be alphabetical, and it does not seem to be random either. It seems that the order is consistent for different object types, but inconsistent between them.
I have pulled the node in the Shell as well and so it seems that the order does not depend on how I get the node, but is not related to the order in which I create the node either.
Properties have no guaranteed order. Do not take any assumptions on a 'maybe' ordering. An upcoming version might change this assumed behaviour and break your code.
I guess it is simpler in Cypher to not return the node itself in favour of a list of properties, e.g.
START node=node(<myid>)
RETURN node.`$type`, node.ModelNumber, node.Name
This has defined columns.
Definitively it seems not to have that functionality.
My workaround for it it is to alias the properties with a prefix in the format aXX_, as in a01_, a02, a03_ and then remove it in the code.
Not pretty, not great, but it works as neo4j respects numerical order.
It needs a letter character at the beggining though, hence the "a" before the numbers.

Storing graph-like structure in Couch DB or do include_docs yourself

I am trying to store network layout in Couch DB, but my solution provides rather randomized graph.
I store a nodes with a document:
{_id ,
nodeName,
group}
and storing links in traditional:
{_id, source_id, target_id, value}
Following multiple tutorials on handling joins and multiple relationship in Couch DB I created view:
function(doc) {
if(doc.type == 'connection') {
if (doc.source_id)
emit("source", {'_id': doc.source_id});
if(doc.target_id)
emit("target", {'_id': doc.target_id});
}
}
which should have emitted sequence of source and target id, then I pass it to the list function with include_docs=true, assumes that source and target come in pairs stitches everything back in a structure like this:
{
"nodes":[
{"nodeName":"Name 1","group":"1"},
{"nodeName":"Name 2","group":"1"},
],
"links": [
{"source":7,"target":0,"value":1},
{"source":7,"target":5,"value":1}
]
}
Although my list produce a proper JSON, view map returns number of rows of source docs and then target docs.
So far I don't have any ideas how to make this thing working properly - I am happy to fetch additional values from document _id in the list, but so far I havn't find any good examples.
Alternative ways of achieving the same goal are welcome. _id values are standard for CouchDB so far.
Update: while writing a question I came up with different view which sorted my immediate problem, but I still would like to see other options.
updated map:
function(doc) {
if(doc.type == 'connection') {
if (doc.source_id)
emit([doc._id,0,"source"], {'_id': doc.source_id});
if(doc.target_id)
emit([doc._id,1,"target"], {'_id': doc.target_id});
}
}
Your updated map function makes more sense. However, you don't need 0 and 1 in your key since you have already "source"and "target".

Resources