Is it possible to apply [Rule Chain] after [Data Converter]? - thingsboard

I am currently working on a POC by using ThingsBoard PE.
Our raw data contains [Asset] [Attributes].
Data flow:
[IoT cloud] --https webhook carry raw data--> [ThingsBoard PE HTTP INTEGRATION] --uplink--> [ThingsBoard PE Data Converter]
My question is: is it possible to apply [Rule Chain] after [ThingsBoard PE Data Converter]? Therefore, the device can auto create relationship with [Asset] by the [Attribute], instead of [AssetName].
Example data after data converter process:
{
"deviceName": "ABC",
"deviceType": "temperature",
"attributes": {
"asset_id": 6 // <-- the id is used in asset attribute
},
"telemetry": {
"temperature": 39.43
}
}

Answering your two, separate questions:
is it possible to apply [Rule Chain] after [ThingsBoard PE Data Converter]?
Yes it is possible. Once your data is successfully integrated and you are receiving it, you can access it using the [Input] Rule Node (the default green one that is always there when you create a Rule) and route it to any other node you need.
Therefore, the device can auto create relationship with [Asset] by the [Attribute], instead of [AssetName].
So, you want the relationship to take your custom attribute and use that as the pattern that identifies the Asset you want to create the relationship from.
The PE edition already has the Create Relation Node. However, seems that as it is one is not able to do what you seek (has no option to specify custom Asset id).
However, two options you got are:
Create a Custom Rule Node that does what you want. For that try checking the Rule Node Development page from Thingsboard. You can use the Create Relation Node as base and work from there. This can be a longer solution than doing...
Enrich your incoming message's metadata, adding your desired attribute. The Create Relation Node allows you to use variables on your message's metadata in your Name and Type patterns, as seen from this screenshot I took from that node:
This allows us a workaround to what you want to do: Add a Script Transformation Node that adds attributes.asset_id to the metadata, for example as metadata.asset_id, so you can then use it as ${asset_id} on your Name and Type patterns.
For example, your Transform() method of such Script Transformation Node should look something like this:
function Transform(msg, metadata, msgType){
//assumming your desired id is msg.attributes.asset_id, add it to the metadata
metadata.asset_id = msg.attributes.asset_id;
//return the message, in your case to the Create Relation Node
return {msg: msg, metadata:metadata, msgType:msgType};
}
Finally, your Rule should be connected like this:
[Input] -> [Script Node] -> [Create Relation Node] -> [...whatever else you like]

Related

Is it possible to add label field into device via Data Converter?

I would like to fill label and description fields without any manual work.
Or what is the best way for TB to fill those fields?
Label & Description in device
You should do it in the rulechain.
You should have in you rulechain tab the Root rule chain, add a script transformation node after the message filter one (you need to connect the two nodes with an Entity created tag if you want it to be done on every device).
to access the label and the description in the node you need "msg.additionalInfo.description" and "msg.label"
add the nodes to he existing root rule chain, and add an originator filter for device if you only want this to be done on devices
You can fill the Label via Data Converter ("deviceLabel" or "assetLabel" key). Description is not supported, yet.
Support was added in version 3.3.2:
https://thingsboard.io/docs/pe/reference/releases/#v332-november-11-2021
// Result object with device/asset attributes/telemetry data
var result = {
// Use deviceName and deviceType or assetName and assetType, but not both.
deviceName: 'Name',
deviceType: "Type",
deviceLabel: "Label",
attributes: {},
telemetry: {}
};

Variable to change label on thingsboard

I would like to see if they can help me with the creation of a variable, where I can change the labels of the MQTT message that is sent from my IoT devices, in order to make it easier and to select the correct parameters when creating a dashboard. .
Example:
This is the message received to my server.
[{"n": "model", "d": "iot-zigbee1783"}, {"n": "Relay", "ap": true}, {"t": "gateway", "ma": "0035DDf45VAIoT215"}]
What I want is to change the label "d" for "deviceIoT" and "ap" for "door sensor" also if it is possible to change the true or false of the door sensor for open and closed.
You can do this with the help of Thingsboards rule-chain.
There is also an official tutorial for this:
https://thingsboard.io/docs/user-guide/rule-engine-2-0/tutorials/transform-incoming-telemetry/
They use the transformation-rule-node called script to convert temperatures from [°F] to [°C].
While this is not your use case, it shows you how to handle incoming telemetry before it is saved to the database.
You could do a mapping of value-keys like this:
var theCustomizedMessage = {};
theCustomizedMessage['customizedKey'] = msg['originalIncomingKey'];
return {msg: theCustomizedMessage, metadata: metadata, msgType: msgType};
Keep in mind that this might be contra-productive since you have to update the rule-node scripts, when something changes.
As an alternative option you can rename the key labels in the widget configuration. This will not help your dashboard developers. But a documentation document will do :)
I strongly recommend against the replacement of boolean values with strings ('closed', 'opened'). This is a job for the widgets (e.g. their value format functions).

How to exclude multiple values in OData call?

I am creating a SAPUI5 application. This application is connected to a backend SAP system via OData. In the SAPUI5 application I use a smart chart control. Out of the box the smart chart lets the user create filters for the underlying data. This works fine - except if you try to use multiple 'not equals' for one property. Is there a way to accomplish this?
I found out that all properties within an 'and_expression' (including nested or_expressions) must have unique name.
The reason why two parameters with the same property don't get parsed into the select options:
/IWCOR/CL_ODATA_EXPR_UTILS=>GET_FILTER_SELECT_OPTIONS takes the expression you pass and parses it into a table of select options.
The select option table returned is of type /IWCOR/IF_ODATA_TYPES=>EDM_SELECT_OPTION_T which is a HASHED TABLE .. WITH UNIQUE KEY property.
From: https://archive.sap.com/discussions/thread/3170195
The problem is that you cannot combine NE terms with OR. Because both parameters after the NE should not be shown in the result set.
So at the end the it_filter_select_options is empty and only the iv_filter_string is filled.
Is there a manual way of facing this problem (evaluation of the iv_filter_string) to handle multiple NE terms?
This would be an example request:
XYZ/SmartChartSet?$filter=(Category%20ne%20%27Smartphone%27%20and%20Category%20ne%20%27Notebook%27)%20and%20Purchaser%20eq%20%27CompanyABC%27%20and%20BuyDate%20eq%20datetime%272018-10-12T02%3a00%3a00%27&$inlinecount=allpages
Normally I want this to exclude items with the category 'Notebook' and 'Smartphone' from my result set that I retrieve from the backend.
If there is a bug inside /iwcor/cl_odata_expr_utils=>get_filter_select_options which makes it unable to treat multiple NE filters of the same component, and you cannot wait for an OSS. I would suggest to wrap it inside a new static method that will make the following logic (if you will be stuck with the ABAP implementation i would try to at least partially implement it when i get time):
Get all instances of <COMPONENT> ne '<VALUE>' inside a () (using REGEX).
Replace each <COMPONENT> with <COMPONENT>_<i> so there will be ( <COMPONENT>_1 ne '<VALUE_1>' and <COMPONENT>_2 ne '<VALUE_2>' and... <COMPONENT>_<n> ne '<VALUE_n>' ).
Call /iwcor/cl_odata_expr_utils=>get_filter_select_options with the modified query.
Modify the rt_select_options result by changing COMPONENT_<i> to <COMPONENT> again.
I can't find the source but I recall that multiple "ne" isn't supported. Isn't that the same thing that happens when you do multiple negatives in SE16, some warning is displayed?
I found this extract for Business ByDesign:
Excluding two values using the OR operator (for example: $filter=CACCDOCTYPE ne ‘1000’ or CACCDOCTYPE ne ‘4000’) is not possible.
The workaround I see is to select the Categories you actively want, not the ones you don't in the UI5 app.
I can also confirm that my code snippet I've used a long time for filtering also has the same problem...
* <SIGNATURE>---------------------------------------------------------------------------------------+
* | Instance Public Method ZCL_MGW_ABS_DATA->FILTERING
* +-------------------------------------------------------------------------------------------------+
* | [--->] IO_TECH_REQUEST_CONTEXT TYPE REF TO /IWBEP/IF_MGW_REQ_ENTITYSET
* | [<-->] CR_ENTITYSET TYPE REF TO DATA
* | [!CX!] /IWBEP/CX_MGW_BUSI_EXCEPTION
* | [!CX!] /IWBEP/CX_MGW_TECH_EXCEPTION
* +--------------------------------------------------------------------------------------</SIGNATURE>
METHOD FILTERING.
FIELD-SYMBOLS <lt_entityset> TYPE STANDARD TABLE.
ASSIGN cr_entityset->* TO <lt_entityset>.
CHECK: cr_entityset IS BOUND,
<lt_entityset> IS ASSIGNED.
DATA(lo_filter) = io_tech_request_context->get_filter( ).
/iwbep/cl_mgw_data_util=>filtering(
exporting it_select_options = lo_filter->get_filter_select_options( )
changing ct_data = <lt_entityset> ).
ENDMETHOD.

Which settings should be used for TokensregexNER

When I try regexner it works as expected with the following settings and data;
props.setProperty("annotators", "tokenize, cleanxml, ssplit, pos, lemma, regexner");
Bachelor of Laws DEGREE
Bachelor of (Arts|Laws|Science|Engineering|Divinity) DEGREE
What I would like to do is that using TokenRegex. For example
Bachelor of Laws DEGREE
Bachelor of ([{tag:NNS}] [{tag:NNP}]) DEGREE
I read that to do this, I should use TokensregexNERAnnotator.
I tried to use it as follows, but it did not work.
Pipeline.addAnnotator(new TokensRegexNERAnnotator("expressions.txt", true));
Or I tried setting annotator in another way,
props.setProperty("annotators", "tokenize, cleanxml, ssplit, pos, lemma, tokenregexner");
props.setProperty("customAnnotatorClass.tokenregexner", "edu.stanford.nlp.pipeline.TokensRegexNERAnnotator");
I tried to different TokenRegex formats but either annotator could not find the expression or I got SyntaxException.
What is the proper way to use TokenRegex (query on tokens with tags) on NER data file ?
BTW I just see a comment in TokensRegexNERAnnotator.java file. Not sure if it is related pos tags does not work with RegexNerAnnotator.
if (entry.tokensRegex != null) {
// TODO: posTagPatterns...
pattern = TokenSequencePattern.compile(env, entry.tokensRegex);
}
First you need to make a TokensRegex rule file (sample_degree.rules). Here is an example:
ner = { type: "CLASS", value: "edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagAnnotation" }
{ pattern: (/Bachelor/ /of/ [{tag:NNP}]), action: Annotate($0, ner, "DEGREE") }
To explain the rule a bit, the pattern field is specifying what type of pattern to match. The action field is saying to annotate every token in the overall match (that is what $0 represents), annotate the ner field (note that we specified ner = ... in the rule file as well, and the third parameter is saying set the field to the String "DEGREE").
Then make this .props file (degree_example.props) for the command:
customAnnotatorClass.tokensregex = edu.stanford.nlp.pipeline.TokensRegexAnnotator
tokensregex.rules = sample_degree.rules
annotators = tokenize,ssplit,pos,lemma,ner,tokensregex
Then run this command:
java -Xmx8g edu.stanford.nlp.pipeline.StanfordCoreNLP -props degree_example.props -file sample-degree-sentence.txt -outputFormat text
You should see that the three tokens you wanted tagged as "DEGREE" will be tagged.
I think I will push a change to the code to make tokensregex link to the TokensRegexAnnotator so you won't have to specify it as a custom annotator.
But for now you need to add that line in the .props file.
This example should help in implementing this. Here are some more resources if you want to learn more:
http://nlp.stanford.edu/software/tokensregex.shtml#TokensRegexRules
http://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/ling/tokensregex/SequenceMatchRules.html
http://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/ling/tokensregex/types/Expressions.html

Neo4j+PopotoJS: filter graph based-on predefined constraints

I have a question about the query based on the predefined constraints in PopotoJs. In this example, the graph can be filtered based on the constraints defined in the search boxes. The sample file in this example visualizations folder, constraint is only defined for "Person" node. It is specified in the sample html file like the following:
"Person": {
"returnAttributes": ["name", "born"],
"constraintAttribute": "name",
// Return a predefined constraint that can be edited in the page.
"getPredefinedConstraints": function (node) {
return personPredefinedConstraints;
},
....
In my graph I would like to apply that query function for more than one node. For example I have 2 nodes: Contact (has "name" attribute) and Delivery (has "address" attribute)
I succeeded it by defining two functions for each nodes. However, I also had to put two search box forms with different input id (like constraint1 and constraint2). And I had to make the queries in the associated search boxes.
Is there a way to make queries which are defined for multiple nodes in one search box? For example searching Contact-name and/or Delivery-adress in the same search box?
Thanks
First I’d like to specify that the predefined constraints feature is still experimental (but fully functional) and doesn’t have any documentation yet.
It is intended to be used in configuration to filter data displayed in nodes and in the example the use of search boxes is just to show dynamically how it works.
A common use of this feature would be to add the list of predefined constraint you want in the configuration for every node types.
Let's take an example:
With the following configuration example the graph will be filtered to show only Person nodes having "born" attribute and only Movie nodes with title in the provided list:
"Person": {
"getPredefinedConstraints": function (node) {
return ["has($identifier.born)"];
},
...
}
"Movie": {
"getPredefinedConstraints": function (node) {
return ["$identifier.title IN [\"The Matrix\", \"The Matrix Reloaded\", \"The Matrix Revolutions\"]"];
},
...
}
The $identifier variable is then replaced during query generation with the corresponding node identifier. In this case the generated query would look like this:
MATCH (person:`Person`) WHERE has(person.born) RETURN person
In your case if I understood your question correctly you are trying to use this feature to implement a search box to filter the data. I'm still working on that feature but it won't be available soon :(
This is a workaround but maybe it could work in your use case, you could keep the search box value in a variable:
var value = d3.select("#constraint")[0][0].value;
inputValue = value;
Then use it in the predefined constraint of all the nodes type you want.
In this example Person will be filtered based on the name attribute and Movie on title:
"Person": {
"getPredefinedConstraints": function (node) {
if (inputValue) {
return ["$identifier.name =~ '(?i).*" + inputValue + ".*'"];
} else {
return [];
}
},
...
}
"Movie": {
"getPredefinedConstraints": function (node) {
if (inputValue) {
return ["$identifier.title =~ '(?i).*" + inputValue + ".*'"];
} else {
return [];
}
},
...
}
Everything is in the HTML page of this example so you can view the full source directly on the page.
#Popoto, thanks for the descriptive reply. I tried your suggestion and it worked pretty much well. With the actual codes, when I make a query it was showing only the queried node and make the other node amount zero. I wanted to make a query which queries only the related node while the number of other nodes are still same.
I tried a temporary solution for my problem. What I did is:
Export the all the node data to JSON file, search my query constraint in the exported JSONs, if the file is existing in JSON, then run the query in the related node; and if not, do nothing.
With that way, of course I needed to define many functions with different variable names (as much as the node amount). Anyhow, it is not a propoer way, bu it worked for now.

Resources