I have the following setup: an IoT Hub ih1 routing messages to Azure Data Explorer adx1. Messages on the IoT hub look like the following:
{
"body": {
{ "value": "myValue" }
},
"messageId": "deadbeef-1234-5678-9abc-0123456789ab",
"properties": {
"myCustomProperty": "myCustomValue"
}
}
(There are more root properties and more custom properties, but the above outline should be clear).
I want to map the custom properties of the IoT Hub message onto a table column. So far, I've only been able to map the system properties and the body property content (in above example: "value": "myValue") to a column.
Using the $.propertyName will only refer to properties within the body property, not on properties on the root of the message.
How can I map custom properties to a column?
Currently this is not implemented in ADX.
As a workaround you can route messages to en Event Hub and then to ADX. Then you should be able to access that properties.
Payload enrichment for IoT Hubs and Event Hubs in ADX is currently not fully implemented and is limited to system properties only.
Full support is on the backlog but we have no ETA at the present time.
Related
Setting up a OData linked service on public OData like at:
https://www.odata.org/odata-services/
works fine in the Azure Data Factory. Hard coding a url in the OData linked service to:
https://services.odata.org/TripPinRESTierService
from an associated data set shows the entities: People, Airlines, Airports. Previewing is helpful when setting up linked services and data sources. Would be great if that was doable with Azure Search OData endpoints.
The Azure Search requires that the api-version exist in the query string. A query string can not be added until either a Copy or LookUp activity (see documentation). Previewing in the LookUp seems not to take with it the query and the preview fails with:
Invalid or missing api-version query string parameter
Debug doesn't show the url or header with the api-key from linked service nore the path from data source. But the output does show the query (with the api-version):
{
"source": {
"type": "ODataSource",
"query": "api-version=2021-04-30-Preview",
"httpRequestTimeout": "00:05:00"
},
"dataset": {
"referenceName": "AzureODataSearch",
"type": "DatasetReference",
"parameters": {}
}
}
Anybody?
I have a gateway fromKhomp manufacturer which delivers packages in the following format (SenML):
message: [
{
"bn": "000D6FFFFE642E09",
"bt": 1611339204
},
{
"n": "model",
"vs": "nir21z"
},
{
"n": "Geladeira Temp",
"u": "Cel",
"v": 4.0
}
When I connect to the Thingsboard platform, the internal GW/Parser breaks as an array before the Input in the Root Rule Chain, and threats as individual packets, but since the first position in this array corresponds to the device ID (MAC) I need to have the whole message to be parser in a script. Does anyone know a way to get the information before the GW parses the message?
If you're using Thingsboard CE then I think you will need to first forward the data to a middleware service to restructure the payload. If you are familiar with AWS Lambda you can do it there.
It would just be a simple script that takes an input payload, restructures, and then forwards to your Thingsboard deployment.
If you're using Thingsboard PE then you can use Integration/Data Converters to do this.
I try to set up a TTN based LoRaWAN Monitoring of my Gateways and devices inside a FIWARE-Environment. Therefore it would be essential to access data not in payload_field of the MQTT-Broker of TTN.
I wonder if it is possible to access field like counter, port, app_id and metadata.
I did not find a possibility yet. Does any of you face the same problem and got a solution to this challenge?
I use the following relevant FIWARE-components in a docker environment:
fiware/orion:2.2.0
fiware/iotagent-lorawan:1.2.3
mongo:3.6.8
If you need to receive metadata directly from LoRaWAN, you will have to customize the code within the LoRaWAN IoT Agent - this just passes measures by default, but the IoT Agent node lib interface is capable of receiving metadata as well.
Alternatively a recent PR Request for the IoT Agent node lib allows for additional static metadata to be added during the provisioning stage and sent as part of the requests to the context broker. You would need to use the latest development code base as the library hasn't been ported to the LoRaWAN IoT Agent yet - amend the iotagent-node-lib dependency in the package.json as shown:
"dependencies": {
...
"iotagent-node-lib": "git://github.com/telefonicaid/iotagent-node-lib.git#master",
...
},
... etc
The documentation can be found here
Attributes with metadata are provisioned with an additional parameter as shown:
"attributes": [
{"object_id": "s", "name": "state", "type":"Text"},
{"object_id": "l", "name": "luminosity", "type":"Integer",
"metadata":{
"unitCode":{"type": "Text", "value" :"CAL"}
}
}
I want to track my custom processes through Zabbix (v2.4.8). I am generating the following json object and sending it through UserParameter=service.value[*],/usr/lib/zabbix/externalscripts/custom1.bash:
{
"data":[
{
"{#NAME}":"ntp",
"{#VALUE}":"1"
},
{
"{#NAME}":"mysql",
"{#VALUE}":"1"
},
{
"{#NAME}":"prometheus",
"{#VALUE}":"0"
},
{
"{#NAME}":"apache2",
"{#VALUE}":"0"
}
]
}
Also, creating an item prototype and graph prototype inside a new template with a new discovery rule, having the following information:
Discovery rule name: Service Graph
Type: Zabbix Agent
key: service.value
Item Prototype name: Service {#NAME} Graph
Type: Zabbix Agent
key: service.value[{#NAME},{#VALUE}]
Type of info: Numeric(Unsigned) & Decimal
When I apply these settings, the items keep giving the following error:
Not supported: Received value [{ "data":[ { "{#NAME}":"ntp", "{#VALUE}":"1" }, { "{#NAME}":"mysql", "{#VALUE}":"1" }, { "{#NAME}":"prometheus", "{#VALUE}":"0" }, { "{#NAME}":"apache2", "{#VALUE}":"0" } ]}] is not suitable for value type [Numeric (unsigned)] and data type [Decimal]
I have to create a graph prototype with these settings, so I cannot mention type as "Text" for obvious reasons.
Another question: The graphs thus generated are not clickable at all like the other existing graphs.
Please let me know where I am going wrong.
If your service.value key generates JSON, that should be used with the LLD rule only. You should not send any values in it. The key to be used in the prototypes should be like any normal key they only returns values it was asked for, do not use the LLD-generating key there.
Your current JSON looks like you might be able to use the built-in items for process monitoring, but that is hard to be sure about without additional detail.
Also note that [*] in the UserParameter definition is not needed if you do not pass parameters to this key.
I am very new to UI5, I am working on an application which requires me to create models based on the request made from the browser (client).
If I consume all the odata services beforehand & use them according to the request made, it will become too heavy unnecessarily.
is there any way, this can be done dynamically?
I think your question title and the question content might be contradictory so I am placing my suggestions separately.
how to use odata services to create model dynamically from manifest.json
In your manifest.json file, locate the "sap.app" section/property and then add a datasource as follows:
"dataSources": { //used data sources -> ui5-related information stored in sap.ui5 namespace (unique inside the app)
"modelalias": { //key is alias which is used below in e.g. sap.ui5 ...
"uri": "/sap/opu/odata/snce/PO_S_SRV;v=2/" , //mandatory; version is part of uri, e.g. ";v=2", default is 1
"type": "OData" , //OData (default)|ODataAnnotation|INA|XML|JSON
"settings": { //data-source-type-specific attributes (key, value pairs)
"odataVersion": "2.0" , //possible values: 2.0 (default), 4.0
"annotations": [ "equipmentanno" ], //filled e.g. for Smart Template
"localUri": "model/metadata.xml" //relative url to local metadata
"maxAge": 360 //time in seconds
}
}
To instantiate this model with the alias "mymodel", you can add an entry into the manifest.json under "sap.ui5" as follows:
"models": {
...
"mymodel": { //empty string "" is the default model
"preload": true; //indicator that the model will be created immediately after the manifest is loaded by component factory and before the component instance is created
"dataSource": "modelalias", //reference of dataSource under sap.app - only enhance it with more settings for UI5 if needed
"settings": {
}
}
},
Now the manifest file will instantiate "mymodel" based on your odata uri in "datasources" and then set the model onto your Component.js. So when your application starts, you can access the model in any controller using:
this.getOwnerComponent().getModel("mymodel")
If I consume all the odata services beforehand & use them according to
the request made, it will become too heavy unnecessarily. is there any
way, this can be done dynamically?
Your assumption is that creating a model will slow the app startup. This may not always be true since:
The model creation is very quick
Reading data is what takes time and model instantiation
ODataModels work asyncronously be default so calling .read or .write are operations that can be managed asynchronously
Special case: If you wish to pre-fetch all your data in advance (at startup) I would advise that you ensure that you utilize filters like $select, $top and $skip on your Gateway service to implement growing list like behavior.
Hope that helps you.
More information on manifest.json: Link
Growing List: Link
ODataModel Examples: Link