I am trying to send the following data via tb-gateway mqtt connector
'{"timestamp": 1670415775, "camera": "testcamera", "alert": "Temperature", "guid": "241ffb1c-5d8f-4959-a34a-02edf32487a1", "highTemp": "false", "temperature": "95.9\u00baF/ 35.5\u00baC", "bottomRightX": 3648, "mask": false, "imgPath": "somedummyurl", "timeSent": 1670415777}'
but it is being split into 2 messages
part1:
{"timestamp": 1670415775, "camera": "testcamera", "alert": "Temperature", "guid": "241ffb1c-5d8f-4959-a34a-02edf32487a1", "highTemp": "false", "temperature": "95.9\u00baF/ 35.5\u00baC", "bottomRightX": 3648}
part2:
{"mask": false, "imgPath": "somedummyurl", "timeSent": 1670415777}
I tried setting maxPayloadSizeBytes: 1024 in tb-gateway.yaml, but it splits the message into individual key-value pairs
I have created a rule chain that works only on the full payload.
How to stop this splitting?
How to aggregate this split telemetry and then invoke my further rule chain?
I am using tb-gateway 3.2.1, tb-ce-3.2.1
Related
I want to publish a message to IoT Hub and also receive the message via the same topic. I connected to IoT Hub and then subscribed to the same topic to which i send the messages, but I don't receive any message. As Client I am using MQTTX windows. Messages are sent and received correctly at the cloud endpoint (telemetry data). Here is a screenshot from my MQTT client: MQTT Client
I see you are using MQTTX as a leaf device to IoT Edge in a transparent gateway pattern, where edge will just pass through messages to IoTHub.
IoTHub does not support custom topic or respond back on the same topic so this "devices/plc2/messages/events/topic" will not work
Subscription to devices/{device_id}/messages/devicebound/# will work but you will need to explicitly send C2D message for that device_id as a response
The Azure IoT Hub is not a generic MQTT broker, see more details here.
I do recommend to read also the doc for Azure IoT Edge MQTT broker.
UPDATE:
Based on your needs, such as a round trip time test, the following is an example how can be achieved this loopback in the Azure IoT Central. Note, that the Iot Central App has a built-in great features to simplify your test without using additional azure resource such as an Azure function.
The concept of this example is based on the following:
REST API for invoking a durable command (Queuing command) to generate a C2D Message.
Using the Data Transformation for exporting a message to the webhook destination
First of all, we have to create a device template (in my example loopback) for our test:
{
"#id": "dtmi:rk2022iotcfree:loopback_24v;1",
"#type": "Interface",
"contents": [
{
"#id": "dtmi:rk2022iotcfree:loopback_24v:message;1",
"#type": "Command",
"displayName": {
"en": "Message"
},
"name": "message",
"request": {
"#type": "CommandPayload",
"displayName": {
"en": "Info"
},
"name": "Info",
"schema": {
"#type": "Object",
"displayName": {
"en": "Object"
},
"fields": [
{
"displayName": {
"en": "ts"
},
"name": "ts",
"schema": "dateTime"
},
{
"displayName": {
"en": "msg"
},
"name": "msg",
"schema": "string"
}
]
}
},
"durable": true
},
{
"#id": "dtmi:rk2022iotcfree:loopback_24v:counter;1",
"#type": "Telemetry",
"displayName": {
"en": "Counter"
},
"name": "counter",
"schema": "double"
},
{
"#id": "dtmi:rk2022iotcfree:loopback_24v:time;1",
"#type": "Telemetry",
"displayName": {
"en": "Timestamp"
},
"name": "time",
"schema": "double"
}
],
"displayName": {
"en": "loopback"
},
"#context": [
"dtmi:iotcentral:context;2",
"dtmi:dtdl:context;2"
]
}
Once we have the device template, we can create a device (in my example device1) with assigning to this device template (loopback).
Now, we need to declare a destination webhook endpoint:
where the Callback URL is:
https://2a92220d-42e3-4a73-b700-2cb858c9c5e7.azureiotcentral.com/api/devices/device1/commands/message?api-version=1.2-preview
and the Authorization token is generated as the IoT Central Api token
Next step is declaring a Data transformation for this destination endpoint:
import "iotc" as iotc;
if .device.id == "device1" then
{
request: {
Info:{
ts:.enqueuedTime,
msg:("Feedback from device: id=" + .device.id + ", counter=" + (.telemetry | iotc::find(.name == "counter").value | tostring) + ", time=" + (.telemetry | iotc::find(.name == "time").value | tostring))
}
}
}
else
empty
end
That's all at the IoT Central App. Based on the above Data transformation, any received telemetry data message from the device1 is exported to the webhook endpoint for invoking a C2D message.
In my example, for the device side is used my tool for virtual MQTT device (either the Azure IoT Hub Tester or Azure IoT Central Tester), but it can be used any other similar tool or MQTT client.
As you can see the above screen snippet, the device1 sent the telemetry data such as a counter and time and the device received the C2D message:
As the above screen snippet shows, there are three timestamps. The two highlighted timestamps are represented a round trip time (from the device to the IoT Central and back to the device). The third one is represented a timestamp at the IoT Central app:
Start device: 2022-05-19T13:50:06.4213024Z
IoT Central App: 2022-05-19T13:50:07.107Z
End device: 2022-05-19T13:50:13.6934397Z
I want to be able to have a list of all users who can view a certain mail item. As an admin on the frontend on exchange online, I can view all my users mail, but when i call to the API I only return my mail. I want to be able to make calls to see exactly who has permission to view each mail item, and cannot find a way through the api.
I can get a list of all users, and a list of all mail for each user, a list of all mailboxes, a list of all groups, but not permissions on each mail item
GET /users/{id | userPrincipalName}/messages
returns all the mail, but mail items come with the following structure:
{
"bccRecipients": [{"#odata.type": "microsoft.graph.recipient"}],
"body": {"#odata.type": "microsoft.graph.itemBody"},
"bodyPreview": "string",
"categories": ["string"],
"ccRecipients": [{"#odata.type": "microsoft.graph.recipient"}],
"changeKey": "string",
"conversationId": "string",
"createdDateTime": "String (timestamp)",
"flag": {"#odata.type": "microsoft.graph.followupFlag"},
"from": {"#odata.type": "microsoft.graph.recipient"},
"hasAttachments": true,
"id": "string (identifier)",
"importance": "String",
"inferenceClassification": "String",
"internetMessageHeaders": [{"#odata.type": "microsoft.graph.internetMessageHeader"}],
"internetMessageId": "String",
"isDeliveryReceiptRequested": true,
"isDraft": true,
"isRead": true,
"isReadReceiptRequested": true,
"lastModifiedDateTime": "String (timestamp)",
"parentFolderId": "string",
"receivedDateTime": "String (timestamp)",
"replyTo": [{"#odata.type": "microsoft.graph.recipient"}],
"sender": {"#odata.type": "microsoft.graph.recipient"},
"sentDateTime": "String (timestamp)",
"subject": "string",
"toRecipients": [{"#odata.type": "microsoft.graph.recipient"}],
"uniqueBody": {"#odata.type": "microsoft.graph.itemBody"},
"webLink": "string",
"attachments": [{"#odata.type": "microsoft.graph.attachment"}],
"extensions": [{"#odata.type": "microsoft.graph.extension"}],
"multiValueExtendedProperties": [{"#odata.type": "microsoft.graph.multiValueLegacyExtendedProperty"}],
"singleValueExtendedProperties": [{"#odata.type": "microsoft.graph.singleValueLegacyExtendedProperty"}]
}
this doesnt contain anything about the full permissions on the item. Does anyone know of a way to get this?
You can't get item level permission as item doesn't store ACL associated with it. You can, however, get Folder level permission by querying PR_NT_SECURITY_DESCRIPTOR (0x0E270102) on the folder.
I actually wrote script for this based on my old REST API client engine: Start-MailboxFolderPermissionReport
I can, if script isn't enough, write C# way of doing it through Graph Managed API
There doesn't appear to be a way to expose mailbox or folder permissions through the Graph API. These are available through the Exchange Online PowerShell module e.g. Get-MailboxFolderPermission.
I have an rails app and I'm trying to configure logging to graylog. Pipeline consists of next steps:
1) Logs are written to file in JSON format by SemanticLogger gem. Log message consist of header info (first level tags) and payload with several levels of hierarchy:
{
"tag": "mortgage",
"app": "sneakers",
"pid": 3448,
"env": "production",
"host": "thesaurus-mortgage",
"thread": "91090300",
"level": "info",
"name": "Sneakers",
"payload": {
"class": "EgrnListenerWorker",
"method": "work",
"json": {
"resource": "kontur",
"action": "request_egrn_done",
"system_code": "thesaurus",
"id": 35883717,
"project_id": "mortgage",
"bank_id": "ab",
"params": {
"egrn": {
"zip": "rosreestr/kontur/kontur_4288_2018-10-11_021848.zip",
"pdf": "rosreestr/kontur/kontur_4288_2018-10-11_021848.pdf",
"xml": "rosreestr/kontur/kontur_4288_2018-10-11_021848.xml"
},
"code": "SUCCESS"
}
},
"valid_json": true
},
"created_at": "2018-10-11T17:44:58.262+00:00"
}
2) File is being read by Filebeat service and sent to Graylog.
And graylog could not parse correctly payload contents:
As you can see - keys are concatenated with ":" in one string in such manner: key1=value1:key2=value2. This is not what I am expected. It would be perfect if I could manage graylog to parse contents of payload into different fields with names payload.key1, payload.key2 and so on (so I could perform search on these fields)
ps: my log data is heterogeneous, i.e. payload contents depend on functionality it was produced by, so I expect that there would be a huge amount of different fields of a kind "payload.xxxxx" - is it ok?
This isn't exactly a filebeat question since filebeat only ships the logs in their original JSON format (zipped, if wanted).
From the Graylog Website: http://docs.graylog.org/en/2.4/pages/extractors.html
Using the JSON extractor
Since version 1.2, Graylog also supports extracting data from messages sent in JSON format.
Using the JSON extractor is easy: once a Graylog input receives
messages in JSON format, you can create an extractor by going to
System -> Inputs and clicking on the Manage extractors button for that
input. Next, you need to load a message to extract data from, and
select the field containing the JSON document. The following page let
you add some extra information to tell Graylog how it should extract
the information.
This should get you going.
I need to know how to query the following invoke(AC(123456A)or (user_type1_c19a5f0b7e) when i have init this:
{
"jsonrpc": "2.0",
"method": "deploy",
"params": {
"type": 1,
"chaincodeID": {
"path": "https://github.com/johndoe/learn-chaincode/finished"
},
"ctorMsg": {
"function": "init",
"args": [
"hi there"
]
},
"secureContext": "user_type1_191b8c2993"
},
"id": 1
}
Here is my (successful) invoke:
{
"jsonrpc": "2.0",
"method": "invoke",
"params": {
"type": 1,
"chaincodeID": {
"name": "b42b506c13fg1157520d585841a691f4b598d9aac53adc95277ed710c7f1f5aaa59a89a8dca8ddf3e46d5241562f3316956a04ad8a33181d30cb01b1c4c9adb2f"
},
"ctorMsg": {
"function": "AC(123456A)",
"args": [
"AcctId:949392949233 ","Year2015 Sem1 ", "IT1234:A ", " IT2345:B+ ", "IT3456:A ", "Year 2015 Sem2 ", "IT4567:B ", "IT5678: B", "IT5678:A ","Year 2016 Sem1 ", "IT6789:C ", "IT7890:C ", "Cumulative GPA: 3"
]
},
"secureContext": "user_type1_c19a5f0b7e"
},
"id": 4
}
Here is the query that I am trying and I can't get the results that I want:
{
"jsonrpc": "2.0",
"method": "query",
"params": {
"type": 1,
"chaincodeID": {
"name": "b42b506c13fg1157520d585841a691f4b598d9aac53adc95277ed710c7f1f5aaa59a89a8dca8ddf3e46d5241562f3316956a04ad8a33181d30cb01b1c4c9adb2f"
},
"ctorMsg": {
"function": "AC(123456A)",
"args": [
"AcctId:949392949233 "
]
},
"secureContext": "user_type1_c19a5f0b7e"
},
"id": 2
}
Here's the response for the query.
{
"jsonrpc": "2.0",
"error": {
"code": -32003,
"message": "Query failure",
"data": "Error when querying chaincode: Error:Transaction or query returned with failure: Received unknown function query"
},
"id": 2
}
Right now, I am trying out this tutorial, which is provided by IBM Bluemix:
https://github.com/IBM-Blockchain/learn-chaincode.
I have tried an init using their example and do invoke(successful) to suit my needs. I tried the Bluemix tutorial, which I mentioned earlier, and I am able to achieve what is stated in the documentation. Therefore, I decided to invoke something to suit my own needs. However, i am unable to query the person AC(123456A) or user_type1_c19a5f0b7e with showing of items such as "Year2015 Sem1", "IT1234:A", "IT3456:B+ " and etc . How can I query (the invoke) to suit my own needs?
Our understanding is that you are using an IBM Blockchain Starter Developer Network on IBM Bluemix. Using this environment, the "Learn Chaincode" tutorial (https://github.com/IBM-Blockchain/learn-chaincode) has been followed. We would like to confirm our understanding of your results when following the tutorial. Our understanding is that when following the tutorial, the Init, Invoke, and Query chaincode functions all operated as expected. Is the understanding correct that the tutorial as written could be followed successfully?
Then we understand that the example chaincode was modified to apply different logic. The Invoke for this modified chaincode that uses the AC(123456A) function with a user of user_type1_c19a5f0b7e appears to have been successful. However, a query that also used the AC(123456A) function was not successful. Is this understanding correct?
Could some additional details be provided so this can be considered further?
Did you write your own customized chaincode? If so, could this chaincode (including the AC(123456A) function definition) be provided?
Is the AC(123456A) function located in the same Go source file as the rest of the chaincode?
We imagine that the user named user_type1_c19a5f0b7e was registered since the Invoke did work. Were steps to register this user followed as noted in the "Secure Enrollment" section of the "Learn Chaincode" tutorial?
Can the AC(123456A) function both write values to the ledger (through the Invoke) and read values from the ledger (as is being attempted through the Query)?
I have retrieved a json object using typhoeus gem.
url = 'www.example.com' <br>
request = ::Typhoeus::Request.get(url,userpwd: username + ":" + pass)<br>
content = JSON.parse(request.body)
I would like to count the occurence of "Priority":"high" including the quotes inside the json response. How do I go about doing this?
Edit:
"priority":"high" is a key value pair. It is deeply nested inside the json tree.(Don't how deeply it is nested). All I need is count of occurence of "priority":"high"
Any and all suggestion is welcome.
Sample data:
"tickets": [{
"url": "https://.zendesk.com/api/v2/tickets/xxxx.json",
"id": xxxxx,
"external_id": null,
"via": {
"channel": "email",
"source": {
"from": {
"address": "#compli.com",
"name": ""
},
"to": {
"name": "organization Global Support",
"address": "support#organization.zendesk.com"
},
"rel": null
}
},
"created_at": "2016-08-04T16:23:13Z",
"updated_at": "2016-08-08T20:26:01Z",
"type": "problem",
"subject": "Problems with abc Connect",
"raw_subject": "Problems with abc Connect",
"description": "Hi – our Tenet ID is 5675.\n\n \n\nThe abc report is not providing the full data when I run the billing preview. I am running it using Chrome. Attached are snapshots of what I’m doing plus the report generated.\n\n \n\nA perfect example of the problem is shown at the bottom of the report generated. Garber Automotive Group, account number A00000490 does not display the data for all of their products. Their data is shown on rows 5658 thru 5712 on the excel file BillingPreviewResult_201620 report run 08.04.16.\n\n \n\nHowever the EXACT same report (all the parameters are the same) run on 07/01/16 included all of Garber’s information. The excel file abc report run 07.01.16 10.13 AM has the data for Garber on rows 6099 – 6182.\n\n \n\nThe report is cutting off a lot of data for some reason. As you can see by comparing the amount of data between the two excel reports there are much fewer lines on the report run on today as opposed to the one run on 07/01, 6182 rows vs 5712 rows.\n\n \n\nThis is a business critical report for us. It is used for cash forecasting, monthly financial reporting, rolling budgeting and ad hoc reporting.\n\n \n\nWe need this problem identified and fixed immediately. It is already causing a problem with finalizing our July results.\n\n \n\nLet me know if you have any questions or need any additional data.\n\n \n\n \n\nRegards,\n\n \n\n \n\n \n\n| Controller\ndesk: 503.963-4239 | fax: 503.294.1200 | \n\nCompli - Cool, Calm and Compliant. TM\n\nVisit() to learn more.\n\n \n\nFollow us on LinkedIn () and Twitter",
"priority": "normal",
"status": "open",
"recipient": "support#organization.zendesk.com",
"requester_id": 1336424406,
"submitter_id": 1336424406,
"assignee_id": null,
"organization_id": 224504969,
"group_id": 21606503,
"collaborator_ids": [560973773, 786229209, 421597631, 539566717, 707192615, 1336424406, 31365392, 719608577, 1817633993],
"forum_topic_id": null,
"problem_id": null,
"has_incidents": false,
"due_at": null,
"tags": ["1_price", "best_practice_advise", "engage_global_services__email_", "escalate", "hard", "internal_escalation", "p0", "yes_escalated", "xxxxx", "zhub"],
"custom_fields": [{
"id": 22024091,
"value": "p0"
}, {
"id": 24212576,
"value": "best_practice_advise"
}, {
"id": 22035048,
"value": "xxx and so on.....