Azure Logic App: Read telemetry data as dynamic content from IoT hub message - iot

I'm routing telemetry messages via IoT Events and event Grid to Logic Apps using a webhook. The logic app lets you input a sample JSON message and then use dynamic content to add information to an email alert I'm sending(O365: Send an Email V2)
I can include System Properties like "iothub-connection-device-id" But when I try to pick temeletry data I get the following error:
InvalidTemplate. Unable to process template language expressions in action 'Send_an_email_(V2)' inputs at line '1' and column '1680': 'The template language expression 'items('For_each')?['data']?['body']?['windingTemp1']' cannot be evaluated because property 'windingTemp1' cannot be selected. Property selection is not supported on values of type 'String'. Please see https://aka.ms/logicexpressions for usage details.'.
When I look at the raw output of the webhook connector it shows the following message but the telemetry points are cleary not there. I'd expect to see them in the "body" property but instead there is just the string: "eyJ3aW5kaW5nVGVtcDEiOjg2LjYzOTYxNzk4MjYxODMzLCJ3aW5kaW5nVGVtcDIiOjc4LjQ1MDc4NTgwMjQyMTUyLCJ3aW5kaW5nVGVtcDMiOjg1LjUzMDYxMDY5OTQ1MzY1LCJMb2FkQSI6MjAyOS44NDgyMTg4ODYxMTEsIkxvYWRCIjoyMDQwLjgxMDk4OTg0MDMzMzgsIkxvYWRWIjoyMDA0LjYxMTkzMjMyNTQ2MTgsIk9pbFRlbXAiOjk5LjA2MjMyNjU2MTY4ODU4fQ=="
Looking for help to determine what could be causing this and how to get the telemetry data passed through correctly so that I can inculde it dynamically in the email alert.
Thanks!
{
"headers": {
"Connection": "Keep-Alive",
"Accept-Encoding": "gzip,deflate",
"Host": "prod-24.northeurope.logic.azure.com",
"aeg-subscription-name": "TEMPALERT",
"aeg-delivery-count": "1",
"aeg-data-version": "",
"aeg-metadata-version": "1",
"aeg-event-type": "Notification",
"Content-Length": "1017",
"Content-Type": "application/json; charset=utf-8"
},
"body": [
{
"id": "c767fb91-3806-324c-ec3c-XXXXXXXXXX",
"topic": "/SUBSCRIPTIONS/XXXXXXXXXXXX",
"subject": "devices/Device-001",
"eventType": "Microsoft.Devices.DeviceTelemetry",
"data": {
"properties": {
"TempAlarm": "true"
},
"systemProperties": {
"iothub-connection-device-id": "Device-001",
"iothub-connection-auth-method": "{\"scope\":\"device\",\"type\":\"sas\",\"issuer\":\"iothub\",\"acceptingIpFilterRule\":null}",
"iothub-connection-auth-generation-id": "637264713410XXXX",
"iothub-enqueuedtime": "2020-06-01T23:05:58.3130000Z",
"iothub-message-source": "Telemetry"
},
"body": "eyJ3aW5kaW5nVGVtcDEiOjg2LjYzOTYxNzk4MjYxODMzLCJ3aW5kaW5nVGVtcDIiOjc4LjQ1MDc4NTgwMjQyMTUyLCJ3aW5kaW5nVGVtcDMiOjg1LjUzMDYxMDY5OTQ1MzY1LCJMb2FkQSI6MjAyOS44NDgyMTg4ODYxMTEsIkxvYWRCIjoyMDQwLjgxMDk4OTg0MDMzMzgsIkxvYWRWIjoyMDA0LjYxMTkzMjMyNTQ2MTgsIk9pbFRlbXAiOjk5LjA2MjMyNjU2MTY4ODU4fQ=="
},
"dataVersion": "",
"metadataVersion": "1",
"eventTime": "2020-06-01T23:05:58.313Z"
}
]
}
Here is the sample input I am using with the trigger:
[{
"id": "9af86784-8d40-fe2g-8b2a-bab65e106785",
"topic": "/SUBSCRIPTIONS/<subscription ID>/RESOURCEGROUPS/<resource group name>/PROVIDERS/MICROSOFT.DEVICES/IOTHUBS/<hub name>",
"subject": "devices/LogicAppTestDevice",
"eventType": "Microsoft.Devices.DeviceTelemetry",
"eventTime": "2019-01-07T20:58:30.48Z",
"data": {
"body": {
"windingTemp1": 95.62818310718433
},
"properties": {
"Status": "Active"
},
"systemProperties": {
"iothub-content-type": "application/json",
"iothub-content-encoding": "utf-8",
"iothub-connection-device-id": "d1",
"iothub-connection-auth-method": "{\"scope\":\"device\",\"type\":\"sas\",\"issuer\":\"iothub\",\"acceptingIpFilterRule\":null}",
"iothub-connection-auth-generation-id": "123455432199234570",
"iothub-enqueuedtime": "2019-01-07T20:58:30.48Z",
"iothub-message-source": "Telemetry"
}
},
"dataVersion": "",
"metadataVersion": "1"
}]

Summary comment to answer to help others who have same problem.
The body you provided is Base64 encoded, you can decode it with Convert.FromBase64String(String) Method.
byte[] newBytes = Convert.FromBase64String(body);
For more details, you could refer to this issue.
Update:
Add the following code in my application will solve the problem.
message.ContentEncoding = "utf-8";
message.ContentType = "application/json";

Related

Filter Mail by Custom InternetMessageHeaders

I am able to send mail and add custom internetMessageHeaders. But what I'm attempting to do now is return messages based on filtering to the custom headers I'm adding.
Example:
If I add a custom header to the message for "x-custom-header-productid = x123z" I want to filter and return all messages that have the custom header for x-custom-header-productid that equals x123z. Is this possible?
Thanks!
The header needs to be provisioned in the Mailbox you want to search for it on eg https://learn.microsoft.com/en-us/exchange/client-developer/exchange-web-services/how-to-provision-x-headers-by-using-ews-in-exchange
So in the Graph if you send a Message using that mailbox that would provision it for use in that Mailbox eg
{
"message": {
"subject": "Meet for lunch?",
"body": {
"contentType": "Text",
"content": "The new cafeteria is open."
},
"toRecipients": [
{
"emailAddress": {
"address": "youmailbox#domain.com"
}
}
],
"singleValueExtendedProperties": [
{
"id": "String {00020386-0000-0000-C000-000000000046} Name X-blah-blah",
"value": "Food"
}
]
}
}
You could then search for any messages that arrived (after you provisioned that header)eg which would mean the property would be promoted in the store.
https://graph.microsoft.com/v1.0/me/mailfolders/inbox/messages?$filter=singleValueExtendedProperties/any(ep:ep/id eq 'String {00020386-0000-0000-C000-000000000046} Name X-blah-blah' and ep/value eq 'Food')

Hasura query action exception

Got a small problem (I guess). I created c# rest web API on docker swarm environment. Rest API is working properly - tested via the postman. Then I tried to compose Hasura service on the same docker swarm environment. The console is working properly also. The problem is with query action.
Code:
Action definition:
type Query {
getWeatherForecast : [WeatherForecastResonse]
}
New types definition:
type WeatherForecastResonse {
date : String
temperatureC : Int
temperature : Int
summary : String
}
Handler:
http://{api ip}:{api port}/WeatherForecast
While trying to execute query:
query MyQuery {
getWeatherForecast {
temperature
summary
date
temperatureC
}
}
All I got from response is error with json:
{
"errors": [
{
"extensions": {
"internal": {
"error": "invalid json: Error in $: not enough input",
"response": {
"status": 405,
"body": "",
"headers": [
{
"value": "Mon, 14 Jun 2021 13:54:00 GMT",
"name": "Date"
},
{
"value": "Kestrel",
"name": "Server"
},
{
"value": "0",
"name": "Content-Length"
},
{
"value": "GET",
"name": "Allow"
}
]
},
"request": {
"body": {
"session_variables": {
"x-hasura-role": "admin"
},
"input": {},
"action": {
"name": "getWeatherForecast"
}
},
"url": "http://{api ip}:{api port}/WeatherForecast",
"headers": []
}
},
"path": "$",
"code": "unexpected"
},
"message": "not a valid json response from webhook"
}
]
}
I got desired response by using postman white calling: http://{api ip}:{api port}/WeatherForecast (GET method)
Where should I improve, to finally get desired result from rest api?
P.S. hasura version: v2.0.0-alpha.4 (tried also with v1.3.3)
UPDATE:
Released a new version of web API. Inside WeatherForecastController included a new method with POST attribute. Query remained the same, but now graphql query returns what I want.
So the question is: Is it possible to call/access web api methods with GET attribute with Hasura action query?
From the version v2.1.0 and above we can do this using the REST Connectors.Hasura Actions RESTConnectors Methods
Go to the Actions tab on the console and create or modify an action. Scroll down to Configure REST Connectors.
In the Configure REST Connectors section, click on Add Request Options Transform
Along with this you can do a lot of other configurations.
No, currently it's not possible, Hasura always makes POST requests to the action handler:
When the action is executed i.e. when the query or the mutation is called, Hasura makes a POST request to the handler with the action arguments and the session variables.
Source: https://hasura.io/docs/latest/graphql/core/actions/action-handlers.html#http-handler

Elastalert Rules for slack integration (message formatting and Attachments)

I'm trying to use message formatting in slack. The Elastalert Testrule.yaml file is partially being parsed. The slack alert shows up with only slack_alert_fields and alert_text fields. I want to send attachments as well in the alerts.
How to use attachments or create buttons fort slack alerts?
es_host: elasticsearch
es_port: 9200
name: Test rule Alert
type: any
index: alerts-*
filter:
- term:
alertType.keyword: "New alert created"
alert:
- "slack"
slack_alert_fields:
- title: Network Name
value: networkName
short: true
- title: Alert Type
value: alertType
short: true
slack_actions:
- name: "network url"
text: "Network URL"
type: "button"
value: networkUrl
alert_text: |
alertData : {0}
alert_text_type: alert_text_only
alert_text_args: ["alertData"]
attachments: [
{
"fallback": "Required plain-text summary of the attachment.",
"color": "#37964f",
"pretext": "New alert created",
"title": alertData.reason ,
"fields": [
{
"title": "Network Name",
"value": networkName,
"short" : true
},
{
"title": "Timestamp",
"value": timestamp,
"short" : true
}
],
"actions": [
{
"name": "network url",
"text": "Network URL",
"type": "button",
"value": networkUrl
},
{
"name": "org_url",
"text": "Organization URL",
"type": "button",
"value": organizationUrl
}
]
}
]
slack_webhook_url:
- "https://hooks.slack.com/xxxxxxxxxxxxxxxxxxxxxxx"
Looking at the official documentation it appears that Elastalert does not support adding custom Slack attachments for alerts, because there is no property for it in the documentation.
In fact it seams that alerts are already formatted as attachment, which is why you can set a title and a title-URL. And also define additional "fields". Something that you can only do with attachments in Slack.
This also means that you can not specify buttons for your alerts (which are a special kind of attachments in Slack).
If you need this functionality I would suggest contacting the developer of Elastalert.

TFS Attachement size 0KB via REST Call

Whenever i tried to attach attachment with TFS WorkItem via REST call, attachment size is 0KB.
First I upload an attachment in Attachment Store using below code.
https://{instance}/DefaultCollection/_apis/wit/attachments?api-version=1.0&filename="{fileName}"
I send data in bytes array through rest call. and after this i attach that attachment with workitem.
Attaching attachment is success but size of an attachment is zero KB
Is there is an issue with TFS or something i am doing wrong?
I am using C# language for programming and REST Sharp for accessing VSTS APIs
Dim restClient = New RestClient("Server URL")
restClient.Authenticator = New HttpBasicAuthenticator("UserId", "Password")
Dim request = New RestRequest("API_Name", Method.POST)
request.AlwaysMultipartFormData = False
request.AddParameter(String.Format("{0}; charset=utf-8", contentType), File.ReadAllBytes(filePath), ParameterType.RequestBody)
request.RequestFormat = DataFormat.Json
Dim response As IRestResponse = restClient.Execute(request)
Return response
I am sending file data in bytes.
Attaching Attachment with WorkItem.
Dim restClient = New RestClient(ACCESS_URL)
restClient.Authenticator = New HttpBasicAuthenticator(USER_NAME, PASSWORD)
Dim request = New
RestRequest("CollectionName}/_apis/wit/workitems/{WorkItem_ID}", Method.PATCH)
request.AddParameter("application/json-patch+json; charset=utf-8",
"post_Data", ParameterType.RequestBody)
request.RequestFormat = DataFormat.Json
Dim response As IRestResponse = restClient.Execute(request)
Return response
Post_Data is an json string which take this type of data
[{
"op": "add",
"path": "/relations/-",
"value": {
"rel": "AttachedFile",
"url": "AttachementURI",
}]
You missed "attributes" section in the Post_Data, try with below:
[{
"op": "add",
"path": "/relations/-",
"value": {
"rel": "AttachedFile",
"url": "AttachementURI",
"attributes": {
}
}]
I can reproduce this issue when keep the { file-contents } as empty.
So, make sure you have specified the { file-contents }.
To attach a file to a work item, upload the attachment to the
attachment store, then attach it to the work item. See Add an attachment for details.
Upload an attachment:
POST https://{instance}/DefaultCollection/_apis/wit/attachments?api-version={version}&filename=Spec.txt
Content-Type: application/octet-stream
{ file-contents }
Add an attachment for specific work item:
PATCH https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/wit/workitems/299?api-version=1.0
Content-Type: application/json-patch+json
[
{
"op": "test",
"path": "/rev",
"value": 3
},
{
"op": "add",
"path": "/fields/System.History",
"value": "Adding the necessary spec"
},
{
"op": "add",
"path": "/relations/-",
"value": {
"rel": "AttachedFile",
"url": "https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/wit/attachments/098a279a-60b9-40a8-868b-b7fd00c0a439?fileName=Spec.txt",
"attributes": {
"comment": "Spec for the work"
}
}
}
]
See below C# sample to upload and add the attachment for a work item:
C# (UploadTextFile method)
C# (AddAttachment method)

Openlayers bbox strategy

I have bbox strategy for one source of data. Code looks like this:
bbox: function newBboxFeatureSource(url, typename) {
return new ol.source.Vector({
loader: function (extent) {
let u = `${url}&TYPENAME=${typename}&bbox=${extent.join(",")}`;
$.ajax(u).then((response) => {
this.addFeatures(
geoJsonFormat.readFeatures(response)
);
});
},
strategy: ol.loadingstrategy.bbox
});
},
I works fine but... When I pan/move the map then this loader is calling again and add another features which fit to new box. But there is a lot of duplicates then because some of new features are just the same as old.
So I wanted first clear all features using this.clear() before add new features but when I add this command then loader is running all the time and I have "infinitive loop". Do you know why? How can I disable loading new features after calling this.clear()?
edit:
my response with features looks like this:
{ "type": "FeatureCollection", "crs": { "type": "name", "properties":
{ "name": "urn:ogc:def:crs:EPSG::3857" } },
"features": [ { "type": "Feature", "properties": { "ogc_fid": "2",
"name": "AL" }, "geometry": { "type": "MultiPolygon" , "coordinates":
[ [ [ ... ] ] ] } }, { "type": "Feature", "properties": { "ogc_fid":
"3", "name": "B" }, "geometry": { "type": "MultiPolygon" ,
"coordinates": [ [ [ ...] ] ] } } ..... and so on
I've removed coordinates because there was too many of them.
My features are generated by mapserver and are configured in .map file which looks like this:
LAYER
NAME "postcode_area_boundaries"
METADATA
"wfs_title" "Postcode area boundaries"
"wfs_srs" "EPSG:3857"
"wfs_enable_request" "*"
"wfs_getfeature_formatlist" "json"
"wfs_geomtype" "multipolygon"
"wfs_typename" "postcode_area_boundaries"
"wms_context_fid" "id"
"wfs_featureid" "id"
"gml_featureid" "id"
"gml_include_items" "id,postarea,wkb_geometry"
"gml_postarea_alias" "name"
"ows_featureid" "id"
"tinyows_table" "postcode_area_boundaries"
"tinyows_retrievable" "1"
"tinyows_include_items" "id,postarea,wkb_geometry"
END
TYPE POLYGON
STATUS ON
CONNECTIONTYPE POSTGIS
CONNECTION "..."
DATA "wkb_geometry FROM postcode_area_boundaries USING UNIQUE id"
DUMP TRUE
END
To summarize the discussion and answer the initial question:
The features sent by the server need an attribute called id, which must be unique and the same for the feature on every request.
{type: "Feature", id: "some-wfs.1234", properties: { "ogc_fid": 2, ...
See this GitHub Issue for the original comment of ahocevar.
In GeoServer this can be achieved if you set an identifier in your layer.
I guess there is something similar to set in MapServer.
Following up on #bennos comment: you can expose the fid in Mapserver using the FORMATOPTION USE_FEATUREID=true as described in: https://mapserver.org/output/ogr_output.html
USE_FEATUREID=true/false
Starting from MapServer v7.0.2. Defaults to false. Include feature ids in the generated output, if the ows_featureid metadata key is set at the layer level. The featureid column to use should be an integer column. Useful if you need to include an “id” attribute to your geojson output. Use with caution as some OGR output drivers may behave strangely when fed with random FIDs.
OUTPUTFORMAT
NAME "application/json"
DRIVER "OGR/GEOJSON"
MIMETYPE "application/json"
FORMATOPTION "FORM=SIMPLE"
FORMATOPTION "FILENAME=ol-query.json"
FORMATOPTION "STORAGE=memory"
FORMATOPTION "USE_FEATUREID=true"
END

Resources