CEP generate Measurement/Event not in UTC time but in Local - iot

All Measurements came with a time-stamp (event time) of when the measurement was created. Some of these measurements are artificial ones, meaning that they are not created by the device itself, but by a CEP rule running inside the CoT.
The "normal" measurements have the time format coded as UTC
[{
"id": "12704547",
"data": {
"data": {
"time": "2016-07-25T15:24:11.000Z",
"id": "1152930",
"self": "http://testTenant.c8y.com/measurement/measurements/1152930",
"source": {
"id": "222812",
"self": "http://testTenant.c8y.com/inventory/managedObjects/222812"
},
"type": "tsystems_cumulocity_energymeter_digital_ping",
"Energieverbrauch": {
"Ping": {
"unit": "Wh",
"value": 1
}
}
},
"realtimeAction": "CREATE"
},
"channel": "/measurements/222812"
}, {
"successful": true,
"channel": "/meta/connect"
}]
But the "artificial" measurements (created by the CEP rule) use a timestamp with local time
[{
"id": "12704578",
"data": {
"data": {
"time": "2016-07-25T17:24:00.952+02:00",
"id": "1152931",
"self": "http://testTenant.c8y.com/measurement/measurements/1152931",
"source": {
"id": "222812",
"self": "http://testTenant.c8y.com/inventory/managedObjects/222812"
},
"type": "tsystems_cumulocity_energymeter_power_consumption",
"Leistung": {
"Aggregation_1min": {
"unit": "W",
"value": 900
}
}
},
"realtimeAction": "CREATE"
},
"channel": "/measurements/222812"
}]
The measurements from one device should always be encoded with the same timezone (UTC preferred) as different timezone can create problems in clients using that data.
I create the 'time' in the CEP with
current_timestamp().toDate() as time

please use:
com.cumulocity.model.util.DateTimeUtils.newUTC(current_timestamp().toDate()) as dateTime,
instead of
current_timestamp().toDate() as time
in your cep rule.
Best regards,
Arkadiusz
Cumulocity Support Team

Related

Vega-Lite Visualization interpreting dates from Google Sheet as long numbers

Pulling data into Google Data Studio from a Google Sheet with dates stored in yyyy-mm-dd format. The dates look correct and calculate correctly with formulas and adjustments everywhere except in a Gantt chart using the Vega-Lite Community Visualization, which shows the date in a long-number format (e.g. 20210520), and is unable to display the data when using "type": "temporal" or using "timeUnit": "utcyearmonthdatehours".
I've ran various tests, including...
Changing the date format for the date columns to plain text, yyyyddmm, yymmdd, yyyy/mm/dd formats.
Replace the current date columns with new columns using the alternate formats in point 1 (above).
Changing the date field formats directly in Google Data Studio to the formats in point 1 (above).
Creating a secondary set of date columns in plain-text using an Arrayformula and Text() function to reformat the actual dates to plain-text.
So far, options 2 & 4 are the only way I've been able to get the gantt to render correctly, reading the data in date format. But option 2 renders the other charts in GDS as unusable, as the other charts cannot translate the plain-text to usable dates.
Option 4 does work, but isn't the ideal route, given the redundant data. I'd prefer to have just 1 column for the Start Date and another for the End Date, rather than 2 columns for both. Feels like I may be missing something obvious here. Is there a way to either properly format the dates in Google Sheets to work properly with both the GDS date fields and Vega-Lite, or is there a way to properly parse the date data in Vega-Lite without needing to use a second set of plain-text columns?
Report replicating the issue: Project Tracking (debug report)
Edit: below is the code for the Vega-lite visualizations using the date fields from Google Sheets, which Vega-lite is not interpreting as dates.
Without timeunit or temporal field type:
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"description": "A bar chart with highlighting on hover and selecting on click. (Inspired by Tableau's interaction style.)",
"config": {
"background": null,
"view": {
"stroke": "transparent"
}
},
"layer": [
{
"layer": [
{
"params": [
{
"name": "grid",
"select": "interval",
"bind": "scales"
}
],
"mark": {
"type": "bar",
"cursor": "pointer",
"tooltip": true,
"point": true,
"cornerRadiusEnd": 5,
"opacity": 0.8
},
"encoding": {
"color": {
"field": "$dimension3",
"title": "$dimension3.name"
}
}
}
],
"encoding": {
"x": {
"field": "$dimension0",
"axis": {
"title": null,
"grid": true
}
},
"y": {
"field": "$dimension1",
"title": "$dimension1.name",
"type": "nominal",
"sort": "x",
"axis": {
"title": null,
"grid": true,
"tickBand": "extent"
}
},
"x2": {
"field": "$dimension2"
},
"yOffset": {
"field": "$dimension3"
}
}
}
]
}
With timeunit and field type temporal:
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"description": "A bar chart with highlighting on hover and selecting on click. (Inspired by Tableau's interaction style.)",
"config": {
"background": null,
"view": {
"stroke": "transparent"
}
},
"layer": [
{
"layer": [
{
"params": [
{
"name": "grid",
"select": "interval",
"bind": "scales"
}
],
"mark": {
"type": "bar",
"cursor": "pointer",
"tooltip": true,
"point": true,
"cornerRadiusEnd": 5,
"opacity": 0.8
},
"encoding": {
"color": {
"field": "$dimension3",
"title": "$dimension3.name"
}
}
}
],
"encoding": {
"x": {
"field": "$dimension0",
"type": "temporal",
"timeUnit": "utcyearmonthdatehours",
"axis": {
"title": null,
"grid": true
}
},
"y": {
"field": "$dimension1",
"title": "$dimension1.name",
"type": "nominal",
"sort": "x",
"axis": {
"title": null,
"grid": true,
"tickBand": "extent"
}
},
"x2": {
"field": "$dimension2"
},
"yOffset": {
"field": "$dimension3"
}
}
}
]
}

Time Series Insights not showing sub-object properties of a key/value pair

I have an application that is pushing data into IoT Hub which is being used as a data source for TSI. Below is an example message:
{
"EnqueuedTimeUtc": "2021-06-17T22:00:47.2170000Z",
"Properties": {},
"SystemProperties": {
"connectionDeviceId": "Device1",
"connectionAuthMethod": "{\"scope\":\"device\",\"type\":\"sas\",\"issuer\":\"iothub\",\"acceptingIpFilterRule\":null}",
"connectionDeviceGenerationId": "637425408342887985",
"contentType": "application/json",
"contentEncoding": "utf-8",
"enqueuedTime": "2021-06-17T22:00:47.2170000Z"
},
"Body": {
"topic": {
"namespace": "spBv1.0",
"edgeNodeDescriptor": "Routed Group/E2",
"groupId": "Routed Group",
"edgeNodeId": "E2",
"deviceId": "D2",
"type": "DBIRTH"
},
"payload": {
"timestamp": "2021-06-17T22:00:47.082Z",
"metrics": [{
"name": "Ramp1",
"timestamp": "2021-06-17T22:00:47.082Z",
"dataType": "Int32",
"metaData": {},
"properties": {
"Quality": {
"type": "Int32",
"value": 192
},
"My Property": {
"type": "String",
"value": "{\"\":\"\"}"
}
},
"value": 77
}],
"seq": 1
}
}
}
I found documentation showing that my array of 'metrics' is supported as shown here:
https://learn.microsoft.com/en-us/azure/time-series-insights/concepts-json-flattening-escaping-rules
With this message, I can see 'Ramp1' show up in TSI with a value and timestamp as expected. However, the 'properties' under each metric do not show up. In this example that is 'Quality' and 'My Property'. Is there a way to get this data into TSI with an association to 'Ramp1'?

Highcharts line chart - incorrect point ordering - boost module

Issue
We are currently implementing a highcharts graph for large datasets that will require the boost module for performance reasons. However, on enabling the boost module we have encountered an intermittent issue that is proving challenging to consistently reproduce and isolate.
Because I am not able to replicate this I cannot create a jsfiddle as of yet. Does anyone have any idea how to isolate and fix this?
The graph should look like this:
Expected graph
But occasionally does this:
Actual graph
This has also happened on another occasion whilst manually testing with different data:
With different data
It has been observed on
Chrome 70.0.3538.67
IE 11.345.17134.0
Highcharts JS & Boost module v6.1.4
Input data
Sample of the type of data:
{
"label": "2018-04-19T15:17:02",
"value": "1"
}, {
"label": "2018-04-19T15:17:05",
"value": "9620035.36877074"
}, {
"label": "2018-04-19T15:17:59",
"value": "9583104.14689662"
}, {
"label": "2018-04-19T15:18:46",
"value": "9604094.84064805"
}, {
"label": "2018-04-19T15:20:17",
"value": "9571679.9536289"
}, {
"label": "2018-04-20T08:18:24",
"value": "7902991.39771514"
}
Or:
{
"label": "2018-04-19T15:17:59",
"value": "1.4132e-007"
}, {
"label": "2018-04-19T15:18:46",
"value": "1.41434e-007"
}, {
"label": "2018-04-19T15:20:17",
"value": "1.41355e-007"
}, {
"label": "2018-04-20T08:18:24",
"value": "1.40532e-007"
}, {
"label": "2018-04-20T08:20:35",
"value": "1.40928e-007"
}, {
"label": "2018-04-20T08:23:51",
"value": "1.4078e-007"
}, {
"label": "2018-04-20T08:24:39",
"value": "1.40901e-007"
}
Graph config
{
"chart": {
"type": "line",
"zoomType": "x",
"panning": true,
"panKey": "shift"
},
"boost": {
"usePreallocated": false
},
...
y-axis label customisation
title
exporting
etc
...
"series": [{
"showInLegend": false,
"name": "",
"data": []
}],
}
Update mechanism
When the HTTP response comes back the following is called on the graph instance:
graphToUpdate.series[0].setData(parsedData, true, true, false);

YouTube API chromeless player token expiration

Summary
I have a Youtube API chromeless player in my Flex/AS3 application. Sometimes I need the player to play movies from Youtube continuously (non-stop).
Issues
After playing for something like 12 or more hours the API token which was generated by the first player initialization request is getting expired.
http://www.youtube.com/apiplayer?version=3&modestbranding=1&autoplay=0
And the issue here is that the API call loadVideoById which is called again after the token is expired can't start playing video, because the token is not getting renewed automatically.
After playing our playlist (each item is getting played by calling to loadVideoById) for something like 4-5 or more hours the video sometimes starts to play and suddenly getting stuck. While no error event is dispatched.
Known Solutions
So for the first issue the solution is to renew the token each 12 hours by unloading the player and loading it again using the API call:
http://www.youtube.com/apiplayer?version=3&modestbranding=1&autoplay=0
For the second issue there are no ideas. Need your help.
The relevant source code
private function CreateYoutubePlayerSWF():void {
...
theYoutubeLoader.contentLoaderInfo.addEventListener(Event.INIT, onLoaderInit);
theYoutubeLoader.load(new URLRequest("http://www.youtube.com/apiplayer?version=3&modestbranding=1&autoplay=0"));
...
}
private function PlayPendingVideo():void {
...
theYoutubePlayer.loadVideoById(videoWaitingToPlayInfo.videoWaitingToPlay);
...
}
Requests & Responses
When the player stops playing this is what I get in the fiddler:
"request": {
"method": "GET",
"url": "http://www.youtube.com/get_video?cpn=Lj5HaLu7MzS5kG-T&fmt=35&splay=1&t=vjVQa1PpcFNwMVrYUDFOTTG-7co1uJFo3oyrB-qoP_k=&video_id=j-vJJSqw5Q4&eurl=http%3A%2F%2Fwatchitoo.com%2Fiframe.php%3Fid%3Dwwa-154%26scale%3Dfalse%26layout%3D14&asv=3&el=embedded&ps=chromeless&ptk=youtube_none&noflv=1",
...
"queryString": [
{
"name": "cpn",
"value": "Lj5HaLu7MzS5kG-T"
},
{
"name": "fmt",
"value": "35"
},
{
"name": "splay",
"value": "1"
},
{
"name": "t",
"value": "vjVQa1PpcFNwMVrYUDFOTTG-7co1uJFo3oyrB-qoP_k"
},
{
"name": "video_id",
"value": "j-vJJSqw5Q4"
},
{
"name": "eurl",
"value": "http%3A%2F%2Fwatchitoo.com%2Fiframe.php%3Fid%3Dwwa-154%26scale%3Dfalse%26layout%3D14"
},
{
"name": "asv",
"value": "3"
},
{
"name": "el",
"value": "embedded"
},
{
"name": "ps",
"value": "chromeless"
},
{
"name": "ptk",
"value": "youtube_none"
},
{
"name": "noflv",
"value": "1"
}
"response": {
"status": 410,
"statusText": "Gone",
"httpVersion": "HTTP/1.1",
"headers": [
{
"name": "Date",
"value": "Wed, 26 Jun 2013 16:32:37 GMT"
},
{
"name": "X-Content-Type-Options",
"value": "nosniff"
},
{
"name": "Server",
"value": "Apache"
},
{
"name": "X-Frame-Options",
"value": "SAMEORIGIN"
},
{
"name": "Content-Type",
"value": "video/x-flv"
},
{
"name": "Cache-Control",
"value": "no-cache"
},
{
"name": "Content-Length",
"value": "0"
},
{
"name": "Expires",
"value": "Tue, 27 Apr 1971 19:44:06 EST"
}

rails geokit - get travel/commute time

Is there a way to use geokit in rails to get the estimated travel time between two locations? Right now, I'm simply showing the distance away but I think a better approach would be to show '15 minutes away' -- gives it a more practical approach to distance (sometimes 2 miles is 10 minutes and sometimes its 3)
A better solution would be to use the Google Maps Directions API using the API is as simple as calling the following URL http://maps.googleapis.com/maps/api/directions/json?origin=Chicago,IL&destination=Los+Angeles,CA&waypoints=Joplin,MO|Oklahoma+City,OK&sensor=false.
Which returns this JSON:
{
"status": "OK",
"routes": [ {
"summary": "I-40 W",
"legs": [ {
"steps": [ {
"travel_mode": "DRIVING",
"start_location": {
"lat": 41.8507300,
"lng": -87.6512600
},
"end_location": {
"lat": 41.8525800,
"lng": -87.6514100
},
"polyline": {
"points": "a~l~Fjk~uOwHJy#P",
"levels": "B?B"
},
"duration": {
"value": 19,
"text": "1 min"
},
"html_instructions": "Head \u003cb\u003enorth\u003c/b\u003e on \u003cb\u003eS Morgan St\u003c/b\u003e toward \u003cb\u003eW Cermak Rd\u003c/b\u003e",
"distance": {
"value": 207,
"text": "0.1 mi"
}
},
...
... additional steps of this leg
...
... additional legs of this route
"duration": {
"value": 74384,
"text": "20 hours 40 mins"
},
"distance": {
"value": 2137146,
"text": "1,328 mi"
},
"start_location": {
"lat": 35.4675602,
"lng": -97.5164276
},
"end_location": {
"lat": 34.0522342,
"lng": -118.2436849
},
"start_address": "Oklahoma City, OK, USA",
"end_address": "Los Angeles, CA, USA"
} ],
"copyrights": "Map data ©2010 Google, Sanborn",
"overview_polyline": {
"points": "a~l~Fjk~uOnzh#vlbBtc~#tsE`vnApw{A`dw#~w\\|tNtqf#l{Yd_Fblh#rxo#b}#xxSfytAblk#xxaBeJxlcBb~t#zbh#jc|Bx}C`rv#rw|#rlhA~dVzeo#vrSnc}Axf]fjz#xfFbw~#dz{A~d{A|zOxbrBbdUvpo#`cFp~xBc`Hk#nurDznmFfwMbwz#bbl#lq~#loPpxq#bw_#v|{CbtY~jGqeMb{iF|n\\~mbDzeVh_Wr|Efc\\x`Ij{kE}mAb~uF{cNd}xBjp]fulBiwJpgg#|kHntyArpb#bijCk_Kv~eGyqTj_|#`uV`k|DcsNdwxAott#r}q#_gc#nu`CnvHx`k#dse#j|p#zpiAp|gEicy#`omFvaErfo#igQxnlApqGze~AsyRzrjAb__#ftyB}pIlo_BflmA~yQftNboWzoAlzp#mz`#|}_#fda#jakEitAn{fB_a]lexClshBtmqAdmY_hLxiZd~XtaBndgC",
"levels": "BBBAAAAABAABAAAAAABBAAABBAAAABBAAABABAAABABBAABAABAAAABABABABBABAABB"
},
"warnings": [ ],
"waypoint_order": [ 0, 1 ],
"bounds": {
"southwest": {
"lat": 34.0523600,
"lng": -118.2435600
},
"northeast": {
"lat": 41.8781100,
"lng": -87.6297900
}
}
} ]
}
Then parse the JSON returned to grab the duration.text and display it on your page.
You may have to comply with terms of service in order to use this API so make sure you read up on googles requirements and how they affect the use of the service.

Resources