TCP input data packets are combined in Dart - dart

I have a simple TCP client in dart:
import 'dart:io';
void main() {
const sendData = "\$I,Z,0.5,5,0*\r\n";
final socket = Socket.connect("192.168.1.100", int.parse("8008"))
.timeout(Duration(seconds: 5))
.whenComplete(() {
print("Connected!");
}).catchError((_) {
print("Error!");
});
socket.then((soc) {
soc.write(sendData);
soc.listen((data) {
print(String.fromCharCodes(data).trim);
});
});
}
This program sends a special message to server and after that, the server sends back a bunch of data every 10 ms. The output is as follows:
$I,1,250,0,206*$I,1,248,0,192*$I,1,246,0,178*$I,1,245,0,165*
$I,1,244,0,153*$I,1,244,0,141*$I,1,244,0,131*$I,1,245,0,121*
$I,1,246,0,113*$I,1,248,0,105*$I,1,250,0,98*
$I,1,253,0,92*$I,2,0,0,86*$I,2,4,0,82*$I,2,8,0,79*$I,2,12,0,76*
$I,2,18,0,74*$I,2,23,0,74*$I,2,29,0,74*$I,2,36,0,75*$I,2,42,0,77*$I,2,50,0,80*$I,2,58,0,84*$I,2,66,0,89*$I,2,74,0,94*
$I,2,83,0,101*$I,2,93,0,109*$I,2,103,0,117*$I,2,113,0,126*$I,2,124,0,136*$I,2,134,0,147*
$I,2,146,0,159*$I,2,157,0,171*$I,2,169,0,185*$I,2,182,0,199*$I,2,194,0,214*$I,2,207,0,230*$I,2,220,0,246*$I,2,233,1,8*$I,2,247,1,26*$I,3,5,1,44*$I,3,19,1,64*$I,3,33,1,84*
$I,3,48,1,105*$I,3,63,1,126*$I,3,77,1,148*$I,3,93,1,171*$I,3,108,1,194*$I,3,123,1,217*
$I,3,138,1,242*$I,3,154,2,10*$I,3,169,2,35*$I,3,185,2,61*$I,3,201,2,87*$I,3,216,2,113*$I,3,232,2,140*$I,3,248,2,167*$I,4,7,2,195*
$I,4,23,2,223*$I,4,39,2,251*$I,4,54,3,23*$I,4,70,3,51*
The server sends data in $I,1,250,0,206* format, i.e. starts with $ and ends with *. As one may note, several consecutive data packages are concatenated incorrectly.
Whenever I increase the interval, for example to 200 ms, everything is ok.
What should I do?
UPDATE
Besides the BrettSutton answer which is true, the contributers in dart github gave a more complete answer here.
I decided to parse the packet in Socket listen handler, split that and append it to a list. As I want to show the data on a chart, I reset the list after 100 data. However the data could be logged as well.
soc.listen((data) {
var av = data.length;
if (av != 0) {
var stList = String.fromCharCodes(data).trim().split("\$");
stList.forEach((str) {
if (str.isNotEmpty) {
var strS = str.split(",");
if (strS != null) y = parseData(strS);
sampleList.add(y);
}
});
print(sampleList);
if (sampleList.length > 100) {
sampleList.clear();
}
print("==========");
}
});

Related

netmq TryReceiveMultipartMessage() works abnormal

I used the netmq (VisualStudio 2022, by Nuget install netmq) as https://www.nuget.org/packages/NetMQ/ described.
One SubscriberSocket one thread to connect and receive message from one publisher. source code like below:
public void ZMQReceiveThread(string serverIP, int port)
{
//Create SubscriberSocket
SubscriberSocket subSocket = new SubscriberSocket();
//Connect to Publisher
subSocket.Connect("tcp://" + serverIP + ":" + port.ToString());
//Subscribe all topics
subSocket.Subscribe("");
//set timeout value
int timeout = 10000 * 300; //300ms
TimeSpan ts = new TimeSpan(timeout);
while (!_isStopEnabled)
{
NetMQMessage recvMessage = null;
bool bSuccess = subSocket.TryReceiveMultipartMessage(ts, ref recvMessage, 1);
if(bSuccess == true) //Recieve data successfully
{
//Handle the recvMessage
}
else //Timeout
{
//output log message
Loger.Error($"TryReceiveMultipartMessage({ts.TotalMilliseconds} ms) timeout...");
continue;
}
}
}
sometimes the subSocket.TryReceiveMultipartMessage() timeout although the publisher sent message continuously (we used another test app written in C language linked libzmq(https://github.com/zeromq/libzmq) which can receive the continuous message).
Any comments about this topic?
thanks a lot in advance.
I looked through the netmq source code(https://github.com/zeromq/netmq) but cannot find any clues about TryReceiveMultipartMessage()

Dart - Get the last or the first value of a stream

I have a stream and I need to use the last value of this stream, and if there is no value emitted by this stream I need to wait for the fist value. I only want to use this value once. What is the correct way to do it?
Sounds like you want the most recent event emitted by a stream (which is presumably a broadcast stream, because otherwise there is no events until you listen), or, if there has been no events before, you want the next event instead.
For a plain Dart Stream, that's impossible. It doesn't remember previous events. You need to have listened to that stream previously in order to know what the most recent event was (but if you do that, it doesn't have to be a broadcast stream anyway).
You can build your own memorizing stream wrapper fairly easily (but as always with asynchronous programming, you need to be careful about race conditions)
// Copyright 2021 Google LLC.
// SPDX-License-Identifier: Apache-2.0
import "dart:async";
/// Listens to [source] to returned stream.
///
/// Each listener on the returned stream receives the most recent
/// event sent on [source] followed by all further events of [source]
/// until they stop listening.
/// If there has been no events on [source] yet, only the further events
/// are forwarded.
Stream<T> mostRecentStream<T>(Stream<T> source) {
var isDone = false;
var hasEvent = false;
T? mostRecentEvent;
List<MultiStreamController>? pendingListeners;
var listeners = <MultiStreamController>[];
void forEachListener(void Function(MultiStreamController) action) {
var active = 0;
var originalLength = listeners.length;
for (var i = 0; i < listeners.length; i++) {
var controller = listeners[i];
if (controller.hasListener) {
listeners[active++] = controller;
if (i < originalLength) action(controller);
}
}
listeners.length = active;
}
source.listen((event) {
mostRecentEvent = event;
hasEvent = true;
forEachListener((controller) {
controller.addSync(event);
});
}, onError: (e, s) {
forEachListener((controller) {
controller.addErrorSync(e, s);
});
}, onDone: () {
isDone = true;
for (var controller in listeners) {
controller.close();
}
listeners.clear();
});
return Stream<T>.multi((controller) {
if (hasEvent) controller.add(mostRecentEvent as T);
if (isDone) {
controller.close();
} else {
listeners.add(controller);
}
});
}
With that, you can simply do var recentStream = mostRecentStream(yourStream) and then later do recentStream.first to get either the most recent event or, if there is none, the next event (if there is one, you get an error if the stream is completely empty).

How to read part of a chunked stream

In Dart, stdin is a Stream<List<int>>. Bytes come in in chunks. What I want is a function that reads from stdin until I get some character (say '\0'), and then returns, so that future readers of stdin get the data after the '\0'.
Unfortunately because of the chunking, the '\0' byte might be in the middle of a chunk, so I kind of want to read a chunk, remove part of it, and then push it back to the start of the stream. But there isn't any way to do this.
Another option would be readByteSync() but reading bytes one at a time is going to be slow and this is in a GUI program so I can't use sync methods.
I think actually because a Stream<> can only ever be listened to once - even if a previous listener cancels its subscription - the only way is to have something permanently filtering the stream's events until the end of time. So you may as well just split the stream into two streams:
import 'dart:async';
class StdioPreambleSplitter {
StdioPreambleSplitter(this._input) {
var preambleFinished = false;
_input.listen((chunk) {
if (preambleFinished) {
_dataStream.add(chunk);
} else {
final nullByte = chunk.indexOf(0);
if (nullByte == -1) {
_dataStream.add(chunk);
} else {
preambleFinished = true;
_preambleStream.add(chunk.sublist(0, nullByte));
_dataStream.add(chunk.sublist(nullByte));
}
}
});
}
Stream<List<int>> preambleStream() {
return _preambleStream.stream;
}
Stream<List<int>> dataStream() {
return _dataStream.stream;
}
final Stream<List<int>> _input;
final StreamController<List<int>> _preambleStream = new StreamController();
final StreamController<List<int>> _dataStream = new StreamController();
}
Hopefully it doesn't add too much overhead.

Safari dropping Web Socket connection due to idle/inactivity when page not in focus

We are facing this issues with our app, only in safari browser, especially on iOS devices.
Current behavior
Not sure if this is a known issue (I tried searching but found nothing). Safari for Mac appears to be silently dropping web socket connections due to inactivity/idle if the page/tab is not in focus.
The biggest issue is that in mobile iOS X is very persistent.
Steps to reproduce
Open Safari > Website loads > Put Safari in Idle and open any application or lock the device.
On wake up, Safari is closing the connection and the data is not displayed anymore, we get infinite loading of the modules where we request the data.
Expected behavior
Websockets should be kept alive via the heartbeat functionality. Not seeing this behavior in other browsers so unlikely to be the code.
Is this possibly some sort of power-saving feature that is overriding/ignoring the heartbeats?
import 'whatwg-fetch';
import Config from "../config/main";
import WS from "./websocket";
import Helpers from "./helperFunctions";
var Zergling = (function (WS, Config) {
'use strict';
var Zergling = {};
var subscriptions = {}, useWebSocket = false, sessionRequestIsInProgress = false, loginInProgress = false,
uiLogggedIn = false, // uiLogggedIn is the login state displayed in UI (sometimes it differs from real one, see delayedLogoutIfNotRestored func)
authData, session, connectionAvailable, isLoggedIn, longPollUrl;
Zergling.loginStates = {
LOGGED_OUT: 0,
LOGGED_IN: 1,
IN_PROGRESS: 2
};
Zergling.codes = { // Swarm response codes
OK: 0,
SESSION_LOST: 5,
NEED_TO_LOGIN: 12
};
function getLanguageCode (lng) {
if (Config.swarm.languageMap && Config.swarm.languageMap[lng]) {
return Config.swarm.languageMap[lng];
}
return lng;
}
//helper func for fetch
function checkStatus (response) {
if (response.status >= 200 && response.status < 300) {
return response;
} else {
var error = new Error(response.statusText);
error.response = response;
throw error;
}
}
//helper func for fetch
function parseJSON (response) {
return response.json();
}
/**
* #description returns randomly selected(taking weight into consideration) long poll url
* #returns {String} long polling URL
*/
function getLongPollUrl () {
if (!longPollUrl) {
longPollUrl = Helpers.getWeightedRandom(Config.swarm.url).url;
console.debug('long Polling URL selected:', longPollUrl);
}
return longPollUrl;
}
/**
* #description
* Applies the diff on object
* properties having null values in diff are removed from object, others' values are replaced.
*
* Also checks the 'price' field for changes and adds new field 'price_change' as sibling
* which indicates the change direction (1 - up, -1 down, null - unchanged)
*
* #param {Object} current current object
* #param {Object} diff received diff
*/
function destructivelyUpdateObject (current, diff) {
if (current === undefined || !(current instanceof Object)) {
throw new Error('wrong call');
}
for (var key in diff) {
if (!diff.hasOwnProperty(key)) continue;
var val = diff[key];
if (val === null) {
delete current[key];
} else if (typeof val !== 'object') {
current[key] = val;
} else { // diff[key] is Object
if (typeof current[key] !== 'object' || current[key] === null) {
current[key] = val;
} else {
var hasPrice = (current[key].price !== undefined);
var oldPrice;
if (hasPrice) {
oldPrice = current[key].price;
}
destructivelyUpdateObject(current[key], val);
if (hasPrice) {
current[key].price_change = (val.price === oldPrice) ? null : (oldPrice < val.price) * 2 - 1;
}
}
}
}
}
This is and iOS feature that protects users against code draining their battery...
Push notifications for background applications should be performed using iOS's push notification system rather than by keeping an open connection alive.
There are hacks around this limitation, but the truth is that the limitation is good for the users and shouldn't be circumvented.
Read the technical note in the link for more details.

Espruino save code and start on initialization

I have making a cool project. I have more or less finalized my code, which will go on a NodeMCU running Espruino. I have trouble to save this code on the Espruino. This code should do this on each boot: connect to wifi, declare all functions and variables. Then the read() function should be run continuously.
As I see from https://www.espruino.com/Saving I have two options. I tried both.
If I put save() at the end of the code, after I restart NodeMCU code continues to run from where it stopped, but this mean that NodeMCU is not connected to wifi.
If I put E.setBootCode(init()); at the end of code and restart the NodeMCU code doesn't run anymore.
Does anybody know how to properly save code on Espruino so it connects to wifi and defines functions and variables each time it is powered on?
My code:
function init() {
var wifi = require("Wifi");
var http = require("http");
wifi.connect("ALHN-096D", {password:"7381491319"}, function(err){
console.log("connected? err=", err, "info=", wifi.getIP());
});
wifi.stopAP();
//NodeMCU.xx is converter to Espruino pins
I2C1.setup({ 'scl': NodeMCU.D2, // pin D4 (in Espruino firmware, different physical pin)
'sda': NodeMCU.D1, // pin D5 (in Espruino firmware, different physical pin)
bitrate: 100000
}); // set bitrate just in case Arduino is talking in a different bitrate
//function to sort and arrange data in normal order
function sort(data) {
//position cursor, points to position in data array
var position = 0;
//creates empty array, later strings will be appended
var string_arr = [];
//first while loop exits when pointer reads 255
while(data[position] != 255) {
//create empty string; important to have "" not just empty!
var string = "";
//second while loop stops when pointer reaches 44 -> in ASCII ","
while(data[position] != 44) {
//inserts last digit first, function converts decimal to string
string = String.fromCharCode(data[position]) + string;
//increments pointer
position++;
}
//increments pointer to position after the "," (44)
position++;
//pushes newly created string in to the array
string_arr.push(string);
}
return string_arr;
}
function sendToServer(sensor) {
http.get("https://xxxxxx.com/send?temp0="+ sensor[0] +"&temp1=" + sensor[1], function(res) {
res.on('data', function(serverData) {
console.log(serverData);
});
});
}
function read() {
//writes data received from Arduino
//I2C1.readFrom(<ID of device>, <number of bytes to receive>);
//ID of device is set in Arduino firmware
//ID in official docs is represented in hex 0x but works as decimal, need to be identical
var rawData = I2C1.readFrom(8, 20);
var sortedData = sort(rawData);
//console logs data
//sort function returns sorted string array with numbers in right order
console.log("Received ... " + rawData);
console.log("Reversing and sorting ... ");
console.log("Received sorted ... " + sortedData);
console.log("Reading5...");
sendToServer(sortedData);
}
//function calls anonymous function each second
setInterval(function() {
console.log("Reading...");
read();
}, 10000);
}
Output of this code:
Reading...
Received ... 49,56,49,44,49,49,57,44,44,255,255,255,255,255,255,255,255,255,255,255
Reversing and sorting ...
Received sorted ... 181,911,
Your best solution is to rename your init function to onInit, and then type save() after upload and it'll magically start working.
The page you found https://www.espruino.com/Saving mentions about onInit automatically getting called at boot.
What you're doing with E.setBootCode(init()); won't work, because it's executing a string. What you're doing is executing the init() function and then putting the return value of that function into setBootCode .
You'd need E.setBootCode("init();"); - but in this case you should really just do the first option - using onInit

Resources