How to use results from different isolates in the main isolate? - dart

I'm really new to Dart and also to programming. I'm trying to develop a a command-line program in Dart using isolates. My intention is to compare it's performance againt the same program, but written in Java with threads.
The Dart program looks like this so far:
main.dart
import "dart:async";
import "dart:isolate";
main() {
var rPort1 = new ReceivePort();
var rPort2 = new ReceivePort();
var p1 = 0;
rPort1.listen((partial) {
print("p1 ${partial}");
p1 = partial;
rPort1.close();
});
var p2 = 0;
rPort2.listen((partial) {
print("p2 ${partial}");
p2 = partial;
rPort2.close();
});
Isolate.spawnUri(new Uri.file("MyIsolate.dart"), [arg0, ...], rPort1.sendPort);
Isolate.spawnUri(new Uri.file("MyIsolate.dart"), [arg0, ...], rPort2.sendPort);
var p3 = p1 + p2;
print("p3 ${p3}");
}
myIsolate.dart
import "dart:async";
import "dart:isolate";
main(args, SendPort port) {
var partial = 0;
// ... do stuff ...
// args are used and partial is updated
port.send(partial);
}
The output looks like this:
p3 0
p1 -0.1168096561671553
p2 0.023709338284264223
As you can see the return values of each isolate comes after the main isolate has finished it's execution. What I want is to use the result of the isolates to further calculation in the main.
I don't know what I'm missing. I'm sure is something very stupid, but I cannot move forward on this problem. In Java is simple to get the result value of each thread, but in Dart I can't figure how to do this in isolates.
Any ideas?

You have to wait until all streams (from your ports) are complete. One of ways for doing that is something like that:
import "dart:async";
import "dart:isolate";
main() {
var rPort1 = new ReceivePort();
var rPort2 = new ReceivePort();
// Defining completers which would complete when Streams are finished
Completer c1 = new Completer();
Completer c2 = new Completer();
var p1 = 0;
rPort1.listen((partial) {
print("p1 ${partial}");
p1 = partial;
rPort1.close();
}, onDone: ()=>c1.complete()); // Notice onDone callback here
var p2 = 0;
rPort2.listen((partial) {
print("p2 ${partial}");
p2 = partial;
rPort2.close();
}, onDone: ()=>c2.complete()); // And here
Isolate.spawnUri(new Uri.file("my_isolate.dart"), [0], rPort1.sendPort);
Isolate.spawnUri(new Uri.file("my_isolate.dart"), [0], rPort2.sendPort);
// Waiting for both streams to complete before summing our results
Future.wait([c1.future,c2.future]).then((_){
var p3 = p1 + p2;
print("p3 ${p3}");
});
}
For your task, if you're waiting for exact values, you may define only Futures for these values you need, and complete them without waiting for your isolates (and their streams) to finish.
For doing this, just move c*.complete(<value>) to corresponding listen() callback. Something like that (not tested):
rPort1.listen((partial) {
print("p1 ${partial}");
c1.complete(partial);
rPort1.close();
});
rPort2.listen((partial) {
print("p2 ${partial}");
c2.complete(partial);
rPort2.close();
});
...
Future.wait([c1.future,c2.future]).then((result){
var p3 = result[0] + result[1];
print("p3 ${p3}");
});

If you want to wait for something in Dart, that something should be a future. You can convert a stream or port event to a future in many different ways. If in doubt, you can always use a Completer to create a future from any other event.
In this case it can be done easier because you just want one event from each stream, and you can use Stream.first (or Stream.last or Stream.single) for that.
import "dart:async";
import "dart:isolate";
main() {
var rPort1 = new ReceivePort();
var rPort2 = new ReceivePort();
Future.wait([
Isolate.spawnUri(new Uri.file("my_isolate.dart"), ["0"], rPort1.sendPort)
.then((_) => rPort1.first,
onError: (_) => rPort1.close()),
Isolate.spawnUri(new Uri.file("my_isolate.dart"), ["0"], rPort2.sendPort)
.then((_) => rPort2.first,
onError: (_) => rPort2.close()),
]).then((ps) {
// Waiting for both streams to complete before summing our results
var p3 = ps[0] + ps[1];
print("p3 ${p3}");
});
}
Here I also wait for the spawnUri return Future because it may contain an error if your isolate didn't spawn correctly.
You can also use some of the helper functions in the isolate package.
import "dart:async";
import "dart:isolate";
import "package:isolate/isolate.dart";
main() async {
// A SingleResponseChannel has a send-port and a result future,
// and completes the future with the first port event.
// Warning: Only closed when event is sent on port!
// Consider setting a time-out on the channel.
var c1 = new SingleResponseChannel();
var c2 = new SingleResponseChannel();
Isolate.spawnUri(new Uri.file("my_isolate.dart"), ["0"], c1.port);
Isolate.spawnUri(new Uri.file("my_isolate.dart"), ["0"], c2.port);
var p3 = await c1.result + await c2.result;
print("p3 ${p3}");
}

Related

Assert all the items emitted by a Stream in order until it is canceled in Dart?

Is there any convenient way to assert all the items emitted by a Stream in order until it is canceled?
If I use:
expectLater(
stream,
emitsInOrder(<String>[
'item1',
'item2',
]),
);
and the Stream emits ['item1', 'item2', 'item3'] , the test won't fail.
The only way I've found so far is the following:
var count = 0;
final expected = ['item1', 'item2', 'item3'];
stream.listen(
expectAsync1(
(final result) {
expect(result, expected[count++]);
},
count: expected.length,
),
);
But it is a bit verbose and not very easy to read. Is there a simpler/more elegant way?
You can collect the items into a list, using toList, then compare it to your own expectation list:
await expectLater(stream.toList(), completion(expected));
This does not handle the case where the stream doesn't close at all (but then, nothing does, you just have to wait for a timeout).
It doesn't catch errors until all events have been emitted, the emitsInOrder approach is better for that. Not shorter, though.
emitsDone can be used if the Stream is closed at some point.
E.g:
test('Test', () async {
final controller = StreamController<String>();
final stream = controller.stream;
final matcher = expectLater(
stream,
emitsInOrder(<dynamic>[
'Item1',
'Item2',
emitsDone,
]),
);
controller
..add('Item1')
..add('Item2')
..add('Item3')
..close();
await matcher;
await controller.close();
});
The test fails with error:
Expected: should do the following in order:
• emit an event that 'Item1'
• emit an event that 'Item2'
• be done
Actual: <Instance of '_ControllerStream<String>'>
Which: emitted • Item1
• Item2
• Item3
x Stream closed.
which didn't be done
As #Irn suggest, a more compact alternative for Streams that complete at some point is using toList:
test('Test', () async {
final controller = StreamController<String>();
final stream = controller.stream;
final matcher = expectLater(stream.toList(), completion(<String>['Item1', 'Item2']));
controller
..add('Item1')
..add('Item2')
..add('Item3')
..close();
await matcher;
await controller.close();
});
If the Stream is never closed, you can add a timeout and check the items that have been emitted in that period:
test('Test3', () async {
final controller = StreamController<String>();
final stream = controller.stream.timeout(const Duration(milliseconds: 200));
final matcher = expectLater(
stream,
emitsInOrder(<dynamic>[
'Item1',
'Item2',
]),
);
controller
..add('Item1')
..add('Item2');
await matcher;
await controller.close();
});

Access http response body from outside

import 'package:http/http.dart' as http;
main() {
String esearch = "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=nucleotide&term=Paenibacillus";
var uidList = [];
http.get(esearch).then((response) {
var pattern = new RegExp(r"<Id>(.*?)</Id>");
var hits = pattern.allMatches(response.body);
hits.forEach((hit) {
print("whole match: " + hit[0] + " first match " + hit[1]);
uidList.add(hit[1]);
});
});
print(uidList.length); // empty, because main thread is faster than query
}
Hello everyone,
I'm playing around with Dart since one day to figure out, whether it's suitable for my needs. In the code I have attached, I want to access the result of the body outside of the http query block. This isn't possible. In another question here, someone writes this is because of Darts async concept.
Is there a way to get access to from outside. This is import because I have to trigger several htttp requests with the resulting data and don't wont to nest them all within the http block.
Or any other suggestions?
Thank you very much.
This doesn't work this way because an async call (http.get()) is scheduled for later execution and than execution proceeds with the next line. Your print is executed before http.get() even started to connect. You need to chain all successive calls with then.
If you have a recent Dart version you can use async/await which makes using async calls easier.
import 'package:http/http.dart' as http;
main() {
String esearch = "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=nucleotide&term=Paenibacillus";
var uidList = [];
return http.get(esearch).then((response) {
var pattern = new RegExp(r"<Id>(.*?)</Id>");
var hits = pattern.allMatches(response.body);
hits.forEach((hit) {
print("whole match: " + hit[0] + " first match " + hit[1]);
uidList.add(hit[1]);
});
return uidList;
}).then((uidList) {
print(uidList.length);
});
}
async/await
import 'package:http/http.dart' as http;
main() async {
String esearch = "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=nucleotide&term=Paenibacillus";
var uidList = [];
var response = await http.get(esearch);
var pattern = new RegExp(r"<Id>(.*?)</Id>");
var hits = pattern.allMatches(response.body);
hits.forEach((hit) {
print("whole match: " + hit[0] + " first match " + hit[1]);
uidList.add(hit[1]);
});
print(uidList.length);
}

Dart: how to append a transformer to an existing stream?

I'm looking a for a way to programmatically add a transformer to an existing stream that's already being listen to.
Example:
Stream numbers = new Stream.fromIterable([0,1,2,3]);
numbers.listen((number) => print(number));
Now in response to some UI event, I'd like to modify this stream by adding a mapping transformer, as if I originally wrote:
numbers.where((number) => number % 2 == 0);
All existing listeners should from now own only receive even numbers, without interruption. How can this be done?
Instead of thinking about it like "how do I dynamically insert a transformer into a stream", one possible way is to think about it like "how do I dynamically control a transformer that I already injected".
Here's an example of using a StreamTransformer:
var onlySendEvenNumbers = false; // controlled by some UI event handler
var originalStream = makeStreamOfStuff();
originalStream = originalStream.transform(new StreamTransformer.fromHandlers(
handleData: (int value, EventSink<int> sink) {
if (onlySendEvenNumber) {
if (value.isEven) {
sink.add(value);
}
} else {
sink.add(value);
}
}));
originalStream.listen(print); // listen on events like normal
One way I can think of doing that is filtering the Stream with a function that calls another function:
var filter = (n) => true;
Stream numbers = new String.fromIterable([0, 1, 2, 3]).where((n) => filter(n));
Then, when you want to change the filtering:
filter = (n) => n % 2 == 0;
A concrete example:
import 'dart:async';
main() {
var filter = (n) => true;
Stream numbers = new Stream.periodic(new Duration(seconds: 1), (n) => n)
.where((n) => filter(n));
numbers.listen((n) => print(n));
new Future.delayed(new Duration(seconds: 4)).then((_) {
filter = (n) => n % 2 == 0;
});
}
This will print:
0
1
2
3
4
6
8
10
12
And so on, for even numbers only, after 4 seconds.
What about rxdart's combineLatest2 ?
It combine two streams, and emit each time when changed both streams.
You can use Switch class for switch on/off with conditions.
class XsBloc {
Api _api = Api();
BehaviorSubject<List<X>> _xs = BehaviorSubject();
BehaviorSubject<Switcher> _switcher =
BehaviorSubject<Switcher>.seeded(Switcher(false, []));
XsBloc() {
Observable.combineLatest2<List<X>, Switcher, List<X>>(
_api.xs(), _switcher, (xs, s) {
if (s.isOn == true) {
return xs.where((x) => s.conditions.contains(x.id)).toList();
} else {
return xs;
}
}).listen((x) => _xs.add(x));
}
Stream<List<X>> get xs => _xs;
ValueObservable<Switcher> get switcher =>
_switcher.stream;
Function(Switcher) get setSwitcher => _switcher.sink.add;
}
class Switcher {
final bool isOn;
final List<String> conditions;
Switcher(this.isOn, this.conditions);
}
var bloc = XsBloc();
bloc.setSwitcher(true, ['A', 'B']);
bloc.setSwitcher(false, []);
bloc.setSwitcher(true, []);

Dart HttpClient.getUrl invoked by Timer without client or http server

EDIT: Problem wasn't related to Timer or HttpServer, it was dart.io sleep function pausing everything. It is clearly described in documentation, my bad.
//
I have weird problem with HttpClient working in server code. I call
client.getUrl(Uri.parse(url)).then((HttpClientRequest response) => response.close()).then(HttpBodyHandler.processResponse).then((HttpClientResponseBody body) {
print(body.response.statusCode);
from Timer object and it never reach print step.
It is almost copy and paste code from previous version, which wasn't called from Timer but from HttpRequest. Working code is in my question [here][1].
It fails on the long line, I suspect that it is a last Future it never reach (HttpClientResponseBody).
Timer object is created like this (just test code):
main() {
t = new Timer.periodic(new Duration(minutes: period), (Timer t) => hit());
}
void hit() {
if (new DateTime.now().hour == 17) {
print("syncing rock");
loadUrlBody(furl + filter).then((content) {
print("content loaded");
//edit:
okay, here is the source, it might be some trivial problem..which I can't figure out for two days :-D
import 'dart:async';
import 'dart:io';
import 'package:http_server/http_server.dart';
import 'package:slack/slack_io.dart' as slack;
Timer t;
bool check;
final period = 1;
final furl = "https://****.tpondemand.com";
final filter = "somefilter";
main() {
t = new Timer.periodic(new Duration(minutes: period), (Timer t) => hit());
}
void hit() {
if (new DateTime.now().hour == 17) {
print("syncing rock");
loadUrlBody(furl + filter).then((content) {
print("content loaded");
Map parsedMap = content.body;
handleMap(parsedMap);
});
sleep(new Duration(minutes: 60));
} else {
print("no time to rock " + new DateTime.now().toString());
sleep(new Duration(minutes: period * 10));
}
}
Future loadUrlBody(String url) {
final c = new Completer();
HttpClient client = new HttpClient();
client.addCredentials(Uri.parse("https://****.tpondemand.com/api"), "tprealm", new HttpClientBasicCredentials("user", "password"));
client.getUrl(Uri.parse(url)).then((HttpClientRequest response) => response.close()).then(HttpBodyHandler.processResponse).then((HttpClientResponseBody body) {
print(body.response.statusCode);
c.complete(body);
});
return c.future;
}
void send2Slack(String m) {
slack.Message message = new slack.Message()..text = m;
slack.token = 'token';
slack.team = 'team';
slack.send(message);
}
void handleMap(Map valueMap) {
final Duration lostInTime = new Duration(days: 30);
var sb = new StringBuffer();
sb.write('K o m p o s t \n');
for (var item in valueMap["Items"]) {
if (item['CreateDate'] == null) item['CreateDate'] = '/Date(1403167885000+0100)/';
if (item['ModifyDate'] == null) item['ModifyDate'] = '/Date(1403167885000+0100)/';
if (item['LastCommentDate'] == null) item['LastCommentDate'] = '/Date(1403167885000+0100)/';
DateTime moonLanding = new DateTime.fromMillisecondsSinceEpoch(int.parse(item['CreateDate'].substring(6, 19)));
DateTime modifyLanding = new DateTime.fromMillisecondsSinceEpoch(int.parse(item['ModifyDate'].substring(6, 19)));
DateTime commentLanding = new DateTime.fromMillisecondsSinceEpoch(int.parse(item['LastCommentDate'].substring(6, 19)));
DateTime lastChangeLanding = (modifyLanding.isBefore(commentLanding)) ? commentLanding : modifyLanding;
Duration difference = new DateTime.now().difference(lastChangeLanding);
if (moonLanding.add(lostInTime).isBefore(new DateTime.now()) && difference.inDays > 4) {
sb
..write('<https://****.tpondemand.com/entity/')
..write(item['Id'])
..write('|')
..write(item['Name'])
..write('> last change: ')
..write(difference.inDays)
..write(' days ago \n');
}
;
}
send2Slack(sb.toString());
print("sent to Slack");
sb.clear();
}
I created similar code but I can't reproduce your problem.
So basically this does work when called from a Timer.
import 'dart:io';
import 'dart:async';
import 'package:http_server/http_server.dart';
Timer t;
final period = 1;
void main(args) {
t = new Timer.periodic(new Duration(minutes: period), (Timer t) => hit());
}
void hit() {
loadUrlBody('http://www.google.com')
.then((HttpClientResponseBody b) => print('hit: ${b.response.statusCode}'));
}
Future loadUrlBody(String url) {
print('executing');
HttpClient client = new HttpClient();
// commented out because I have no server where I can use it
// HttpClient client = new HttpClient()
// ..addCredentials(Uri.parse("https://****.tpondemand.com/api"), "tprealm", new HttpClientBasicCredentials("user", "password"));
return client.getUrl(Uri.parse(url)) // <== return is important here
.then((HttpClientRequest response) => response.close())
.then(HttpBodyHandler.processResponse)
.then((HttpClientResponseBody body) {
print('body: (${new DateTime.now()}) ${body.response.statusCode}');
return body; // <== this value is the value the next 'then' receives.
// for example x in: loadUrlBody('http://someurl').then(x) => doSomething(x));
});
}
You don't need to use a Completer. Completer are for more complicated used cases where for example one method returns a Completer and for example an eventHandler completes it.
You just have to ensure that you return a Future everywhere. then always returns a Future. The value of the returned Future is the value returned inside then.

async Future StreamSubscription Error

Could someone please explain what's wrong with the following code. I'm making two calls to the function fInputData. The first works ok, the second results in an error :
"unhandled exception"
"Bad state: Stream already has subscriber"
I need to write a test console program that inputs multiple parameters.
import "dart:async" as async;
import "dart:io";
void main() {
fInputData ("Enter Nr of Iterations : ")
.then((String sResult){
int iIters;
try {
iIters = int.parse(sResult);
if (iIters < 0) throw new Exception("Invalid");
} catch (oError) {
print ("Invalid entry");
exit(1);
}
print ("In Main : Iterations selected = ${iIters}");
fInputData("Continue Processing? (Y/N) : ") // this call bombs
.then((String sInput){
if (sInput != "y" && sInput != "Y")
exit(1);
fProcessData(iIters);
print ("Main Completed");
});
});
}
async.Future<String> fInputData(String sPrompt) {
async.Completer<String> oCompleter = new async.Completer();
stdout.write(sPrompt);
async.Stream<String> oStream = stdin.transform(new StringDecoder());
async.StreamSubscription oSub;
oSub = oStream.listen((String sInput) {
oCompleter.complete(sInput);
oSub.cancel();
});
return oCompleter.future;
}
void fProcessData(int iIters) {
print ("In fProcessData");
print ("iIters = ${iIters}");
for (int iPos = 1; iPos <= iIters; iPos++ ) {
if (iPos%100 == 0) print ("Processed = ${iPos}");
}
print ("In fProcessData - completed ${iIters}");
}
Some background reading:
Streams comes in two flavours: single or multiple (also known as
broadcast) subscriber. By default, our stream is a single-subscriber
stream. This means that if you try to listen to the stream more than
once, you will get an exception, and using any of the callback
functions or future properties counts as listening.
You can convert the single-subscriber stream into a broadcast stream
by using the asBroadcastStream() method.
So you've got two options - either re-use a single subscription object. i.e. call listen once, and keep the subscription object alive.
Or use a broadcast stream - note there are a number of differences between broadcast streams and single-subscriber streams, you'll need to read about those and make sure they suit your use-case.
Here's an example of reusing a subscriber to ask multiple questions:
import 'dart:async';
import 'dart:io';
main() {
var console = new Console();
var loop;
loop = () => ask(console).then((_) => loop());
loop();
}
Future ask(Console console) {
print('1 + 1 = ...');
return console.readLine().then((line) {
print(line.trim() == '2' ? 'Yup!' : 'Nope :(');
});
}
class Console {
StreamSubscription<String> _subs;
Console() {
var input = stdin
.transform(new StringDecoder())
.transform(new LineTransformer());
_subs = input.listen(null);
}
Future<String> readLine() {
var completer = new Completer<String>();
_subs.onData(completer.complete);
return completer.future;
}
}

Resources