Dart Streams: How to load balacing? - dart

Is it possible to make a stream in Dart to load balance its emited datas?
Example:
I have one Stream and 3 listeners (A), (B) and (C):
Emits 0, (A) receives but not (B) and (C)
Then it emits 1, (B) receives but not (A) and (C)
Then it emits 2, (C) receives but not (A) and (B)
How to acomplish it in Dart ?

This is just a draft solution but you can do something like this (and properly add some utility methods like e.g. removing streams when they are stopping listening):
import 'dart:async';
void main() {
final splitter = StreamSplitter(
Stream<int>.periodic(const Duration(seconds: 1), (i) => i));
splitter
.getLoadBalancedStream()
.forEach((element) => print('Called Stream 1 with: $element'));
splitter
.getLoadBalancedStream()
.forEach((element) => print('Called Stream 2 with: $element'));
splitter
.getLoadBalancedStream()
.forEach((element) => print('Called Stream 3 with: $element'));
// Called Stream 1 with: 0
// Called Stream 2 with: 1
// Called Stream 3 with: 2
// Called Stream 1 with: 3
// Called Stream 2 with: 4
// Called Stream 3 with: 5
// Called Stream 1 with: 6
}
class StreamSplitter<T> {
List<StreamController<T>> streamControllers = [];
StreamSplitter(Stream<T> stream) {
final controller = _controllers().iterator;
stream.listen((event) => controller
..moveNext()
..current.add(event));
}
Stream<T> getLoadBalancedStream() {
final controller = StreamController<T>();
streamControllers.add(controller);
return controller.stream;
}
Iterable<StreamController<T>> _controllers() sync* {
// ignore: literal_only_boolean_expressions
while (true) {
for (var i = 0; i < streamControllers.length; i++) {
yield streamControllers[i];
}
}
}
}

Related

Dart Stream: How to merge emitted items, when they're short after eachother?

Let's assume I have a Stream<int> emitting integers in different time deltas i.e. between 5ms and 1000ms.
When the delta is <= 50ms I want to merge them. for example:
3, (delta:100) 5, (delta:27) 6, (delta:976) 3
I want to consume: 3, 11(merged using addition), 3.
Is this possible?
You can use the debounceBuffer stream transformer from the stream_transform package.
stream
.transform(debounceBuffer(const Duration(milliseconds: 50)))
.map((list) => list.fold(0, (t, e) => t + e))
You can write that easily enough yourself:
Stream<int> debounce(
Stream<int> source, Duration limit, int combine(int a, int b)) async* {
int prev;
var stopwatch;
await for (var event in source) {
if (stopwatch == null) {
// First event.
prev = event;
stopwatch = Stopwatch()..start();
} else {
if (stopwatch.elapsed < limit) {
prev = combine(prev, event);
} else {
yield prev;
prev = event;
}
stopwatch.reset();
}
}
// If any event, yield prev.
if (stopwatch != null) yield prev;
}

How to cancel a Stream when using Stream.periodic?

I'm having trouble canceling a stream that is created using the Stream.periodic constructor. Below is my attempt at canceling the stream. However, I'm having a hard time extracting out the 'count' variable from the internal scope. Therefore, I can't cancel the subscription.
import 'dart:async';
void main() {
int count = 0;
final Stream newsStream = new Stream.periodic(Duration(seconds: 2), (_) {
return _;
});
StreamSubscription mySubscribedStream = newsStream.map((e) {
count = e;
print(count);
return 'stuff $e';
}).listen((e) {
print(e);
});
// count = 0 here because count is scoped inside mySubscribedStream
// How do I extract out 'count', so I can cancel the stream?
if (count > 5) {
mySubscribedStream.cancel();
mySubscribedStream = null;
}
}
I'd rather use take(5) instead of checking > 5 and then cancel
final Stream newsStream = new Stream.periodic(Duration(seconds: 2), (_) => count++);
newsStream.map((e) {
count = e;
print(count);
return 'stuff $e';
}).take(5).forEach((e) {
print(e);
});

How can I merge multiple Streams into a higher level Stream?

I have two streams, Stream<A> and Stream<B>. I have a constructor for a type C that takes an A and a B. How do I merge the two Streams into a Stream<C>?
import 'dart:async' show Stream;
import 'package:async/async.dart' show StreamGroup;
main() async {
var s1 = stream(10);
var s2 = stream(20);
var s3 = StreamGroup.merge([s1, s2]);
await for(int val in s3) {
print(val);
}
}
Stream<int> stream(int min) async* {
int i = min;
while(i < min + 10) {
yield i++;
}
}
See also http://news.dartlang.org/2016/03/unboxing-packages-async-part-2.html
prints
10
20
11
21
12
22
13
23
14
24
15
25
16
26
17
27
18
28
19
29
You can use StreamZip in package:async to combine two streams into one stream of pairs, then create the C objects from that.
import "package:async" show StreamZip;
...
Stream<C> createCs(Stream<A> as, Stream<B> bs) =>
new StreamZip([as, bs]).map((ab) => new C(ab[0], ab[1]));
If you need to react when either Stream<A> or Stream<B> emits an event and use the latest value from both streams, use combineLatest.
Stream<C> merge(Stream<A> streamA, Stream<B> streamB) {
return streamA
.combineLatest(streamB, (a, b) => new C(a, b));
}
For people that need to combine more than two streams of different types and get all latest values on each update of any stream.
import 'package:stream_transform/stream_transform.dart';
Stream<List> combineLatest(Iterable<Stream> streams) {
final Stream<Object> first = streams.first.cast<Object>();
final List<Stream<Object>> others = [...streams.skip(1)];
return first.combineLatestAll(others);
}
The combined stream will produce:
streamA: a----b------------------c--------d---|
streamB: --1---------2-----------------|
streamC: -------&----------%---|
combined: -------b1&--b2&---b2%---c2%------d2%-|
Why not StreamZip? Because StreamZip would produce:
streamA: a----b------------------c--------d---|
streamB: --1---------2-----------------|
streamC: -------&----------%---|
combined: -------a1&-------b2%--|
Usage:
Stream<T> sA;
Stream<K> sB;
Stream<Y> sC;
combineLatest([sA, sB, sC]).map((data) {
T resA = data[0];
K resB = data[1];
Y resC = data[2];
return D(resA, resB, resC);
});
To get combined two streams when the second takes a result from the first one use asyncExpand
Stream<UserModel?> getCurrentUserModelStream() {
return FirebaseAuth.instance.authStateChanges().asyncExpand<UserModel?>(
(currentUser) {
if (currentUser == null) {
return Stream.value(null);
}
return FirebaseFirestore.instance
.collection('users')
.doc(currentUser.uid)
.snapshots()
.map((doc) {
final userData = doc.data();
if (userData == null) {
return null;
}
return UserModel.fromJson(userData);
});
},
);
}
Using rxdart, you can use CombineLatestStream to achieve what you want. Note that the new stream doesn't return any value until all streams emitted at least one event:
You can create the combined stream using CombineLatestStream.list():
import 'package:rxdart/rxdart.dart';
Stream<A> s1 = Stream.fromIterable([A()]);
Stream<B> s2 = Stream.fromIterable([B()]);
Stream<dynamic> s3 = CombineLatestStream.list<dynamic>([s1, s2])
..listen(
(value) {
A a = value[0];
B b = value[1];
},
);
Since Dart doesn't support union types, a downside of CombineLatestStream.list() is that events from streams of different types should be casted afterwards (due to List<dynamic>). Another approach is to use CombineLatestStream.combine2() (e.g. with a combiner that creates a Tuple2) to keep the types.

Dart: how to append a transformer to an existing stream?

I'm looking a for a way to programmatically add a transformer to an existing stream that's already being listen to.
Example:
Stream numbers = new Stream.fromIterable([0,1,2,3]);
numbers.listen((number) => print(number));
Now in response to some UI event, I'd like to modify this stream by adding a mapping transformer, as if I originally wrote:
numbers.where((number) => number % 2 == 0);
All existing listeners should from now own only receive even numbers, without interruption. How can this be done?
Instead of thinking about it like "how do I dynamically insert a transformer into a stream", one possible way is to think about it like "how do I dynamically control a transformer that I already injected".
Here's an example of using a StreamTransformer:
var onlySendEvenNumbers = false; // controlled by some UI event handler
var originalStream = makeStreamOfStuff();
originalStream = originalStream.transform(new StreamTransformer.fromHandlers(
handleData: (int value, EventSink<int> sink) {
if (onlySendEvenNumber) {
if (value.isEven) {
sink.add(value);
}
} else {
sink.add(value);
}
}));
originalStream.listen(print); // listen on events like normal
One way I can think of doing that is filtering the Stream with a function that calls another function:
var filter = (n) => true;
Stream numbers = new String.fromIterable([0, 1, 2, 3]).where((n) => filter(n));
Then, when you want to change the filtering:
filter = (n) => n % 2 == 0;
A concrete example:
import 'dart:async';
main() {
var filter = (n) => true;
Stream numbers = new Stream.periodic(new Duration(seconds: 1), (n) => n)
.where((n) => filter(n));
numbers.listen((n) => print(n));
new Future.delayed(new Duration(seconds: 4)).then((_) {
filter = (n) => n % 2 == 0;
});
}
This will print:
0
1
2
3
4
6
8
10
12
And so on, for even numbers only, after 4 seconds.
What about rxdart's combineLatest2 ?
It combine two streams, and emit each time when changed both streams.
You can use Switch class for switch on/off with conditions.
class XsBloc {
Api _api = Api();
BehaviorSubject<List<X>> _xs = BehaviorSubject();
BehaviorSubject<Switcher> _switcher =
BehaviorSubject<Switcher>.seeded(Switcher(false, []));
XsBloc() {
Observable.combineLatest2<List<X>, Switcher, List<X>>(
_api.xs(), _switcher, (xs, s) {
if (s.isOn == true) {
return xs.where((x) => s.conditions.contains(x.id)).toList();
} else {
return xs;
}
}).listen((x) => _xs.add(x));
}
Stream<List<X>> get xs => _xs;
ValueObservable<Switcher> get switcher =>
_switcher.stream;
Function(Switcher) get setSwitcher => _switcher.sink.add;
}
class Switcher {
final bool isOn;
final List<String> conditions;
Switcher(this.isOn, this.conditions);
}
var bloc = XsBloc();
bloc.setSwitcher(true, ['A', 'B']);
bloc.setSwitcher(false, []);
bloc.setSwitcher(true, []);

async Future StreamSubscription Error

Could someone please explain what's wrong with the following code. I'm making two calls to the function fInputData. The first works ok, the second results in an error :
"unhandled exception"
"Bad state: Stream already has subscriber"
I need to write a test console program that inputs multiple parameters.
import "dart:async" as async;
import "dart:io";
void main() {
fInputData ("Enter Nr of Iterations : ")
.then((String sResult){
int iIters;
try {
iIters = int.parse(sResult);
if (iIters < 0) throw new Exception("Invalid");
} catch (oError) {
print ("Invalid entry");
exit(1);
}
print ("In Main : Iterations selected = ${iIters}");
fInputData("Continue Processing? (Y/N) : ") // this call bombs
.then((String sInput){
if (sInput != "y" && sInput != "Y")
exit(1);
fProcessData(iIters);
print ("Main Completed");
});
});
}
async.Future<String> fInputData(String sPrompt) {
async.Completer<String> oCompleter = new async.Completer();
stdout.write(sPrompt);
async.Stream<String> oStream = stdin.transform(new StringDecoder());
async.StreamSubscription oSub;
oSub = oStream.listen((String sInput) {
oCompleter.complete(sInput);
oSub.cancel();
});
return oCompleter.future;
}
void fProcessData(int iIters) {
print ("In fProcessData");
print ("iIters = ${iIters}");
for (int iPos = 1; iPos <= iIters; iPos++ ) {
if (iPos%100 == 0) print ("Processed = ${iPos}");
}
print ("In fProcessData - completed ${iIters}");
}
Some background reading:
Streams comes in two flavours: single or multiple (also known as
broadcast) subscriber. By default, our stream is a single-subscriber
stream. This means that if you try to listen to the stream more than
once, you will get an exception, and using any of the callback
functions or future properties counts as listening.
You can convert the single-subscriber stream into a broadcast stream
by using the asBroadcastStream() method.
So you've got two options - either re-use a single subscription object. i.e. call listen once, and keep the subscription object alive.
Or use a broadcast stream - note there are a number of differences between broadcast streams and single-subscriber streams, you'll need to read about those and make sure they suit your use-case.
Here's an example of reusing a subscriber to ask multiple questions:
import 'dart:async';
import 'dart:io';
main() {
var console = new Console();
var loop;
loop = () => ask(console).then((_) => loop());
loop();
}
Future ask(Console console) {
print('1 + 1 = ...');
return console.readLine().then((line) {
print(line.trim() == '2' ? 'Yup!' : 'Nope :(');
});
}
class Console {
StreamSubscription<String> _subs;
Console() {
var input = stdin
.transform(new StringDecoder())
.transform(new LineTransformer());
_subs = input.listen(null);
}
Future<String> readLine() {
var completer = new Completer<String>();
_subs.onData(completer.complete);
return completer.future;
}
}

Resources