How can I merge multiple Streams into a higher level Stream? - dart

I have two streams, Stream<A> and Stream<B>. I have a constructor for a type C that takes an A and a B. How do I merge the two Streams into a Stream<C>?

import 'dart:async' show Stream;
import 'package:async/async.dart' show StreamGroup;
main() async {
var s1 = stream(10);
var s2 = stream(20);
var s3 = StreamGroup.merge([s1, s2]);
await for(int val in s3) {
print(val);
}
}
Stream<int> stream(int min) async* {
int i = min;
while(i < min + 10) {
yield i++;
}
}
See also http://news.dartlang.org/2016/03/unboxing-packages-async-part-2.html
prints
10
20
11
21
12
22
13
23
14
24
15
25
16
26
17
27
18
28
19
29

You can use StreamZip in package:async to combine two streams into one stream of pairs, then create the C objects from that.
import "package:async" show StreamZip;
...
Stream<C> createCs(Stream<A> as, Stream<B> bs) =>
new StreamZip([as, bs]).map((ab) => new C(ab[0], ab[1]));

If you need to react when either Stream<A> or Stream<B> emits an event and use the latest value from both streams, use combineLatest.
Stream<C> merge(Stream<A> streamA, Stream<B> streamB) {
return streamA
.combineLatest(streamB, (a, b) => new C(a, b));
}

For people that need to combine more than two streams of different types and get all latest values on each update of any stream.
import 'package:stream_transform/stream_transform.dart';
Stream<List> combineLatest(Iterable<Stream> streams) {
final Stream<Object> first = streams.first.cast<Object>();
final List<Stream<Object>> others = [...streams.skip(1)];
return first.combineLatestAll(others);
}
The combined stream will produce:
streamA: a----b------------------c--------d---|
streamB: --1---------2-----------------|
streamC: -------&----------%---|
combined: -------b1&--b2&---b2%---c2%------d2%-|
Why not StreamZip? Because StreamZip would produce:
streamA: a----b------------------c--------d---|
streamB: --1---------2-----------------|
streamC: -------&----------%---|
combined: -------a1&-------b2%--|
Usage:
Stream<T> sA;
Stream<K> sB;
Stream<Y> sC;
combineLatest([sA, sB, sC]).map((data) {
T resA = data[0];
K resB = data[1];
Y resC = data[2];
return D(resA, resB, resC);
});

To get combined two streams when the second takes a result from the first one use asyncExpand
Stream<UserModel?> getCurrentUserModelStream() {
return FirebaseAuth.instance.authStateChanges().asyncExpand<UserModel?>(
(currentUser) {
if (currentUser == null) {
return Stream.value(null);
}
return FirebaseFirestore.instance
.collection('users')
.doc(currentUser.uid)
.snapshots()
.map((doc) {
final userData = doc.data();
if (userData == null) {
return null;
}
return UserModel.fromJson(userData);
});
},
);
}

Using rxdart, you can use CombineLatestStream to achieve what you want. Note that the new stream doesn't return any value until all streams emitted at least one event:
You can create the combined stream using CombineLatestStream.list():
import 'package:rxdart/rxdart.dart';
Stream<A> s1 = Stream.fromIterable([A()]);
Stream<B> s2 = Stream.fromIterable([B()]);
Stream<dynamic> s3 = CombineLatestStream.list<dynamic>([s1, s2])
..listen(
(value) {
A a = value[0];
B b = value[1];
},
);
Since Dart doesn't support union types, a downside of CombineLatestStream.list() is that events from streams of different types should be casted afterwards (due to List<dynamic>). Another approach is to use CombineLatestStream.combine2() (e.g. with a combiner that creates a Tuple2) to keep the types.

Related

Merging Multiple GLTF having some common nodes using gltf-transform

This issue is an Extension of Multiple GLTF loading and Merging on server side.
I am trying to merge multiple GLTF files that have some common nodes too.
The answer helped me for the merging the files and I combined the scenes by following code and it rendered perfectly
const scenes = root.listScenes()
const scene0 = scenes[0]
root.setDefaultScene(scene0);
if (scenes.length > 1) {
for (let i = 1; i < scenes.length; i++) {
let scene = scenes[i];
let nodes = scene.listChildren()
for (let j = 0; j < nodes.length; j++) {
scene0.addChild(nodes[j]);
}
}
}
root.listScenes().forEach((b, index) => index > 0 ? b.dispose() : null);
My issue is all the data that was common in the GLTFs is duplicated and this will create issue in animation when root bones are required to change. Is there a way to merge so that the common nodes are not duplicated ? I am also trying for some custom merge.
const gltfLoader = () => {
const document = new Document();
const root = document.getRoot();
document.merge(io.read(filePaths[0]));
let model;
for (let i = 1; i < filePaths.length; i++) {
const inDoc = new Document();
inDoc.merge(io.read(filePaths[i]));
model = inDoc.getRoot().listScenes()[0];
model.listChildren().forEach((child) => {
mergeStructure(root.listScenes()[0], child);
});
}
io.write('output.gltf', document);
}
const mergeStructure = (parent, childToMerge) => {
let contains = false;
parent.listChildren().forEach((child) => {
if (child.t === childToMerge.t && !contains && child.getName() === childToMerge.getName()) {
childToMerge.listChildren().forEach((subChild) => {
mergeStructure(child, subChild);
});
contains = true;
}
});
if (!contains) {
console.log("Adding " + childToMerge.getName() + " to " + parent.getName() + " as child")
parent.addChild(childToMerge);
}
}
But this merge is not working due to Error: Cannot link disconnected graphs/documents.
I am newbie to 3D modelling. Some direction would be great.
Thanks!
The error you're seeing above is occurring because the code attempts to move individual resources — e.g. Nodes — from one glTF Document to another. This isn't possible, each Document manages its resource graph internally, but an equivalent workflow would be:
Load N files and merge into one document (with N scenes).
import { Document, NodeIO } from '#gltf-transform/core';
const io = new NodeIO();
const document = new Document();
for (const path of paths) {
document.merge(io.read(path));
}
Iterate over all of the scenes, moving their children to some common scene:
const root = document.getRoot();
const mainScene = root.listScenes()[0];
for (const scene of root.listScenes()) {
if (scene === mainScene) continue;
for (const child of scene.listChildren()) {
// If conditions are met, append child to `mainScene`.
// Doing so will automatically detach it from the
// previous scene.
}
scene.dispose();
}
Clean up any remaining unmerged resources.
import { prune } from '#gltf-transform/functions';
await document.transform(prune());

How to search for text in a binary file in Dart

I am trying to search the binary data of a file in dart, to find the index of a substring. I have working js code but I am unable to convert it to dart. This is the js snippet:
var rp = require('request-promise');
async function test(){
const uri = "https://file-examples-com.github.io/uploads/2017/04/file_example_MP4_480_1_5MG.mp4"
const result = await rp({ uri });
const position = Buffer.from(result).indexOf('dnlu');
console.log(position);
}
test() //outputs 2631582
What would be the dart equivalent of this function?
You'll typically fetch the bytes as a Uint8List.
The Dart Uint8List does not have an indexOf method which works on sublists, so you'll have to search the old fashioned way - by looking at it.
I assume that the bytes represent UTF-8 or Latin-1 characters, and since your string contains only ASCII, you can search for the code units directly.
Maybe you could add something like:
extension IndexOfListExtension<T> on List<T> {
int indexOfAll(List<T> needle, [int start = 0]) {
if (needle.length == 0) return start;
var first = needle[0];
var end = this.length - needle.length;
for (var i = start; i <= end; i++) {
match:
if (this[i] == first) {
for (var j = 1; j < needle.length; j++) {
if (this[i + j] != needle[j]) break match;
}
return i;
}
}
return -1;
}
}
Then you would be able to do:
var bytes = await fetch_the_bytes(uri); // However you want to do this.
var position = bytes.indexOfAll("dnlu".codeUnits);
...

Dart Stream: How to merge emitted items, when they're short after eachother?

Let's assume I have a Stream<int> emitting integers in different time deltas i.e. between 5ms and 1000ms.
When the delta is <= 50ms I want to merge them. for example:
3, (delta:100) 5, (delta:27) 6, (delta:976) 3
I want to consume: 3, 11(merged using addition), 3.
Is this possible?
You can use the debounceBuffer stream transformer from the stream_transform package.
stream
.transform(debounceBuffer(const Duration(milliseconds: 50)))
.map((list) => list.fold(0, (t, e) => t + e))
You can write that easily enough yourself:
Stream<int> debounce(
Stream<int> source, Duration limit, int combine(int a, int b)) async* {
int prev;
var stopwatch;
await for (var event in source) {
if (stopwatch == null) {
// First event.
prev = event;
stopwatch = Stopwatch()..start();
} else {
if (stopwatch.elapsed < limit) {
prev = combine(prev, event);
} else {
yield prev;
prev = event;
}
stopwatch.reset();
}
}
// If any event, yield prev.
if (stopwatch != null) yield prev;
}

Dart: how to append a transformer to an existing stream?

I'm looking a for a way to programmatically add a transformer to an existing stream that's already being listen to.
Example:
Stream numbers = new Stream.fromIterable([0,1,2,3]);
numbers.listen((number) => print(number));
Now in response to some UI event, I'd like to modify this stream by adding a mapping transformer, as if I originally wrote:
numbers.where((number) => number % 2 == 0);
All existing listeners should from now own only receive even numbers, without interruption. How can this be done?
Instead of thinking about it like "how do I dynamically insert a transformer into a stream", one possible way is to think about it like "how do I dynamically control a transformer that I already injected".
Here's an example of using a StreamTransformer:
var onlySendEvenNumbers = false; // controlled by some UI event handler
var originalStream = makeStreamOfStuff();
originalStream = originalStream.transform(new StreamTransformer.fromHandlers(
handleData: (int value, EventSink<int> sink) {
if (onlySendEvenNumber) {
if (value.isEven) {
sink.add(value);
}
} else {
sink.add(value);
}
}));
originalStream.listen(print); // listen on events like normal
One way I can think of doing that is filtering the Stream with a function that calls another function:
var filter = (n) => true;
Stream numbers = new String.fromIterable([0, 1, 2, 3]).where((n) => filter(n));
Then, when you want to change the filtering:
filter = (n) => n % 2 == 0;
A concrete example:
import 'dart:async';
main() {
var filter = (n) => true;
Stream numbers = new Stream.periodic(new Duration(seconds: 1), (n) => n)
.where((n) => filter(n));
numbers.listen((n) => print(n));
new Future.delayed(new Duration(seconds: 4)).then((_) {
filter = (n) => n % 2 == 0;
});
}
This will print:
0
1
2
3
4
6
8
10
12
And so on, for even numbers only, after 4 seconds.
What about rxdart's combineLatest2 ?
It combine two streams, and emit each time when changed both streams.
You can use Switch class for switch on/off with conditions.
class XsBloc {
Api _api = Api();
BehaviorSubject<List<X>> _xs = BehaviorSubject();
BehaviorSubject<Switcher> _switcher =
BehaviorSubject<Switcher>.seeded(Switcher(false, []));
XsBloc() {
Observable.combineLatest2<List<X>, Switcher, List<X>>(
_api.xs(), _switcher, (xs, s) {
if (s.isOn == true) {
return xs.where((x) => s.conditions.contains(x.id)).toList();
} else {
return xs;
}
}).listen((x) => _xs.add(x));
}
Stream<List<X>> get xs => _xs;
ValueObservable<Switcher> get switcher =>
_switcher.stream;
Function(Switcher) get setSwitcher => _switcher.sink.add;
}
class Switcher {
final bool isOn;
final List<String> conditions;
Switcher(this.isOn, this.conditions);
}
var bloc = XsBloc();
bloc.setSwitcher(true, ['A', 'B']);
bloc.setSwitcher(false, []);
bloc.setSwitcher(true, []);

async Future StreamSubscription Error

Could someone please explain what's wrong with the following code. I'm making two calls to the function fInputData. The first works ok, the second results in an error :
"unhandled exception"
"Bad state: Stream already has subscriber"
I need to write a test console program that inputs multiple parameters.
import "dart:async" as async;
import "dart:io";
void main() {
fInputData ("Enter Nr of Iterations : ")
.then((String sResult){
int iIters;
try {
iIters = int.parse(sResult);
if (iIters < 0) throw new Exception("Invalid");
} catch (oError) {
print ("Invalid entry");
exit(1);
}
print ("In Main : Iterations selected = ${iIters}");
fInputData("Continue Processing? (Y/N) : ") // this call bombs
.then((String sInput){
if (sInput != "y" && sInput != "Y")
exit(1);
fProcessData(iIters);
print ("Main Completed");
});
});
}
async.Future<String> fInputData(String sPrompt) {
async.Completer<String> oCompleter = new async.Completer();
stdout.write(sPrompt);
async.Stream<String> oStream = stdin.transform(new StringDecoder());
async.StreamSubscription oSub;
oSub = oStream.listen((String sInput) {
oCompleter.complete(sInput);
oSub.cancel();
});
return oCompleter.future;
}
void fProcessData(int iIters) {
print ("In fProcessData");
print ("iIters = ${iIters}");
for (int iPos = 1; iPos <= iIters; iPos++ ) {
if (iPos%100 == 0) print ("Processed = ${iPos}");
}
print ("In fProcessData - completed ${iIters}");
}
Some background reading:
Streams comes in two flavours: single or multiple (also known as
broadcast) subscriber. By default, our stream is a single-subscriber
stream. This means that if you try to listen to the stream more than
once, you will get an exception, and using any of the callback
functions or future properties counts as listening.
You can convert the single-subscriber stream into a broadcast stream
by using the asBroadcastStream() method.
So you've got two options - either re-use a single subscription object. i.e. call listen once, and keep the subscription object alive.
Or use a broadcast stream - note there are a number of differences between broadcast streams and single-subscriber streams, you'll need to read about those and make sure they suit your use-case.
Here's an example of reusing a subscriber to ask multiple questions:
import 'dart:async';
import 'dart:io';
main() {
var console = new Console();
var loop;
loop = () => ask(console).then((_) => loop());
loop();
}
Future ask(Console console) {
print('1 + 1 = ...');
return console.readLine().then((line) {
print(line.trim() == '2' ? 'Yup!' : 'Nope :(');
});
}
class Console {
StreamSubscription<String> _subs;
Console() {
var input = stdin
.transform(new StringDecoder())
.transform(new LineTransformer());
_subs = input.listen(null);
}
Future<String> readLine() {
var completer = new Completer<String>();
_subs.onData(completer.complete);
return completer.future;
}
}

Resources