I have a web component which subscribes to a stream.
Since the web component is re-created each time it's displayed, I have to clean up the subscriber and redo it.
Right now I am adding all subscribers to a list and in removed() life-cycle method I'm doing :
subscriptions.forEach((sub) => sub.cancel());
Now, to the problem: when the web component isn't displayed, there's no one listening to the stream. The issue is that the component is missing data/events when it's not displayed.
What I need is buffering. Events need to be buffered and sent at once when a listener is registered. According to the documentation, buffering happens until a listener is registered:
The controller will buffer all incoming events until the subscriber is registered.
This works, but the problem is that the listener will at some point removed, and re-registered, and it appears this does not trigger buffering.
It appears that buffering happens only initially, not later on even if all listeners are gone.
So the question is: how do I buffer in this situation where listeners may be gone and back?
Note: normally you shouldn't be able to resubscribe to a Stream that has already been closed. This seems to be a bug we forgot to fix.
I'm unfamiliar with web-components but I hope I'm addressing your problem with the following suggestion.
One way (and there are of course many) would be to create a new Stream for every subscriber (like html-events do) that pauses the original stream.
Say origin is the original Stream. Then implement a stream getter that returns a new Stream that is linked to origin:
Untested code.
Stream origin;
var _subscription;
final _listeners = new Set<StreamController>();
_addListener(controller) {
_listeners.add(controller);
if (_subscription == null) {
_subscription = origin.listen((event) {
// When we emit the event we want listeners to be able to unsubscribe
// or add new listeners. In order to avoid ConcurrentModificationErrors
// we need to make sure that the _listeners set is not modified while
// we are iterating over it with forEach. Here we just create a copy with
// toList().
// Alternatively (more efficient) we could also queue subscription
// modification requests and do them after the forEach.
_listeners.toList().forEach((c) => c.add(event));
});
}
_subscription.resume(); // Just in case it was paused.
}
_removeListener(controller) {
_listeners.remove(controller);
if (_listeners.isEmpty) _subscription.pause();
}
Stream get stream {
var controller;
controller = new StreamController(
onListen: () => _addListener(controller),
onCancel: () => _removeListener(controller));
return controller.stream;
}
If you need to buffer events immediately you need to start the subscription right away and not lazily as in the sample code.
Related
I'm writing a flutter app which sends commands via BlueTooth (FlutterBlue) to a device. The device controlls some LEDs.
The communication is working in general quite well but:
On the UI I have a slider controlling the light intensity. When I pull the slider there are more values generated than the bluetooth backend can handle.
In my first implementation I was sending the data directly to the bluetooth characteristic, resulting in exceptions from the bluetooth backend and some values get lost. It's hard to fade light down to zero.
In my second approach I'm using a stream and an await for loop to send the data. Now all values are send without any exceptions but it takes several seconds after releasing the slider until all values are send. Since I want direct visual feedback on the LEDs, this is not an option.
Since there are multiple commands of the same type to be send, I can skip all commands of the same type which were added while the bluetooth send routine was processing a write event.
I saw that there is a Stream.Distinct method but: It returns a new stream. So I have to exit my await for loop and handle the new stream.
Is there a way of removing undesired events from an existing stream without creating a new stream where I have to listen to?
Here is what I'm doing:
class MyBlueToothDevice {
BluetoothDevice _device;
List<BluetoothCharacteristic> _characteristics =
List<BluetoothCharacteristic>();
final _sendStream = StreamController<Tuple2<SendCommands, List<int>>>();
MyBlueToothDevice(this._device) {
_writeNext();
}
Future<void> write(SendCommands command, List<int> value) async {
if (isConnected) {
_sendStream.add(Tuple2<SendCommands, List<int>>(command, value));
// await _characteristics[command.index].write(value).catchError((value) {
// print("Characteristics.write error: $value");
// });
}
}
Future<void> _writeNext() async {
await for (var tuple in _sendStream.stream) {
await _characteristics[tuple.item1.index]
.write(tuple.item2)
.catchError((value) {
print("Characteristics.write error: $value");
});
}
}
}
The best solution is to use application state management to receive all the events from your slider. The state manager will then rate-limit the messages to the device to something it can handle, and also ensure that the most recent message is not lost.
A very basic solution would receive the slider value and update the value in the state manager. A periodic timer with a suitable rate could then update that value to the device; possibly only if the value actually changed since the last time it was sent.
I have this StreamController:
StreamController<Workout> _selectedWorkoutSubject = new StreamController()
I listen to it in a StreamBuilder like this:
StreamBuilder(
stream: workoutBloc.selectedWorkoutStream,
builder: (BuildContext context, AsyncSnapshot<Workout> snapshot) {
if (snapshot.hasData) {
return ...
} else {
return Text('loading...');
}
}
);
When I leave the current page and return to it, I get the Exception: Bad state: Stream has already been listened to. I know I can only once at a time but isn't that what I'm doing? When I leave the page, the StreamBuilder should get Garbage collected which frees the stream, so something else could listen to it.
Or can a Stream only be listend to once in general (Not once at a time)? I know I can just do
StreamController<Workout> _selectedWorkoutSubject = new StreamController.broadcast();
But I'm trying to understand why it doesn't work the way I originally did it.
It's not "one at a time". A traditional non-broadcast stream can only be listened to once, ever. It only allows one listener, and when that listener cancels, the stream cleans up after itself and won't work again.
A traditional broadcast stream can be listened to many times, but all concurrent listeners get the same events.
Most Stream constructors and StreamControllers create either traditional "single-subscription" streams or broadcast streams.
There has been added another way to create a stream which can be listened to more than once: The Stream.multi constructor.
With that, you can treat each listen call individually, and send whichever events you want to them, not necessarily the same values all listeners (each listen call gets its own indivudual StreamController).
You can implement the behavior of a broadcast stream using that, but you don't have to.
Other than that, using a broadcast stream is your best choice. The broadcast stream controller does get onCancel and onListen events when it stops having listeners and starts having listeners again, so you can avoid doing work when there are no listeners.
With Stream.multi, you have to keep record of all the listeners yourself.
Try a few things:
Set the initial data of the stream builder
Change the StreamController to a BehaviorSubject (RxDart lib)
I think the first tip will help you out,
I know I have to cancel Stream Subscriptions when I no longer want to receive any events.
Do I have to this even after I receive a 'Done' event? Or do I get memory leaks?
What happens to Streams that are passed to addStream of another Stream? Are they automatically canceled?
Same Question on the StreamSink side do I have to close them if the stream is already done?
Short-answer: no, but you should. Nothing in the contract of either StreamSubscription or StreamSink requires closing the resources, but some use cases can lead to memory leaks if you don't close them, even though in some cases, doing so might be confusing. Part of the confusion around these classes is that they are overloaded, and handle two fairly distinct use cases:
Resource streams (like file I/O, network access)
Event streams (like click handlers)
Let's tackle these subjects one at a time, first, StreamSubscription:
StreamSubscription
When you listen to a Stream, you receive a StreamSubscription. In general, when you are done listening to that Stream, for any reason, you should close the subscription. Not all streams will leak memory if choose not to, but, some will - for example, if you are reading input from a file, not closing the stream means the handle to the file may remain open.
So, while not strictly required, I'd always cancel when done accessing the stream.
StreamSink
The most common implementation of StreamSink is StreamController, which is a programmatic interface to creating a Stream. In general, when your stream is complete (i.e. all data emitted), you should close the controller.
Here is where it gets a little confusing. Let's look at those two cases:
File I/O
Imagine you were creating an API to asynchronously read a File line-by-line:
Stream<String> readLines(String path);
To implement this, you might use a StreamController:
Stream<String> readLines(String path) {
SomeFileResource someResource;
StreamController<String> controller;
controller = new StreamController<String>(
onListen: () {
someResource = new SomeFileResource(path);
// TODO: Implement adding to the controller.
},
);
return controller.stream;
}
In this case, it would make lots of sense to close the controller when the last line has been read. This gives a signal to the user (a done event) that the file has been read, and is meaningful (you can close the File resource at that time, for example).
Events
Imagine you were creating an API to listen to news articles on HackerNews:
Stream<String> readHackerNews();
Here it makes less sense to close the underlying sink/controller. Does HackerNews ever stop? Event streams like this (or click handlers in UI programs) don't traditionally "stop" without the user accessing for it (i.e cancelling the StreamSubscription).
You could close the controller when you are done, but it's not required.
Hope that makes sense and helps you out!
I found in my case that if I have code like this:
Stream<String> readHackerNews(String path) {
StreamController<String> controller = StreamController<String>();
......
return controller.stream;
}
I see a warning message "Close instance of 'dart.core.Sink'." in the Visual Studio Code.
In order to fix this warning I added
controller.close()
to the event handler for the OnCancel event, see below:
Stream<String> readHackerNews(String path) {
StreamController<String> controller = StreamController<String>();
//TODO: your code here
controller.onCancel = () {
controller.close();
};
return controller.stream;
}
Hope this helps!
I'm not sure why, but for some reason when using the observable that is created via concat I will always get all values that are pushed from my list (works as intended). Where as with the normal subscribe it seems that some values never make it to those who have subscribed to the observable (only in certain conditions).
These are the two cases that I am using. Could anyone attempt to explain why in certain cases when subscribing to the second version not all values are received? Are they not equivalent? The intent here is to rewind the stream. What are some reasons that could explain why Case 2 fails while Case 1 does not.
Replay here is just a list of the ongoing stream.
Case 1.
let observable =
Observable.Create(fun (o:IObserver<'a>) ->
let next b =
for v in replay do
o.OnNext(v.Head)
o.OnNext(b)
o.OnCompleted()
someOtherObs.Subscribe(next, o.OnError, o.OnCompleted))
let toReturn = observable.Concat(someOtherObs).Publish().RefCount()
Case 2.
let toReturn =
Observable.Create(fun (o:IObserver<'a>) ->
for v in replay do
o.OnNext(v.Head)
someOtherObs.Subscribe(o)
).Publish().RefCount()
Caveat! I don't use F# regularly enough to be 100% comfortable with the syntax, but I think I see what's going on.
That said, both of these cases look odd to me and it greatly depends on how someOtherObs is implemented, and where (in terms of threads) things are running.
Case 1 Analysis
You apply concat to a source stream which appears to work like this:
It subscribes to someOtherObs, and in response to the first event (a) it pushes the elements of replay to the observer.
Then it sends event (a) to the observer.
Then it completes. At this point the stream is finished and no further events are sent.
In the event that someOtherObs is empty or just has a single error, this will be propagated to the observer instead.
Now, when this stream completes, someOtherObs is concatenated on to it. What happens now is a little unpreditcable - if someOtherObs is cold, then the first event would be sent a second time, if someOtherObs is hot, then the first event is not resent, but there's a potential race condition around which event of the remainder will go next which depends on how someOtherObs is implemented. You could easily miss events if it's hot.
Case 2 Analysis
You replay all the replay events, and then send all the events of someOtherObs - but again there's a race condition if someOtherObs is hot because you only subscribe after pushing replay, and so might miss some events.
Comments
In either case, it seems messy to me.
This looks like an attempt to do a merge of a state of the world (sotw) and a live stream. In this case, you need to subscribe to the live stream first, and cache any events while you then acquire and push the sotw events. Once sotw is pushed, you push the cached events - being careful to de-dupe events that may been read in the sotw - until you are caught up with live at which point you can just pass live events though.
You can often get away with naive implementations that flush the live cache in an OnNext handler of the live stream subscription, effectively blocking the source while you flush - but you run the risk of applying too much back pressure to the live source if you have a large history and/or a fast moving live stream.
Some considerations for you to think on that will hopefully set you on the right path.
For reference, here is an extremely naïve and simplistic C# implementation I knocked up that compiles in LINQPad with rx-main nuget package. Production ready implementations I have done in the past can get quite complex:
void Main()
{
// asynchronously produce a list from 1 to 10
Func<Task<List<int>>> sotw =
() => Task<List<int>>.Run(() => Enumerable.Range(1, 10).ToList());
// a stream of 5 to 15
var live = Observable.Range(5, 10);
// outputs 1 to 15
live.MergeSotwWithLive(sotw).Subscribe(Console.WriteLine);
}
// Define other methods and classes here
public static class ObservableExtensions
{
public static IObservable<TSource> MergeSotwWithLive<TSource>(
this IObservable<TSource> live,
Func<Task<List<TSource>>> sotwFactory)
{
return Observable.Create<TSource>(async o =>
{
// Naïve indefinite caching, no error checking anywhere
var liveReplay = new ReplaySubject<TSource>();
live.Subscribe(liveReplay);
// No error checking, no timeout, no cancellation support
var sotw = await sotwFactory();
foreach(var evt in sotw)
{
o.OnNext(evt);
}
// note naive disposal
// and extremely naive de-duping (it really needs to compare
// on some unique id)
// we are only supporting disposal once the sotw is sent
return liveReplay.Where(evt => !sotw.Any(s => s.Equals(evt)))
.Subscribe(o);
});
}
}
Why dart calls my function "aFunction" after Step2? If I execute this code this text below in console:
Step2
Step1
My code:
void main()
{
...
stream.listen(aFunction);
print("Step2");
...
}
void aFunction()
{
print("Step1");
}
Thanks for help.
One of the few promises that a Dart Stream makes is that it generates no events in response to a listen call.
The events may come at a later time, but the code calling 'listen' is allowed to continue, and complete, before the first event is fired.
We originally allowed streams to fire immediately on a listen, but when we tried to program with that, it was completely impossible to control in practice.
The same is true for listening on a future, for example with 'then'. The callback will never come immediately.
Events should generally act as if they were fired by the top-level event loop, so the event handler doesn't have to worry if other code is running - other code that might not be reentrant.
That is not always the case in practice. One event handler may trigger other events through a synchronous stream controller, effectively turning one event into anoter. That requires the event handler to know what it is doing. Synchronous controllers are intended for internal use inside, e.g., a stream transformer, and using a synchronous stream controller isn't recommended in general.
So, no, you can't have the listen call immediately trigger the callback.
You can listen to a stream synchronously if you created a StreamController with the sync option enabled. Here is an example to get what you describe:
var controller = new StreamController<String>(sync: true);
var stream = controller.stream.asBroadcastStream();
stream.listen((text) => print(text));
controller.add("Step1");
print("Step2");