I was wondering, is it possible to send a Future through a stream that can get resolved after "reception" by a listener?
Cheers!
Futures are just values, so you can make a Stream<Future>. The receiver of the future can then wait on the future normally, and you can complete it at any point in between
It's generally frowned upon, though, because of the double asynchrony.
What it does is to make the receiver wait for a stream event, which you emit at one point, then have the receiver wait again for the actual result which may come at an even later point.
The most urgent issue with that is that you don't always know whether the future has been received yet when you complete it. Maybe the receiver paused the stream, maybe you are quicker than you expected. In any case, if you complete the future with an error before it has been received, then that error is probably going to end up uncaught, which may crash your entire program.
It also has bad usability. If you instead waited for the future on the sending side and only sent the event when the result was ready, it's easier and simpler for the receiver (they just get the result as normal), and it's usually just as good at achieving what you want to achieve.
If you really have a situation where a number of asynchronous results (futures) can complete in any order, but the receiver needs to know the original order of the futures themselves, then I guess a Stream<Future<X>> can be the answer (but do consider whether your solution is just needlessly complicated).
Example (in full generality):
Stream<Future<int>> randomDelays() {
var controller = StreamController<Future<String>>();
controller.onListen = () {
var rng = Random();
for (int i = 0; i < 10; i++) {
var delay = rng.nextInt(10);
var completer = Completer<int>();
controller.add(completer.future);
Timer(Duration(seconds: delay), () {
completer.complete(i);
});
}
controller.close();
}
}
or simpler:
Future<Stream<int>> randomDelays() async* {
var rng = Random();
for (int i = 0; i < 10; i++) {
var delay = rng.nextInt(10);
yield Future.delayed(Duration(seconds: delay), () => i);
}
}
I am not sure why your want that but sure you can do that:
import 'dart:async';
void main() {
final controller = StreamController<FutureOr<int>>();
controller.sink.add(Future.delayed(Duration(seconds: 1), () => 5));
print(controller.stream.first.runtimeType); // _Future<FutureOr<int>>
}
When you add a Future to the sink it will not be automatically awaited. So what you get out from the Stream are Future objects if you put Future objects in the sink.
Related
I've enabled the dart 2.8 experimental null saftey.
I have the following exiting code.
StreamSubscription<String> subscription;
subscription =
response.transform(Utf8Decoder()).transform(LineSplitter()).listen(
(line) async {
result += line;
},
onDone: () async {
unawaited(subscription.cancel());
completer.complete(result);
},
);
With null saftey enabled I get a error in the 'onDone' method where it calls subscription.cancl
"The expression is nullable and must be null-checked before it can be used.
Try checking that the value isn't null before using it.",
I can fix the problem by putting a conditional before the call to cancel, but this seems unnecessary as in reality subscription can never be null.
Is there a coding pattern that allows subscription to be declared as non-null?
The problem here is that the read of subscription happens at a place where it's still potentially unassigned. It isn't, actually, but we only know that because the listen method promises not to call any of the callbacks before returning. The compiler can't see that. So, you need to move the reading to after the assignment.
What I'd do to make this listen call work:
var buffer = StringBuffer(result);
var subscription = response
.transform(Utf8Decoder())
.transform(LineSplitter())
.listen((line) {
buffer.write(line);
});
subscription.onDone(() {
completer.complete(buffer.toString());
});
I removed the async from the callbacks because it is not needed. All it does to make these functions async is to return a future that no-one would ever look at.
In general, the callbacks on Stream and Future should have non-async callbacks.
I also removed the subscription.cancel from the onDone event handler. If you get a "done" event, the subscription is done, there is no need to cancel it.
I also added a string buffer to avoid the quadratic time and space complexity of repeated string concatenation.
Looking at the code, you seem to be concatenating lines right after splitting them, maybe all you need is:
response.transform(Utf8Decoder()).join("").then(completer.complete);
I'll assume for now that the splitting+joining is necessary.
In that case, what I'd actually prefer to do instead is of using listen is:
var buffer = StringBuffer();
response
.transform(Utf8Decoder())
.transform(LineSplitter())
.forEach((line) {
buffer.write(line);
}).then(() {
completer.complete(buffer.toString());
}, onError: (e, s) {
completer.completeError(e, s);
});
or, if in an async function:
try {
var buffer = StringBuffer();
await for (var line in response.transform(Utf8Decoder()).transform(LineSplitter())) {
buffer.write(line);
}
completer.complete(buffer.toString());
} catch(e, s) {
completer.completeError(e, s);
}
I am just trying to understand how timeout works in Futures, but I can't interrupt anything a litle more complex. Look at this really simple code , I thought that it would get an exception if that loop take longer than 1 second but it does not happen. Anyone could please explain what happens here?
I am trying using this tool: https://dartpad.dartlang.org/
import 'dart:async';
void main() {
new Future(() {
var sum = 0;
for (var i = 0; i < 500000000; i++) {
sum += i;
}
return sum;
}).timeout(new Duration(seconds: 1)).then(print).catchError(print);
}
First of all Future.timeout does not interrupt any computation.
It creates a new Future which completes either with the value of the original Future, or with the result of timing out, if the original future didn't complete in time. It's more like it takes two futures: The original one and one created using Future.delayed, and then creates a third Future which will complete with the result of the first the other two to complete.
There is no signal going back to the computation which will eventually complete the original future. It will not stop unless you make it stop.
The use-case for Future.timeout is, say, a network connection that doesn't appear to give a result in time. You can't stop that request, but you can stop waiting for the answer, which is what timeout does.
What happens in your example is:
You create a Future, f1 with the Future constructor. This schedules a timer with zero duration to call the argument function.
You then call timeout on f1. This starts a timer with a duration of one second, and returns a future f2.
You then call then on f2 with print as argument. This puts a listener on f2 and returns a future f3.
You then call catchError on f3 which puts an error listener on f3 and returns a future f4 (which is then ignored).
Then control returns to the event loop, and the next event is the zero-duration timer.
The argument to the Future constructor is called. It counts to 500000000, then returns a value v and completes f1 with that.
The result of f1 is propagated to f2 which also completes with v.
The result of f2 is printed by print, and f3 completes with null.
The result of f4 is ignored by the catchError and f4 completes with null.
Then control returns to the event loop. The next event is the 1-second timer started by Future.timeout. More than one second may have passed already, but this is the earliest opportunity for it to run.
The Future.timeout timer callback sees that f2 is already completed and does nothing.
Control returns to the event loop, and since it is empty, the program ends.
i think you need to call this way timeout(Duration timeLimit, onTimeout());
so timeout(new Duration(seconds: 1),printFunction())
import 'dart:async';
_onTimeout() => print("Time Out occurs");
void main() {
new Future(() {
var sum = 0;
for (var i = 0; i < 500000000; i++) {
sum += i;
}
return sum;
}).timeout(new Duration(seconds: 1),onTimeout: _onTimeout());
}
Your code is not working because your future value is being processing synchronously, so the event loop cannot process anything else but the for loop you're running. I suggest to read through these articles to understand better https://medium.com/hackernoon/are-futures-in-dart-threads-2cdc5bd8063a https://medium.com/dartlang/dart-asynchronous-programming-isolates-and-event-loops-bffc3e296a6a of Futures and the event loop works.
A quick fix to your issue would be to use dart Isolates (together with the package:isolate library), for example:
import 'dart:async';
import 'package:isolate/isolate.dart';
Future<void> main() async {
var isolate = await IsolateRunner.spawn();
var value = await isolate.run(func, null, timeout: Duration(seconds: 1));
print(value);
}
int func(_) {
var sum = 0;
for (var i = 0; i < 500000000; i++) {
sum += i;
}
return sum;
}
It will throw a TimeoutException if the func() takes more than 1 second to complete, you can set an onTimeout function to be run when the timeout is hit.
I have just started using durable functions and needs some advise for how to do fan out pattern correctly. I have a FTP server where from I read all the files. I want to start an Activity function for each file. As I understand it the orchestrator function will be called everytime an Activity function is being executed. I just want to read the files once. To avoid calling the code that read the files and starts the activity functions multiple times, what is the recommended approach? Is it having an activity function that that add's all the activity functions or is it using the IsReplaying property, or something different?
[FunctionName("OrchestrationMoveFilesToBlob")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context)
{
var outputs = new List<string>();
if (!context.IsReplaying)
{
// Do you call your database here and make a call to CallActivityAsync for each row?
}
// doing it here is properly very wrong as it will be called multiple times
var tasks = new Task<string>[7];
for (int i = 0; i < 7; i++)
{
tasks[i] = context.CallActivityAsync<string>("E2_CopyFileToBlob",""); }
await Task.WhenAll(tasks);
return outputs;
}
When looking into the sample in the link below this actually calls it directly in the orchestrator function? Is this not really bad? It continue adding same activities again and again .... ?
https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-cloud-backup
Not sure I understand what you try to achieve but your code looks not bad so far. An orchestration is just called once (and maybe some times more for replay but this is not your problem here). From your orchestration you can call in a fan out all your activity functions (gathering a file from an ftp) each activity function one file. await Task.WhenAll(tasks) is your fan in. (you can use a List<Task> instead of the array and call .Add(task) on it if you want. In order to not edit your code I copied it here and added some comments and questions (feel free to edit here):
[FunctionName("OrchestrationMoveFilesToBlob")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context)
{
var outputs = new List<string>();
if (!context.IsReplaying)
{
// just needed for things that should not happen twice like logging....
}
// if your work isn't a fixed list just call an activity
// which replies with the list of work here (e.g. list of filenames)
var tasks = new Task<string>[7]; // can be a List<Task> too
for (int i = 0; i < 7; i++)
{
tasks[i] = context.CallActivityAsync<string>("E2_CopyFileToBlob","");
}
await Task.WhenAll(tasks);
return outputs; // currently an empty list. What do you want to give back?
}
I suspect that my understanding of Streams in Dart might have a few holes in it...
I have a situation in which I'd like a Dart app to respond to intermittent input (which immediately suggests the use of Streamss -- or Futures, maybe). I can implement the behavior I want with listener functions but I was wondering how to do this in a better, more Dartesque way.
As a simple example, the following (working) program listens to keyboard input from the user and adds a div element to the document containing what has been typed since the previous space, whenever the space bar is hit.
import 'dart:html';
main() {
listenForSpaces(showInput);
}
void listenForSpaces(void Function(String) listener) {
var input = List<String>();
document.onKeyDown.listen((keyboardEvent) {
var key = keyboardEvent.key;
if (key == " ") {
listener(input.join());
input.clear();
} else {
input.add(key.length > 1 ? "[$key]" : key);
}
});
}
void showInput(String message) {
document.body.children.add(DivElement()..text = message);
}
What I'd like to be able to do is to create a new Stream from the Stream that I'm listening to (in the example above, to create a new Stream from onKeyDown). In other words, I might set the program above out as:
var myStream = ...
myStream.listen(showInput);
I suspect that there is a way to create a Stream and then, at different times and places, insert elements to it or call for it to emit a value: it feels as though I am missing something simple. In any case, any help or direction to documentation would be appreciated.
Creating a new stream from an existing stream is fairly easy with an async* function.
For a normal stream, I would just do:
Stream<String> listenForSpaces() async* {
var input = <String>[];
await for (var keyboardEvent in document.onKeyDown) {
var key = keyboardEvent.key;
if (key == " ") {
yield input.join();
input.clear();
} else {
input.add(key.length > 1 ? "[$key]" : key);
}
}
}
The async* function will propagate pauses through to the underlying stream, and it may potentially pause the source during the yield.
That may or may not be what you want, since pausing a DOM event stream can cause you to miss events. For a DOM stream, I'd probably prefer to go with the StreamController based solution above.
There are several methods and there is a whole package rxdart to allow all kinds of things.
Only the final consumer should use listen and only if you need to explicitly want to unsubscribe, otherwise use forEach
If you want to manipulate events like in your example, use map.
I wasn't originally planning to answer my own question but I have since found a very simple answer to this question in the dartlang creating streams article; in case it's helpful to others:
Specifically, if we'd like to create a stream that we can insert elements into at arbitrary times and places in the code, we can do so via the StreamController class. Instances of this class have an add method; we can simply use the instance's stream property as our stream.
As an example, the code in my question could be rewritten as:
import 'dart:html';
import 'dart:async';
main() async {
// The desired implementation stated in the question:
var myStream = listenForSpaces();
myStream.listen(showInput);
}
Stream<String> listenForSpaces() {
// Use the StreamController class.
var controller = StreamController<String>();
var input = List<String>();
document.onKeyDown.listen((keyboardEvent) {
var key = keyboardEvent.key;
if (key == " ") {
// Add items to the controller's stream.
controller.add(input.join());
input.clear();
} else {
input.add(key.length > 1 ? "[$key]" : key);
}
});
// Listen to the controller's stream.
return controller.stream;
}
void showInput(String message) {
document.body.children.add(DivElement()..text = message);
}
(As mentioned in the article, we need to be careful if we want to set up a stream from scratch like this because there is nothing to stop us from inserting items to streams that don't have associated, active subscribers; inserted items would in that case be buffered, which could result in a memory leak.)
how to wait for the completion of Future without 'async' and 'futures'?
In the library that I use all functions are asynchronous.
// must return <bool>
bool my_func(int x){
//returns Future<int>
var tmp = somelib.somefunc( ... );
//wait y
return x == y;
}
I tried to write my 'await', but
waiting for a result with a while loop freezes everything.
dynamic my_await(Future f) {
dynamic x;
bool completed = false;
f.then((v){
x = v;
completed = true;
});
do {} while (!completed);
return x;
}
Dart VM version: 1.24.3 (Mon Dec 18 16:57:48 2017) on "linux_x64"
A synchronous function, or really, any Dart function, returns a value immediately when you call them. If you want to return a boolean immediately, and the value of that boolean depends on the result that some future completes with, then there is no way to compute that boolean in time.
If you need to wait for a future, then your function is asynchronous. You need to return something immediately, even if you don't know the result yet. That's what a Future is. It's not magical in any way, it's just an object that you can set a callback on which gets called when some result is ready.
So, you need to return a Future<bool> for this to work.
Dart is single-threaded. Without using isolates, there is no concurrency. Instead asynchronous functions work by taking turns, giving time for other code to run, e.g., while they wait on a future. If you just do a do {} while (!completed); then no other code gets to run, which means that nothing will be able to set completed to true.
I'm new to dart, so not sure if this is the correct way of doing it, but I've solved this issue by using the function whenCompleted() on the Future returned by the async method I'm calling.
Here openDatabase returns a Future.
abstract class IBaseDatabaseHandler {
Database sqliteDbHandler;
IBaseDatabaseHandler.sqlite(String dataBasePath) {
sqfliteFfiInit();
var databaseFactory = databaseFactoryFfi;
databaseFactory
.openDatabase(dataBasePath)
.whenComplete(() => sqliteDbHandler);
}
}