Is it possible to encapsulate repeated send/responses to the same dart isolate within a single asynchronous function?
Background:
In order to design a convenient API, I would like to have a function asynchronously return the result generated by an isolate, e.g.
var ans = await askIsolate(isolateArgs);
This works fine if I directly use the response generated by a spawnUri call, e.g
Future<String> askIsolate(Map<String,dynamic> isolateArgs) {
ReceivePort response = new ReceivePort();
var uri = Uri.parse(ISOLATE_URI);
Future<Isolate> remote = Isolate.spawnUri(uri, [JSON.encode(isolateArgs)], response.sendPort);
return remote.then((i) => response.first)
.catchError((e) { print("Failed to spawn isolate"); })
.then((msg) => msg.toString());
}
The downside of the above approach, however, is that if I need to repeatedly call askIsolate, the isolate must be spawned each time.
I would instead like to communicate with a running isolate, which is certainly possible by having the isolate return a sendPort to the caller. But I believe since the 2013 Isolate refactoring , this requires the caller to listen to subsequent messages on the receivePort, making encapsulation within a single async function impossible.
Is there some mechanism to accomplish this that I'm missing?
The answer depend on how you intend to use the isolate
Do you intend to keep it running indefinitely, sending it inputs and expecting to receive answers asynchronously?
Do you want to send the isolate many (but finite) inputs at once, expect to receive answers asynchronously, then close the isolate?
I'm guessing the latter, and your askIsolate() function needs to immediately return a Future than completes when it receives all the answers.
The await for loop can be used to listen to a stream and consume events until it closes.
I'm not familiar with isolates, so I hope this is OK, I have not tested it. I've assumed that the isolate terminates and response closes.
String askIsolate(Map<String,dynamic> isolateArgs) async {
ReceivePort response = new ReceivePort();
var uri = Uri.parse(ISOLATE_URI);
Isolate.spawnUri(uri, [JSON.encode(isolateArgs)], response.sendPort)
.catchError((e)) {
throw ...;
});
List<String> answers = new List<String>;
await for(var answer in response) {
out.add(answer.toString());
}
return answers;
}
Note:
response is the stream you are listening to for answers. It's created before spawning the isolate so you don't need to (and probably should not) wait for the isolate future to complete before listening to it.
I made askIsolate() async because that makes it very easy to immediately return a future which completes when the function returns - without all that tedious mucking about with .then(...) chains, which I personally find confusing and hard to read.
BTW, your original then(...).catchError(...) style code would be better written like this:
Isolate.spawnUri(uri, [JSON.encode(isolateArgs)], response.sendPort)
.catchError((e) { ... });
return response.first)
.then((msg) => msg.toString());
I believe that delaying attaching a catchError handler to the line after the isolate's creation might allow the future to complete with an error before the handler is in place.
See: https://www.dartlang.org/articles/futures-and-error-handling/#potential-problem-failing-to-register-error-handlers-early .
I also recommend looking at IsolateRunner in package:isolate, it is intended to solve problems like this - calling a function in the same isolate several times instead of just once when the isolate is created.
If you don't want that, there are other, more primitive, options
Async functions can wait on futures or streams and a ReceivePort is a stream.
For a quick hack, you can probably do something with an await for on the response stream, but it won't be very convenient.
Wrapping the ReceivePort in a StreamQueue from package:async is a better choice. That allows you to convert the individual events into futures. Something like:
myFunc() async {
var responses = new ReceivePort();
var queue = new StreamQueue(responses);
// queryFunction sends its own SendPort on the port you pass to it.
var isolate = await isolate.spawn(queryFunction, [], responses.sendPort);
var queryPort = await queue.next();
for (var something in somethingToDo) {
queryPort.send(something);
var response = await queue.next();
doSomethingWithIt(response);
}
queryPort.send("shutdown command");
// or isolate.kill(), but it's better to shut down cleanly.
responses.close(); // Don't forget to close the receive port.
}
A quick working example based on lrn's comment above follows. The example initializes an isolate via spawnURI, and then communicates with the isolate by passing a new ReceivePort upon which a reply is expected. This allows the askIsolate to directly return a response from a running spawnURI isolate.
Note error handling has been omitted for clarity.
Isolate code:
import 'dart:isolate';
import 'dart:convert' show JSON;
main(List<String> initArgs, SendPort replyTo) async {
ReceivePort receivePort = new ReceivePort();
replyTo.send(receivePort.sendPort);
receivePort.listen((List<dynamic> callArgs) async {
SendPort thisResponsePort = callArgs.removeLast(); //last arg must be the offered sendport
thisResponsePort.send("Map values: " + JSON.decode(callArgs[0]).values.join(","));
});
}
Calling code:
import 'dart:async';
import 'dart:isolate';
import 'dart:convert';
const String ISOLATE_URI = "http://localhost/isolates/test_iso.dart";
SendPort isolateSendPort = null;
Future<SendPort> initIsolate(Uri uri) async {
ReceivePort response = new ReceivePort();
await Isolate.spawnUri(uri, [], response.sendPort, errorsAreFatal: true);
print("Isolate spawned from $ISOLATE_URI");
return await response.first;
}
Future<dynamic> askIsolate(Map<String,String> args) async {
if (isolateSendPort == null) {
print("ERROR: Isolate has not yet been spawned");
isolateSendPort = await initIsolate(Uri.parse(ISOLATE_URI)); //try again
}
//Send args to the isolate, along with a receiveport upon which we listen for first response
ReceivePort response = new ReceivePort();
isolateSendPort.send([JSON.encode(args), response.sendPort]);
return await response.first;
}
main() async {
isolateSendPort = await initIsolate(Uri.parse(ISOLATE_URI));
askIsolate({ 'foo':'bar', 'biz':'baz'}).then(print);
askIsolate({ 'zab':'zib', 'rab':'oof'}).then(print);
askIsolate({ 'One':'Thanks', 'Two':'lrn'}).then(print);
}
Output
Isolate spawned from http://localhost/isolates/test_iso.dart
Map values: bar,baz
Map values: zib,oof
Map values: Thanks,lrn
I’ve implemented my own binary message protocol for simple request/response objects from a Dart client to a Java server. These are encoded in Dart as an Uint8List and on the remote server in Java as a ByteBuffer. The round trip works for the WebSocket in [dart:io] because the websocket.listen stream handler in the Dart client command app is passed data typed as Uint8List.
In [dart:html] the response data in MessageEvent.data received from the websocket.onMessage stream is typed as Blob. I’m not finding a way to convert the Blob to Uint8List. Because the response will often be a large binary array of numbers (double) that will be supplying data to a virtual scrolling context, I want to minimize copying. Could someone please point me in the right direction.
According to this article, you need to use a FileReader to do this.
This example seems to work, the result type is a Uint8List when I tested this in Chrome.
var blob = new Blob(['abc']);
var r = new FileReader();
r.readAsArrayBuffer(blob);
r.onLoadEnd.listen((e) {
var data = r.result;
print(data.runtimeType);
});
Another option is to set WebSocket.binaryType to "arraybuffer". Then MessageEvent.data will return a ByteBuffer which can be turned into a Uint8List. See the example below.
import 'dart:html';
import 'dart:typed_data';
void main() {
var ws = new WebSocket('...')..binaryType = 'arraybuffer';
ws.onMessage.listen((MessageEvent e) {
ByteBuffer buf = e.data;
var data = buf.asUint8List();
// ...
});
}
How would you simply implement this function:
String fetchUrlBodyAsString(String url) {
...
}
Usage:
String schema = fetchUrlBodyAsString("http://json-schema.org/draft-04/schema#");
This thread Using dart to download a file explains a good way to get to the data from a main function. But if you try it you see that the real work happens after leaving main. I think that the synchronous function that I want to create is difficult using HttpClient because it is trying to get an async api to work synchronously. According to this thread that may not be possible: https://groups.google.com/a/dartlang.org/d/msg/misc/kAgayQyaPhQ/wonJ776_FGIJ
What is a Dart way to implement this in a non-browser/console setting?
The using of asynchronous methods is really infectious. Once you start using Future inside a function you have to return a Future as result. So your fetchUrlBodyAsString function can look like :
import 'dart:io';
import 'dart:async';
Future<String> fetchUrlBodyAsString(String url) =>
new HttpClient().getUrl(Uri.parse(url))
.then((HttpClientRequest request) => request.close())
.then((HttpClientResponse response) =>
response.transform(new StringDecoder()).join());
main() {
final url = "http://json-schema.org/draft-04/schema#";
Future<String> schema = fetchUrlBodyAsString(url);
schema.then(handleContent);
}
handleContent(String content) {
print(content); // or do what you want with content.
}
or with async/await:
import 'dart:io';
import 'dart:async';
Future<String> fetchUrlBodyAsString(String url) async {
var request = await new HttpClient().getUrl(Uri.parse(url));
var response = await request.close();
return response.transform(new StringDecoder()).join();
}
main() async {
final url = "http://json-schema.org/draft-04/schema#";
handleContent(await fetchUrlBodyAsString(url));
}
handleContent(String content) {
print(content); // or do what you want with content.
}
You can make synchronous http requests by using HttpRequest.open and setting async to false.
https://api.dartlang.org/apidocs/channels/stable/dartdoc-viewer/dart-dom-html.HttpRequest#id_open
import 'dart:html';
String fetchUrlBodyAsString(String url) {
var request = new HttpRequest()
..open('GET', url, async: false)
..send();
return request.responseText;
}
String schema = fetchUrlBodyAsString("http://json-schema.org/draft-04/schema#");
There is no way to turn an async API into a sync API; once you have a Future as a result, that is what you will have to deal with.
For your specific example, the only way to achieve what you want would be to build your own synchronous HTTP library from the ground up. Using asynchronous APIs in a synchronous manner is not possible.
So I'm working on a web API that a Roku Channel will interact with to send and receive data. Roku SDK has a built in XML Parser that is easy to use, but the only problem is that Roku will only parse XML wrapped in an <rsp stat="ok"></rsp> element. I don't see how or where to override the XML Output on the web API to wrap it with the <rsp> element.
So my question is, how can I override the XML Formatter and insert <rsp stat="ok"> before the output, and </rsp> after?
If you are ensuring that you will return only XML by removing JSON formatter like this
config.Formatters.Remove(config.Formatters.JsonFormatter);
you can use a message handler to add the envelope blindly for all responses like this.
public class MyHandler : DelegatingHandler
{
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
var response = await base.SendAsync(request, cancellationToken);
string responseBody = "<rsp stat=\"ok\">" +
await response.Content.ReadAsStringAsync() +
"</rsp>";
response.Content = new StringContent(
responseBody, Encoding.UTF8, "application/xml");
return response;
}
}
At the moment I'm using a simple HttpRequest to retrieve a JSON file:
void getConfigData(String url) {
var request = new HttpRequest.get(url, this.onSuccess);
}
void onSuccess(HttpRequest req) {
JsonObject conf = new JsonObject.fromJsonString(req.responseText);
MenuItemCollection top_bar = new MenuItemCollection();
// and parse the JSON data ...
}
What I would like to know is if I should be using Futures instead of the callback?
You don't really have the choice between Futures or callbacks, this choice is made by the API you are using. Sometimes you have to give a callback like with HttpRequest.get and sometimes you get a Future like File.create.