Yield values from generator from async functions without awaiting - dart

I need to run many async functions concurrently and yield the results as they complete, order doesn't matter.
Here is what i have in a simplified example, of course this does not work right because it's waiting for every response before moving to the next request.
Stream<String> stringGenerator(List<http.Request> requests) async* {
final httpClient = http.Client();
for (var req in requests) {
final response = await httpClient.send(req);
yield response.headers['example'];
}
}

Could you try and see if this is working for you?
Stream<String> stringGenerator(List<http.Request> requests) {
final controller = StreamController<String>();
final httpClient = http.Client();
Future.wait(requests.map((req) => httpClient
.send(req)
.then((response) => controller.add(response.headers['example']!))))
.whenComplete(() => controller.close());
return controller.stream;
}
More correct would be this, since we don't want to generate events before we are listening for them according to the documentation for StreamController. It is really not an issue for internal usage since StreamController does buffer events until a listener are subscribed:
Stream<String> stringGenerator(List<http.Request> requests) {
final controller = StreamController<String>();
controller.onListen = () {
final httpClient = http.Client();
Future.wait(requests.map((req) => httpClient
.send(req)
.then((response) => controller.add(response.headers['example']!))))
.whenComplete(() => controller.close());
};
return controller.stream;
}

A generalized alternative to #julemand101's solution which works for any kind of futures:
Stream<T> fromFutures<T>(Iterable<Future<T>> futures) {
var pending = 0;
var controller = Controller<T>();
for (var future in futures) {
pending++;
future.then((v) {
controller.add(v);
if (--pending == 0) controller.close();
}, onError: (e, s) {
controller.addError(e, s);
if (--pending == 0) controller.close();
});
}
return controller.stream;
}
You can use this the specify stringGenerator as:
Stream<String> stringGenerator(List<http.Request> requests) async* {
var client = http.Client();
yield* fromFutures(requests.map(client.send));
}

Related

How to respond with a stream in a Sveltekit server load function

Below I try to respond with a stream when I receive ticker updates.
+page.server.js:
import YahooFinanceTicker from "yahoo-finance-ticker";
const ticker = new YahooFinanceTicker();
const tickerListener = await ticker.subscribe(["BTC-USD"])
const stream = new ReadableStream({
start(controller) {
tickerListener.on("ticker", (ticker) => {
console.log(ticker.price);
controller.enqueue(ticker.price);
});
}
});
export async function load() {
return response????
};
Note: The YahooFinanceTicker can't run in the browser.
How to handle / set the response in the Sveltekit load function.
To my knowledge, the load functions cannot be used for this as their responses are JS/JSON serialized. You can use an endpoint in +server to return a Response object which can be constructed from a ReadableStream.
Solution: H.B. comment showed me the right direction to push unsollicited price ticker updates the client.
api route: yahoo-finance-ticker +server.js
import YahooFinanceTicker from "yahoo-finance-ticker";
const ticker = new YahooFinanceTicker();
const tickerListener = await ticker.subscribe(["BTC-USD"])
/** #type {import('./$types').RequestHandler} */
export function GET({ request }) {
const ac = new AbortController();
console.log("GET api: yahoo-finance-ticker")
const stream = new ReadableStream({
start(controller) {
tickerListener.on("ticker", (ticker) => {
console.log(ticker.price);
controller.enqueue(String(ticker.price));
}, { signal: ac.signal });
},
cancel() {
console.log("cancel and abort");
ac.abort();
},
})
return new Response(stream, {
headers: {
'content-type': 'text/event-stream',
}
});
}
page route: +page.svelte
<script>
let result = "";
async function getStream() {
const response = await fetch("/api/yahoo-finance-ticker");
const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
while (true) {
const { value, done } = await reader.read();
console.log("resp", done, value);
if (done) break;
result += `${value}<br>`;
}
}
getStream();
</script>
<section>
<p>{#html result}</p>
</section>

How to use dart shelf_router in web only dart project

From the example given at this place :
https://pub.dev/packages/shelf_router/example
I have written the router part like this :
class Service {
Handler get handler {
final router = Router();
router.get('/say-hi/<name>', (Request request, String name) {
return Response.ok('hi $name');
});
router.get('/user/<userId|[0-9]+>', (Request request, String userId) {
return Response.ok('User has the user-number: $userId');
});
router.get('/wave', (Request request) async {
await Future.delayed(Duration(milliseconds: 100));
return Response.ok('_o/');
});
router.mount('/api/', Api().router);
router.all('/<ignored|.*>', (Request request) {
return Response.notFound('Page not found');
});
return router;
}
}
class Api {
Future<Response> _messages(Request request) async {
return Response.ok('[]');
}
Router get router {
final router = Router();
router.get('/messages', _messages);
router.get('/messages/', _messages);
uter.all('/<ignored|.*>', (Request request) => Response.notFound('null'));
return router;
}
}
and from the main method it tells to use it like this:
void main() async {
final service = Service();
final server = await shelf_io.serve(service.handler, 'localhost', 8080);
print('Server running on localhost:${server.port}');
}
but in web-only project we don't do : final server = await shelf_io.serve(service.handler, 'localhost', 8080);
I was thinking of creating single page application using a shelf router, I saw from the documentation it says that it is compatible with the dart web js platform
My expectation was :
if I write this in web :
router.get('/say-hi/<name>', (Request request, String name) {
return Response.ok('hi $name');
});
then when I will hit /say-hi/ram, then it should have returned "hi ram" in the browser

Dart append to file when i use transform in read text from file

in this simple code i can show all fetched ids when finished reading file and get id from text file, but i want to append this fetched id inside JsonObjectTransformer class, not finished reading file
Future<void> main() async {
final ids = await File('sample.json')
.openRead()
.transform(const Utf8Decoder())
.transform<dynamic>(JsonObjectTransformer())
.map((dynamic json) => json['id'] as String)
.toList();
print(ids); // [#123456, #123456]
}
class JsonObjectTransformer extends StreamTransformerBase<String, dynamic> {
static final _openingBracketChar = '{'.codeUnitAt(0);
static final _closingBracketChar = '}'.codeUnitAt(0);
#override
Stream<dynamic> bind(Stream<String> stream) async* {
final sb = StringBuffer();
var bracketsCount = 0;
await for (final string in stream) {
for (var i = 0; i < string.length; i++) {
final current = string.codeUnitAt(i);
sb.writeCharCode(current);
if (current == _openingBracketChar) {
bracketsCount++;
}
if (current == _closingBracketChar && --bracketsCount == 0) {
yield json.decode(sb.toString());
sb.clear();
}
}
}
/*for example this line*/
//new File('test.txt').writeAsStringSync(sb.toString(), mode: FileMode.APPEND);
}
}
how can i do that?
There are multiple ways to do this but a simple way is to change the JsonObjectTransformer like this:
class JsonObjectTransformer extends StreamTransformerBase<String, dynamic> {
static final _openingBracketChar = '{'.codeUnitAt(0);
static final _closingBracketChar = '}'.codeUnitAt(0);
#override
Stream<dynamic> bind(Stream<String> stream) async* {
final sb = StringBuffer();
var bracketsCount = 0;
final ioSink = File('test.txt').openWrite(mode: FileMode.append);
await for (final string in stream) {
for (var i = 0; i < string.length; i++) {
final current = string.codeUnitAt(i);
sb.writeCharCode(current);
if (current == _openingBracketChar) {
bracketsCount++;
}
if (current == _closingBracketChar && --bracketsCount == 0) {
final dynamic jsonObject = json.decode(sb.toString());
ioSink.writeln(jsonObject['id'] as String);
yield jsonObject;
sb.clear();
}
}
}
await ioSink.flush();
await ioSink.close();
}
}
A more clean solution (since we want some separate of concern) would be to make use of the Stream in your main to write the ID's as each object are parsed. An example how to do that would be:
Future<void> main() async {
final file = File('test.txt').openWrite(mode: FileMode.append);
final ids = <String>[];
await File('sample.json')
.openRead()
.transform(const Utf8Decoder())
.transform<dynamic>(JsonObjectTransformer())
.map((dynamic json) => json['id'] as String)
.forEach((id) {
file.writeln(id);
ids.add(id);
});
await file.flush();
await file.close();
print(ids); // [#123456, #123456]
}

xamarin android receiving one simple sentence/string from webservice

I am trying to receive one simple sentence from a webservice, but I have something wrong.
This is my async task to request from the webservice:
private async Task<string> GetData (string url)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create (new Uri(url));
request.ContentType = "text/plain";
request.Method = "GET";
using (WebResponse response = await request.GetResponseAsync())
{
using (Stream stream = response.GetResponseStream())
{
string doc = await Task.Run(() => stream.ToString());
return doc;
}
}
}
And this is my button:
cmd02.Click += async (sender, e) => {
string sentence = await GetData(url);
txt01.Text = sentence;
};
I get only "System.Net.WebConnectionStream" into my TextView and donĀ“t know which function I should use. Or maybe everthing is wrong?
Maybe somebody has an idea?
public static async Task<string> SendGetRequestAsync (string url) {
string responseString = "";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create (url);
request.Method = WebRequestMethods.Http.Get;
HttpWebResponse response;
await Task.Run (() => {
try {
response = request.GetResponse () as HttpWebResponse;
using (var reader = new StreamReader (response.GetResponseStream ())) {
responseString = reader.ReadToEnd ();
}
} catch (WebException ex) {
Console.WriteLine (ex);
}
});
return responseString;
}

Why use a completer when you can return a async callback function?

Is there any benefit in using completers when you can return an asynchronous callback function(which will return a future).
Example:
Future function() {
return this.socket.request(successCallBack: (response) {
.......
return true;
}); // async call.
against
Future function() {
Completer c = new Completer();
this.socket.request( .. (...){// async callback.
c.complete(xyz);
});
return c.future;
}
Here, The futures return xyz value in both instances. Is it a style preference?
A completer is for more complex scenarios, for example when you want to complete a future in another method than where you create it. In your example the completer is redundant.
class MyComponent {
Completer _initDoneCompleter;
MyComponent() {
Completer _initDoneCompleter = new Completer();
someStream.listen(_eventHandler);
}
void _eventHandler(e) {
if(e.someProperty == 'someValue') {
_initDoneCompleter.complete(e.someProperty);
}
}
static Future<MyComponent> createNew() async {
var c = new MyComponent();
await c.initDoneCompleter.future;
return c;
}
}
void main() async {
var c = await MyComponent.createNew();
}

Resources