This question already has an answer here:
How to wait for forEach to complete with asynchronous callbacks?
(1 answer)
Closed 1 year ago.
Now I am facing a problem, this is my Dart code:
Future<List<Music>?> getAvaliableFmMusics(List<Music>? recommand) async {
final List<Music> resultMusic = List.empty(growable: true);
if (recommand != null) {
recommand.forEach((element) async {
bool isLegacyMusic = await ReddwarfMusic.legacyMusic(element);
if (!isLegacyMusic) {
resultMusic.add(element);
}
});
}
return resultMusic;
}
I get a recommand music list, the next step I check the music list is contains legacy music(user already listening for many times), then add the filtered music to a new list. Finnaly return the filtered music list to the user UI. The problem is that the legacy music will send an async http requst, the resultMusic always return null. What I want to do is waiting the foreach complete then return the filtered resultMusic,what should I do to make it work like that? is it possible to do like this?
To work with forEach and Futures you should use the Future.forEach
Like so:
await Future.forEach(yourList,(e) async {
await asyncCall(e);
});
Related
I have just started using durable functions and needs some advise for how to do fan out pattern correctly. I have a FTP server where from I read all the files. I want to start an Activity function for each file. As I understand it the orchestrator function will be called everytime an Activity function is being executed. I just want to read the files once. To avoid calling the code that read the files and starts the activity functions multiple times, what is the recommended approach? Is it having an activity function that that add's all the activity functions or is it using the IsReplaying property, or something different?
[FunctionName("OrchestrationMoveFilesToBlob")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context)
{
var outputs = new List<string>();
if (!context.IsReplaying)
{
// Do you call your database here and make a call to CallActivityAsync for each row?
}
// doing it here is properly very wrong as it will be called multiple times
var tasks = new Task<string>[7];
for (int i = 0; i < 7; i++)
{
tasks[i] = context.CallActivityAsync<string>("E2_CopyFileToBlob",""); }
await Task.WhenAll(tasks);
return outputs;
}
When looking into the sample in the link below this actually calls it directly in the orchestrator function? Is this not really bad? It continue adding same activities again and again .... ?
https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-cloud-backup
Not sure I understand what you try to achieve but your code looks not bad so far. An orchestration is just called once (and maybe some times more for replay but this is not your problem here). From your orchestration you can call in a fan out all your activity functions (gathering a file from an ftp) each activity function one file. await Task.WhenAll(tasks) is your fan in. (you can use a List<Task> instead of the array and call .Add(task) on it if you want. In order to not edit your code I copied it here and added some comments and questions (feel free to edit here):
[FunctionName("OrchestrationMoveFilesToBlob")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context)
{
var outputs = new List<string>();
if (!context.IsReplaying)
{
// just needed for things that should not happen twice like logging....
}
// if your work isn't a fixed list just call an activity
// which replies with the list of work here (e.g. list of filenames)
var tasks = new Task<string>[7]; // can be a List<Task> too
for (int i = 0; i < 7; i++)
{
tasks[i] = context.CallActivityAsync<string>("E2_CopyFileToBlob","");
}
await Task.WhenAll(tasks);
return outputs; // currently an empty list. What do you want to give back?
}
I suspect that my understanding of Streams in Dart might have a few holes in it...
I have a situation in which I'd like a Dart app to respond to intermittent input (which immediately suggests the use of Streamss -- or Futures, maybe). I can implement the behavior I want with listener functions but I was wondering how to do this in a better, more Dartesque way.
As a simple example, the following (working) program listens to keyboard input from the user and adds a div element to the document containing what has been typed since the previous space, whenever the space bar is hit.
import 'dart:html';
main() {
listenForSpaces(showInput);
}
void listenForSpaces(void Function(String) listener) {
var input = List<String>();
document.onKeyDown.listen((keyboardEvent) {
var key = keyboardEvent.key;
if (key == " ") {
listener(input.join());
input.clear();
} else {
input.add(key.length > 1 ? "[$key]" : key);
}
});
}
void showInput(String message) {
document.body.children.add(DivElement()..text = message);
}
What I'd like to be able to do is to create a new Stream from the Stream that I'm listening to (in the example above, to create a new Stream from onKeyDown). In other words, I might set the program above out as:
var myStream = ...
myStream.listen(showInput);
I suspect that there is a way to create a Stream and then, at different times and places, insert elements to it or call for it to emit a value: it feels as though I am missing something simple. In any case, any help or direction to documentation would be appreciated.
Creating a new stream from an existing stream is fairly easy with an async* function.
For a normal stream, I would just do:
Stream<String> listenForSpaces() async* {
var input = <String>[];
await for (var keyboardEvent in document.onKeyDown) {
var key = keyboardEvent.key;
if (key == " ") {
yield input.join();
input.clear();
} else {
input.add(key.length > 1 ? "[$key]" : key);
}
}
}
The async* function will propagate pauses through to the underlying stream, and it may potentially pause the source during the yield.
That may or may not be what you want, since pausing a DOM event stream can cause you to miss events. For a DOM stream, I'd probably prefer to go with the StreamController based solution above.
There are several methods and there is a whole package rxdart to allow all kinds of things.
Only the final consumer should use listen and only if you need to explicitly want to unsubscribe, otherwise use forEach
If you want to manipulate events like in your example, use map.
I wasn't originally planning to answer my own question but I have since found a very simple answer to this question in the dartlang creating streams article; in case it's helpful to others:
Specifically, if we'd like to create a stream that we can insert elements into at arbitrary times and places in the code, we can do so via the StreamController class. Instances of this class have an add method; we can simply use the instance's stream property as our stream.
As an example, the code in my question could be rewritten as:
import 'dart:html';
import 'dart:async';
main() async {
// The desired implementation stated in the question:
var myStream = listenForSpaces();
myStream.listen(showInput);
}
Stream<String> listenForSpaces() {
// Use the StreamController class.
var controller = StreamController<String>();
var input = List<String>();
document.onKeyDown.listen((keyboardEvent) {
var key = keyboardEvent.key;
if (key == " ") {
// Add items to the controller's stream.
controller.add(input.join());
input.clear();
} else {
input.add(key.length > 1 ? "[$key]" : key);
}
});
// Listen to the controller's stream.
return controller.stream;
}
void showInput(String message) {
document.body.children.add(DivElement()..text = message);
}
(As mentioned in the article, we need to be careful if we want to set up a stream from scratch like this because there is nothing to stop us from inserting items to streams that don't have associated, active subscribers; inserted items would in that case be buffered, which could result in a memory leak.)
Scenario
If I want to read from a file and store the data in a Map, and if that map is being used multiple times for validation.
Is it possible for me to do this without having to change the return type of all methods, that use the above mentioned map, to Future?
Example:
Map metadata = null
Future readFromFile async {
.... metadata = await File.readingfromFile(...);
}
Future getRegion(..) async {
if(metadata == null) { await readFromFile() }
return metadata["region"]
}
Using the above code if a method(like isValidRegion,etc) that uses and needs getRegion(..) to complete, then the return type of isValidRegion should be converted to Future.
Future<bool> isValidRegion(..) async {
return ((await getRegionData(...)) != null )
}
If that isValidRegion is present within another methods, then the return type of them have to be changed to Future as well.
Future<String> parse(...) async {
....
if(await isValidRegion()) {
...
}
...
}
What is an elegant way to avoid this chain of futures as return types?
Async execution is contagious, there is nothing you can do to get back from async to sync execution.
What you can do is to do the read from the file synchronous to avoid the problem in the first place (if this is possible, if you read it from a network connection, this might not be possible).
I'm using the Lawndart library to access browser data, and want to collect the results of a set of queries. Here's what I thought should work:
numberOfRecordsPerSection(callback) {
var map = new Map();
db_sections.keys().forEach((_key) {
db_sections.getByKey(_key).then((Map _section) {
int count = _section.length;
map[_key] = count;
});
}).then(callback(map));
}
However, when the callback is called, map is still empty (it gets populated correctly, but later, after all the Futures have completed). I assume the problem is that the Futures created by the getByKey() calls are not "captured by" the Futures created by the forEach() calls.
How can I correct my code to capture the result correctly?
the code from How do I do this jquery pattern in dart? looks very similar to yours
For each entry of _db.keys() a future is added to an array and then waited for all of them being finished by Future.wait()
Not sure if this code works (see comments on the answer on the linked question)
void fnA() {
fnB().then((_) {
// Here, all keys should have been loaded
});
}
Future fnB() {
return _db.open().then((_) {
List<Future> futures = [];
return _db.keys().forEach((String key_name) {
futures.add(_db.getByKey(key_name).then((String data) {
// do something with data
return data;
}));
}).then((_) => Future.wait(futures));
});
}
I am trying to write a multi threading search and then display all the results once the tasks have finished running but currently I don't understand how to process the results once all the tasks are complete
My code is as follows:
private async void DoSearchAsync()
{
var productResults = await SearchProductsAsync(CoreCache.AllProducts);
var brochureResults = await SearchBrochuresAsync(CoreCache.AllBrochures);
_searchResults.AddRange(productResults);
_searchResults.AddRange(brochureResults);
ResultsCount = _searchResults.Count;
}
Where _searchResults is a List<SearchResult>
My understanding is that it will do both of the awaits simultaneously and then add the products to the search results. However when I call this in my controller:
public ActionResult Index(string searchText)
{
SearchHelper helper = new SearchHelper(searchText);
helper.DoSearchAsync();
return View(helper);
}
It is displaying the page before the searches are complete so no results are showing up. How do I make it wait for the results to finish before showing the page?
I've had a look into Tasks.Wait but don't know how to apply it to the above as it expects an array of tasks
private Task<List<SearchResult>> SearchProductsAsync(IEnumerable<Product> products)
{
return Task<List<SearchResult>>.Factory.StartNew(() => GetProducts(products));
}
private Task<List<SearchResult>> SearchBrochuresAsync(IEnumerable<Assets> brochures)
{
return Task<List<SearchResult>>.Factory.StartNew(() => GetBrochures(brochures));
}
Every time you call Factory.StartNew or Task.Run inside an ASP.NET controller, you grab a thread from ThreadPool. That thread could be otherwise serving another incoming HTTP request. So, your're really hurting your web app scalability. This may be a serious issue, depending on how many concurrent HTTP requests your web app is expected to receive.
Are you OK with that? If so, the code could look like this:
private async Task DoSearchAsync()
{
var productResults = SearchProductsAsync(CoreCache.AllProducts);
var brochureResults = SearchBrochuresAsync(CoreCache.AllBrochures);
await Task.WhenAll(productResults, brochureResults);
_searchResults.AddRange(productResults.Result);
_searchResults.AddRange(brochureResultsbrochure.Results);
ResultsCount = _searchResults.Count;
}
public async Task<ActionResult> Index(string searchText)
{
SearchHelper helper = new SearchHelper(searchText);
await helper.DoSearchAsync();
return View(helper);
}
Note I changed async void to async Task for DoSearchAsync, and made your controller method async, so it returns Task<ActionResult>.
My understanding is that it will do both of the awaits simultaneously
This is not correct Pete. The await means that it will pause execution of the current method until such time that the Async method being called completes. So your 2 searches will not run in Parallel.
For clarification, see the first line of the MSDN documentation....
The await operator is applied to a task in an asynchronous method to suspend the execution of the method until the awaited task completes.
You should use Task.WhenAll to wait for both searches to complete. As WhenAll supports returning Task<T> and both of your Async searches return a List<SearchResult> you can combine the results of both searches and await them both in one statement...
var searchProducts = Task.Factory.StartNew(() => GetProducts(CoreCache.AllProducts));
var searchBrochure = Task.Factory.StartNew(() => GetBrochures(CoreCache.AllBrochures));
var allResults = await Task.WhenAll(new [] { searchProducts, searchBrochure });
//allResults is List<SearchResult>[]
For completeness, its worth noting that allResults will contain 2 result sets. It does not perform a union on the 2 sets.
A complete working example for LinqPad can be found here on GitHub
You can use a task factory, then call wait on the resulting task:
Task taskA = Task.Factory.StartNew(() => DoSomeWork(10000000));
taskA.Wait();
Console.WriteLine("taskA has completed.");
http://msdn.microsoft.com/en-us/library/dd537610(v=vs.110).aspx
You can make it wait for the results to finish before displaying the page by making your function NOT be async.
Or if you want async you could move the search functions to be called by an AJAX call triggered when your page is loaded
You need to wait for the function to complete. In order to do that, first make it return a task:
private async Task DoSearchAsync()
then await it:
public async Task<ActionResult> Index(string searchText)
{
SearchHelper helper = new SearchHelper(searchText);
await helper.DoSearchAsync();
return View(helper);
}
For more information, see http://www.asp.net/mvc/tutorials/mvc-4/using-asynchronous-methods-in-aspnet-mvc-4