I want to stop/sleep executing to simulate long time process, unfortunately I can't find information about it. I've read the following topic (How can I "sleep" a Dart program), but it isn't what I look for.
For example sleep() function from dart:io packages isn't applicable, because this package is not available in a browser.
For example:
import 'dart:html';
main() {
// I want to "sleep"/hang executing during several seconds
// and only then run the rest of function's body
querySelect('#loading').remove();
...other functions and actions...
}
I know that there is Timer class to make callbacks after some time, but still it doesn't prevent the execution of program as a whole.
There is no way to stop execution. You can either use a Timer, Future.delayed, or just use an endless loop which only ends after certain time has passed.
If you want a stop the world sleeping function, you could do it entirely yourself. I will mention that I don't recommend you do this, it's a very bad idea to stop the world, but if you really want it:
void sleep(Duration duration) {
var ms = duration.inMilliseconds;
var start = new DateTime.now().millisecondsSinceEpoch;
while (true) {
var current = new DateTime.now().millisecondsSinceEpoch;
if (current - start >= ms) {
break;
}
}
}
void main() {
print("Begin.");
sleep(new Duration(seconds: 2));
print("End.");
}
Related
I want to set up a queue of functions in Darts. Queuing should be asynchronous, allowing multiple functions to run concurrently. However, a maximum of three functions should be executed simultaneously. How can I achieve this?
I already tied working off a list but i am struggeling at adding a limit on same time running functions
List<String> queue = new List();
main(){
queue.add("...");
queue.add("...");
queue.add("...");
for(String q in queue){
await crawl(q);
}
}
crawl(String) async{
...
}
I would use a queue:
import "dart:collection";
final queue = Queue<String>();
main() {
queue
..add("...")
..add("...")
..add("...");
while (queue.isNotEmpty) {
await crawl(queue.removeFirst());
}
}
crawl(String x) async {
.... queue.add(...) ...
}
This should work. It will not do concurrent crawling because await each operation. If you want concurrent crawling, I recommend being a little more clever. Look for worker pools or similar structures to ensure that you only have a certain number of operations running at the same time.
I have an Apache Beam pipeline running on Google Dataflow whose job is rather simple:
It reads individual JSON objects from Pub/Sub
Parses them
And sends them via HTTP to some API
This API requires me to send the items in batches of 75. So I built a DoFn that accumulates events in a list and publish them via this API once they I get 75. This results to be too slow, so I thought instead of executing those HTTP requests in different threads using a thread pool.
The implementation of what I have right now looks like this:
private class WriteFn : DoFn<TheEvent, Void>() {
#Transient var api: TheApi
#Transient var currentBatch: MutableList<TheEvent>
#Transient var executor: ExecutorService
#Setup
fun setup() {
api = buildApi()
executor = Executors.newCachedThreadPool()
}
#StartBundle
fun startBundle() {
currentBatch = mutableListOf()
}
#ProcessElement
fun processElement(processContext: ProcessContext) {
val record = processContext.element()
currentBatch.add(record)
if (currentBatch.size >= 75) {
flush()
}
}
private fun flush() {
val payloadTrack = currentBatch.toList()
executor.submit {
api.sendToApi(payloadTrack)
}
currentBatch.clear()
}
#FinishBundle
fun finishBundle() {
if (currentBatch.isNotEmpty()) {
flush()
}
}
#Teardown
fun teardown() {
executor.shutdown()
executor.awaitTermination(30, TimeUnit.SECONDS)
}
}
This seems to work "fine" in the sense that data is making it to the API. But I don't know if this is the right approach and I have the sense that this is very slow.
The reason I think it's slow is that when load testing (by sending a few million events to Pub/Sub), it takes it up to 8 times more time for the pipeline to forward those messages to the API (which has response times of under 8ms) than for my laptop to feed them into Pub/Sub.
Is there any problem with my implementation? Is this the way I should be doing this?
Also... am I required to wait for all the requests to finish in my #FinishBundle method (i.e. by getting the futures returned by the executor and waiting on them)?
You have two interrelated questions here:
Are you doing this right / do you need to change anything?
Do you need to wait in #FinishBundle?
The second answer: yes. But actually you need to flush more thoroughly, as will become clear.
Once your #FinishBundle method succeeds, a Beam runner will assume the bundle has completed successfully. But your #FinishBundle only sends the requests - it does not ensure they have succeeded. So you could lose data that way if the requests subsequently fail. Your #FinishBundle method should actually be blocking and waiting for confirmation of success from the TheApi. Incidentally, all of the above should be idempotent, since after finishing the bundle, an earthquake could strike and cause a retry ;-)
So to answer the first question: should you change anything? Just the above. The practice of batching requests this way can work as long as you are sure the results are committed before the bundle is committed.
You may find that doing so will cause your pipeline to slow down, because #FinishBundle happens more frequently than #Setup. To batch up requests across bundles you need to use the lower-level features of state and timers. I wrote up a contrived version of your use case at https://beam.apache.org/blog/2017/08/28/timely-processing.html. I would be quite interested in how this works for you.
It may simply be that the extremely low latency you are expecting, in the low millisecond range, is not available when there is a durable shuffle in your pipeline.
I have been looking a lot for this but haven't found any packages or a way to schedule background tasks in Flutter. Like in Android there is WorkManager,AlarmManager.
I know I can access those classes using MethodChannel, but I want something that works for iOS and Android both.
(Its very disappointing that a mobile framework doesn't have the ability to schedule background tasks).
There is a Medium blogpost that explains how to do this.
However we thought it was way too complicated to set up so it just happens we created a plugin that aids you with this.
//Provide a top level function or static function.
//This function will be called by Android and will return the value you provided when you registered the task.
//See below
void callbackDispatcher() {
Workmanager.executeTask((task) {
print("Native echoed: $task");
return Future.value(true);
});
}
Workmanager.initialize(
callbackDispatcher, //the top level function.
isInDebugMode: true //If enabled it will post a notification whenever the job is running. Handy for debugging jobs
)
We support Android's Workmanager and iOS performFetch
For now it only works for Android project, but we are looking at iOS soon.
I'll update this answer when it is available.
We have iOS support now too. It is still early alpha, but give a go.
We wrote a complimentary Medium post too.
SOLUTION 1: android_alarm_manager_plus is the best solution to schedule background tasks. But the only disadvantage is only support Android.
Note: if you are using android_alarm_manager version, please migrate to the Plus version :D
Let's start:
After importing this plugin to your project as usual, add the following to your AndroidManifest.xml within the tags:
<uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED"/>
<uses-permission android:name="android.permission.WAKE_LOCK"/>
Next, within the tags, add:
<service
android:name="io.flutter.plugins.androidalarmmanager.AlarmService"
android:permission="android.permission.BIND_JOB_SERVICE"
android:exported="false"/>
<receiver
android:name="io.flutter.plugins.androidalarmmanager.AlarmBroadcastReceiver"
android:exported="false"/>
<receiver
android:name="io.flutter.plugins.androidalarmmanager.RebootBroadcastReceiver"
android:enabled="false">
<intent-filter>
<action android:name="android.intent.action.BOOT_COMPLETED"></action>
</intent-filter>
</receiver>
Then in Dart code add:
import 'package:android_alarm_manager/android_alarm_manager.dart';
void printHello() {
final DateTime now = DateTime.now();
final int isolateId = Isolate.current.hashCode;
print("[$now] Hello, world! isolate=${isolateId} function='$printHello'");
}
main() async {
final int helloAlarmID = 0;
await AndroidAlarmManager.initialize();
runApp(...);
await AndroidAlarmManager.periodic(const Duration(minutes: 1), helloAlarmID, printHello);
}
If you want to schedule any task every day at a specific time, you need to do something like this:
if (Platform.isAndroid) {
await AndroidAlarmManager.periodic(
const Duration(hours: 24), //Do the same every 24 hours
helloAlarmID, //Different ID for each alarm
printHello,
wakeup: true, //the device will be woken up when the alarm fires
startAt: DateTime(DateTime.now().year, DateTime.now().month, DateTime.now().day, 5, 0), //Start whit the specific time 5:00 am
rescheduleOnReboot: true, //Work after reboot
);
}
If alarm callbacks will need access to other Flutter plugins, including the alarm manager plugin itself, it may be necessary to inform the background service how to initialize plugins depending on which Flutter Android embedding the application is using.
This is done by giving the AlarmService a callback to call the application's onCreate method.
In particular, its Application class is as follows:
public class Application extends FlutterApplication implements PluginRegistrantCallback {
#Override
public void onCreate() {
super.onCreate();
AlarmService.setPluginRegistrant(this);
}
#Override
public void registerWith(PluginRegistry registry) {
//add AndroidAlarmManagerPlugin plugin register if you work with arlarm
AndroidAlarmManagerPlugin.registerWith(registry.registrarFor("io.flutter.plugins.androidalarmmanager.AndroidAlarmManagerPlugin"));
}
}
Which must be reflected in the application's AndroidManifest.xml. E.g.:
<application
android:name=".Application"
...
SOLUTION 2:
Cron is another best solution to schedule background tasks. Cron run tasks periodically at fixed times, dates or intervals. But the onliy disadvantage of corn is once the app kill, cron not working in background as expected.
A simple usage example:
import 'package:cron/cron.dart';
main() {
var cron = new Cron();
cron.schedule(new Schedule.parse('*/3 * * * *'), () async {
print('every three minutes');
});
cron.schedule(new Schedule.parse('8-11 * * * *'), () async {
print('between every 8 and 11 minutes');
});
}
How to setup a cronjob in general: information
Test cronjob: crontab
Use WorkManager lib
dependencies:
workmanager: ^0.2.3
WorkManager comes under two parts, which run the task in the background.
1. Delayed background work
registerOneOffTask runs the task only once with an initial delay of 10 seconds. This is useful when we need to perform any background work only once.
Example:
Workmanager.registerOneOffTask(
"1",
"registerOneOffTask",
initialDelay: Duration(seconds: 10),
);
2. Periodic background work
This task runs periodically, Since we have not provided a frequency it will be the default 15 minutes.
Example:
Workmanager.registerPeriodicTask(
"2",
"registerPeriodicTask",
initialDelay: Duration(seconds: 10),
);
For a detailed explanation follow a blog:
https://medium.com/flutterworld/flutter-perform-background-job-9bce92f0a21e
I want to test a function that calls exit.
Basicly, I have a console application, that asks the user if he is sure that he wants a directory to be overwritten. When the users answers "No", the directory won't be overwritten, and the program should exit.
promptToDeleteRepo() {
bool okToDelete = ...
if(okToDelete) {
deleteRepo();
} else {
exit(0);
}
}
So I want to test that if the user answers "No", that the program really exits. But if I test this, my test runner exits.
In python I seem to be able to do something like:
with pytest.raises(SystemExit):
promptToDeleteRepo();
Is there something like this in Dart?
You can inject a custom exit function during the tests.
import 'dart:io' as io;
typedef ExitFn(int code);
ExitFn exit = io.exit;
promptToDeleteRepo() {
bool okToDelete = ...
if(okToDelete) {
deleteRepo();
} else {
exit(0);
}
}
and in your test :
int exitCodeUsed;
mylib.exit = (int code) {exitCodeUsed = code};
mylib.promptToDeleteRepo();
A better solution whould have to use zones but there doesn't seem to be possible to handle exit. It could be worth to file an issue.
One option that comes to my mind is to run the code you want to test in a new process Process.run() or Process.start() and check the exit code at the end. You can use stdin/stdout to communicate with the process (send keyboard input, read output)
I'm learning Dart's Future, and have read some articles about the Future.
It says Dart is single-thread, and we can use Future to make some expensive functions run later, e.g. reading files.
Suppose reading a file will cost 10 seconds, and I have 3 files to read.
My dart code:
main() {
readFile("aaa.txt");
readFile("bbb.txt");
readFile("ccc.txt");
print("Will print the content of the files later");
}
readFile(String filename) {
File file = new File(filename);
file.readAsString().then((content) {
print("File content:\n");
print(content);
});
}
Since reading a file will cost 10 seconds, so the above code will cost at least 30 seconds, right? Using futures to read files just to make the expensive tasks run later one by one, without blocking current code, but won't reduce the total cost?
If in java, I can make a thread pool, and make 3 future tasks running in parallel, the total cost will be between 10 and 20 seconds.
Is it possible to do the same in Dart? Is using Dart's isolate the only solution?
I would expect that this could take 10 seconds, as it will start three reads, each of which will queue an callback to the "then" function when the read is complete. It is entirely possible that the three files will load in parallel and all complete after 10 seconds. The callbacks will be called on the main thread sequentially though.
Although the user code in dart is single threaded (assuming you don't use isolates or web workers), nothing says that the implementation can't create threads or use the operating system's asynchronous loading to perform tasks in parallel as long as the future's run sequentially in the main thread.
That's correct. If you start an new async path with new Timer(), new Future(), or scheduleMicrotask() it will be scheduled for later execution.
When one of your async paths is waiting for a network request or the file system returning data, another async path may jump in and run in the meantime. So you might get a runtime less than 30 seconds, but you can't reduce runtime by adding a CPU.
I have to admit, that I don't know details about when scheduling takes place and how it works exactly.
Dart has no threads, so if you want to run code in parallel you need isolates.
Almost 30 seconds.
I had just run the code with dart 2.16.2, and the result is almost 30 seconds.
here is my code:
import 'dart:async';
import 'dart:convert';
import 'dart:io';
import 'dart:isolate';
main() async {
print('main start');
printCurrentTime("main before all future");
Future(() => readFile(0));
Future(() => readFile(1));
Future(() => readFile(2));
Future(() {
printCurrentTime("future last");
});
print('main end');
printCurrentTime("main");
}
printCurrentTime(String name) {
print("$name ${DateTime.now().millisecondsSinceEpoch}");
}
readFile(number) {
print("start read file $number");
var watch = Stopwatch();
watch.start();
var filename = r"path/to/file";
File file = File(filename);
file.readAsBytes().then((content) {
printCurrentTime("\nfuture#$number start");
print("File $number content:");
print(content.toString().length);
printCurrentTime("future#$number finish");
print("finish read file $number");
});
}
And here is the result:
main start
main before all future 1652964314276
main end
main 1652964314278
// all the event queue start to run
start read file 0
start read file 1
start read file 2
future last 1652964314290
// the dart system read file parallelly, after finish read file
// they put the future to the event queue, and dart start running all
// those event task one by one:
future#0 start 1652964314343
File 0 content:
241398625
future#0 finish 1652964317457
finish read file 0
future#1 start 1652964317457
File 1 content:
241398625
future#1 finish 1652964320470
finish read file 1
future#2 start 1652964320471
File 2 content:
241398625
future#2 finish 1652964323403
finish read file 2
As we can see:
file.readAsBytes() take about 53ms (or 100ms sometime during my test)
content.toString() take about 3s or more
So we can come to this conclusion:
file.readAsBytes() all run in the other thread parallelly, and the value the return Future<Uint8List> is added to the Event Task dequeue which is run synchronously, that's why we can see the future#1 start... print one by one.