I have been looking a lot for this but haven't found any packages or a way to schedule background tasks in Flutter. Like in Android there is WorkManager,AlarmManager.
I know I can access those classes using MethodChannel, but I want something that works for iOS and Android both.
(Its very disappointing that a mobile framework doesn't have the ability to schedule background tasks).
There is a Medium blogpost that explains how to do this.
However we thought it was way too complicated to set up so it just happens we created a plugin that aids you with this.
//Provide a top level function or static function.
//This function will be called by Android and will return the value you provided when you registered the task.
//See below
void callbackDispatcher() {
Workmanager.executeTask((task) {
print("Native echoed: $task");
return Future.value(true);
});
}
Workmanager.initialize(
callbackDispatcher, //the top level function.
isInDebugMode: true //If enabled it will post a notification whenever the job is running. Handy for debugging jobs
)
We support Android's Workmanager and iOS performFetch
For now it only works for Android project, but we are looking at iOS soon.
I'll update this answer when it is available.
We have iOS support now too. It is still early alpha, but give a go.
We wrote a complimentary Medium post too.
SOLUTION 1: android_alarm_manager_plus is the best solution to schedule background tasks. But the only disadvantage is only support Android.
Note: if you are using android_alarm_manager version, please migrate to the Plus version :D
Let's start:
After importing this plugin to your project as usual, add the following to your AndroidManifest.xml within the tags:
<uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED"/>
<uses-permission android:name="android.permission.WAKE_LOCK"/>
Next, within the tags, add:
<service
android:name="io.flutter.plugins.androidalarmmanager.AlarmService"
android:permission="android.permission.BIND_JOB_SERVICE"
android:exported="false"/>
<receiver
android:name="io.flutter.plugins.androidalarmmanager.AlarmBroadcastReceiver"
android:exported="false"/>
<receiver
android:name="io.flutter.plugins.androidalarmmanager.RebootBroadcastReceiver"
android:enabled="false">
<intent-filter>
<action android:name="android.intent.action.BOOT_COMPLETED"></action>
</intent-filter>
</receiver>
Then in Dart code add:
import 'package:android_alarm_manager/android_alarm_manager.dart';
void printHello() {
final DateTime now = DateTime.now();
final int isolateId = Isolate.current.hashCode;
print("[$now] Hello, world! isolate=${isolateId} function='$printHello'");
}
main() async {
final int helloAlarmID = 0;
await AndroidAlarmManager.initialize();
runApp(...);
await AndroidAlarmManager.periodic(const Duration(minutes: 1), helloAlarmID, printHello);
}
If you want to schedule any task every day at a specific time, you need to do something like this:
if (Platform.isAndroid) {
await AndroidAlarmManager.periodic(
const Duration(hours: 24), //Do the same every 24 hours
helloAlarmID, //Different ID for each alarm
printHello,
wakeup: true, //the device will be woken up when the alarm fires
startAt: DateTime(DateTime.now().year, DateTime.now().month, DateTime.now().day, 5, 0), //Start whit the specific time 5:00 am
rescheduleOnReboot: true, //Work after reboot
);
}
If alarm callbacks will need access to other Flutter plugins, including the alarm manager plugin itself, it may be necessary to inform the background service how to initialize plugins depending on which Flutter Android embedding the application is using.
This is done by giving the AlarmService a callback to call the application's onCreate method.
In particular, its Application class is as follows:
public class Application extends FlutterApplication implements PluginRegistrantCallback {
#Override
public void onCreate() {
super.onCreate();
AlarmService.setPluginRegistrant(this);
}
#Override
public void registerWith(PluginRegistry registry) {
//add AndroidAlarmManagerPlugin plugin register if you work with arlarm
AndroidAlarmManagerPlugin.registerWith(registry.registrarFor("io.flutter.plugins.androidalarmmanager.AndroidAlarmManagerPlugin"));
}
}
Which must be reflected in the application's AndroidManifest.xml. E.g.:
<application
android:name=".Application"
...
SOLUTION 2:
Cron is another best solution to schedule background tasks. Cron run tasks periodically at fixed times, dates or intervals. But the onliy disadvantage of corn is once the app kill, cron not working in background as expected.
A simple usage example:
import 'package:cron/cron.dart';
main() {
var cron = new Cron();
cron.schedule(new Schedule.parse('*/3 * * * *'), () async {
print('every three minutes');
});
cron.schedule(new Schedule.parse('8-11 * * * *'), () async {
print('between every 8 and 11 minutes');
});
}
How to setup a cronjob in general: information
Test cronjob: crontab
Use WorkManager lib
dependencies:
workmanager: ^0.2.3
WorkManager comes under two parts, which run the task in the background.
1. Delayed background work
registerOneOffTask runs the task only once with an initial delay of 10 seconds. This is useful when we need to perform any background work only once.
Example:
Workmanager.registerOneOffTask(
"1",
"registerOneOffTask",
initialDelay: Duration(seconds: 10),
);
2. Periodic background work
This task runs periodically, Since we have not provided a frequency it will be the default 15 minutes.
Example:
Workmanager.registerPeriodicTask(
"2",
"registerPeriodicTask",
initialDelay: Duration(seconds: 10),
);
For a detailed explanation follow a blog:
https://medium.com/flutterworld/flutter-perform-background-job-9bce92f0a21e
Related
I have an Apache Beam pipeline running on Google Dataflow whose job is rather simple:
It reads individual JSON objects from Pub/Sub
Parses them
And sends them via HTTP to some API
This API requires me to send the items in batches of 75. So I built a DoFn that accumulates events in a list and publish them via this API once they I get 75. This results to be too slow, so I thought instead of executing those HTTP requests in different threads using a thread pool.
The implementation of what I have right now looks like this:
private class WriteFn : DoFn<TheEvent, Void>() {
#Transient var api: TheApi
#Transient var currentBatch: MutableList<TheEvent>
#Transient var executor: ExecutorService
#Setup
fun setup() {
api = buildApi()
executor = Executors.newCachedThreadPool()
}
#StartBundle
fun startBundle() {
currentBatch = mutableListOf()
}
#ProcessElement
fun processElement(processContext: ProcessContext) {
val record = processContext.element()
currentBatch.add(record)
if (currentBatch.size >= 75) {
flush()
}
}
private fun flush() {
val payloadTrack = currentBatch.toList()
executor.submit {
api.sendToApi(payloadTrack)
}
currentBatch.clear()
}
#FinishBundle
fun finishBundle() {
if (currentBatch.isNotEmpty()) {
flush()
}
}
#Teardown
fun teardown() {
executor.shutdown()
executor.awaitTermination(30, TimeUnit.SECONDS)
}
}
This seems to work "fine" in the sense that data is making it to the API. But I don't know if this is the right approach and I have the sense that this is very slow.
The reason I think it's slow is that when load testing (by sending a few million events to Pub/Sub), it takes it up to 8 times more time for the pipeline to forward those messages to the API (which has response times of under 8ms) than for my laptop to feed them into Pub/Sub.
Is there any problem with my implementation? Is this the way I should be doing this?
Also... am I required to wait for all the requests to finish in my #FinishBundle method (i.e. by getting the futures returned by the executor and waiting on them)?
You have two interrelated questions here:
Are you doing this right / do you need to change anything?
Do you need to wait in #FinishBundle?
The second answer: yes. But actually you need to flush more thoroughly, as will become clear.
Once your #FinishBundle method succeeds, a Beam runner will assume the bundle has completed successfully. But your #FinishBundle only sends the requests - it does not ensure they have succeeded. So you could lose data that way if the requests subsequently fail. Your #FinishBundle method should actually be blocking and waiting for confirmation of success from the TheApi. Incidentally, all of the above should be idempotent, since after finishing the bundle, an earthquake could strike and cause a retry ;-)
So to answer the first question: should you change anything? Just the above. The practice of batching requests this way can work as long as you are sure the results are committed before the bundle is committed.
You may find that doing so will cause your pipeline to slow down, because #FinishBundle happens more frequently than #Setup. To batch up requests across bundles you need to use the lower-level features of state and timers. I wrote up a contrived version of your use case at https://beam.apache.org/blog/2017/08/28/timely-processing.html. I would be quite interested in how this works for you.
It may simply be that the extremely low latency you are expecting, in the low millisecond range, is not available when there is a durable shuffle in your pipeline.
I'm trying with Apache Beam 2.1.0 to consume simple data (key,value) from google PubSub and group by key to be able to treat batches of data.
With default trigger my code after "GroupByKey" never fires (I waited 30min).
If I defined custom trigger, code is executed but I would like to understand why default trigger is never fired. I tried to define my own timestamp with "withTimestampLabel" but same issue. I tried to change duration of windows but same issue too (1second, 10seconds, 30seconds etc).
I used command line for this test to insert data
gcloud beta pubsub topics publish test A,1
gcloud beta pubsub topics publish test A,2
gcloud beta pubsub topics publish test B,1
gcloud beta pubsub topics publish test B,2
From documentation it says that we can do one or the other but not necessarily both
If you are using unbounded PCollections, you must use either
non-global windowing OR an aggregation trigger in order to perform a
GroupByKey or CoGroupByKey
It looks to be similar to
Consuming unbounded data in windows with default trigger
Scio: groupByKey doesn't work when using Pub/Sub as collection source
My code
static class Compute extends DoFn<KV<String, Iterable<Integer>>, Void> {
#ProcessElement
public void processElement(ProcessContext c) {
// Code never fires
System.out.println("KEY:" + c.element().getKey());
System.out.println("NB:" + c.element().getValue().spliterator().getExactSizeIfKnown());
}
}
public static void main(String[] args) {
Pipeline p = Pipeline.create(PipelineOptionsFactory.create());
p.apply(PubsubIO.readStrings().fromSubscription("projects/" + args[0] + "/subscriptions/test"))
.apply(Window.into(FixedWindows.of(Duration.standardMinutes(1))))
.apply(
MapElements
.into(TypeDescriptors.kvs(TypeDescriptors.strings(), TypeDescriptors.integers()))
.via((String row) -> {
String[] parts = row.split(",");
System.out.println(Arrays.toString(parts)); // Code fires
return KV.of(parts[0], Integer.parseInt(parts[1]));
})
)
.apply(GroupByKey.create())
.apply(ParDo.of(new Compute()));
p.run();
}
I have a Pub/Sub topic + subscription and want to consume and aggregate the unbounded data from the subscription in a Dataflow. I use a fixed window and write the aggregates to BigQuery.
Reading and writing (without windowing and aggregation) works fine. But when I pipe the data into a fixed window (to count the elements in each window) the window is never triggered. And thus the aggregates are not written.
Here is my word publisher (it uses kinglear.txt from the examples as input file):
public static class AddCurrentTimestampFn extends DoFn<String, String> {
#ProcessElement public void processElement(ProcessContext c) {
c.outputWithTimestamp(c.element(), new Instant(System.currentTimeMillis()));
}
}
public static class ExtractWordsFn extends DoFn<String, String> {
#ProcessElement public void processElement(ProcessContext c) {
String[] words = c.element().split("[^a-zA-Z']+");
for (String word:words){ if(!word.isEmpty()){ c.output(word); }}
}
}
// main:
Pipeline p = Pipeline.create(o); // 'o' are the pipeline options
p.apply("ReadLines", TextIO.Read.from(o.getInputFile()))
.apply("Lines2Words", ParDo.of(new ExtractWordsFn()))
.apply("AddTimestampFn", ParDo.of(new AddCurrentTimestampFn()))
.apply("WriteTopic", PubsubIO.Write.topic(o.getTopic()));
p.run();
Here is my windowed word counter:
Pipeline p = Pipeline.create(o); // 'o' are the pipeline options
BigQueryIO.Write.Bound tablePipe = BigQueryIO.Write.to(o.getTable(o))
.withSchema(o.getSchema())
.withCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_IF_NEEDED)
.withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND);
Window.Bound<String> w = Window
.<String>into(FixedWindows.of(Duration.standardSeconds(1)));
p.apply("ReadTopic", PubsubIO.Read.subscription(o.getSubscription()))
.apply("FixedWindow", w)
.apply("CountWords", Count.<String>perElement())
.apply("CreateRows", ParDo.of(new WordCountToRowFn()))
.apply("WriteRows", tablePipe);
p.run();
The above subscriber will not work, since the window does not seem to trigger using the default trigger. However, if I manually define a trigger the code works and the counts are written to BigQuery.
Window.Bound<String> w = Window.<String>into(FixedWindows.of(Duration.standardSeconds(1)))
.triggering(AfterProcessingTime
.pastFirstElementInPane()
.plusDelayOf(Duration.standardSeconds(1)))
.withAllowedLateness(Duration.ZERO)
.discardingFiredPanes();
I like to avoid specifying custom triggers if possible.
Questions:
Why does my solution not work with Dataflow's default trigger?
How do I have to change my publisher or subscriber to trigger windows using the default trigger?
How are you determining the trigger never fires?
Your PubSubIO.Write and PubSubIO.Read transforms should both specify a timestamp label using withTimestampLabel, otherwise the timestamps you've added will not be written to PubSub and the publish times will be used.
Either way, the input watermark of the pipeline will be derived from the timestamps of the elements waiting in PubSub. Once all inputs have been processed, it will stay back for a few minutes (in case there was a delay in the publisher) before advancing to real time.
What you are likely seeing is that all the elements are published in the same ~1 second window (since the input file is pretty small). These are all read and processed relatively quickly, but the 1-second window they are put in will not trigger until after the input watermark has advanced, indicating that all data in that 1-second window has been consumed.
This won't happen until several minutes, which may make it look like the trigger isn't working. The trigger you wrote fired after 1 second of processing time, which would fire much earlier, but there is no guarantee all the data has been processed.
Steps to get better behavior from the default trigger:
Use withTimestampLabel on both the write and read pubsub steps.
Have the publisher spread the timestamps out further (eg., run for several minutes and spread the timestamps out across that range)
I want to stop/sleep executing to simulate long time process, unfortunately I can't find information about it. I've read the following topic (How can I "sleep" a Dart program), but it isn't what I look for.
For example sleep() function from dart:io packages isn't applicable, because this package is not available in a browser.
For example:
import 'dart:html';
main() {
// I want to "sleep"/hang executing during several seconds
// and only then run the rest of function's body
querySelect('#loading').remove();
...other functions and actions...
}
I know that there is Timer class to make callbacks after some time, but still it doesn't prevent the execution of program as a whole.
There is no way to stop execution. You can either use a Timer, Future.delayed, or just use an endless loop which only ends after certain time has passed.
If you want a stop the world sleeping function, you could do it entirely yourself. I will mention that I don't recommend you do this, it's a very bad idea to stop the world, but if you really want it:
void sleep(Duration duration) {
var ms = duration.inMilliseconds;
var start = new DateTime.now().millisecondsSinceEpoch;
while (true) {
var current = new DateTime.now().millisecondsSinceEpoch;
if (current - start >= ms) {
break;
}
}
}
void main() {
print("Begin.");
sleep(new Duration(seconds: 2));
print("End.");
}
I'm learning Dart's Future, and have read some articles about the Future.
It says Dart is single-thread, and we can use Future to make some expensive functions run later, e.g. reading files.
Suppose reading a file will cost 10 seconds, and I have 3 files to read.
My dart code:
main() {
readFile("aaa.txt");
readFile("bbb.txt");
readFile("ccc.txt");
print("Will print the content of the files later");
}
readFile(String filename) {
File file = new File(filename);
file.readAsString().then((content) {
print("File content:\n");
print(content);
});
}
Since reading a file will cost 10 seconds, so the above code will cost at least 30 seconds, right? Using futures to read files just to make the expensive tasks run later one by one, without blocking current code, but won't reduce the total cost?
If in java, I can make a thread pool, and make 3 future tasks running in parallel, the total cost will be between 10 and 20 seconds.
Is it possible to do the same in Dart? Is using Dart's isolate the only solution?
I would expect that this could take 10 seconds, as it will start three reads, each of which will queue an callback to the "then" function when the read is complete. It is entirely possible that the three files will load in parallel and all complete after 10 seconds. The callbacks will be called on the main thread sequentially though.
Although the user code in dart is single threaded (assuming you don't use isolates or web workers), nothing says that the implementation can't create threads or use the operating system's asynchronous loading to perform tasks in parallel as long as the future's run sequentially in the main thread.
That's correct. If you start an new async path with new Timer(), new Future(), or scheduleMicrotask() it will be scheduled for later execution.
When one of your async paths is waiting for a network request or the file system returning data, another async path may jump in and run in the meantime. So you might get a runtime less than 30 seconds, but you can't reduce runtime by adding a CPU.
I have to admit, that I don't know details about when scheduling takes place and how it works exactly.
Dart has no threads, so if you want to run code in parallel you need isolates.
Almost 30 seconds.
I had just run the code with dart 2.16.2, and the result is almost 30 seconds.
here is my code:
import 'dart:async';
import 'dart:convert';
import 'dart:io';
import 'dart:isolate';
main() async {
print('main start');
printCurrentTime("main before all future");
Future(() => readFile(0));
Future(() => readFile(1));
Future(() => readFile(2));
Future(() {
printCurrentTime("future last");
});
print('main end');
printCurrentTime("main");
}
printCurrentTime(String name) {
print("$name ${DateTime.now().millisecondsSinceEpoch}");
}
readFile(number) {
print("start read file $number");
var watch = Stopwatch();
watch.start();
var filename = r"path/to/file";
File file = File(filename);
file.readAsBytes().then((content) {
printCurrentTime("\nfuture#$number start");
print("File $number content:");
print(content.toString().length);
printCurrentTime("future#$number finish");
print("finish read file $number");
});
}
And here is the result:
main start
main before all future 1652964314276
main end
main 1652964314278
// all the event queue start to run
start read file 0
start read file 1
start read file 2
future last 1652964314290
// the dart system read file parallelly, after finish read file
// they put the future to the event queue, and dart start running all
// those event task one by one:
future#0 start 1652964314343
File 0 content:
241398625
future#0 finish 1652964317457
finish read file 0
future#1 start 1652964317457
File 1 content:
241398625
future#1 finish 1652964320470
finish read file 1
future#2 start 1652964320471
File 2 content:
241398625
future#2 finish 1652964323403
finish read file 2
As we can see:
file.readAsBytes() take about 53ms (or 100ms sometime during my test)
content.toString() take about 3s or more
So we can come to this conclusion:
file.readAsBytes() all run in the other thread parallelly, and the value the return Future<Uint8List> is added to the Event Task dequeue which is run synchronously, that's why we can see the future#1 start... print one by one.