I am using this function to add a timeout callback (repeated) to a specific GMainContext.
guint GstThreadHelper::timeoutAdd(guint delay, GSourceFunc function, gpointer data) {
// See https://developer.gnome.org/programming-guidelines/stable/main-contexts.html.en#implicit-use-of-the-global-default-main-context
// It is important that all thread functions we invoke don't implicitly decide a maincontext.
// We must manually provide one.
GSource *source = NULL;
guint id;
source = g_timeout_source_new(delay);
g_source_set_callback (source, function, data, NULL);
id = g_source_attach (source, priv->mainContext);
g_source_unref (source);
return id;
}
Later, I use the returned id to cancel the callback.
void GstThreadHelper::timeoutRemove(guint id) {
g_source_remove(id);
}
However, the callback still gets called. Here is my callback.
static gboolean position_update (gpointer user_data)
{
Player::PrivateData* priv = (Player::PrivateData*)user_data;
gint64 pos = 0;
if (gst_element_query_position (priv->playbin, GST_FORMAT_TIME, &pos)) {
pos = pos / 1000000;
priv->callback->PositionChanged(pos);
}
// Call me again
return TRUE;
}
I understand I am returning TRUE, but my understanding is that it still should be stopped. If I cancel callbacks by returning FALSE, I wouldn't bother with the g_source_remove call.
Why doesn't g_source_remove stop my callback from being raised?
EDIT
If I replace my timeoutAdd method with this...
guint GstThreadHelper::timeoutAdd(guint delay, GSourceFunc function, gpointer data) {
return g_timeout_add(delay, function, data);
}
...it works. However, I can't use this, because it doesn't trigger the callbacks on a specific GMainContext, as opposed to the default global GMainContext.
EDIT2
I copied the default source for g_timeout_add_seconds_full into my function, and it worked.
However, the moment I changed g_source_attach to use my private GMainContext, it failed.
The issue is something to do with calling g_source_remove for timeouts added on non-default GMainContexts.
It turns out that g_source_remove operates under the assumption that you are using the global/default GMainContext, which in this case, I am not.
I don't remember reading this in the docs.
Anyways, Here is the solution.
void GstThreadHelper::timeoutRemove(guint id) {
GSource* source = g_main_context_find_source_by_id(priv->mainContext, id);
if (source)
g_source_destroy (source);
}
This is essentially what g_source_remove is doing, but instead using our private GMainContext.
Related
I've enabled the dart 2.8 experimental null saftey.
I have the following exiting code.
StreamSubscription<String> subscription;
subscription =
response.transform(Utf8Decoder()).transform(LineSplitter()).listen(
(line) async {
result += line;
},
onDone: () async {
unawaited(subscription.cancel());
completer.complete(result);
},
);
With null saftey enabled I get a error in the 'onDone' method where it calls subscription.cancl
"The expression is nullable and must be null-checked before it can be used.
Try checking that the value isn't null before using it.",
I can fix the problem by putting a conditional before the call to cancel, but this seems unnecessary as in reality subscription can never be null.
Is there a coding pattern that allows subscription to be declared as non-null?
The problem here is that the read of subscription happens at a place where it's still potentially unassigned. It isn't, actually, but we only know that because the listen method promises not to call any of the callbacks before returning. The compiler can't see that. So, you need to move the reading to after the assignment.
What I'd do to make this listen call work:
var buffer = StringBuffer(result);
var subscription = response
.transform(Utf8Decoder())
.transform(LineSplitter())
.listen((line) {
buffer.write(line);
});
subscription.onDone(() {
completer.complete(buffer.toString());
});
I removed the async from the callbacks because it is not needed. All it does to make these functions async is to return a future that no-one would ever look at.
In general, the callbacks on Stream and Future should have non-async callbacks.
I also removed the subscription.cancel from the onDone event handler. If you get a "done" event, the subscription is done, there is no need to cancel it.
I also added a string buffer to avoid the quadratic time and space complexity of repeated string concatenation.
Looking at the code, you seem to be concatenating lines right after splitting them, maybe all you need is:
response.transform(Utf8Decoder()).join("").then(completer.complete);
I'll assume for now that the splitting+joining is necessary.
In that case, what I'd actually prefer to do instead is of using listen is:
var buffer = StringBuffer();
response
.transform(Utf8Decoder())
.transform(LineSplitter())
.forEach((line) {
buffer.write(line);
}).then(() {
completer.complete(buffer.toString());
}, onError: (e, s) {
completer.completeError(e, s);
});
or, if in an async function:
try {
var buffer = StringBuffer();
await for (var line in response.transform(Utf8Decoder()).transform(LineSplitter())) {
buffer.write(line);
}
completer.complete(buffer.toString());
} catch(e, s) {
completer.completeError(e, s);
}
I'm making a Future method that lives inside a seperate class, that fetches a bunch XKCD comics, and puts them in a List and returns it.
And that is all fine and dandy, but I would like to notify back when a single comic has been fetched, so I can show a progress dialog, on how far we are.
This is my code:
// This is inside my class ComicManager
Future<List<ComicModel>> generateComicList() async {
List<ComicModel> comicList = new List<ComicModel>();
ComicModel latestComic = await getLatestComic();
for (var i = 1; i <= latestComic.num; i++) {
try {
http.Response response =
await http.get('https://xkcd.com/${i}/info.0.json');
Map comicmap = json.decode(response.body);
var comic = new ComicModel.fromJson(comicmap);
comicList.add(comic);
print(comic.num);
// Notify here that we have fetched a comic
} catch (ex) {
// Comic could apparently not be parsed, skip it.
}
}
return comicList;
}
How should I solve this?
There seems no particularly elegant way to do this. From some flutter code samples, it seems using VoidCallBack listeners is an accepted way.
First register callback functions in a Set
Set<VoidCallBack> listeners
Then define the callback functions you needed. And add them to the set
void fun()
//...
listeners.add(fun);//Or you can define a method to do this or simply pass the function through the constructor of this class.
Finally, write a notifyListeners function or its equivalent and call it wherever you want
void notifyListeners(){
for(final listener in listeners){
listener();
}
}
If you want callback functions to carry an argument, just change the VoidCallBack to whatever function types.
Found a solution.
I just used Streams like so:
Stream<ComicProgressModel> getAllComicsStream() async* {
// Do what you need to do here
// This will respond back when you are listening to the stream
yield stuffToYield; // Can be anything, and you can yield as many times you want
// When you reach the end of the method, the onDone method will be called.
// So if you are running a for loop, and call yield multiple times it onDone is only called the the this method ends
}
Then I can just listen to events like so:
Stream comicStream =
ComicManager().getAllComicsStream().asBroadcastStream();
StreamSubscription comicsub = comicStream.listen((onData) {
// Do what i need
});
Super easy to be honest.
how to wait for the completion of Future without 'async' and 'futures'?
In the library that I use all functions are asynchronous.
// must return <bool>
bool my_func(int x){
//returns Future<int>
var tmp = somelib.somefunc( ... );
//wait y
return x == y;
}
I tried to write my 'await', but
waiting for a result with a while loop freezes everything.
dynamic my_await(Future f) {
dynamic x;
bool completed = false;
f.then((v){
x = v;
completed = true;
});
do {} while (!completed);
return x;
}
Dart VM version: 1.24.3 (Mon Dec 18 16:57:48 2017) on "linux_x64"
A synchronous function, or really, any Dart function, returns a value immediately when you call them. If you want to return a boolean immediately, and the value of that boolean depends on the result that some future completes with, then there is no way to compute that boolean in time.
If you need to wait for a future, then your function is asynchronous. You need to return something immediately, even if you don't know the result yet. That's what a Future is. It's not magical in any way, it's just an object that you can set a callback on which gets called when some result is ready.
So, you need to return a Future<bool> for this to work.
Dart is single-threaded. Without using isolates, there is no concurrency. Instead asynchronous functions work by taking turns, giving time for other code to run, e.g., while they wait on a future. If you just do a do {} while (!completed); then no other code gets to run, which means that nothing will be able to set completed to true.
I'm new to dart, so not sure if this is the correct way of doing it, but I've solved this issue by using the function whenCompleted() on the Future returned by the async method I'm calling.
Here openDatabase returns a Future.
abstract class IBaseDatabaseHandler {
Database sqliteDbHandler;
IBaseDatabaseHandler.sqlite(String dataBasePath) {
sqfliteFfiInit();
var databaseFactory = databaseFactoryFfi;
databaseFactory
.openDatabase(dataBasePath)
.whenComplete(() => sqliteDbHandler);
}
}
Whenever I use addListenerForSingleValueEvent with setPersistenceEnabled(true), I only manage to get a local offline copy of DataSnapshot and NOT the updated DataSnapshot from the server.
However, if I use addValueEventListener with setPersistenceEnabled(true), I can get the latest copy of DataSnapshot from the server.
Is this normal for addListenerForSingleValueEvent as it only searches DataSnapshot locally (offline) and removes its listener after successfully retrieving DataSnapshot ONCE (either offline or online)?
Update (2021): There is a new method call (get on Android and getData on iOS) that implement the behavior you'll like want: it first tries to get the latest value from the server, and only falls back to the cache when it can't reach the server. The recommendation to use persistent listeners still applies, but at least there's a cleaner option for getting data once even when you have local caching enabled.
How persistence works
The Firebase client keeps a copy of all data you're actively listening to in memory. Once the last listener disconnects, the data is flushed from memory.
If you enable disk persistence in a Firebase Android application with:
Firebase.getDefaultConfig().setPersistenceEnabled(true);
The Firebase client will keep a local copy (on disk) of all data that the app has recently listened to.
What happens when you attach a listener
Say you have the following ValueEventListener:
ValueEventListener listener = new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot snapshot) {
System.out.println(snapshot.getValue());
}
#Override
public void onCancelled(FirebaseError firebaseError) {
// No-op
}
};
When you add a ValueEventListener to a location:
ref.addValueEventListener(listener);
// OR
ref.addListenerForSingleValueEvent(listener);
If the value of the location is in the local disk cache, the Firebase client will invoke onDataChange() immediately for that value from the local cache. If will then also initiate a check with the server, to ask for any updates to the value. It may subsequently invoke onDataChange() again if there has been a change of the data on the server since it was last added to the cache.
What happens when you use addListenerForSingleValueEvent
When you add a single value event listener to the same location:
ref.addListenerForSingleValueEvent(listener);
The Firebase client will (like in the previous situation) immediately invoke onDataChange() for the value from the local disk cache. It will not invoke the onDataChange() any more times, even if the value on the server turns out to be different. Do note that updated data still will be requested and returned on subsequent requests.
This was covered previously in How does Firebase sync work, with shared data?
Solution and workaround
The best solution is to use addValueEventListener(), instead of a single-value event listener. A regular value listener will get both the immediate local event and the potential update from the server.
A second solution is to use the new get method (introduced in early 2021), which doesn't have this problematic behavior. Note that this method always tries to first fetch the value from the server, so it will take longer to completely. If your value never changes, it might still be better to use addListenerForSingleValueEvent (but you probably wouldn't have ended up on this page in that case).
As a workaround you can also call keepSynced(true) on the locations where you use a single-value event listener. This ensures that the data is updated whenever it changes, which drastically improves the chance that your single-value event listener will see the current value.
So I have a working solution for this. All you have to do is use ValueEventListener and remove the listener after 0.5 seconds to make sure you've grabbed the updated data by then if needed. Realtime database has very good latency so this is safe. See safe code example below;
public class FirebaseController {
private DatabaseReference mRootRef;
private Handler mHandler = new Handler();
private FirebaseController() {
FirebaseDatabase.getInstance().setPersistenceEnabled(true);
mRootRef = FirebaseDatabase.getInstance().getReference();
}
public static FirebaseController getInstance() {
if (sInstance == null) {
sInstance = new FirebaseController();
}
return sInstance;
}
Then some method you'd have liked to use "addListenerForSingleEvent";
public void getTime(final OnTimeRetrievedListener listener) {
DatabaseReference ref = mRootRef.child("serverTime");
ref.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
if (listener != null) {
// This can be called twice if data changed on server - SO DEAL WITH IT!
listener.onTimeRetrieved(dataSnapshot.getValue(Long.class));
}
// This can be called twice if data changed on server - SO DEAL WITH IT!
removeListenerAfter2(ref, this);
}
#Override
public void onCancelled(DatabaseError databaseError) {
removeListenerAfter2(ref, this);
}
});
}
// ValueEventListener version workaround for addListenerForSingleEvent not working.
private void removeListenerAfter2(DatabaseReference ref, ValueEventListener listener) {
mHandler.postDelayed(new Runnable() {
#Override
public void run() {
HelperUtil.logE("removing listener", FirebaseController.class);
ref.removeEventListener(listener);
}
}, 500);
}
// ChildEventListener version workaround for addListenerForSingleEvent not working.
private void removeListenerAfter2(DatabaseReference ref, ChildEventListener listener) {
mHandler.postDelayed(new Runnable() {
#Override
public void run() {
HelperUtil.logE("removing listener", FirebaseController.class);
ref.removeEventListener(listener);
}
}, 500);
}
Even if they close the app before the handler is executed, it will be removed anyways.
Edit: this can be abstracted to keep track of added and removed listeners in a HashMap using reference path as key and datasnapshot as value. You can even wrap a fetchData method that has a boolean flag for "once" if this is true it would do this workaround to get data once, else it would continue as normal.
You're Welcome!
You can create transaction and abort it, then onComplete will be called when online (nline data) or offline (cached data)
I previously created function which worked only if database got connection lomng enough to do synch. I fixed issue by adding timeout. I will work on this and test if this works. Maybe in the future, when I get free time, I will create android lib and publish it, but by then it is the code in kotlin:
/**
* #param databaseReference reference to parent database node
* #param callback callback with mutable list which returns list of objects and boolean if data is from cache
* #param timeOutInMillis if not set it will wait all the time to get data online. If set - when timeout occurs it will send data from cache if exists
*/
fun readChildrenOnlineElseLocal(databaseReference: DatabaseReference, callback: ((mutableList: MutableList<#kotlin.UnsafeVariance T>, isDataFromCache: Boolean) -> Unit), timeOutInMillis: Long? = null) {
var countDownTimer: CountDownTimer? = null
val transactionHandlerAbort = object : Transaction.Handler { //for cache load
override fun onComplete(p0: DatabaseError?, p1: Boolean, data: DataSnapshot?) {
val listOfObjects = ArrayList<T>()
data?.let {
data.children.forEach {
val child = it.getValue(aClass)
child?.let {
listOfObjects.add(child)
}
}
}
callback.invoke(listOfObjects, true)
}
override fun doTransaction(p0: MutableData?): Transaction.Result {
return Transaction.abort()
}
}
val transactionHandlerSuccess = object : Transaction.Handler { //for online load
override fun onComplete(p0: DatabaseError?, p1: Boolean, data: DataSnapshot?) {
countDownTimer?.cancel()
val listOfObjects = ArrayList<T>()
data?.let {
data.children.forEach {
val child = it.getValue(aClass)
child?.let {
listOfObjects.add(child)
}
}
}
callback.invoke(listOfObjects, false)
}
override fun doTransaction(p0: MutableData?): Transaction.Result {
return Transaction.success(p0)
}
}
In the code if time out is set then I set up timer which will call transaction with abort. This transaction will be called even when offline and will provide online or cached data (in this function there is really high chance that this data is cached one).
Then I call transaction with success. OnComplete will be called ONLY if we got response from firebase database. We can now cancel timer (if not null) and send data to callback.
This implementation makes dev 99% sure that data is from cache or is online one.
If you want to make it faster for offline (to don't wait stupidly with timeout when obviously database is not connected) then check if database is connected before using function above:
DatabaseReference connectedRef = FirebaseDatabase.getInstance().getReference(".info/connected");
connectedRef.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot snapshot) {
boolean connected = snapshot.getValue(Boolean.class);
if (connected) {
System.out.println("connected");
} else {
System.out.println("not connected");
}
}
#Override
public void onCancelled(DatabaseError error) {
System.err.println("Listener was cancelled");
}
});
When workinkg with persistence enabled, I counted the times the listener received a call to onDataChange() and stoped to listen at 2 times. Worked for me, maybe helps:
private int timesRead;
private ValueEventListener listener;
private DatabaseReference ref;
private void readFB() {
timesRead = 0;
if (ref == null) {
ref = mFBDatabase.child("URL");
}
if (listener == null) {
listener = new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
//process dataSnapshot
timesRead++;
if (timesRead == 2) {
ref.removeEventListener(listener);
}
}
#Override
public void onCancelled(DatabaseError databaseError) {
}
};
}
ref.removeEventListener(listener);
ref.addValueEventListener(listener);
}
There are some database operations I need to execute before the end of the final attempt of my Hangfire background job (I need to delete the database record related to the job)
My current job is set with the following attribute:
[AutomaticRetry(Attempts = 5, OnAttemptsExceeded = AttemptsExceededAction.Delete)]
With that in mind, I need to determine what the current attempt number is, but am struggling to find any documentation in that regard from a Google search or Hangfire.io documentation.
Simply add PerformContext to your job method; you'll also be able to access your JobId from this object. For attempt number, this still relies on magic strings, but it's a little less flaky than the current/only answer:
public void SendEmail(PerformContext context, string emailAddress)
{
string jobId = context.BackgroundJob.Id;
int retryCount = context.GetJobParameter<int>("RetryCount");
// send an email
}
(NB! This is a solution to the OP's problem. It does not answer the question "How to get the current attempt number". If that is what you want, see the accepted answer for instance)
Use a job filter and the OnStateApplied callback:
public class CleanupAfterFailureFilter : JobFilterAttribute, IServerFilter, IApplyStateFilter
{
public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
try
{
var failedState = context.NewState as FailedState;
if (failedState != null)
{
// Job has finally failed (retry attempts exceeded)
// *** DO YOUR CLEANUP HERE ***
}
}
catch (Exception)
{
// Unhandled exceptions can cause an endless loop.
// Therefore, catch and ignore them all.
// See notes below.
}
}
public void OnStateUnapplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
// Must be implemented, but can be empty.
}
}
Add the filter directly to the job function:
[CleanupAfterFailureFilter]
public static void MyJob()
or add it globally:
GlobalJobFilters.Filters.Add(new CleanupAfterFailureFilter ());
or like this:
var options = new BackgroundJobServerOptions
{
FilterProvider = new JobFilterCollection { new CleanupAfterFailureFilter () };
};
app.UseHangfireServer(options, storage);
Or see http://docs.hangfire.io/en/latest/extensibility/using-job-filters.html for more information about job filters.
NOTE: This is based on the accepted answer: https://stackoverflow.com/a/38387512/2279059
The difference is that OnStateApplied is used instead of OnStateElection, so the filter callback is invoked only after the maximum number of retries. A downside to this method is that the state transition to "failed" cannot be interrupted, but this is not needed in this case and in most scenarios where you just want to do some cleanup after a job has failed.
NOTE: Empty catch handlers are bad, because they can hide bugs and make them hard to debug in production. It is necessary here, so the callback doesn't get called repeatedly forever. You may want to log exceptions for debugging purposes. It is also advisable to reduce the risk of exceptions in a job filter. One possibility is, instead of doing the cleanup work in-place, to schedule a new background job which runs if the original job failed. Be careful to not apply the filter CleanupAfterFailureFilter to it, though. Don't register it globally, or add some extra logic to it...
You can use OnPerforming or OnPerformed method of IServerFilter if you want to check the attempts or if you want you can just wait on OnStateElection of IElectStateFilter. I don't know exactly what requirement you have so it's up to you. Here's the code you want :)
public class JobStateFilter : JobFilterAttribute, IElectStateFilter, IServerFilter
{
public void OnStateElection(ElectStateContext context)
{
// all failed job after retry attempts comes here
var failedState = context.CandidateState as FailedState;
if (failedState == null) return;
}
public void OnPerforming(PerformingContext filterContext)
{
// do nothing
}
public void OnPerformed(PerformedContext filterContext)
{
// you have an option to move all code here on OnPerforming if you want.
var api = JobStorage.Current.GetMonitoringApi();
var job = api.JobDetails(filterContext.BackgroundJob.Id);
foreach(var history in job.History)
{
// check reason property and you will find a string with
// Retry attempt 3 of 3: The method or operation is not implemented.
}
}
}
How to add your filter
GlobalJobFilters.Filters.Add(new JobStateFilter());
----- or
var options = new BackgroundJobServerOptions
{
FilterProvider = new JobFilterCollection { new JobStateFilter() };
};
app.UseHangfireServer(options, storage);
Sample output :