I'm doing some work with libevent, version 2.0.22, and I'm struggling with dealing with bufferevent_free and making sure sockets close at the right time. This is within an iOS app built with Xcode 6.4, running on iOS 8.4.
Each socket is managed by a struct bufferevent, and I also have a data structure which keeps track of application state for that socket:
bev = bufferevent_socket_new(evbase, -1, BEV_OPT_CLOSE_ON_FREE);
bufferevent_socket_connect_hostname(bev, dns_base, AF_UNSPEC, host, port_number);
struct stream_data *data = malloc(sizeof(struct stream_data));
/* initialize stream data here */
bufferevent_setcb(bev, read_cb, write_cb, event_cb, data);
In a callback from some other buffer socket, I decide I don't need the buffer I just tried to open. This happens before I get the connected callback on the bev in question. Because I created it with BEV_OPT_CLOSE_ON_FREE, I just free it. Then I delete the data structure I'm using. Looks like this:
bufferevent_free(bev);
free(stream_data); // the data corresponding to that bev
In this case, though, the socket actually finished connecting in the meantime. So my event callback fires:
void event_cb(struct bufferevent *bev, short what, void *ctx)
{
struct stream_data *data = ctx;
// data now points to already freed memory
}
And now I've got a pointer to already freed memory. I've confirmed with debugger breakpoints, NSLog, etc that the event callback is firing after the free above.
Is this expected behavior? If so, how can I ever tell that a bufferevent I freed is well and truly gone, making it safe to remove my own data structures?
Yes, this is expected libevent behavior: after bufferevent_free() it still may invoke your callbacks. From libevent book:
Bufferevents are internally reference-counted, so if the bufferevent has pending deferred callbacks when you free it, it won’t be deleted until the callbacks are done.
The easiest solutions is to remove all the callbacks before freeing the bufferevent object:
bufferevent_setcb(bev, NULL, NULL, NULL, NULL);
bufferevent_free(bev);
Related
I am working on exposing an audio library (C library) for Dart. To trigger the audio engine, it requires a few initializations steps (non blocking for UI), then audio processing is triggered with a perform function, which is blocking (audio processing is a heavy task). That is why I came to read about Dart isolates.
My first thought was that I only needed to call the performance method in the isolate, but it doesn't seem possible, since the perform function takes the engine state as first argument - this engine state is an opaque pointer ( Pointer in dart:ffi ). When trying to pass engine state to a new isolate with compute function, Dart VM returns an error - it cannot pass C pointers to an isolate.
I could not find a way to pass this data to the isolate, I assume this is due to the separate memory of main isolate and the one I'm creating.
So, I should probably manage the entire engine state in the isolate which means :
Create the engine state
Initialize it with some options (strings)
trigger the perform function
control audio at runtime
I couldn't find any example on how to perform this actions in the isolate, but triggered from main thread/isolate. Neither on how to manage isolate memory (keep the engine state, and use it). Of course I could do
Here is a non-isolated example of what I want to do :
Pointer<Void> engineState = createEngineState();
initEngine(engineState, parametersString);
startEngine(engineState);
perform(engineState);
And at runtime, triggered by UI actions (like slider value changed, or button clicked) :
setEngineControl(engineState, valueToSet);
double controleValue = getEngineControl(engineState);
The engine state could be encapsulated in a class, I don't think it really matters here.
Whether it is a class or an opaque datatype, I can't find how to manage and keep this state, and perform triggers from main thread (processed in isolate). Any idea ?
In advance, thanks.
PS: I notice, while writing, that my question/explaination may not be precise, I have to say I'm a bit lost here, since I never used Dart Isolates. Please tell me if some information is missing.
EDIT April 24th :
It seems to be working with creating and managing object state inside the Isolate. But the main problem isn't solved. Because the perform method is actually blocking while it is not completed, there is no way to still receive messages in the isolate.
An option I thought first was to use the performBlock method, which only performs a block of audio samples. Like this :
while(performBlock(engineState)) {
// listen messages, and do something
}
But this doesn't seem to work, process is still blocked until audio performance finishes. Even if this loop is called in an async method in the isolate, it blocks, and no message are read.
I now think about the possibility to pass the Pointer<Void> managed in main isolate to another, that would then be the worker (for perform method only), and then be able to trigger some control methods from main isolate.
The isolate Dart package provides a registry sub library to manage some shared memory. But it is still impossible to pass void pointer between isolates.
[ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: Invalid argument(s): Native objects (from dart:ffi) such as Pointers and Structs cannot be passed between isolates.
Has anyone already met this kind of situation ?
It is possible to get an address which this Pointer points to as a number and construct a new Pointer from this address (see Pointer.address and Pointer.fromAddress()). Since numbers can freely be passed between isolates, this can be used to pass native pointers between them.
In your case that could be done, for example, like this (I used Flutter's compute to make the example a bit simpler but that would apparently work with explicitly using Send/ReceivePorts as well)
// Callback to be used in a backround isolate.
// Returns address of the new engine.
int initEngine(String parameters) {
Pointer<Void> engineState = createEngineState();
initEngine(engineState, parameters);
startEngine(engineState);
return engineState.address;
}
// Callback to be used in a backround isolate.
// Does whichever processing is needed using the given engine.
void processWithEngine(int engineStateAddress) {
final engineState = Pointer<Void>.fromAddress(engineStateAddress);
process(engineState);
}
void main() {
// Initialize the engine in a background isolate.
final address = compute(initEngine, "parameters");
final engineState = Pointer<Void>.fromAddress(address);
// Do some heavy computation in a background isolate using the engine.
compute(processWithEngine, engineState.address);
}
I ended up doing the processing of callbacks inside the audio loop itself.
while(performAudio())
{
tasks.forEach((String key, List<int> value) {
double val = getCallback(key);
value.forEach((int element) {
callbackPort.send([element, val]);
});
});
}
Where the 'val' is the thing you want to send to callback. The list of int 'value' is a list of callback index.
Let's say you audio loop performs with vector size of 512 samples, you will be able to pass your callbacks after every 512 audio samples are processed, which means 48000 / 512 times per second (assuming you sample rate is 48000). This method is not the best one but it works, I still have to see if it works in very intensive processing context though. Here, it has been thought for realtime audio, but it could work the same for audio rendering.
You can see the full code here : https://framagit.org/johannphilippe/csounddart/-/blob/master/lib/csoundnative.dart
I am working with Rime, more specifically with the runicast example. Once a message is received i store it in a linked list, then i post an event to a process which is in charge of extracting messages from the linked list and processing them. My code is something like this:
static void recv_runicast(struct runicast_conn *c,
const linkaddr_t *from, uint8_t seqno)
{
/*code to insert the message into the linked list*/
...
/*Post an event to the process which extracts messages from the linked list */
process_post(&extract_msg_from_linked_list, PROCESS_EVENT_CONTINUE, NULL);
}
My question is: Is it safe to use process_post within the callback function recv_runicast? or should i use process_poll?
Thanks in advance
Yes, it's safe. The network stack operations are done in process context, and Contiki processes are not preemptive. So pretty much any process-related operations are "safe".
The main differences between process_post and process_poll is that the former will put a new event in the process event buffer, while the latter will simply set a flag. So the second options is slightly more efficient. Also, in theory the event buffer can get full and events start to get lost, but that's very unlikely to be a problem.
I would use none of these functions at all, but do the processing directly in the callback to simplify the execution flow.
I'm using PLCrashReporter in my iOS project and I'm curious, is it possible to use Core Foundation code in my custom crash callback. The thing, that handle my needs is CFPreferences.Here is part of code, that I create:
void LMCrashCallback(siginfo_t* info, ucontext_t* uap, void* context) {
CFStringRef networkStatusOnCrash;
networkStatusOnCrash = (CFStringRef)CFPreferencesCopyAppValue(networkStatusKey, kCFPreferencesCurrentApplication);
CFStringRef additionalInfo = CFStringCreateWithFormat(
NULL, NULL, CFSTR( "Additional Crash Properties:[Internet: %#]", networkStatusOnCrash);
CFPreferencesSetAppValue(additionalInfoKey, additionalInfo,
kCFPreferencesCurrentApplication);
CFPreferencesAppSynchronize(kCFPreferencesCurrentApplication);
}
My target is to collect some system information, just in time when app crashed, e.g Internet connection type.
I know it is not good idea to create own crash callback due to async-safe functions, but this can help.
Also as other option: Is there a way to extend somehow PLCrashReportSystemInfo class?
This is very dangerous. In particular the call to CFStringCreateWithFormat allocates memory. Allocating memory in the middle of a crash handler can lead to battery-draining deadlock (yep; had that bug…) For example, if you were in the middle of free() (which is not an uncommon place to crash), you may already be holding a spinlock on the heap. When you call malloc to get some memory, you may spinlock the heap again and deadlock in a tight-loop. The heap needs to be locked so often and for such short periods of time that it doesn't use a blocking lock. It does the equivalent of while (locked) {}.
You seem to just be reading a preference and copying it to another preference. There's no reason to do that inside a crash handler. Just check hasPendingCrashReport during startup (which I assume you're doing already), and read the key then. It's not clear what networkStatusKey is, but it should still be there when you start up again.
If for any reason it's modified very early (before you call hasPendingCrashReport), you can grab it in main() before launching the app. Or you can grab it in a +load method, which is called even earlier.
NB: The entire code base for this project is so large that posting any meaningful amount wold render this question too localised, I have tried to distil any code down to the bare-essentials. I'm not expecting anyone to solve my problems directly but I will up vote those answers I find helpful or intriguing.
This project uses a modified version of AudioStreamer to playback audio files that are saved to locally to the device (iPhone).
The stream is set up and scheduled on the current loop using this code (unaltered from the standard AudioStreamer project as far as I know):
CFStreamClientContext context = {0, self, NULL, NULL, NULL};
CFReadStreamSetClient(
stream,
kCFStreamEventHasBytesAvailable | kCFStreamEventErrorOccurred | kCFStreamEventEndEncountered,
ASReadStreamCallBack,
&context);
CFReadStreamScheduleWithRunLoop(stream, CFRunLoopGetCurrent(), kCFRunLoopCommonModes);
The ASReadStreamCallBack calls:
- (void)handleReadFromStream:(CFReadStreamRef)aStream
eventType:(CFStreamEventType)eventType
On the AudioStreamer object, this all works fine until the stream is read using this code:
BOOL hasBytes = NO; //Added for debugging
hasBytes = CFReadStreamHasBytesAvailable(stream);
length = CFReadStreamRead(stream, bytes, kAQDefaultBufSize);
hasBytes is YES but when CFReadStreamRead is called execution stops, the App does not crash it just stops exciting, any break points below the CFReadStreamRead call are not hit and ASReadStreamCallBack is not called again.
I am at a loss to what might cause this, my best guess is the thread is being terminated? But the hows and whys is why I'm asking SO.
Has anyone seen this behaviour before? How can I track it down and ideas on how I might solve it will be very much welcome!
Additional Info Requested via Comments
This is 100% repeatable
CFReadStreamHasBytesAvailable was added by me for debugging but removing it has no effect
First, I assume that CFReadStreamScheduleWithRunLoop() is running on the same thread as CFReadStreamRead()?
Is this thread processing its runloop? Failure to do this is my main suspicion. Do you have a call like CFRunLoopRun() or equivalent on this thread?
Typically there is no reason to spawn a separate thread for reading streams asynchronously, so I'm a little confused about your threading design. Is there really a background thread involved here? Also, typically CFReadStreamRead() would be in your client callback (when you receive the kCFStreamEventHasBytesAvailable event (which it appears to be in the linked code), but you're suggesting ASReadStreamCallBack is never called. How have you modified AudioStreamer?
It is possible that the stream pointer is just corrupt in some way. CFReadStreamRead should certainly not block if bytes are available (it certainly would never block for more than a few milliseconds for local files). Can you provide the code you use to create the stream?
Alternatively, CFReadStreams send messages asynchronously but it is possible (but not likely) that it's blocking because the runloop isn't being processed.
If you prefer, I've uploaded my AudioPlayer inspired by Matt's AudioStreamer hosted at https://code.google.com/p/audjustable/. It supports local files (as well as HTTP). I think it does what you wanted (stream files from more than just HTTP).
Consider the following program flow:
pthread_rwlock_rdlock( &mylock);
... compute a lot, maybe be the target of a pthread_cancel() ...
pthread_rwlock_unlock( &mylock);
that is going to leave the lock in a rdlock state if thread is canceled.
It appears that the "right" thing to do is to use pthread_cleanup_push() and pthread_cleanup_pop() and do the unlock inside my cleanup function, but there doesn't seem to be a valid order for the function calls:
void my_cleanup(void *arg) { pthread_rwlock_unlock(&mylock); }
...
pthread_cleanup_push( my_cleanup, 0);
/* A */
pthread_rwlock_rdlock( &mylock);
... compute a lot, maybe be the target of a pthread_cancel() ...
pthread_cleanup_pop( 1);
... that appears nearly correct, except that if the pthread_cancel() hits at "A" then the cleanup will unlock a mylock which is not yet locked leading to undefined behavior.
The whole answer may be:
void my_cleanup(void *arg) { pthread_rwlock_unlock(&mylock); }
...
pthread_setcancelstate( PTHREAD_CANCEL_DISABLE, &oldstate);
pthread_cleanup_push( my_cleanup, 0);
pthread_rwlock_rdlock( &mylock);
pthread_setcancelstate( oldstate, 0);
... compute a lot, maybe be the target of a pthread_cancel() ...
pthread_cleanup_pop( 1);
but at that point it seems like I'm wrapping some well defined primitives in bandages.
So is there a better idiom for this?
Unless you allow asynchronous cancel, the pending pthread_cancel() will be processed only at known points. Therefore it should be safe to push just after successfully locking, and the pop just before unlocking.
I took a look at the glibc/nptl source and I don't see a better way than what you suggested. There's not even enough state in a rwlock for you to "cheat" and trust that the unlock side will know if you got the lock or not. If you didn't acquire the lock but someone else did, a spurious unlock will corrupt the state (rather than returning error).
In fact, offhand I can't find a reason why a well-placed SIGCANCEL couldn't interrupt pthread_rwlock_rdlock itself and leave the lll_lock (glibc internal) lock held, causing all other access to the rwlock to hang. So your PTHREAD_CANCEL_DISABLE might be even more important. Add one around the cleanup as well if you really plan to create/cancel many of these threads. It would be worth writing a test program to stress-test it.