NSURLSession request body passed by slow NSInputStream (bandwidth management) - ios

Hi based on this answer I wrote subclass of NSInputStream and it works pretty well.
Now It turned out that I have scenario where I'm feeding to server large amount of data and to prevent starvation of other services I need control speed of feeding data. So I improved functinality of my subclass with following conditions:
when data should be postponed, hasBytesAvailable returns NO and reading attempts ends with zero bytes read
when data can be send, - read:maxLength: allows to read some maximum amount data at once (by default 2048).
when - read:maxLength: returns zero bytes read, needed delay is calculated and after that delay NSStreamEventHasBytesAvailable event is posted.
Here is interesting parts of code (it is mixed with C++):
- (NSInteger)read:(uint8_t *)buffer maxLength:(NSUInteger)len {
if (![self isOpen]) {
return kOperationFailedReturnCode;
}
int delay = 0;
NSInteger readCount = (NSInteger)self.cppInputStream->Read(buffer, len, delay);
if (readCount<0) {
return kOperationFailedReturnCode;
}
LOGD("Stream") << __PRETTY_FUNCTION__
<< " len: " << len
<< " readCount: "<< readCount
<< " time: " << (int)(-[openDate timeIntervalSinceNow]*1000)
<< " delay: " << delay;
if (!self.cppInputStream->IsEOF()) {
if (delay==0)
{
[self enqueueEvent: NSStreamEventHasBytesAvailable];
} else {
NSTimer *timer = [NSTimer timerWithTimeInterval: delay*0.001
target: self
selector: #selector(notifyBytesAvailable:)
userInfo: nil
repeats: NO];
[self enumerateRunLoopsUsingBlock:^(CFRunLoopRef runLoop) {
CFRunLoopAddTimer(runLoop, (CFRunLoopTimerRef)timer, kCFRunLoopCommonModes);
}];
}
} else {
[self setStatus: NSStreamStatusAtEnd];
[self enqueueEvent: NSStreamEventEndEncountered];
}
return readCount;
}
- (void)notifyBytesAvailable: (NSTimer *)timer {
LOGD("Stream") << __PRETTY_FUNCTION__ << "notifyBytesAvailable time: " << (int)(-[openDate timeIntervalSinceNow]*1000);
[self enqueueEvent: NSStreamEventHasBytesAvailable];
}
- (BOOL)hasBytesAvailable {
bool result = self.cppInputStream->HasBytesAvaible();
LOGD("Stream") << __PRETTY_FUNCTION__ << ": " << result << " time: " << (int)(-[openDate timeIntervalSinceNow]*1000);
return result;
}
I wrote some test for that and it worked.
Problem appeared when I used this stream with NSURLSession as source of body of HTTP request. From logs I can see that NSURLSession tries to read everything at once. On first read I return limited portion of data. Immediately after that NSURLSession asks if there are bytes available (I return NO).
After some time (for example 170 ms), I'm sending notification that bytes are now available but NSURLSession doesn't respond to that and do not invoke any method of my stream class.
Here is what I see in logs (when running some test):
09:32:14990[0x7000002a0000] D/Stream: -[CSCoreFoundationCppInputStreamWrapper open]
09:32:14990[0x7000002a0000] D/Stream: -[CSCoreFoundationCppInputStreamWrapper hasBytesAvailable]: 1 time: 0
09:32:14990[0x7000002a0000] D/Stream: -[CSCoreFoundationCppInputStreamWrapper read:maxLength:] len: 32768 readCount: 2048 time: 0 delay: 170
09:32:14990[0x7000002a0000] D/Stream: -[CSCoreFoundationCppInputStreamWrapper hasBytesAvailable]: 0 time: 0
09:32:14990[0x7000002a0000] D/Stream: -[CSCoreFoundationCppInputStreamWrapper hasBytesAvailable]: 0 time: 0
09:32:14990[0x7000002a0000] D/Stream: -[CSCoreFoundationCppInputStreamWrapper hasBytesAvailable]: 0 time: 0
09:32:15161[0x7000002a0000] D/Stream: -[CSCoreFoundationCppInputStreamWrapper notifyBytesAvailable:]notifyBytesAvailable time: 171
Where time is amount of milliseconds since stream has been opened.
Looks looks NSURLSession is unable to handle input streams with limited data rate.
Does anyone else had similar problem?
Or has alternative concept how to achieve bandwidth management on NSURLSession?

solutions tha I can support is:
using NSURLSessionStreamTask, from iOS9 and OSX10.11.
using ASIHTTPRequest instead.

Unfortunately, NSInputStream is a class cluster. That makes subclassing hard. And in the case of NSInputStream, any subclasses are completely unsupported and are likely to fail in fascinating ways. (See http://blog.bjhomer.com/2011/04/subclassing-nsinputstream.html for details.)
Instead of subclassing NSInputStream, you should use a bound pair of streams and create your own data provider class to feed data into it. To do this:
Call CFStreamCreateBoundPair.
Cast the resulting CFReadStream object to an NSInputStream pointer.
Cast the CFWriteStream object to an NSOutputStream pointer.
Pass the input stream when you create the upload task or request object.
Create a class that uses a timer to periodically pass the next chunk of data to the output stream.
If you do this, the data your data provider class passes to the NSOutputStream will become available for reading from the NSInputStream on the other end.

Related

Is there any way to save mulaw audio stream from twilio in a file

I am using Twilio voice stream feature and i don't want to use Twilio record functionality. When Twilio starts sending voice stream to my server i want to store it into disk as an audio file in realtime.
I was running into the same issue today and figured a way to generate a WAVE Header for the mu-law header:
If you're following Twilio's blog post, that's the code I ended implementing:
wss.on('connection', (socket) => {
socket.on('message', (msg) => {
const { event, ...message } = JSON.parse(msg);
switch (event) {
case 'start':
let streamSid = message.start.streamSid;
socket.wstream = fs.createWriteStream(__dirname + `/${Date.now()}.wav`, { encoding: 'binary' });
// This is a mu-law header for a WAV-file compatible with twilio format
socket.wstream.write(Buffer.from([
0x52,0x49,0x46,0x46,0x62,0xb8,0x00,0x00,0x57,0x41,0x56,0x45,0x66,0x6d,0x74,0x20,
0x12,0x00,0x00,0x00,0x07,0x00,0x01,0x00,0x40,0x1f,0x00,0x00,0x80,0x3e,0x00,0x00,
0x02,0x00,0x04,0x00,0x00,0x00,0x66,0x61,0x63,0x74,0x04,0x00,0x00,0x00,0xc5,0x5b,
0x00,0x00,0x64,0x61,0x74,0x61,0x00,0x00,0x00,0x00, // Those last 4 bytes are the data length
]));
break;
case 'media':
// decode the base64-encoded data and write to stream
socket.wstream.write(Buffer.from(message.media.payload, 'base64'));
break;
case 'stop':
// Now the only thing missing is to write the number of data bytes in the header
socket.wstream.write("", () => {
let fd = fs.openSync(socket.wstream.path, 'r+'); // `r+` mode is needed in order to write to arbitrary position
let count = socket.wstream.bytesWritten;
count -= 58; // The header itself is 58 bytes long and we only want the data byte length
console.log(count)
fs.writeSync(
fd,
Buffer.from([
count % 256,
(count >> 8) % 256,
(count >> 16) % 256,
(count >> 24) % 256,
]),
0,
4, // Write 4 bytes
54, // starts writing at byte 54 in the file
);
});
break;
}
});
});
You can use FFmpeg to convert the Twilio mulaw to a regular WAV file.
If you use the class below, then you will just need to send the Twilio stream buffers when they arrive.
Something like:
recording = StreamAudioRecording(tempDirectory)
recording.start_recording(call_id)
<loop over buffer packets arriving>
recording.write_buffer(buffer)
recording_audio_path = recording.stop_recording()
This is the class implementation
import os
RAW_AUDIO_FILE_EXTENSION = "ulaw"
CONVERTED_AUDIO_FILE_EXTENSION = "wav"
class StreamAudioRecording:
def __init__(self, audio_recording_path):
self.audio_recording_path = audio_recording_path
self.f = None
self.audio_file_path = None
def start_recording(self, call_id):
self.audio_file_path = os.path.join(self.audio_recording_path, f" .
{call_id}.{RAW_AUDIO_FILE_EXTENSION}")
self.f = open(self.audio_file_path, 'wb')
def write_buffer(self, buffer):
self.f.write(buffer)
def stop_recording(self):
self.f.close()
converted_audio_path =
self.audio_file_path.replace(RAW_AUDIO_FILE_EXTENSION,
CONVERTED_AUDIO_FILE_EXTENSION)
self.convert_call_recording(self.audio_file_path, converted_audio_path)
return converted_audio_path
#staticmethod
def convert_call_recording(mulaw_path, wav_path):
command = f"ffmpeg -y -loglevel panic -f mulaw -ar 8k -ac 1 -i {mulaw_path} {wav_path}"
os.system(command)
If your using Node.js, the solution that #tdeo provided works great. I was inspired and I made a simple library using this solution. It's available now on npm.
https://www.npmjs.com/package/twilio-media-stream-save-audio-file

Using WaitForMultipleObjects() with ACE_SOCK_Stream - get event only when there's data

Is it possible to use WaitForMultipleObjects() with ACE_SOCK_Stream, and make it return only when there's data to read from it?
I tried to following:
// set some params
DWORD handlesCount = 1;
DWORD timeoutMs = 5 * 1000;
HANDLE* handles = new HANDLE[handlesCount];
handles[0] = sock_stream.get_handle();
while (true) {
int ret = WaitForMultipleObjects(handlesCount, handles, false, timeoutMs);
std::cout << "Result: " << ret << std::endl;
But the WaitForMultipleObjects() returns immediately the socket stream index, indicating that its ready (it prints 0 in an endless loop).
The socket is accepted via a ACE_SOCK_Acceptor (ACE_SOCK_Acceptor->accept()).
How do I make WaitForMultipleObjects() wait until the socket has data to read?
The socket handle is not suitable for use in WFMO. You should use WSAEventSelect to associate the desired event(s) with an event handle that's registered with WFMO.
Since you are also familiar with ACE, you can check the source code for ace/WFMO_Reactor.cpp, register_handler() method to see a use-case and how it works with WFMO.

CAPL Script for Diagnostic Services

I am writing the CAPL script to Automise the Diagnostic services. I have read some DIDs which are bigger than 8 bytes in size. Till 8 bytes I can capture correctly the data in my CAPL script but when the data size exceeds the 8 bytes, then I get some garbage values 00 for remaining bytes.
The complete read data I can see in CANoe Trace but I am not able to capture it in my CAPL script. If someone has any ideas or solution, please share with me.
In Belo script, the issue is that I can capture value till this.byte(7) correctly. But for this.byte(8) and this.byte(9) I read 00 although the actual value in CANoe Trace is 0x54 and 0x66. So it means I cannot read more than 8 bytes in CAPL from CAN.
My script looks like:
variables
{
//Please insert your code below this comment
byte test_num;
message DTOOL_to_UPA msg_tester;
mstimer readTimerDID_2001;
mstimer defaultSession;
byte readBuf2001[8];
}
// read request
on key 'd'
{
test_num = 0;
msg_tester.dlc = 8;
msg_tester.dir = tx;
msg_tester.can = 1;
settimer(defaultSession, 2000);
}
on timer defaultSession // Request DID: 10 01
{
msg_tester.byte(0) = 0x02;
msg_tester.byte(1) = 0x10;
msg_tester.byte(2) = 0x01;
output(msg_tester);
settimer(readTimerDID_2001, 100);
canceltimer(defaultSession);
}
on timer readTimerDID_2001 // Read Request DID: 22 20 01
{
msg_tester.byte(0) = 0x03;
msg_tester.byte(1) = 0x22;
msg_tester.byte(2) = 0x20;
msg_tester.byte(3) = 0x01;
output(msg_tester);
canceltimer(readTimerDID_2001);
}
on message UPA_to_DTOOL
{
if (this.DIR == RX)
{
// Response Data for DID 2001
if (
(this.byte(0)== 0x04)&&(this.byte(1)== 0x62)&&(this.byte(2)==0x20)&&
(this.byte(3)== 0x01)&&(this.byte(4)== 0x23) &&(this.byte(5)== 0x00)&&
(this.byte(6)== 0x44)&&(this.byte(7)== 0x22) &&(this.byte(8)==0x54)&&
(this.byte(9)== 0x66)
)
{
readDID2001();
}
}
}
on message UPA_to_DTOOL
is reacting on the CAN message UPA_to_DTOOL, and this you can only access the 8 bytes of the CAN message.
If you want to react on diagnostic messages you should use
on diagResponse <serviceName>
inside of this handler you can then access the complete data of the diagnostic message
I had a similar problem accessing j1939 PGN with data length code (DLC) > 8 byte. These messages were transmitted as a j1939 Frame (DLC > 8 byte) instead of a CAN frame (DLC = 8 byte) in the trace window. I had to make use of the getThisMessage(pg pg_variable, int length) function in an on pg event like this.
on pg UPA_to_DTOOL {
pg UPA_to_DTOOL UPA_to_DTOOL_pg;
getThisMessage(UPA_to_DTOOL, UPA_to_DTOOL.dlc);
write("byte 9 = %X", UPA_to_DTOOL.byte(9));
}
Because messages with DLC > 8 are transmitted in a special way, the getThisMessage had to be used in my case, which let me access all the message bytes. I am not sure this solution for j1939 PGNs helps you because I do not know whether you have a license for j1939 in your canoe installation.

Heap corruption detected - iPhone 5S only

I am developing an app that listens for frequency/pitches, it works fine on iPhone4s, simulator and others but not iPhone 5S. This is the message I am getting:
malloc: *** error for object 0x178203a00: Heap corruption detected, free list canary is damaged
Any suggestion where should I start to dig into?
Thanks!
The iPhone 5s has an arm64/64-bit CPU. Check all the analyze compiler warnings for trying to store 64-bit pointers (and other values) into 32-bit C data types.
Also make sure all your audio code parameter passing, object messaging, and manual memory management code is thread safe, and meets all real-time requirements.
In case it helps anyone, I had exactly the same problem as described above.
The cause in my particular case was pthread_create(pthread_t* thread, ...) on ARM64 was putting the value into *thread at some time AFTER the thread was started. On OSX, ARM32 and on the simulator, it was consistently filling in this value before the start_routine was called.
If I performed a pthread_detach operation in the running thread before that value was written (even by using pthread_self() to get the current thread_t), I would end up with the heap corruption message.
I added a small loop in my thread dispatcher that waited until that value was filled in -- after which the heap errors went away. Don't forget 'volatile'!
Restructuring the code might be a better way to fix this -- it depends on your situation. (I noticed this in a unit test that I'd written, I didn't trip up on this issue on any 'real' code)
Same problem. but my case is I malloc 10Byte memory, but I try to use 20Byte. then it Heap corruption.
## -64,7 +64,7 ## char* bytesToHex(char* buf, int size) {
* be converted to two hex characters, also add an extra space for the terminating
* null byte.
* [size] is the size of the buf array */
- int len = (size * 2) + 1;
+ int len = (size * 3) + 1;
char* output = (char*)malloc(len * sizeof(char));
memset(output, 0, len);
/* pointer to the first item (0 index) of the output array */
char *ptr = &output[0];
int i;
for (i = 0; i < size; i++) {
/* "sprintf" converts each byte in the "buf" array into a 2 hex string
* characters appended with a null byte, for example 10 => "0A\0".
*
* This string would then be added to the output array starting from the
* position pointed at by "ptr". For example if "ptr" is pointing at the 0
* index then "0A\0" would be written as output[0] = '0', output[1] = 'A' and
* output[2] = '\0'.
*
* "sprintf" returns the number of chars in its output excluding the null
* byte, in our case this would be 2. So we move the "ptr" location two
* steps ahead so that the next hex string would be written at the new
* location, overriding the null byte from the previous hex string.
*
* We don't need to add a terminating null byte because it's been already
* added for us from the last hex string. */
ptr += sprintf(ptr, "%02X ", buf[i] & 0xFF);
}
return output;

Is it possible to have zlib read from and write to the same memory buffer?

I have a character buffer that I would like to compress in place. Right now I have it set up so there are two buffers and zlib's deflate reads from the input buffer and writes to the output buffer. Then I have to change the input buffer pointer to point to the output buffer and free the old input buffer. This seems like an unnecessary amount of allocation. Since zlib is compressing, the next_out pointer should always lag behind the next_in pointer. Anyway, I can't find enough documentation to verify this and was hoping someone had some experience with this. Thanks for your time!
It can be done, with some care. The routine below does it. Not all data is compressible, so you have to handle the case where the output data catches up with the input data. It takes a lot of incompressible data, but it can happen (see comments in code), in which case you have to allocate a buffer to temporarily hold the remaining input.
/* Compress buf[0..len-1] in place into buf[0..*max-1]. *max must be greater
than or equal to len. Return Z_OK on success, Z_BUF_ERROR if *max is not
enough output space, Z_MEM_ERROR if there is not enough memory, or
Z_STREAM_ERROR if *strm is corrupted (e.g. if it wasn't initialized or if it
was inadvertently written over). If Z_OK is returned, *max is set to the
actual size of the output. If Z_BUF_ERROR is returned, then *max is
unchanged and buf[] is filled with *max bytes of uncompressed data (which is
not all of it, but as much as would fit).
Incompressible data will require more output space than len, so max should
be sufficiently greater than len to handle that case in order to avoid a
Z_BUF_ERROR. To assure that there is enough output space, max should be
greater than or equal to the result of deflateBound(strm, len).
strm is a deflate stream structure that has already been successfully
initialized by deflateInit() or deflateInit2(). That structure can be
reused across multiple calls to deflate_inplace(). This avoids unnecessary
memory allocations and deallocations from the repeated use of deflateInit()
and deflateEnd(). */
int deflate_inplace(z_stream *strm, unsigned char *buf, unsigned len,
unsigned *max)
{
int ret; /* return code from deflate functions */
unsigned have; /* number of bytes in temp[] */
unsigned char *hold; /* allocated buffer to hold input data */
unsigned char temp[11]; /* must be large enough to hold zlib or gzip
header (if any) and one more byte -- 11
works for the worst case here, but if gzip
encoding is used and a deflateSetHeader()
call is inserted in this code after the
deflateReset(), then the 11 needs to be
increased to accomodate the resulting gzip
header size plus one */
/* initialize deflate stream and point to the input data */
ret = deflateReset(strm);
if (ret != Z_OK)
return ret;
strm->next_in = buf;
strm->avail_in = len;
/* kick start the process with a temporary output buffer -- this allows
deflate to consume a large chunk of input data in order to make room for
output data there */
if (*max < len)
*max = len;
strm->next_out = temp;
strm->avail_out = sizeof(temp) > *max ? *max : sizeof(temp);
ret = deflate(strm, Z_FINISH);
if (ret == Z_STREAM_ERROR)
return ret;
/* if we can, copy the temporary output data to the consumed portion of the
input buffer, and then continue to write up to the start of the consumed
input for as long as possible */
have = strm->next_out - temp;
if (have <= (strm->avail_in ? len - strm->avail_in : *max)) {
memcpy(buf, temp, have);
strm->next_out = buf + have;
have = 0;
while (ret == Z_OK) {
strm->avail_out = strm->avail_in ? strm->next_in - strm->next_out :
(buf + *max) - strm->next_out;
ret = deflate(strm, Z_FINISH);
}
if (ret != Z_BUF_ERROR || strm->avail_in == 0) {
*max = strm->next_out - buf;
return ret == Z_STREAM_END ? Z_OK : ret;
}
}
/* the output caught up with the input due to insufficiently compressible
data -- copy the remaining input data into an allocated buffer and
complete the compression from there to the now empty input buffer (this
will only occur for long incompressible streams, more than ~20 MB for
the default deflate memLevel of 8, or when *max is too small and less
than the length of the header plus one byte) */
hold = strm->zalloc(strm->opaque, strm->avail_in, 1);
if (hold == Z_NULL)
return Z_MEM_ERROR;
memcpy(hold, strm->next_in, strm->avail_in);
strm->next_in = hold;
if (have) {
memcpy(buf, temp, have);
strm->next_out = buf + have;
}
strm->avail_out = (buf + *max) - strm->next_out;
ret = deflate(strm, Z_FINISH);
strm->zfree(strm->opaque, hold);
*max = strm->next_out - buf;
return ret == Z_OK ? Z_BUF_ERROR : (ret == Z_STREAM_END ? Z_OK : ret);
}

Resources