H264 Video Streaming over RTMP on iOS - ios

Question

With a bit of digging, I have found a library that extracts NAL units from .mp4 file while it is being written. I'm attempting to packetize this information to flv over RTMP using libavformat and libavcodec. I setup a video stream using:
-(void)setupVideoStream {
int ret = 0;
videoCodec = avcodec_find_decoder(STREAM_VIDEO_CODEC);
if (videoCodec == nil) {
NSLog(#"Could not find encoder %i", STREAM_VIDEO_CODEC);
return;
}
videoStream = avformat_new_stream(oc, videoCodec);
videoCodecContext = videoStream->codec;
videoCodecContext->codec_type = AVMEDIA_TYPE_VIDEO;
videoCodecContext->codec_id = STREAM_VIDEO_CODEC;
videoCodecContext->pix_fmt = AV_PIX_FMT_YUV420P;
videoCodecContext->profile = FF_PROFILE_H264_BASELINE;
videoCodecContext->bit_rate = 512000;
videoCodecContext->bit_rate_tolerance = 0;
videoCodecContext->width = STREAM_WIDTH;
videoCodecContext->height = STREAM_HEIGHT;
videoCodecContext->time_base.den = STREAM_TIME_BASE;
videoCodecContext->time_base.num = 1;
videoCodecContext->gop_size = STREAM_GOP;
videoCodecContext->has_b_frames = 0;
videoCodecContext->ticks_per_frame = 2;
videoCodecContext->qcompress = 0.6;
videoCodecContext->qmax = 51;
videoCodecContext->qmin = 10;
videoCodecContext->max_qdiff = 4;
videoCodecContext->i_quant_factor = 0.71;
if (oc->oformat->flags & AVFMT_GLOBALHEADER)
videoCodecContext->flags |= CODEC_FLAG_GLOBAL_HEADER;
videoCodecContext->extradata = avcCHeader;
videoCodecContext->extradata_size = avcCHeaderSize;
ret = avcodec_open2(videoStream->codec, videoCodec, NULL);
if (ret < 0)
NSLog(#"Could not open codec!");
}
Then I connect, and each time the library extracts a NALU, it returns an array holding one or two NALUs to my RTMPClient. The method that handles the actual streaming looks like this:
-(void)writeNALUToStream:(NSArray*)data time:(double)pts {
int ret = 0;
uint8_t *buffer = NULL;
int bufferSize = 0;
// Number of NALUs within the data array
int numNALUs = [data count];
// First NALU
NSData *fNALU = [data objectAtIndex:0];
int fLen = [fNALU length];
// If there is more than one NALU...
if (numNALUs > 1) {
// Second NALU
NSData *sNALU = [data objectAtIndex:1];
int sLen = [sNALU length];
// Allocate a buffer the size of first data and second data
buffer = av_malloc(fLen + sLen);
// Copy the first data bytes of fLen into the buffer
memcpy(buffer, [fNALU bytes], fLen);
// Copy the second data bytes of sLen into the buffer + fLen + 1
memcpy(buffer + fLen + 1, [sNALU bytes], sLen);
// Update the size of the buffer
bufferSize = fLen + sLen;
}else {
// Allocate a buffer the size of first data
buffer = av_malloc(fLen);
// Copy the first data bytes of fLen into the buffer
memcpy(buffer, [fNALU bytes], fLen);
// Update the size of the buffer
bufferSize = fLen;
}
// Initialize the packet
av_init_packet(&pkt);
//av_packet_from_data(&pkt, buffer, bufferSize);
// Set the packet data to the buffer
pkt.data = buffer;
pkt.size = bufferSize;
pkt.pts = pts;
// Stream index 0 is the video stream
pkt.stream_index = 0;
// Add a key frame flag every 15 frames
if ((processedFrames % 15) == 0)
pkt.flags |= AV_PKT_FLAG_KEY;
// Write the frame to the stream
ret = av_interleaved_write_frame(oc, &pkt);
if (ret < 0)
NSLog(#"Error writing frame %i to stream", processedFrames);
else {
// Update the number of frames successfully streamed
frameCount++;
// Update the number of bytes successfully sent
bytesSent += pkt.size;
}
// Update the number of frames processed
processedFrames++;
// Update the number of bytes processed
processedBytes += pkt.size;
free((uint8_t*)buffer);
// Free the packet
av_free_packet(&pkt);
}
After about 100 or so frames, I get an error:
malloc: *** error for object 0xe5bfa0: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
Which I cannot seem to stop from happening. I've tried commenting out the av_free_packet() method and the free() along with trying to use av_packet_from_data() rather than initializing the packet and setting the data and size values.
My question is; how can I stop this error from happening, and according to wireshark, these are proper RTMP h264 packets, but they do not play anything more than a black screen. Is there some glaring error that I am overlooking?

Answer 1

It looks to me like you are overflowing your buffer and corrupting you stream here:
memcpy(buffer + fLen + 1, [sNALU bytes], sLen);
You are allocating fLen + sLen bytes then writing fLen + sLen + 1 bytes. Just get rid of the + 1.
Because your AVPacket is allocated on the stack av_free_packet() is not needed.
Finally, it is considered good practice to to allocate extra bytes for libav. av_malloc(size + FF_INPUT_BUFFER_PADDING_SIZE )