Force GStreamer for IOS to use ffmpeg library outside of the framework - ios

Gstreamer 1.0 for IOS is delivered in a static framework, the source to build the framework is around 1.2g , this framework is huge and tries to provide for any decoding server scenario you may have. Trouble is it tries to do to much and IMHO not enough thought was put into the IOS port.
Here's the problem we have an application that uses the GSTreamer avdec_h264 plugin for displaying an RTP over UDP stream . This works rather well. recently we were required to do some special recording functions so we introduced an api that had its own version of ffmpeg. Gstreamer has Libav compiled into the framework. When we place our api into the application with the gst_IOS_RESTICTED_PLUGINS disabled the code runs fine when we introduce the GStreamer.framework into the application code similar to that shown below fails with a protocol not found error.
The problem is that the internal version of libav seems to disable all the protocols that ffmpeg supplies. because GSTreamer uses its own custom AVIO callback based on ffmpeg pipe protocol.
According to Gstreamer support that has been somewhat helpful
) Add a new recipe with the libav version you want to use and disable the build of the internal libav in gst-libav-1.0 with:
configure_options = '--with-system-libav'
You might need to comment out this part to prevent libav being packaged in the framewiork or make sure that your libav recipe creates these files in the correct place to include them in the framework:
42 for f in ['libavcodec', 'libavformat', 'libavutil', 'libswscale']:
43 for ext in ['.a', '.la']:
44 path = os.path.join('lib', f + ext)
45 self.files_plugins_codecs_restricted_devel.append(path)
2) Update libav submodule gst-libav to use the correct version you need.
https://bugs.freedesktop.org/show_bug.cgi?id=77399
The first method didn't work , the recipe kept getting overwritten even after applying a patch for a bug fix that was made as result of this bug report.
And I have no idea how to do the second method. Which is what I'd like some help with.
Has anyone with GStreamer 1.0 for iOS
1) Built the get-libav plugin against an external to to the framework set of ffmpeg static libs (.a)
2) built the internal libav to allow for RTP , UDP and TCP protocols, or written a custom AVIO callback using the FFPipe protocol.
3) just managed to somehow get the below code working with GStreamer.
I don't ask many questions, I've kind of implemented all kinds of encoders/decoders using ffmpeg , lib555 and a few hardware decoders. But this GStreamer issue is causing me more sleepless nights than I've had in a long time.
AVFormatContext * avctx;
avctx = avformat_alloc_context();
av_register_all();
avformat_network_init();
avcodec_register_all();
avdevice_register_all();
// Set the RTSP Options
AVDictionary *opts = 0;
av_dict_set(&opts, "rtsp_transport", "udp", 0);
int err = 0;
err = avformat_open_input(&avctx, "rtsp://184.72.239.149/vod/mp4:BigBuckBunny_115k.mov", NULL, &opts);
av_dict_free(&opts);
if (err) {
NSLog(#"Error: Could not open stream: %d", err);
char errbuf[400];
av_strerror(err,errbuf,400);
NSLog(#"%s failed with error %s","avformat_open_input",errbuf);
}
else {
NSLog(#"Opened stream");
}
err = avformat_find_stream_info(avctx, NULL);
if( err < 0)
{
char errbuf[400];
av_strerror(err,errbuf,400);
NSLog(#"%s failed with error %s","avformat_find_stream_info",errbuf);
return ;
}else {
NSLog(#"found stream info");
}
}

Related

iOS ffmpeg how to run a command to trim remote url video?

I was initially using the AVFoundation libraries to trim video but it has a limitation that it can't do it for remote URLs and only works for local URLs.
So after further research I found ffmpeg library which can be included in a Xcode project for iOS.
I have tested the following commands to trim a remote video on command line:
ffmpeg -y -ss 00:00:01.000 -i "http://i.imgur.com/gQghRNd.mp4" -t 00:00:02.000 -async 1 cut.mp4
which will trim the .mp4 from 1 second to 3 second mark. This works perfectly via command line on my mac.
I have been successfully able to compile and include ffmpeg library into a xcode project but not sure how to proceed further.
Now I am trying to figure out how to run this command on an iOS app using the ffmpeg libraries. How can I do this?
If you can point me to some helpful direction, I would really appreciate it! If I can get it resolved using your solution, I will award a bounty (in 2 days when it gives me the option).
I have some idea about this. However, I have very limited exp on iOS and not sure whether my thought is the best way:
As far as I know, generally it is impossible to run the cmd tools on iOS. Maybe you have to write some code linked to ffmpeg libs.
Here's all the jobs needed to do:
Open input file and init some ffmpeg context.
Get the video stream and seek to the timestamp you want. This may be complicated. See ffmpeg tutorial for some help, or check this to seek precisely and dealing with the troublesome key frames.
Decode some frames. Until the frame match the end timestamp.
Meanwhile with above, encode the frames to a new file as output.
The examples in ffmpeg source is very good to learn how to do this.
Some maybe useful codes:
av_register_all();
avformat_network_init();
AVFormatContext* fmt_ctx;
avformat_open_input(&fmt_ctx, "http://i.imgur.com/gQghRNd.mp4", NULL, NULL);
avformat_find_stream_info(fmt_ctx, NULL);
AVCodec* dec;
int video_stream_index = av_find_best_stream(fmt_ctx, AVMEDIA_TYPE_VIDEO, -1, -1, &dec, 0);
AVCodecContext* dec_ctx = avcodec_alloc_context3(NULL);
avcodec_parameters_to_context(dec_ctx, fmt_ctx->streams[video_stream_index]->codecpar)
// If there is audio you need, it should be decoded/encoded too.
avcodec_open2(dec_ctx, dec, NULL);
// decode initiation done
av_seek_frame(fmt_ctx, video_stream_index, frame_target, AVSEEK_FLAG_FRAME);
// or av_seek_frame(fmt_ctx, video_stream_index, timestamp_target, AVSEEK_FLAG_ANY)
// and for most time, maybe you need AVSEEK_FLAG_BACKWARD, and skipping some following frames too.
AVPacket packet;
AVFrame* frame = av_frame_alloc();
int got_frame, frame_decoded;
while (av_read_frame(fmt_ctx, &packet) >= 0 && frame_decoded < second_needed * fps) {
if (packet.stream_index == video_stream_index) {
got_frame = 0;
ret = avcodec_decode_video2(dec_ctx, frame, &got_frame, &packet);
// This is old ffmpeg decode/encode API, will be deprecated later, but still working now.
if (got_frame) {
// encode frame here
}
}
}

OpenCV error loading video file on different machine

I have developed an application using wxWidgets and OpenCV 2.4.8. I am simply loading avi files from the disk using VideoCapture. The application works perfectly fine on the machine it is developed on. But when run on a different machine, cap.open(fileName) return false as seen in the code snippet below. Does anyone have an idea what might be the problem here?
bool Data::loadVideoFile(const char *fileName)
{
VideoCapture cap;
if ( !cap.open(fileName) ) // if not success, return false
{
return false;
}
else
{
return true;
}
}
I assume that your paths are correct.
You need to make sure that the machine you are running on has the proper codecs and DLLs.
If you do not have the opencv_ffmpeg DLLs, then Highgui falls back on DirectX or VfW codecs. If, in that case, you do not have the proper CODECs, then opening the file will fail.
This is unlike the other OpenCV DLLs which generate a missing DLL error when they are inaccessible.

Error while capturing video via RTSP from a network camera using OpenCV and FFMPEG

I am using OpenCV and FFMPEG to capture frames from a network camera using RTSP. The point is that OpenCV successfully loads the FFMPEG .dll but icvCreateFileCapture_FFMPEG_p returns false in the following code of cap_ffmpeg.cpp:
virtual bool open( const char* filename )
{
close();
icvInitFFMPEG();
if( !icvCreateFileCapture_FFMPEG_p )
return false;
ffmpegCapture = icvCreateFileCapture_FFMPEG_p( filename );
return ffmpegCapture != 0;
}
Probably the stream is not ready or you have problems with the network address/access.
Check this if you have followed the correct way of doing it. Try pining the network resource first and see if it is available or not. The camera also must allow un-authenticated access, set via its web interface. Sometimes MJPEG works and MPEG4 has problems.

alternative to cvWaitKey() function in iOS

ok, what i am trying to do is retrieve a frame from an existing video file, do some work on the frame and then save it to a new file,
what actually happens is that it writes some frames and then crashes as the code is quite fast,
if i don't put cvWaitKey() i get the same error i get when writing video frames with AVFoundation Library without using
AVAssetWriterInput.readyForMoreMediaData
OpenCV video writer is implemented using AVFoundation classes but we lose access to
AVAssetWriterInput.readyForMoreMediaData
or am i missing something ?
here is the code similar to what i'm trying to do,
while (grabResult&&frameResult) {
grabResult = cvGrabFrame(capture); // capture a frame
if(grabResult){
img = cvRetrieveFrame(capture, 0); // retrieve the captured frame
cvFlip(img,NULL,0); // edit img
frameResult = cvWriteFrame(writer,img); // add the frame to the file
cvWaitKey(-1); or anything that helps to finish adding the previous frame
}
}
I am trying to convert a video file using OpenCV (without displaying)
in my iPhone/iPad app, everything works except cvWaitKey() function
and I get this error:
OpenCV Error: Unspecified error (The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script) in cvWaitKey,
Without this function frames are dropped as there's no way to know if
the video writer is ready, is there an alternative to my problem?
I am using OpenCV 2.4.2 and I get same error with the latest
precompiled version of OpenCV.
Repaint the UIImageView:
[[NSRunLoop currentRunLoop]runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.0f]];
followed by your imshow or cvShowImage
I don't know anything about iOS development, but did you try calling the C++ interface method waitKey()?

OpenCV and iPhone

I am writing an application to create a movie file from a bunch of images on an iPhone. I am using OpenCv. I downloaded OpenCv static libraries for ARM(iPhone's native instruction architecture) and the libraries were generated just fine. There were no problems linking to them libraries.
As a first step, I was trying to create a .avi file using one image, to see if it works. But cvCreateVideoWriter always returns me a NULL value. I did some searching and I believe its due to the codec not being present. I am trying this on the iPhone simulator. This is what i do:
- (void)viewDidLoad {
[super viewDidLoad];
UIImage *anImage = [UIImage imageNamed:#"1.jpg"];
IplImage *img_color = [self CreateIplImageFromUIImage:anImage];
//The image gets created just fine
CvVideoWriter *writer =
cvCreateVideoWriter("out.avi",CV_FOURCC('P','I','M','1'),
25,cvSize(320,480),1);
//writer is always null
int result = cvWriteFrame(writer, img_color);
NSLog(#"\n%d",result);
//hence this is also 0 all the time
cvReleaseVideoWriter(&writer);
}
I am not sure about the second parameter. What sort of codec or what exactly does it do...
I am a n00B at this. Any suggestions?
On *nix flavors, OpenCV uses ffmpeg under the covers to encode video files, so you need to make sure your static libraries are built with ffmpeg support. The second parameter, CV_FOURCC('P','I','M','1'), is the FOURCC code describing the video format/codec you are requesting, in this case the MPEG1 codec. Check out fourcc.org for a complete listing (not all of which work in ffmpeg).

Resources