I'm on my project about rtsp streams, merging and stuff and one of things I need to do is to pass frames from rtsp stream to a handler or buffer or something so another person could process it with f.e. OpenCV.
how could I do it with gstreamer?
Thanks!
GStreamer already has opencv based plugins (. So the best way is to write a similar plugin that applys your opencv code. There are elements called appsrc and appsink to source data from ann app or receive data by an app, but there is not appfilter element. One could use a pad-probe for it, but it is not a good approach.
Related
I need to build a web application which creates video call (point to point and conference call) via web RTC. While the video is being live streamed in real time, I've to create a queue of frames and feed it to an algorithm. I plan to use Open CV for this purpose. Is this approach fine or are there better ways to accomplish this?
opencv can't stream out anything on its own.
you'll probably have to dive into ffmpeg / gstreamer for this.
I have a .swf file, which i want to embed in my opencv and overlay over camera stream and display it to the user. Until now i have not found a solution by simple google search. I would appreciate if anyone has any idea how to approach this.
Thanks
OpenCV doesn't deal with .swf files, so you need to use some other technology like FFMPEG or GStreamer to retrieve the frames and decode them to BGR to be able to create a valid IplImage (or cv::Mat if you are insterested in the C++ interface).
GStreamer also provides a simple mechanism to stream video over the network.
I need some ideas about how to stream video feed coming from opencv to a webpage. I currently have gStreamer, but I don't know if this is the right tool for the job. Any advice on using gStreamer or any hyperlinks to tutorials would be helpful and appreciated!
Thanks!
OpenCV doesn't provide an interface for streaming video, which means that you'll need to use some other techonology for this purpose.
I've used GStreamer in several professional projects: this is the droid you are looking for.
I do not have any experience w/ streaming OpenCV output to a website. However I'm sure this is possible using gstreamer.
Using a gstreamer stream, it is possible to get data and convert the data in to OpenCV format. I recommend you read up on GstAppSink and GstBuffer.
Basically, if I remember correctly, you must run a pipeline in the a background thread. Then using some function in gst_app_sink, you can get the buffer data from the sink.
A quick lookup on the issue, you had to use GST_BUFFER_DATA for this
I remember having to convert the result from yCBCr to bgr, a collegue had problems as the conversion of opencv was inadequate. So you might have to write your own. (This was back in the IplImage* days)
I try to record a Video with the OpenCV Framework an would like to save that into an Matroska(mkv) Container together with some additional data streams.
First I thought using FFmpeg is the way that.
But while looking into the OpenCV Sourcecode and searching in the web I found GStreamer.
Because the documentation in GStreamer is much better than the FFmpeg documentation I would prefer using this Framework.
In my understanding GStreamer is primarily used for Streaming, but could also rncode and mux video data.
Is there any disadvantage when using GStreamer instead of FFmpeg?
Thanks in advance
Horst
I try to record a Video with the OpenCV Framework an would like to save that into an Matroska(mkv)
I don't think OpenCV can store video as MKV,
together with some additional data streams.
OpenCV doesn't provide features for this operation.
An easy workaround is to simply call ffmpeg's or gstreamer's cmd-line application to do the conversion for you.
GStreamer has indeed a decent documentation and it can also do the job. The obvious disadvantage is that if you know how to work with FFmpeg, changing to GStreamer will require some extra time to understand how it works since both have completely different APIs: GStreamer architecture was inspired by DirectShow and Quicktime.
The advantage is that GStreamer (besides being cross-platform as well) is used on several big projects and getting to know GStreamer will certainly add a great skill to your programming arsenal.
You can use an OpenCv VideoWriter using either ffmpeg or gstreamer backend and compare. For example (you may adapt to your platform plugins and video mode):
# Using ffmpeg backend
cv::VideoWriter ffmpeg_h264_writer ("test-ffmpeg-h264-writer.mkv",
cv::CAP_FFMPEG,
cv::VideoWriter::fourcc ('X', '2', '6', '4'),
fps,
cv::Size (width, height));
# Using gstreamer backend:
cv::VideoWriter gst_omxh264_writer ("appsrc ! queue ! videoconvert ! video/x-raw,format=I420 ! queue ! omxh264enc ! video/x-h264,format=byte-stream ! matroskamux ! filesink location=test-gstreamer-omxh264-writer.mkv ",
cv::CAP_GSTREAMER,
0,
fps,
cv::Size (width, height));
where width and height are integer values and fps is a float value.
I see no disadvantage using gstreamer. In fact the way I see is gstreamer is a framework not just as a tool or a single purpose library. You can use it to develop your own plugins that can seamlessly hook into gstreamer pipeline.
I was planning to use the vlc library to decode an H.264 based RTSP stream and extract each frame from it (convert vlc picture to IplImage). I have done a bit of exploration of the vlc code and concluded that there is a function called libvlc_video_take_snapshot which does a similar thing. However the captured frame in this case is saved on the hard disk which I wish to avoid due to the real time nature of my application. What would be the best way to do this? Would it be possible without modifying the vlc source (I want to avoid recompilation if possible). I have heard of vmem etc but could not really figure out what it does and how to use it.
The picture_t structure is internal to the library, how can we get an access to the same.
Awaiting your response.
P.S. Earlier I tried doing this using FFMPEG, however the ffmpeg library has a lot of issues while decoding an H.264 based RTSP stream on windows and hence I had to switch to VLC.
Regards,
Saurabh Gandhi