I'm trying to open a video stream in opencv but I'm having some difficulties. I can start a stream with :
gst-launch -v v4l2src device=/dev/video0 ! 'video/x-raw-yuv,width=640,height=480' ! jpegenc quality=30 ! rtpjpegpay ! udpsink host=127.0.0.1 port=1234
`
and I can open it with:
gst-launch udpsrc port=1234 ! "application/x-rtp, payload=127" ! rtpjpegdepay ! jpegdec ! xvimagesink sync=false
But when I tried to open it in my code with
VideoCapture cv_cap;
cv_cap.open("rtp:127.0.0.1:1234/");
I get an error about a missing SDP files. I know what an SDP file is and that I should get the info for it from the gstreamer output but I don't understand exactly how to parse the output.
Related
I am using below list of version.
opencv-4.2.0, Gstreamer-1.20.2, python 3.7, windows 10.
I want to play video using Gstreamer rtsp camera.why this gst-pipeline it's not run on vscode python but these gst-pipeline perfectly run on cmd.
Pipeline:----
rtspsrc location=rtsp://... ! rtph264depay ! queue ! h264parse ! d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory), format=(string)NV12 ! appsink
The error i am getting is as follows:-
[ WARN:0] global opencv-4.2.0\modules\videoio\src\cap_gstreamer.cpp (1759) cv::handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module udpsrc1 reported: Internal data stream error.
[ WARN:0] global opencv-4.2.0\modules\videoio\src\cap_gstreamer.cpp (888) cv::GStreamerCapture::open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global opencv-4.2.0\modules\videoio\src\cap_gstreamer.cpp (480) cv::GStreamerCapture::isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Can you please let me know.How to solve this issue??
You may try adding caps after udpsrc:
rtspsrc location=rtsp://... ! application/x-rtp,encoding-name=H264 ! rtph264depay ! ...
This was a stupid advice. Specifying these caps make sense with udpsrc, but are not required for rtspsrc.
Re-reading it now, the issue is probably the memory space. Opencv may only expect system memory for now, so you may try:
rtspsrc location=rtsp://... ! rtph264depay ! queue ! h264parse ! d3d11h264dec ! d3d11convert ! video/x-raw ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1
Here converting to BGR as most opencv color algorithms expect, but if you intend to process NV12 frames, just change format (not sure videoconvert is required, but if not the overhead may be low).
Hardware:
Apalis IMX8 CPU(SOM)
and
Sensoray model-1012 video frame grabber
I am trying to save analog video with h264 coding and play it.
The code has 3 parts. Reading camera, saving video with coding and playing video with decoding.
The problem i am facing is on decoding part. When i decode and show video it corrupts.
How i read analog video(Works fine):
cap = cv::VideoCapture(" v4l2src device=/dev/video4 ! video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=30/1, interlace-mode=interleaved ! deinterlace fields=1 method=2 ! videoconvert ! appsink ",cv::CAP_GSTREAMER);
How i compress and save video(Works fine):
cv::VideoWriter fixedVideo;
QString pipeTmp = "appsrc ! videoconvert ! v4l2h264enc ! h264parse ! qtmux ! filesink location="+ FixedIMG_recordName +" sync=false ";
std::string pipe = pipeTmp.toUtf8().constData();
isOpen = fixedVideo.open(pipe , cv::CAP_GSTREAMER, (double)30, cv::Size(720,480), true);
How i decode and open video(DOESNT Work Fine):
cv::VideoCapture cap_reader;
QString pipeTmp = " filesrc location=" + device + " ! qtdemux ! h264parse ! video/x-h264, width=720, height=480 ! v4l2h264dec ! videoconvert ! appsink ";
std::string pipe = pipeTmp.toUtf8().constData();
cap_reader.open(pipe , cv::CAP_GSTREAMER);
when i open cap_read pipe line to play same video from gstreamer command line pipe it works fine with some warnings. I put the GST_DEBUG output in log.txt Pipe line:
GST_DEBUG=3 gst-launch-1.0 -v filesrc location=test.mp4 ! qtdemux ! h264parse ! 'video/x-h264, width=720, height=480, framerate=30/1' ! v4l2h264dec ! videoconvert ! autovideosink
When i open the video from VLC it also works fine. But when i open the video from VideoCapture and the pipeline that i gave above it corrupts. The corrupted image example.
There is no problem in compression. When you decode mp4 file, you should use "imxvideoconvert_g2d".
Decode pipeline should be filesrc location=" + device + " ! qtdemux ! h264parse ! video/x-h264, width=720, height=480 ! v4l2h264dec ! imxvideoconverter_g2d ! video/x-raw,format=UYVY,width=720,height=480 ! videoconvert ! appsink
I'm trying to combine two RTSP streams using gst-launch-1.0 but I'm already stuck already at trying to record/play one RTSP stream. The stream contains both audio and video.
The command line I'm using is:
gst-launch-1.0 ^
rtspsrc location=<location-omitted> protocols=GST_RTSP_LOWER_TRANS_TCP latency=5000 name=s_0 ^
s_0. ! application/x-rtp,media=audio ! rtpjitterbuffer ! decodebin ! audioconvert ! autoaudiosink ^
s_0. ! application/x-rtp,media=video ! rtpjitterbuffer ! decodebin ! videoconvert ! autovideosink
If I change the autoaudiosink to fakesink, the video plays. If I remove the video sink (didn't test with fakesink), the audio plays. But if I add both (like above), the video shows 1 frame and then freezes. [not sure if it matters, but I am on Windows]
I actually suspect that (for some reason) the pipeline goes on pause, but I'm at a loss on how to debug this issue.
I've tried with/without rtpjitterbuffer, I've tried with/without queue's at various places. I've tried with combinations of rtp...depay/parse. Although I've tried so much in the past few hours (ohoh) that I'm not sure if I overlooked anything.
Also tried with various options for rtspsrc (sync/etc).
But the end result is pretty much always the same, playing either audio or video works fine, playing both at the same time (or muxing them into a single file) fails.
Writing each to their own file works fine, for example:
gst-launch-1.0 ^
rtspsrc location=<location-omitted> protocols=GST_RTSP_LOWER_TRANS_TCP latency=5000 name=s_0 ^
s_0. ! application/x-rtp,media=audio ! rtpjitterbuffer ! decodebin ! audioconvert ! avenc_aac ! flvmux ! filesink location=audio.flv ^
s_0. ! application/x-rtp,media=video ! rtpjitterbuffer ! decodebin ! videoconvert ! x264enc ! flvmux ! filesink location=video.flv
If I mux the above into one file the same thing happens as with trying to play the file (using the auto-sink), the output file stays at 0-bytes.
The output of gst-launch-1.0 when trying to play is the same as when writing to two files:
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to <location-omitted>
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Redistribute latency...
Redistribute latency...
Somehow I think it's some type of sync-issue, but I can't seem to figure out how to solve it.
Since this question is more than 1 year ago, I hope someone with same problem still need the help (as I do, I used many hours to figure this out, and thank you to the poster he gives me the hints)
You need to edit as this
gst-launch-1.0 ^
rtspsrc location=<location-omitted> protocols=GST_RTSP_LOWER_TRANS_TCP latency=5000 name=s_0 ^
s_0. ! application/x-rtp,media=audio ! rtpjitterbuffer ! decodebin ! audioconvert ! avenc_aac ! flvmux name=mux ^
s_0. ! application/x-rtp,media=video ! rtpjitterbuffer ! decodebin ! videoconvert ! x264enc ! mux. ^
mux. ! filesink location=video.flv
I'm trying to open an IP camera in OpenCV using gstreamer pipleine.
I can open the IPcamera using Gstreamer in terminal, using :
gst-launch-1.0 -v rtspsrc location="rtsp://192.168.0.220:554/user=admin&password=admin&channel=1&stream=0.sdp?real_stream--rtp-caching=10" latency=10 ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! xvimagesink
Now with this how can I open the same camera in OpenCV videoCapture().
Any help is appreciated.
You can copy the same pipe and use it in VideoCapture (if you built OpenCV with gstreamer modules).
Important point is you need to finish the pipe with an appsink element.
const char* pipe = "rtspsrc location=\"rtsp://192.168.0.220:554/user=admin&password=admin&channel=1&stream=0.sdp?real_stream--rtp-caching=10\" latency=10 ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! appsink";
VideoCapture cap(pipe);
I use following 2 commands to stream video from Raspberry Pi
RaPi
raspivid -t 999999 -h 720 -w 1080 -fps 25 -hf -b 2000000 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=$RA-IP-ADDR port=5000
Linux Box
gst-launch-1.0 -v tcpclientsrc host=$RA-IP-ADDR port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false
But what kind of stream is it? Can I read it with OpenCV? or convert with avconv|ffmpeg nc $RA-IP-ADDR 5000 | avconv? or watch with VLC ?
The stream appears to be an RTP stream encapsulated in a GDP stream, the latter of which which appears to be proprietary to GStreamer. You might be able to remove the gdppay and gdpdepay elements from your pipeline and use other RTP tools (there are plenty out there; I believe VLC supports RTP directly), but you could also use a GStreamer pipeline to pipe the depayloaded GDP stream (in this case, the H.264 stream it contains) from the RPi to a file on the Linux Box side, like so:
gst-launch-1.0 tcpclientsrc host=$RA-IP-ADDR port=5000 ! gdpdepay ! rtph264depay ! filesink location=$FILENAME
or, to pipe it to stdout:
gst-launch-1.0 tcpclientsrc host=$RA-IP-ADDR port=5000 ! gdpdepay ! rtph264depay ! fdsink
One or the other of these should let you operate on the H.264 video at a stream level.
GStreamer 1.0 can also interact with libav more or less directly if you have the right plugin. Use gst-inspect-1.0 libav to see the elements supported. The avdec_h264 element already in your pipeline is one of these libav elements.