I use some python code to decode H264 Video on NVIDIA JETSON NANO everything works fine.
I see some times that I get this warning:
[ WARN:0] global /home/ubuntu/opencv/modules/videoio/src/cap_gstreamer.cpp (961) open
OpenCV | GStreamer warning: Cannot query video position: status=1, value=253, duration=-1
I do not rely sure that this message is the problem but,
When I think to get this warning I get the performance of 7~7.5FPS, sometimes I don't get this warning, and my performance increase to 10.5FPS.
I would be happy to understand the problem I play some pcap video using colasoft player and capture the video in jetson using this python script.
class Video:
def __init__(self, urlName='udp://127.0.0.1:46002'):
print(f"Initialize Vieo, url: {urlName}")
self.pipeline = 'udpsrc port=46002 multicast-group=234.0.0.0 ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1'
# self.cap = cv2.VideoCapture(urlName, cv2.CAP_FFMPEG)
self.cap = cv2.VideoCapture(self.pipeline, cv2.CAP_GSTREAMER)
if not self.cap.isOpened():
print('VideoCapture not opened')
exit(-1)
def readVideo(self):
"""
This Function read h.264 stream and calculate image info
:return: image, data_image
"""
try:
ret, frame = self.cap.read()
while frame == None:
ret, frame = self.cap.read()
except Exception as e:
pass
shape = frame.shape
data = {"image_width": shape[1], "image_height": shape[0], "channels": shape[2], "size_in_bytes": shape[0] * shape[1] * shape[2], "bits_per_pixel": shape[2] * 8} # data Video
return frame, data
Related
I have a python script that receives jpeg encoded data using following pipeline and sends the jpeg data on a port.
rtspsrc location=rtsp://192.168.100.40:8554/ latency=0 retry=50 ! rtph265depay ! h265parse ! avdec_h265 ! videoscale ! videorate ! video/x-raw, framerate=10/1, width=1920, height=1080 ! jpegenc quality=85 ! image/jpeg ! appsink
At the receiver end I want to save the incoming data as a video, as described in this link
https://gstreamer.freedesktop.org/documentation/jpeg/jpegenc.html
gst-launch-1.0 videotestsrc num-buffers=50 ! video/x-raw, framerate='(fraction)'5/1 ! jpegenc ! avimux ! filesink location=mjpeg.avi
I have tried using opencv's VideoWriter with CAP_GSTREAMER
pipeline = f'appsrc ! avimux ! filesink location=recording.avi'
cap_write = cv2.VideoWriter(pipeline,cv2.CAP_GSTREAMER,0, 1, (1920,1080), False)
...
cap_write.write(jpgdata)
but it gives a runtime warning
[ WARN:0] global ../opencv/modules/videoio/src/cap_gstreamer.cpp (1948) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline
If I modify the pipeline and use
pipeline = f'appsrc ! videoconvert ! videorate ! video/x-raw, framerate=1/1 ! filesink location=recording.avi'
The code does save the incoming frames as video but the saved video is too big with no bitrate and duration information in it.
ffmpeg -i recording.avi
...
[mjpeg # 0x560f4a408600] Format mjpeg detected only with low score of 25, misdetection possible!
Input #0, mjpeg, from 'recording.avi':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 1920x1080 [SAR 1:1 DAR 16:9], 25 tbr, 1200k tbn, 25 tbc
At least one output file must be specified
Need help!
At this moment I have the VideoImageTrack class I show bellow (adapted from here) that returns an av VideoFrame. The script I show works fine. My problem is that the frame encoding step:
VideoFrame.from_ndarray(image, format="bgr24")
is quite slow.
Is there a gstreamer pipeline that outputs the already encoded frame and iterable with python-opencv read()?
class VideoImageTrack(VideoStreamTrack):
"""
A video stream track that returns a rotating image.
"""
def __init__(self):
super().__init__() # don't forget this!
self.video = cv2.VideoCapture("v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480,framerate=15/1,bitrate=250000 ! appsink")
async def recv(self):
pts, time_base = await self.next_timestamp()
retval, image = self.video.read()
frame = VideoFrame.from_ndarray(image, format="bgr24")
frame.pts = pts
frame.time_base = time_base
return frame
Although I don't know if processing will be faster but you can try to use GStreamer's videoconvert that transcodes video frames between many formats.
Example pipeline:
v4l2src device=/dev/video0 ! videoconvert ! video/x-raw,format=BGR,width=640,height=480,framerate=15/1 ! appsink
I have an RTP/RTSP stream that's running at 25fps, as verified by ffprobe -i <URI>. Also, VLC plays back the RTSP stream at a real-time rate, but doesn't show me the FPS in the Media Information window.
However, when I use OpenCV 4.1.1.26 to retrieve the input stream's frame rate, it is giving me a response of 90000.0.
Question: How can I use OpenCV to probe for the correct frame rate of the RTSP stream? What would cause it to report 90000.0 instead of 25?
Here's my Python function to retrieve the frame rate:
import cv2
vid : cv2.VideoCapture = cv2.VideoCapture('rtsp://192.168.1.10/cam1/mpeg4')
def get_framerate(video: cv2.VideoCapture):
fps = video.get(cv2.CAP_PROP_FPS)
print('FPS is {0}'.format(fps))
get_framerate(vid)
MacOS Catalina
Python 3.7.4
I hope this helps you somehow. It is a simple calculator that takes cont captures and measure the beginning and the ending time. Then with the rule of three, i converted it to fps.
Related to you second question i read here that it could be due to bad installation. Also, you can check that your camera is working properly by printing ret variable. If it is true then you should be able to see the fps, if it is false then you can have an unpredictable result.
cv2.imshow() and key = cv2.waitKey(1) should be commented as it adds ping/delay resulting in bad measurement.
I post this as a comment because i do not have enough reputation points.
img = cv2.VideoCapture('rtsp://192.168.1.10/cam1/mpeg4')
while True:
if cont == 50:
a = datetime.now() - start
b = (a.seconds * 10e6 + a.microseconds)
print((a.seconds * 10e6 + a.microseconds), "fps = ", (50 * 10e6)/ b)
break
ret, frame = img.read()
# Comment for best test
cv2.imshow('fer', frame)
key = cv2.waitKey(1)
if key == ord('q'):
break
cont+=1
img.release()
cv2.destroyAllWindows()`
Python 3.5.2, anaconda 4.2.0 on Windows 10.
OpenCV installed from conda, version 3.1.0.
I'm trying to process a video file by opening it, transforming each frame, and putting the result into a new video file. The output file is created, but the size is about 800 bytes and its empty. The input file has ~4,000 frames and it's about 150 MB.
Here's my code, which follows the guide on the OpenCV documentation pretty closely.
import cv2
import progressbar
# preprocess video
# args.input is a valid file name
outname = 'foo.mp4'
cap = cv2.VideoCapture(args.input)
codec = int(cap.get(cv2.CAP_PROP_FOURCC))
framerate = app_config.camera.framerate #240
size = (app_config.camera.width, app_config.camera.height) #1080 x 720
vw = cv2.VideoWriter(filename=outname, fourcc=codec, fps=framerate, frameSize=size, isColor=False)
curframe = 0
with progressbar.ProgressBar(min_value=0, max_value=int(cap.get(cv2.CAP_PROP_FRAME_COUNT))) as pb:
while cap.isOpened():
ret, frame = cap.read()
if ret:
#update the progress bar
curframe += 1
pb.update(curframe)
# convert to greyscale
grey = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# invert colors
inverted = cv2.bitwise_not(grey)
vw.write(inverted)
#cv2.imshow('right', right)
#if cv2.waitKey(1) & 0xFF == ord('q'):
# break
else:
break
cap.release()
vw.release()
cv2.destroyAllWindows()
I receive the following error:
OpenCV: FFMPEG: tag 0x7634706d/'mp4v' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x00000020/' ???'
I receive similar errors (as well as a warning that I have an incorrect environment variable for h.264 library path) if i try to set codec = cv2.VideoWriter_fourcc(*'H264').
Ensure that the dimensions of inverted match the dimensions of the size parameter in the videoWriter definition.
Also use 'M','P','4','V' codec with the .mp4 container.
im using gstreamer 1.2 to feed frames from my IP camera to opencv program
the stream is (640*368 YUVj420p) and i want to convert it to RBG888 to be able to use it in my opencv program
so is there a way to use gstreamer to do that conversion ?
or do i have to do it by myself?
if so please give me the equation that do this conversion
After some trials with gstreamer i decided to do the conversion myself and it worked
First we have to understand the YUVj420p pixel format
As shown in the above image, the Y', U and V components in Y'UV420 are encoded separately in sequential blocks. A Y' value is stored for every pixel, followed by a U value for each 2×2 square block of pixels, and finally a V value for each 2×2 block. Corresponding Y', U and V values are shown using the same color in the diagram above. Read line-by-line as a byte stream from a device, the Y' block would be found at position 0, the U block at position x×y (6×4 = 24 in this example) and the V block at position x×y + (x×y)/4 (here, 6×4 + (6×4)/4 = 30).(copied)
here is the code to do it (python)
This code will show how to inject frame to opencv using gstreamer and make the converstion
import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst
import numpy as np
import cv2
GObject.threads_init()
Gst.init(None)
def YUV_stream2RGB_frame(data):
w=640
h=368
size=w*h
stream=np.fromstring(data,np.uint8) #convert data form string to numpy array
#Y bytes will start form 0 and end in size-1
y=stream[0:size].reshape(h,w) # create the y channel same size as the image
#U bytes will start from size and end at size+size/4 as its size = framesize/4
u=stream[size:(size+(size/4))].reshape((h/2),(w/2))# create the u channel its size=framesize/4
#up-sample the u channel to be the same size as the y channel and frame using pyrUp func in opencv2
u_upsize=cv2.pyrUp(u)
#do the same for v channel
v=stream[(size+(size/4)):].reshape((h/2),(w/2))
v_upsize=cv2.pyrUp(v)
#create the 3-channel frame using cv2.merge func watch for the order
yuv=cv2.merge((y,u_upsize,v_upsize))
#Convert TO RGB format
rgb=cv2.cvtColor(yuv,cv2.cv.CV_YCrCb2RGB)
#show frame
cv2.imshow("show",rgb)
cv2.waitKey(5)
def on_new_buffer(appsink):
sample = appsink.emit('pull-sample')
#get the buffer
buf=sample.get_buffer()
#extract data stream as string
data=buf.extract_dup(0,buf.get_size())
YUV_stream2RGB_frame(data)
return False
def Init():
CLI="rtspsrc name=src location=rtsp://192.168.1.20:554/live/ch01_0 latency=10 !decodebin ! appsink name=sink"
#simplest way to create a pipline
pipline=Gst.parse_launch(CLI)
#getting the sink by its name set in CLI
appsink=pipline.get_by_name("sink")
#setting some important properties of appsnik
appsink.set_property("max-buffers",20) # prevent the app to consume huge part of memory
appsink.set_property('emit-signals',True) #tell sink to emit signals
appsink.set_property('sync',False) #no sync to make decoding as fast as possible
appsink.connect('new-sample', on_new_buffer) #connect signal to callable func
def run():
pipline.set_state(Gst.State.PLAYING)
GObject.MainLoop.run()
Init()
run()
How exactly are you getting the frames from your camera? And how you inject it into your opencv application?
Supposing you get your frames outside of gstreamer you should use a pipeline like:
appsrc caps="video/x-raw, format=I420, width=640, height=368" ! videoconvert ! capsfilter caps="video/x-raw, format=RGB" ! appsink
And then use appsrc to inject the data and use appsink to receive it back. If you are getting your data from camera from http or v4l2 you can replace appsrc with souphttpsrc or v4l2src.