I'm working on a project in which a picamera records to stdout.
I'm adding object detection to the video and need to write these frames to stdout in h264.
So far I've got this:
for frame in self.camera.capture_continuous(rawCapture, format="bgr",use_video_port=True):
#while True:
# grab the raw NumPy array representing the image, then
# initialize the timestamp and occupied/unoccupied text
frame = frame.array
if frame is None:
break
object_detector_frame = self.object_detector.draw_boxes(frame)
sys.stdout.write(object_detector_frame.tostring())
#writer.write(frame)
rawCapture.truncate(0)
CV2's video writer doesn't seem to be able to write bytes straight to stdout.
Thanks in advance.
Related
I am trying to do some image processing on a video, then save the resulting video using opencv on colab. However, I am not able to access the resulting video file that I am writing to.
import cv2
from google.colab.patches import cv2_imshow
import numpy as np
fourcc = cv2.VideoWriter_fourcc(*'H264')
cap = cv2.VideoCapture(vid_file)
out = cv2.VideoWriter('output.mp4',fourcc,30.0,(1124,1080))
cnt = 0
ret = True
while(ret):
ret,frame = cap.read()
print(cnt,end=' ')
# check if prey was tracked on this frame
match = np.where(prey_frames==cnt)[0]
if match:
prey_frame = match[0]
# print(prey_frame)
image = cv2.circle(frame,(int(prey_px[prey_frame].x),95+int(prey_px[prey_frame].y)),
radius=5,color=(255,0,255),thickness=2)
else:
image = frame
out.write(image)
cnt += 1
out.release()
cap.release()
cv2.destroyAllWindows()
From what I understand, this should write to a file called 'output.mp4'. This code runs without error, but there is no file in the current directory, and no file of that name available to download (using files.download('output.mp4') returns 'cannot find file' error).
Any help would be appreciated!
I've hit this problem a few times and I believe that it has to do with the fact that Colab's operating environment only supports a few video encodings.
I was able to get the video writer working with the following:
fourcc = cv2.VideoWriter_fourcc('F','M','P','4')
I have an RTP/RTSP stream that's running at 25fps, as verified by ffprobe -i <URI>. Also, VLC plays back the RTSP stream at a real-time rate, but doesn't show me the FPS in the Media Information window.
However, when I use OpenCV 4.1.1.26 to retrieve the input stream's frame rate, it is giving me a response of 90000.0.
Question: How can I use OpenCV to probe for the correct frame rate of the RTSP stream? What would cause it to report 90000.0 instead of 25?
Here's my Python function to retrieve the frame rate:
import cv2
vid : cv2.VideoCapture = cv2.VideoCapture('rtsp://192.168.1.10/cam1/mpeg4')
def get_framerate(video: cv2.VideoCapture):
fps = video.get(cv2.CAP_PROP_FPS)
print('FPS is {0}'.format(fps))
get_framerate(vid)
MacOS Catalina
Python 3.7.4
I hope this helps you somehow. It is a simple calculator that takes cont captures and measure the beginning and the ending time. Then with the rule of three, i converted it to fps.
Related to you second question i read here that it could be due to bad installation. Also, you can check that your camera is working properly by printing ret variable. If it is true then you should be able to see the fps, if it is false then you can have an unpredictable result.
cv2.imshow() and key = cv2.waitKey(1) should be commented as it adds ping/delay resulting in bad measurement.
I post this as a comment because i do not have enough reputation points.
img = cv2.VideoCapture('rtsp://192.168.1.10/cam1/mpeg4')
while True:
if cont == 50:
a = datetime.now() - start
b = (a.seconds * 10e6 + a.microseconds)
print((a.seconds * 10e6 + a.microseconds), "fps = ", (50 * 10e6)/ b)
break
ret, frame = img.read()
# Comment for best test
cv2.imshow('fer', frame)
key = cv2.waitKey(1)
if key == ord('q'):
break
cont+=1
img.release()
cv2.destroyAllWindows()`
I have a Garmin VIRB XE camera, and want to get a live stream and interact with camera like getting GPS data. I could get the live stream by VLC media player and also could post commands to the camera by CURL from windows command prompt, but i can't get the live stream using OpenCV and interact with camera using requests library in python.
I can get a live stream from "rtsp://192.168.1.35/livePreviewStream" using VLC media player's network streaming feature, also could interact with camera, for example by "curl --data "{\"command\":\"startRecording\"}" http://192.168.1.35/virb" from command prompt I could start the recording, but the following codes are not working.
'''
import simplejson
import requests
url='http://192.168.1.37:80/virb'
data = {'command':'startRecording'}
r=requests.post(url, simplejson.dumps(data))
'''
or
'''
import cv2
capture = cv2.VideoCapture("rtsp://192.168.1.35/livePreviewStream")
'''
The post return the error
"ProxyError: HTTPConnectionPool(host='127.0.0.1', port=8000): Max retries exceeded with url: http://192.168.1.37:80/virb (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response')))".
Also the capture could not get any frames.
Since you have already confirmed that your RTSP link works with VLC player, here's a IP camera video streaming widget using OpenCV and cv2.VideoCapture.read(). This implementation uses threading for obtaining frames in a different thread since read() is a blocking operation. By putting this operation into a separate that that just focuses on obtaining frames, it improves performance by I/O latency reduction. I used my own IP camera RTSP stream link. Change stream_link to your own IP camera link.
Depending on your IP camera, your RTSP stream link will vary, here's an example of mine:
rtsp://username:password#192.168.1.49:554/cam/realmonitor?channel=1&subtype=0
rtsp://username:password#192.168.1.45/axis-media/media.amp
Code
from threading import Thread
import cv2
class VideoStreamWidget(object):
def __init__(self, src=0):
# Create a VideoCapture object
self.capture = cv2.VideoCapture(src)
# Start the thread to read frames from the video stream
self.thread = Thread(target=self.update, args=())
self.thread.daemon = True
self.thread.start()
def update(self):
# Read the next frame from the stream in a different thread
while True:
if self.capture.isOpened():
(self.status, self.frame) = self.capture.read()
def show_frame(self):
# Display frames in main program
if self.status:
self.frame = self.maintain_aspect_ratio_resize(self.frame, width=600)
cv2.imshow('IP Camera Video Streaming', self.frame)
# Press Q on keyboard to stop recording
key = cv2.waitKey(1)
if key == ord('q'):
self.capture.release()
cv2.destroyAllWindows()
exit(1)
# Resizes a image and maintains aspect ratio
def maintain_aspect_ratio_resize(self, image, width=None, height=None, inter=cv2.INTER_AREA):
# Grab the image size and initialize dimensions
dim = None
(h, w) = image.shape[:2]
# Return original image if no need to resize
if width is None and height is None:
return image
# We are resizing height if width is none
if width is None:
# Calculate the ratio of the height and construct the dimensions
r = height / float(h)
dim = (int(w * r), height)
# We are resizing width if height is none
else:
# Calculate the ratio of the 0idth and construct the dimensions
r = width / float(w)
dim = (width, int(h * r))
# Return the resized image
return cv2.resize(image, dim, interpolation=inter)
if __name__ == '__main__':
stream_link = 'your stream link!'
video_stream_widget = VideoStreamWidget(stream_link)
while True:
try:
video_stream_widget.show_frame()
except AttributeError:
pass
So I want to read individual frames of picamera using while loop. Here is what I found for doing it using for loop:
# import the necessary packages
from picamera.array import PiRGBArray
from picamera import PiCamera
import time
import cv2
# initialize the camera and grab a reference to the raw camera capture
camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 32
rawCapture = PiRGBArray(camera, size=(640, 480))
# allow the camera to warmup
time.sleep(0.1)
# capture frames from the camera
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
# grab the raw NumPy array representing the image, then initialize the timestamp
# and occupied/unoccupied text
image = frame.array
# show the frame
cv2.imshow("Frame", image)
key = cv2.waitKey(1) & 0xFF
# clear the stream in preparation for the next frame
rawCapture.truncate(0)
# if the `q` key was pressed, break from the loop
if key == ord("q"):
break
cv2.destroyAllWindows()
Now when I use the above code, I get the video feed but I am planning to do the same using a while loop. Going from the same logic I added a while loop something like this:
while True: frame1=camera.capture_continious(rawCapture,format="bgr",use_video_port=True)
image1 = frame1.array
# show the frame
cv2.imshow("Frame1", image1)
# clear the stream in preparation for the next frame
rawCapture.truncate(0)
But still I am getting an error as frame1 is a generator and doesn't contain such attributes while the same code is running well with for loop. What modifications can I make?
The function capture_continuous() returns an infinite iterator of images captured continuously from the camera. It does not return a single image. That's is why it does work with a for loop.
In your while loop you should use the capture() function, which does return an image.
You can (and should ;) ) read more about it in this documentation
im using gstreamer 1.2 to feed frames from my IP camera to opencv program
the stream is (640*368 YUVj420p) and i want to convert it to RBG888 to be able to use it in my opencv program
so is there a way to use gstreamer to do that conversion ?
or do i have to do it by myself?
if so please give me the equation that do this conversion
After some trials with gstreamer i decided to do the conversion myself and it worked
First we have to understand the YUVj420p pixel format
As shown in the above image, the Y', U and V components in Y'UV420 are encoded separately in sequential blocks. A Y' value is stored for every pixel, followed by a U value for each 2×2 square block of pixels, and finally a V value for each 2×2 block. Corresponding Y', U and V values are shown using the same color in the diagram above. Read line-by-line as a byte stream from a device, the Y' block would be found at position 0, the U block at position x×y (6×4 = 24 in this example) and the V block at position x×y + (x×y)/4 (here, 6×4 + (6×4)/4 = 30).(copied)
here is the code to do it (python)
This code will show how to inject frame to opencv using gstreamer and make the converstion
import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst
import numpy as np
import cv2
GObject.threads_init()
Gst.init(None)
def YUV_stream2RGB_frame(data):
w=640
h=368
size=w*h
stream=np.fromstring(data,np.uint8) #convert data form string to numpy array
#Y bytes will start form 0 and end in size-1
y=stream[0:size].reshape(h,w) # create the y channel same size as the image
#U bytes will start from size and end at size+size/4 as its size = framesize/4
u=stream[size:(size+(size/4))].reshape((h/2),(w/2))# create the u channel its size=framesize/4
#up-sample the u channel to be the same size as the y channel and frame using pyrUp func in opencv2
u_upsize=cv2.pyrUp(u)
#do the same for v channel
v=stream[(size+(size/4)):].reshape((h/2),(w/2))
v_upsize=cv2.pyrUp(v)
#create the 3-channel frame using cv2.merge func watch for the order
yuv=cv2.merge((y,u_upsize,v_upsize))
#Convert TO RGB format
rgb=cv2.cvtColor(yuv,cv2.cv.CV_YCrCb2RGB)
#show frame
cv2.imshow("show",rgb)
cv2.waitKey(5)
def on_new_buffer(appsink):
sample = appsink.emit('pull-sample')
#get the buffer
buf=sample.get_buffer()
#extract data stream as string
data=buf.extract_dup(0,buf.get_size())
YUV_stream2RGB_frame(data)
return False
def Init():
CLI="rtspsrc name=src location=rtsp://192.168.1.20:554/live/ch01_0 latency=10 !decodebin ! appsink name=sink"
#simplest way to create a pipline
pipline=Gst.parse_launch(CLI)
#getting the sink by its name set in CLI
appsink=pipline.get_by_name("sink")
#setting some important properties of appsnik
appsink.set_property("max-buffers",20) # prevent the app to consume huge part of memory
appsink.set_property('emit-signals',True) #tell sink to emit signals
appsink.set_property('sync',False) #no sync to make decoding as fast as possible
appsink.connect('new-sample', on_new_buffer) #connect signal to callable func
def run():
pipline.set_state(Gst.State.PLAYING)
GObject.MainLoop.run()
Init()
run()
How exactly are you getting the frames from your camera? And how you inject it into your opencv application?
Supposing you get your frames outside of gstreamer you should use a pipeline like:
appsrc caps="video/x-raw, format=I420, width=640, height=368" ! videoconvert ! capsfilter caps="video/x-raw, format=RGB" ! appsink
And then use appsrc to inject the data and use appsink to receive it back. If you are getting your data from camera from http or v4l2 you can replace appsrc with souphttpsrc or v4l2src.