I want to get both depth and video from streams from the kinect to my opencv code. I am working in Linux. I have installed libfreenect module for depth. However, there is only one device listed in /dev/. Now, when I connect the Kinect to my pc and run
camorama -d /dev/video0
I get the depth map. Then, I access the device using videocapture in opencv and I get the rgb video. Now, if I again run the camorama command, I get the rgb video this time. I can't figure out what's happening. I basically want both the stream in my opencv code. Please help.
Run this python script:
import freenect
import cv2
import numpy as np
from functions import *
def nothing(x):
pass
kernel = np.ones((5, 5), np.uint8)
def pretty_depth(depth):
np.clip(depth, 0, 2**10 - 1, depth)
depth >>= 2
depth = depth.astype(np.uint8)
return depth
while 1:
orig = freenect.sync_get_video()[0]
orig = cv2.cvtColor(orig,cv2.COLOR_BGR2RGB)
dst = pretty_depth(freenect.sync_get_depth()[0])#input from kinect
cv2.imshow('Disparity', dst)
cv2.imshow('RGB',orig)
if cv2.waitKey(1) & 0xFF == ord('b'):
break
Related
I'm new to coding and using a raspberry pi. I've searched through many tutorials online and found a how to get OpenCV library into the pi itself and downloaded VSC on my laptop and the pi. The issue that I'm having is the code that I used on my laptop doesn't work the same on the pi. I've been getting errors on my code that dosen't show in in my laptop VSC.
the purpose is to display a live feed from the camera in the raspberry pi
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
ret, frame = cap.read()
frame = cv2.resize(frame, (0,0), fx=0.5,fy=0.5)
cv2.imshow("Frame",frame)
ch = cv2.waitKey(1)
if ch & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
(line 10) error: (-206:Bad flag (parameter or structure field)) Unrecognized or unsupported array type in function 'cvGetMat'
(line 9) error: (-215:Assertion failed) !ssize.empty() in function 'resize'
I think you have to use different device id in cv.VideoCapture(id)
because raspberry is using linux and at some point the id is not always 0
I have been trying to segment biological cells in an image using watershed algorithm. I found an excellent article on pyimagesearch which clearly gives an overview of the algorithm and its implementation in python. The code uses both opencv and scikit-image for processing the image.
My goal is to convert the whole code into pure opencv. But the issues is that there's a function called scipy.feature.peak_local_max in scikit-image which does the job of finding local peaks in an image very efficiently. I couldn't find or devise such function in OpenCV.
Original Code(I have documented this snippet according to my understanding, please correct if am wrong):
import the necessary packages
from skimage.feature import peak_local_max
from skimage.morphology import watershed
from scipy import ndimage
import numpy as np
import argparse
import imutils
import cv2
from matplotlib import pyplot as plt
# load the image and perform pyramid mean shift filtering
# to aid the thresholding step
image = cv2.imread("test2.png")
shifted = cv2.pyrMeanShiftFiltering(image, 21, 51)
# Apply grayscale
gray = cv2.cvtColor(shifted, cv2.COLOR_BGR2GRAY)
# Convert to binary
thresh = cv2.threshold(gray, 0, 255,cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
# Watershed starts from here
# compute the exact Euclidean distance from every binary
# pixel to the nearest zero pixel, then find peaks in this
# distance map
D = ndimage.distance_transform_edt(thresh)
localMax = peak_local_max(D, indices=False, min_distance=10,labels=thresh)
# perform a connected component analysis on the local peaks,
# using 8-connectivity, then appy the Watershed algorithm
markers = ndimage.label(localMax, structure=np.ones((3, 3)))[0]
# Apply segmentation
labels = watershed(-D, markers, mask=thresh)
print("[INFO] {} unique segments found".format(len(np.unique(labels)) - 1))
cv2.imwrite("labels.png",labels)
# Contouring
for label in np.unique(labels):
# if the label is zero, we are examining the 'background'
# so simply ignore it
if label == 0:
continue
# otherwise, allocate memory for the label region and draw
# it on the mask
mask = np.zeros(gray.shape, dtype="uint8")
mask[labels == label] = 255
cnts = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
c = max(cnts, key=cv2.contourArea)
# draw a circle enclosing the object
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.018 * peri, True)
cv2.drawContours(image, [approx], -1, (0,0,255), 2)
cv2.imwrite("output.jpg",image)
Pure OpenCV Code till finding distance map:
# import the necessary packages
import numpy as np
import cv2
# load the image and perform pyramid mean shift filtering
# to aid the thresholding step
image = cv2.imread("1.png")
shifted = cv2.pyrMeanShiftFiltering(image, 21, 51)
# Apply grayscale
gray = cv2.cvtColor(shifted, cv2.COLOR_BGR2GRAY)
# Convert to binary
thresh = cv2.threshold(gray, 0, 255,cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
# Watershed starts from here
# compute the exact Euclidean distance from every binary
# pixel to the nearest zero pixel, then find peaks in this
# distance map
D = cv2.distanceTransform(thresh,cv2.DIST_L2,0)
The point till D, both the original code and the pure opencv code which I have tried have exactly the same outputs, the issue is I dont exactly have a clear idea on how to implement peak_local_max in opencv which would give identical result as scikit's function.
It would be really helpful if someone who has relavent knowledge could explain how this function works in finding those peaks in such a fine grained manner.
Input Image:
Peak Local max output in scikit-image(BGR format image):
Required output:
I'm trying to capture video from an in-build webcam on a laptop (or external USB camera) using opencv, specifically VideoCapture with the DSHOW argument.
I know there is a way to set the resolution and even FPS, however the DirectShow argument for the API returns none when I included it in the code.
For example;
# returns my webcam's stream, but all optional arguments are ignored
camera = cv2.VideoCapture(0)
camera = cv2.VideoCapture(0, cv2.CAP_V4L2)
# returns none and loops infinitely or errors out when *if im.any()*
camera = cv2.VideoCapture(0, cv2.CAP_DSHOW)
This is the code that follows after the above;
# should set resolution, settings are always ignored
camera.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
while(True):
retval, im = camera.read()
if im.any(): # errors out when image is none
cv2.imshow("image", im)
k = cv2.waitKey(33)
if k==27: # Esc key press
print('Resolution: {0}x and {1}y'.format(im.shape[1],im.shape[0]))
print('FPS: {0}'.format(camera.get(cv2.CAP_PROP_FPS)))
break
camera.release()
cv2.destroyAllWindows()
Is the DSHOW the correct API to use and is it the only API to use that can change resolution and FPS of a camera stream using opencv? Or is there something else I'm doing incorrectly?
More details about the system.
Ubuntu 18.04.6
python 3.9.5
opencv-python 4.5.2.52
Thank you in advance for the help!
Regards, Tiz
DSHOW (and MSMF) are windows only.
on linux, use V4L, FFMPEG or GSTREAMER
also, please check the return val of capture.set(),
not all properties/values will be supported on any given machine
This code runs as expected, and gives the expected output
import multiprocessing
import cv2
import os
path = r"/home/pi/Desktop/calibration.jpg"
image = cv2.imread(path)
def cvtcolor(img):
print "converting to gray ..."
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
print "converted to gray"
if True:
p = multiprocessing.Process(name='test',
target=cvtcolor,
kwargs={'img':image}
)
p.start()
p2 = multiprocessing.Process(name='test',
target=cvtcolor,
kwargs={'img':image}
)
p2.start()
outputs:
converting to gray ...
converting to gray ...
converted to gray
converted to gray
However, this code hangs when executed
import multiprocessing
import cv2
import os
path = r"/home/pi/Desktop/calibration.jpg"
image = cv2.imread(path)
def cvtcolor(img):
print "converting to gray ..."
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
print "converted to gray"
cvtcolor(image)
if True:
p = multiprocessing.Process(name='test',
target=cvtcolor,
kwargs={'img':image}
)
p.start()
the function executed in the main process proceeds, but the function executed in the "test" process hangs forever
converting to gray ...
converted to gray
converting to gray ...
I am using OpenCV version 3.2.0, installed as detailed here on Raspbian Jessie (raspberry pi)
Does anyone has an explanation / solution for this?
have a look at what is returned. If you try the BGR2GRAY directly, you will get an array with shape attribute same as input image but with only 1 color, e.g. gray. When you run the same function using multiprocessing you do not get an array returned. It will have no shape attribute, try printing the output to see what form it is in, then maybe reconstruct an image from this.
im using gstreamer 1.2 to feed frames from my IP camera to opencv program
the stream is (640*368 YUVj420p) and i want to convert it to RBG888 to be able to use it in my opencv program
so is there a way to use gstreamer to do that conversion ?
or do i have to do it by myself?
if so please give me the equation that do this conversion
After some trials with gstreamer i decided to do the conversion myself and it worked
First we have to understand the YUVj420p pixel format
As shown in the above image, the Y', U and V components in Y'UV420 are encoded separately in sequential blocks. A Y' value is stored for every pixel, followed by a U value for each 2×2 square block of pixels, and finally a V value for each 2×2 block. Corresponding Y', U and V values are shown using the same color in the diagram above. Read line-by-line as a byte stream from a device, the Y' block would be found at position 0, the U block at position x×y (6×4 = 24 in this example) and the V block at position x×y + (x×y)/4 (here, 6×4 + (6×4)/4 = 30).(copied)
here is the code to do it (python)
This code will show how to inject frame to opencv using gstreamer and make the converstion
import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst
import numpy as np
import cv2
GObject.threads_init()
Gst.init(None)
def YUV_stream2RGB_frame(data):
w=640
h=368
size=w*h
stream=np.fromstring(data,np.uint8) #convert data form string to numpy array
#Y bytes will start form 0 and end in size-1
y=stream[0:size].reshape(h,w) # create the y channel same size as the image
#U bytes will start from size and end at size+size/4 as its size = framesize/4
u=stream[size:(size+(size/4))].reshape((h/2),(w/2))# create the u channel its size=framesize/4
#up-sample the u channel to be the same size as the y channel and frame using pyrUp func in opencv2
u_upsize=cv2.pyrUp(u)
#do the same for v channel
v=stream[(size+(size/4)):].reshape((h/2),(w/2))
v_upsize=cv2.pyrUp(v)
#create the 3-channel frame using cv2.merge func watch for the order
yuv=cv2.merge((y,u_upsize,v_upsize))
#Convert TO RGB format
rgb=cv2.cvtColor(yuv,cv2.cv.CV_YCrCb2RGB)
#show frame
cv2.imshow("show",rgb)
cv2.waitKey(5)
def on_new_buffer(appsink):
sample = appsink.emit('pull-sample')
#get the buffer
buf=sample.get_buffer()
#extract data stream as string
data=buf.extract_dup(0,buf.get_size())
YUV_stream2RGB_frame(data)
return False
def Init():
CLI="rtspsrc name=src location=rtsp://192.168.1.20:554/live/ch01_0 latency=10 !decodebin ! appsink name=sink"
#simplest way to create a pipline
pipline=Gst.parse_launch(CLI)
#getting the sink by its name set in CLI
appsink=pipline.get_by_name("sink")
#setting some important properties of appsnik
appsink.set_property("max-buffers",20) # prevent the app to consume huge part of memory
appsink.set_property('emit-signals',True) #tell sink to emit signals
appsink.set_property('sync',False) #no sync to make decoding as fast as possible
appsink.connect('new-sample', on_new_buffer) #connect signal to callable func
def run():
pipline.set_state(Gst.State.PLAYING)
GObject.MainLoop.run()
Init()
run()
How exactly are you getting the frames from your camera? And how you inject it into your opencv application?
Supposing you get your frames outside of gstreamer you should use a pipeline like:
appsrc caps="video/x-raw, format=I420, width=640, height=368" ! videoconvert ! capsfilter caps="video/x-raw, format=RGB" ! appsink
And then use appsrc to inject the data and use appsink to receive it back. If you are getting your data from camera from http or v4l2 you can replace appsrc with souphttpsrc or v4l2src.