Grabbing Pixel data from a video strem - opencv

I am having trouble if understanding certain coding i am sorry if this comes off as stupid but i have a code to capture a video from my webcam i want to get the RGB valuee from the frame, if this is impossible would have to to save a frame as a picture and then get values from it?
const char window_name[]="Webcam";
int main(int argc, char* argv[])
{
/* attempt to capture from any connected device */
CvCapture *capture=cvCaptureFromCAM(CV_CAP_ANY);
if(!capture)
{
printf("Failed to initialise webcam\n");
return -1;
}
/* create the output window */
cvNamedWindow(window_name, CV_WINDOW_NORMAL);
do
{
/* attempt to grab a frame */
IplImage *frame=cvQueryFrame(capture);
if(!frame)
{
printf("Failed to get frame\n");
break;
}
COLORREF myColAbsolute = GetPixel(frame, 10,10);//error in saying frame is not compatible with HDC.
cout << "Red - " << (int)GetRValue(myColAbsolute) << endl;
cout << "Green - " << (int)GetGValue(myColAbsolute) << endl;
cout << "Blue - " << (int)GetBValue(myColAbsolute) << endl;
/* show the frame */
cvShowImage(window_name, frame);

ha ! ( obviously caught with a copy & paste bummer )
GetPixel() is a windows function, not an opencv one. same for GetRValue() and sisters.
you'd use them in the native win32 api, to get a pixel from an HDC, but it won't work with opencv/highgui, since neither HDC, nor HWND are exposed.
since you're obviously a beginner(nothing wrong with that, again!) let me try to talk you out of using the old, 1.0 opencv api(IplImages, cv*Functions) as well,
you should be using the new one(cv::Mat, namespace cv::Functions) instead.
#include "opencv2/opencv.hpp"
#include "opencv2/highgui/highgui.hpp"
using namespace cv;
using namespace std;
int main()
{
Mat frame;
namedWindow("video", 1);
VideoCapture cap(0);
while ( cap.isOpened() )
{
cap >> frame;
if(frame.empty()) break;
int x=3, y=5;
// Ladies and Gentlemen, the PIXEL!
Vec3b pixel = frame.at<Vec3b>(y,x); // row,col, not x,y!
cerr << "b:" << int(pixel[0]) << " g:" << int(pixel[1]) << " r:" << int(pixel[2]) << endl;
imshow("video", frame);
if(waitKey(30) >= 0) break;
}
return 0;
}

Related

video capturing works in the example code, but doesn't in my own project

I built opencv with openni2 using Cmake, and I succeeded to run the example 'openni_capture' which is in OpenCV.sln. It clearly shows the video being captured. I'm using Orbbec Astra camera.
But when I try to make my own project, copy and paste the code, and run it, it says 'can not open a capture object' even if it was successfully built.
The code is like below. The problem is that 'capture.isOpened()' is TRUE in the example project, but it is FALSE in my own project which has exactly same code as the example project.
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>
using namespace cv;
using namespace std;
static void colorizeDisparity( const Mat& gray, Mat& rgb, double maxDisp=-1.f, float S=1.f, float V=1.f )
{
.
.
.
}
static float getMaxDisparity( VideoCapture& capture )
{
.
.
.
}
static void printCommandLineParams()
{
cout << "-cd= Colorized disparity? (0 or 1; 1 by default) Ignored if disparity map is not selected to show." << endl;
cout << "-fmd= Fixed max disparity? (0 or 1; 0 by default) Ignored if disparity map is not colorized (-cd 0)." << endl;
cout << "-mode= image mode: resolution and fps, supported three values: 0 - CAP_OPENNI_VGA_30HZ, 1 - CAP_OPENNI_SXGA_15HZ," << endl;
cout << " 2 - CAP_OPENNI_SXGA_30HZ (0 by default). Ignored if rgb image or gray image are not selected to show." << endl;
cout << "-m= Mask to set which output images are need. It is a string of size 5. Each element of this is '0' or '1' and" << endl;
cout << " determine: is depth map, disparity map, valid pixels mask, rgb image, gray image need or not (correspondently), ir image" << endl ;
cout << " By default -m=010100 i.e. disparity map and rgb image will be shown." << endl ;
cout << "-r= Filename of .oni video file. The data will grabbed from it." << endl ;
}
static void parseCommandLine( int argc, char* argv[], bool& isColorizeDisp, bool& isFixedMaxDisp, int& imageMode, bool retrievedImageFlags[],
string& filename, bool& isFileReading )
{
filename.clear();
cv::CommandLineParser parser(argc, argv, "{h help||}{cd|0|}{fmd|0|}{mode|-1|}{m|000100|}{r||}");
if (parser.has("h"))
{
help();
printCommandLineParams();
exit(0);
}
isColorizeDisp = (parser.get<int>("cd") != 0);
isFixedMaxDisp = (parser.get<int>("fmd") != 0);
imageMode = parser.get<int>("mode");
int flags = parser.get<int>("m");
isFileReading = parser.has("r");
if (isFileReading)
filename = parser.get<string>("r");
if (!parser.check())
{
parser.printErrors();
help();
exit(-1);
}
if (flags % 1000000 == 0)
{
cout << "No one output image is selected." << endl;
exit(0);
}
for (int i = 0; i < 6; i++)
{
retrievedImageFlags[5 - i] = (flags % 10 != 0);
flags /= 10;
}
}
int main( int argc, char* argv[] )
{
bool isColorizeDisp, isFixedMaxDisp;
int imageMode;
bool retrievedImageFlags[6];
string filename;
bool isVideoReading;
parseCommandLine( argc, argv, isColorizeDisp, isFixedMaxDisp, imageMode, retrievedImageFlags, filename, isVideoReading );
cout << "Device opening ..." << endl;
VideoCapture capture;
if( isVideoReading )
capture.open( filename );
else
{
capture.open( CAP_OPENNI2 );
if (!capture.isOpened())
{
capture.open(CAP_OPENNI);
}
}
cout << "done." << endl;
if( !capture.isOpened() )
{
cout << "Can not open a capture object." << endl;
return -1;
}
.
.
.
I added to VC++ directory-include directory that
C:\OpenCV_end\Source\opencv-3.4.0\build\install\include ,C:\Program Files\OpenNI2\Include
I added to VC++ directory-library directory that
C:\OpenCV_end\Source\opencv-3.4.0\build\install\x64\vc14\lib ,C:\Program Files\OpenNI2\Lib
I added to Linker-input that
opencv_world340d.lib ,OpenNI2.lib
and I copied the dll files to the folder which contains my project source. opencv_world340d.dll and all the files which are in C:\Program Files\OpenNI2\Redist
but it never wants to work.. Please help me
Thank you.

OpenCV / HighGUI draws new windows for each attached object

I'm working through Learning OpenCV 3 by Kaehler & Bradski.
I've applied all errata fixes to this code per http://www.oreilly.com/catalog/errata.csp?isbn=0636920044765.
Expected behavior: the trackbar should be attached to the namedWindow "Example2-4", then frames from a video file (specified via command-line argument) should be drawn in the same window.
Actual behavior: the trackbar is drawn in one window named "Example2-4", then the frames are drawn in a second window named "Example2-4". Closing either window causes the program to hang. Otherwise the behavior is correct.
Platform: Windows 10, x64, OpenCV 3.3
Edit 1:
I tried adding a string literal to the top of main():
char *window_name = "Window";
and replacing every instance of "Example2-4" with window_name. This didn't change the behavior.
Edit 2:
This code is the first time I've tried to add an interactive widget, but adding images and video frames to windows in previous examples also generated two windows with the same name. I suspect I'm up against a configuration issue, but I still have no idea how to solve it.
Edit 3: I've added code for the simplest program that will demonstrate the problem behavior, and changed the title of this post to characterize it better.
Code:
/* Windows precompiled headers */
//#include "stdafx.h"
/* C++ */
#include <iostream>
#include <fstream>
/* OpenCv */
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
using namespace std;
int g_slider_position = 0;
int g_run = 1, g_dontset = 0; // start out in single step mode
cv::VideoCapture g_cap;
void onTrackBarSlide(int pos, void *);
int main(int argc, char **argv)
{
cv::namedWindow("Example2-4", CV_WINDOW_AUTOSIZE);
g_cap.open(string(argv[1]));
int frames = (int)g_cap.get(CV_CAP_PROP_FRAME_COUNT);
int tmpw = (int)g_cap.get(CV_CAP_PROP_FRAME_WIDTH);
int tmph = (int)g_cap.get(CV_CAP_PROP_FRAME_HEIGHT);
cout << "Video has " << frames << " frames of dimensions ("
<< tmpw << ", " << tmph << ")." << endl;
cv::createTrackbar("Position", "Example2-4", &g_slider_position, frames,
onTrackBarSlide);
cv::Mat frame;
for (;;) {
if (g_run != 0) {
g_cap >> frame;
if (frame.empty())
break;
int current_pos = (int)g_cap.get(CV_CAP_PROP_POS_FRAMES);
g_dontset = 1;
cv::setTrackbarPos("Position", "Example2-4", current_pos);
cv::imshow("Example2-4", frame);
g_run -= 1;
}
char c = (char)cv::waitKey(10);
if (c == 's') { // single step
g_run = 1;
cout << "Single step, run = " << g_run << endl;
}
if (c == 'r') { // run mode
g_run = -1;
cout << "Run mode, run = " << g_run << endl;
}
if (c == 27)
break;
}
return 0;
}
void onTrackBarSlide(int pos, void *) {
g_cap.set(CV_CAP_PROP_POS_FRAMES, pos);
if (!g_dontset)
g_run = 1;
g_dontset = 0;
}
Simplest code which results in problem behavior. Gets an image via command line argument:
#include <opencv2\opencv.hpp>
int main(int argc, char **argv)
{
char *window_name = "Window";
cv::Mat img = cv::imread(argv[1], -1);
if (img.empty())
return -1;
cv::namedWindow(window_name, CV_WINDOW_AUTOSIZE);
cv::imshow(window_name, img);
cv::waitKey(0);
cv::destroyWindow(window_name);
return 0;
}

frame subtraction in video

I am trying to do Frame subtraction in a video.Steps I am following
Get a image , convert it into grayscale.
Subtract it from previous frame grayscale.
All I see in diff2(and diff also) a complete black image.One observation I made is that pixel value of gray1 and gray2 become equal.
My code
#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/objdetect/objdetect.hpp"
#include <opencv2/video/background_segm.hpp>
#include <iostream>
using namespace cv;
using namespace std;
RNG rng(12345);
int main( int argc, const char** argv )
{
VideoCapture cap(0);
if ( !cap.isOpened() )
{
cout << "Cannot open the web cam" << endl;
return -1;
}
Mat img1,img2,diff,gray1,gray2,diff2;
bool bSuccess = cap.read(img1); // read a new frame from video
if (!bSuccess) //if not success, break loop
{
cout << "Cannot read a frame from video stream" << endl;
return -1;
}
cvtColor( img1,gray1, CV_BGR2GRAY );
while (true)
{
bSuccess = cap.read(img2); // read a new frame from video
if (!bSuccess) //if not success, break loop
{
cout << "Cannot read a frame from video stream" << endl;
break;
}
cvtColor( img2,gray2, CV_BGR2GRAY );
absdiff(gray2,gray1,diff);
threshold(diff, diff2, 150, 255, CV_THRESH_BINARY);
cout<<gray2.at<uchar>(100,200) <<endl;
cout<<gray1.at<uchar>(100,200) <<endl;
gray1=gray2;
imshow("1",gray1);
imshow("2",diff2);
if (waitKey(1000) == 27) //wait for 'esc' key press for 30ms. If 'esc' key is pressed, break loop
{
cout << "esc key is pressed by user" << endl;
break;
}
}
return -1;
}
please try this code. It looks like you're overwriting gray1 so that gray1 and gray2 use the very same data memory positions.
You could either use gray1=gray2.clone(); instead or use some real "swapping" of the buffers instead of overwriting. My code should perform a simple buffer swapping and has some comments about the problem.
#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/objdetect/objdetect.hpp"
#include <opencv2/video/background_segm.hpp>
#include <iostream>
using namespace cv;
using namespace std;
RNG rng(12345);
int main( int argc, const char** argv )
{
VideoCapture cap(0);
if ( !cap.isOpened() )
{
cout << "Cannot open the web cam" << endl;
return -1;
}
Mat img1,img2,diff,gray1,gray2,diff2;
Mat tmp; // used to swap the buffers
bool bSuccess = cap.read(img1); // read a new frame from video
if (!bSuccess) //if not success, break loop
{
cout << "Cannot read a frame from video stream" << endl;
return -1;
}
// this will allocate memory of gray1 if not allocated yet
cvtColor( img1,gray1, CV_BGR2GRAY );
while (true)
{
bSuccess = cap.read(img2); // read a new frame from video
if (!bSuccess) //if not success, break loop
{
cout << "Cannot read a frame from video stream" << endl;
break;
}
// memory for gray2 won't be allocated if it is present already => if gray2 and gray1 use the same data memory, you'll overwrite gray1's pixels here and obviously gray1 and gray2 will have the same pixel values then
cvtColor( img2,gray2, CV_BGR2GRAY );
absdiff(gray2,gray1,diff);
threshold(diff, diff2, 150, 255, CV_THRESH_BINARY);
cout<<gray2.at<uchar>(100,200) <<endl;
cout<<gray1.at<uchar>(100,200) <<endl;
// don't lose the memory of gray1
tmp = gray1;
// this means gray1 and gray2 will use the same data memory location
gray1=gray2;
// give gray2 a new data memory location. Since previous gray1 memory is still present but wont be used anymore, use it here.
gray2=tmp;
imshow("1",gray1);
imshow("2",diff2);
if (waitKey(1000) == 27) //wait for 'esc' key press for 30ms. If 'esc' key is pressed, break loop
{
cout << "esc key is pressed by user" << endl;
break;
}
}
return -1;
}
in addition, a thres difference threshold of 150 might be very high for common tasks, but it might be ok for your special task. Typical difference values/thresholds in background subtraction for foreground extraction are around 20 to 30 from my experience, but at the end it depends on your task/problem/domain.

Decode QR code, from low quality image (with ZBar)

I tried to decode QR codes from images like this:
Based on the following code,
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <zbar.h>
#include <iostream>
using namespace cv;
using namespace std;
using namespace zbar;
//g++ main.cpp /usr/local/include/ /usr/local/lib/ -lopencv_highgui.2.4.8 -lopencv_core.2.4.8
int main(int argc, char* argv[])
{
VideoCapture cap(0); // open the video camera no. 0
// cap.set(CV_CAP_PROP_FRAME_WIDTH,800);
// cap.set(CV_CAP_PROP_FRAME_HEIGHT,640);
if (!cap.isOpened()) // if not success, exit program
{
cout << "Cannot open the video cam" << endl;
return -1;
}
ImageScanner scanner;
scanner.set_config(ZBAR_NONE, ZBAR_CFG_ENABLE, 1);
double dWidth = cap.get(CV_CAP_PROP_FRAME_WIDTH); //get the width of frames of the video
double dHeight = cap.get(CV_CAP_PROP_FRAME_HEIGHT); //get the height of frames of the video
cout << "Frame size : " << dWidth << " x " << dHeight << endl;
namedWindow("MyVideo",CV_WINDOW_AUTOSIZE); //create a window called "MyVideo"
while (1)
{
Mat frame;
bool bSuccess = cap.read(frame); // read a new frame from video
if (!bSuccess) //if not success, break loop
{
cout << "Cannot read a frame from video stream" << endl;
break;
}
Mat grey;
cvtColor(frame,grey,CV_BGR2GRAY);
int width = frame.cols;
int height = frame.rows;
uchar *raw = (uchar *)grey.data;
// wrap image data
Image image(width, height, "Y800", raw, width * height);
// scan the image for barcodes
int n = scanner.scan(image);
// extract results
for(Image::SymbolIterator symbol = image.symbol_begin();
symbol != image.symbol_end();
++symbol) {
vector<Point> vp;
// do something useful with results
cout << "decoded " << symbol->get_type_name() << " symbol \"" << symbol->get_data() << '"' <<" "<< endl;
int n = symbol->get_location_size();
for(int i=0;i<n;i++){
vp.push_back(Point(symbol->get_location_x(i),symbol->get_location_y(i)));
}
RotatedRect r = minAreaRect(vp);
Point2f pts[4];
r.points(pts);
for(int i=0;i<4;i++){
line(frame,pts[i],pts[(i+1)%4],Scalar(255,0,0),3);
}
//cout<<"Angle: "<<r.angle<<endl;
}
imshow("MyVideo", frame); //show the frame in "MyVideo" window
if (waitKey(30) == 27) //wait for 'esc' key press for 30ms. If 'esc' key is pressed, break loop
{
cout << "esc key is pressed by user" << endl;
break;
}
}
return 0;
}
The naive ZLib approach fails 100%. But the zxing barcode scanner app can decode it from the computer screen, so it's definitely contains all the necessary information.
Any idea how to make the scanning more robust?

opencv cvblob -- render blobs other than white

I am trying to detect objects with cvblob. Somehow, my code only marks the white objects. How to mark objects of other colors, like a can of beer or a bottle of water.
Here is my code:
#include "highgui.h"
#include "cv.h"
#include "cvaux.h"
#include "iostream"
#include <stdio.h>
#include <ctype.h>
#include <cvblob.h>
using namespace cv;
using namespace std;
using namespace cvb;
int main(int argc, char** argv) {
CvTracks tracks;
cvNamedWindow("frame", CV_WINDOW_AUTOSIZE);
cvMoveWindow("frame", 50, 100);
CvCapture* capture;
IplImage* frame = 0;
capture = cvCreateCameraCapture( 0 ); //capture frames from cam on index 0: /dev/video0/
if (!capture) {
return -1;
}
cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH, 320);
cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, 240);
frame = cvQueryFrame(capture);
while(frame) {
IplImage *gray = cvCreateImage(cvGetSize(frame), IPL_DEPTH_8U, 1);
cvCvtColor(frame, gray, CV_BGR2GRAY);
cvThreshold(gray, gray, 150, 255, CV_THRESH_BINARY);
IplImage *labelImg=cvCreateImage(cvGetSize(gray), IPL_DEPTH_LABEL, 1);
CvBlobs blobs;
unsigned int result=cvLabel(gray, labelImg, blobs);
cvFilterByArea(blobs, 500, 1000000);
// cvRenderBlobs(labelImg, blobs, frame, frame, CV_BLOB_RENDER_BOUNDING_BOX);
cvRenderBlobs(labelImg, blobs, frame, frame, CV_BLOB_RENDER_CENTROID);
cvUpdateTracks(blobs, tracks, 200., 5);
cvRenderTracks(tracks, frame, frame, CV_TRACK_RENDER_ID|CV_TRACK_RENDER_BOUNDING_BOX);
for (CvBlobs::const_iterator it=blobs.begin(); it!=blobs.end(); ++it) {
cout << "Blob #" << it->second->label << ": Area=" << it->second->area << ", Centroid=(" << it->second->centroid.x << ", " << it->second->centroid.y << ")" << endl;
}
cvShowImage("frame", frame);
cout << "----------------------------" << endl;
frame = cvQueryFrame(capture);
char c = cvWaitKey(10);
if(c==27)
break;
}
}
Any tip is appreciated.
Milo
That's the option by default and you cannot change it if you don't change the source code in cvblob library.
If you really want to change this is so easy, you can create a copy of the same method adding a new input var like CvScalar to select output color. It's so easy.
The method cvRenderBlob will be in cvcontour.cpp.
I've been made many improvement in cvblob library and in next months I will push it to the creator.
Try adding:
"cvInRangeS(hsvframe,cvScalar(23,41,133),cvScalar(40,150,255),threshy);//for yellow"
Before Filtering the blobs. Its a range of HSV(instead of RGB) values that defines the threshold of the desire color.
Hope it helps.

Resources