I bought a raw data output camera sensor, it's data sheet in this website:
https://www.leopardimaging.com/uploads/LI-USB30-IMX225C_datasheet.pdf
it's resolution is 1312x992, 12 bits raw stream.
I have wrote a ISP program, it's input should be a raw picture, such as a.dng or b.raw and so on.Then i want to use this program and camera sensor to do preview.
simple code like this:
int main() {
VideoCapture cap(0);
Mat src;
if (cap.isOpened()) {
while (1) {
cap.read(src);
namedWindow("capture", 0);
imshow("capture", src);
waitKey(10);
}
}
}
there are some problem, color and resolution are incorrect.
first, color is green, it's abnormal.And it's resolution is 320x240.
Can anyone know why is this, and help me to fix it.
Thank you!
Example image
Related
Hi I'm working on getting a raw image data from AVCaptureVideoDataoutput.
I have lots of experience in using avfoundation and worked lots of projects but this time I'm working on image processing project which I don't have any experience.
public func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
}
I know I'm getting CMSampleBuffer right here in delegate callback function.
MY Questions.
is CMSampleBuffer is 'RAW' data which is from image sensor?
I want to know whole flow of making CMSampleBuffer to here. But I could not find any detail flow.
I found some possible solutions searching on stackoverflow
(Raw image data from camera like "645 PRO")
in this question, the author tried using opencv and got what he wanted.
but.. what is exactly different the result by opencv from CMSamplebuffer?(why the opencv result is real raw data the author said)
And also (How to get the Y component from CMSampleBuffer resulted from the AVCaptureSession?)
if i set as below,
if self.session.canAddOutput(self.captureOutput) {
self.session.addOutput(self.captureOutput)
captureOutput.videoSettings = [
kCVPixelBufferPixelFormatTypeKey : kCVPixelFormatType_32BGRA
] as [String : Any]
captureOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "capture"))
captureOutput.alwaysDiscardsLateVideoFrames = true
}
by setting format key _32BGRA,, AM i now getting raw data from samplebuffer?
It's not RAW in your case. All modern sensors build on Bayer filter, so you get an image converted from the Bayer format. You can't get raw image with this api. There is a format called kCVPixelFormatType_14Bayer_BGGR, but the camera probably won't support it.
Maybe on WWDC 419 session you will find answer. I don't know
It's the same; cv::Mat is just a wrapper around the image data from CMSampleBuffer. If you save your data as PNG, you will not lose any quality. The TIFF format saves without any compression, but you can also use PNG without compression.
If you use RGBA format, behind the scenes it is converted from Bayer to RGBA. To get the Y channel, you need to additionally apply an RGBA to YUV conversion and take the Y channel. Or you can use the kCVPixelFormatType_420YpCbCr8BiPlanarFullRange format and get the Y channel from the first plane. Also note that VideoRange has different chroma output range.
// ObjC code
int width = CVPixelBufferGetWidth(imageBuffer);
int height = CVPixelBufferGetHeight(imageBuffer);
uint8_t *yBuffer = CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 0);
size_t yPitch = CVPixelBufferGetBytesPerRowOfPlane(imageBuffer, 0);
cv::Mat yChannel(height, width, CV_8UC1, yBuffer, yPitch);
I'm creating a program in OpenCV (2.4.8) which should read video files and do some calculations on them. For these calculations I don't need the high-res frames, I'm perfectly fine with 640*360 as resolution.
In early tests I had my webcam attached and I used:
VideoCapture cap(0);
cap.set(CV_CAP_PROP_FRAME_WIDTH, 640);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 360);
Mat image;
cap.read(image);
namedWindow("firstframe", 1);
imshow("firstframe", image);
waitKey(0);
Which resized the image perfectly. Now I'm getting to the next step where I want to use my program for stored video instead of a live feed (which I used for testing).
When I change the '0' with the source file path (string), the file is loaded, but the resolution remains 1920*1080.
Did I do anything wrong? Is there a way to load the video at a lower resolution 'on the fly'?
I've read the OpenCV documentation. Some of the settings are labeled 'cameras only' but this setting isn't:
http://docs.opencv.org/modules/highgui/doc/reading_and_writing_images_and_video.html#videocapture-videocapture
I'm using OpenCV on a mac, and installed it with MacPorts.
Let me know if any more details are needed.
Thanks ahead for your help
edit:
I've realised that the cap.set(...) functions return a boolean, so I've tried printing them out and they both return 0. This of course confirms that the frame isn't resized. Still no clue as to why...
edit 2 :
So now, for a temporary solution I use the following line after read(image):
resize(image, image, Size(640, 360), 0, 0, INTER_CUBIC);
And this works. But I'm guessing this isn't really the most optimal solution.
using
resize(image, image, Size(640, 360), 0, 0, INTER_CUBIC);
after read(image) seems to be the best solution to solve this problem.
So the total (test) code becomes:
VideoCapture cap("path/to/file");
cap.set(CV_CAP_PROP_FRAME_WIDTH, 640);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 360);
Mat image;
cap.read(image);
resize(image, image, Size(640, 360), 0, 0, INTER_CUBIC);
namedWindow("firstframe", 1);
imshow("firstframe", image);
waitKey(0);
If anyone knows of a better way, please let me know.
My question is similar to OpenCV: Detect blinking lights in a video feed
openCV detect blinking lights
I want to detect LED on/off status from any image which will have LED object. LED object can be of any size ( but mostly circle ). It is important to get the location of all the LEDs in that image although it can be ON or OFF. First of all I would like to get the status and position of LEDs which are ON only. Right now image source is static for my work out but it must be from video of any product having glowing LEDs. So there is no chance of having template image to substract the background.
I have tried using OpenCV (new to OpenCV) with threshold, Contours and Circles methods but not found sucessful. Please share if any source code or solution. The solution can be anything not only using OpenCV which would give result for me. It would be greatly appreciated.
The difference from other two question is that I want to get the number of LEDs in the image whether it can be ON or OFF and status of all LEDs. I know this is very complex. First of all I was trying to detect glowing LEDs in the image. I have implemented the code which i shared below. I had different implementations but below code is able to show me the glowing LEDs just by drawing the contours but number of contours are more than the glowing LEDs. So I am not able to get the total number of LEDs glowing atleast. Please suggest me your inputs.
int main(int argc, char* argv[])
{
IplImage* newImg = NULL;
IplImage* grayImg = NULL;
IplImage* contourImg = NULL;
float minAreaOfInterest = 180.0;
float maxAreaOfInterest = 220.0;
//parameters for the contour detection
CvMemStorage * storage = cvCreateMemStorage(0);
CvSeq * contours = 0;
int mode = CV_RETR_EXTERNAL;
mode = CV_RETR_CCOMP; //detect both outside and inside contour
cvNamedWindow("src", 1);
cvNamedWindow("Threshhold",1);
//load original image
newImg = cvLoadImage(argv[1], 1);
IplImage* imgHSV = cvCreateImage(cvGetSize(newImg), 8, 3);
cvCvtColor(newImg, imgHSV, CV_BGR2HSV);
cvNamedWindow("HSV",1);
cvShowImage( "HSV", imgHSV );
IplImage* imgThreshed = cvCreateImage(cvGetSize(newImg), 8, 1);
cvInRangeS(newImg, cvScalar(20, 100, 100), cvScalar(30, 255, 255), imgThreshed);
cvShowImage( "src", newImg );
cvShowImage( "Threshhold", imgThreshed );
//make a copy of the original image to draw the detected contour
contourImg = cvCreateImage(cvGetSize(newImg), IPL_DEPTH_8U, 3);
contourImg=cvCloneImage( newImg );
cvNamedWindow("Contour",1);
//find the contour
cvFindContours(imgThreshed, storage, &contours, sizeof(CvContour), mode, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));
int i = 0;
for (; contours != 0; contours = contours->h_next)
{
i++;
//ext_color = CV_RGB( rand()&255, rand()&255, rand()&255 ); //randomly coloring different contours
cvDrawContours(contourImg, contours, CV_RGB(0, 255, 0), CV_RGB(255, 0, 0), 2, 2, 8, cvPoint(0,0));
}
printf("Total Contours:%d\n", i);
cvShowImage( "Contour", contourImg );
cvWaitKey(0);
cvDestroyWindow( "src" ); cvDestroyWindow( "Threshhold" );
cvDestroyWindow( "HSV" );
cvDestroyWindow( "Contour" );
cvReleaseImage( &newImg ); cvReleaseImage( &imgThreshed );
cvReleaseImage( &imgHSV );
cvReleaseImage( &contourImg );
}
I has some time yesterday night, here is a (very) simple and partial solution that works fine for me.
I created a git repository that you can directly clone :
git://github.com/jlengrand/image_processing.git
and run using Python
$ cd image_processing/LedDetector/
$ python leddetector/led_highlighter.py
You can see the code here
My method :
Convert to one channel image
Search for brightest pixel, assuming that we have at least one LED on and a dark background as on your image
Create a binary image with the brightest part of the image
Extract the blobs from the image, retrieve their center and the number of leds.
The code only takes an image into account at this point, but you can enhance it with a loop to take a batch of images (I already provides some example images in my repo.)
You simply have to play around a bit with the center found for LEDs, as they might not be one pixel accurate from one image to another (center could be slightly shifted).
In order to get the algorithm more robust (know whether there is a LED on or not, find an automatic and not hard coded margin value), you can play around a bit with the histogram (placed in extract_bright).
I already created the function for that you should just have to enhance it a bit.
Some more information concerning the input data :
Opencv does only accept avi files for now, so you will have to convert the mp4 file to avi (uncompressed in my case). I used this, that worked perfectly.
For some reason, the queryframe function caused memory leaks on my computer. That is why I created the grab_images functions, that takes the avi file as input and creates a batch of jpg images that you can use easier.
Here is the result for an image :
Input image :
Binary image :
Final result :
Hope this helps. . .
EDIT :
Your problem is slightly more complex if you want to use this image. The method I posted could still be used, but needs to be a bit complexified.
You want to detect the leds that display 'an information' (status, bandwidth, . . . ) and discard the design part.
I see three simple solutions to this :
you have a previous knowledge of the position of the leds. In this case, you can apply the very same method, but on a precise part of the whole image (using cv.SetImageROI).
you have a previsous knowledge of the color of the leds (you can see on the image that there are two different colors). Then you can search the whole image, and then apply a color filter to restrain your choice.
you have no previous knowledge. In this case, things get a bit more complex. I would tend to say that leds that are not useful should all have the same color, and that status leds usually blink. This means that by adding a learning step to the method, you might be able to see which leds actually have to be selected as useful.
Hope this brings some more food for thoughts
I am facing a problem while writing image from cvMat.
This is what I have done.
IplImage* low_threshold_mask = cvCreateImage(cvSize(width, height), IPL_DEPTH_8U, 1);
CvMat* labelMat = cvCreateMat(low_threshold_mask->height,low_threshold_mask->width,CV_32F);
/* I populate elements of labelMat inside a function. Its done like this: cvmSet(labelMat,r,c,label); // where label is of type long */
To check the values I dump each pixel value in a text file and also dump the image.
IplImage* labelImg;
IplImage imageHeader;
labelImg = cvGetImage(labelMat, &imageHeader);
Now when I cross-check pixel intensity with corresponding value in dumped text file, I find mis-match. I feel I have got correct values in text file but wrong ones in image.
Can anyone help in figuring out the mistake?
---------------------New addition-------------------
I am still facing the problem. I have uploaded my programs. I will explain where exactly I am facing the error.
Libraries used: Along with openCV, I am using disjoint_sets of boost library.
Basically I am doing connected component labeling.
For debugging purpose, for 20th frame, I have dumped the label info of each pixel both in a)text file as well b) an image with intensity levels same as the final label of the pixels. So i am expecting the values same in both text and image. But that's not happening. And I am unable to figure out why. The text files shows the correct values but not the image. I am checking pixel values of image in Matlab(i had taken care of indices in matlab starts with 1 not 0).
My text files
a) (frame20final.txt) gets populated in GrimsonGMM.cpp/ConCompLabeling().
b) (frame20image.txt) gets populated in main.cpp
My dumped image(frame-ccs.jpg) gets populated in main.cpp.
Both the text files get same values. So there must be some mistake in writing the image from CvMat.
Test Video: person15_walking_d1_uncomp.avi
You can try with any other video also.
Thanks in advance,
Kaushik
I understood why I was getting the error. I was dumping in .jpg image which was doing compression. This was resolved when I used .png
Your question its so easy.
You want to work with CvMat and after do operations with CvMat you want to plot your CvMat like it was an image.
You must create imageHeader, something like this.
CvMat* mat = cvCreateMatHeader(rows, cols, type);
mat->step = 4 * (mat->cols * CV_ELEM_SIZE1(mat->type) * CV_MAT_CN(mat->type) / 4 + 1);//critical
cvCreateData(mat);
In OpenCV 2.0 and below C++ interface it's not really necessary to change from Mat to IplImage.
You can plot your image using cvShowImage and if you want to convert intoIplImage
just do a easy cast IplImage *img = labelMat;
I'm new to OpenCV and just started sifting through the APIs. I intend to fetch the color, intensity and texture values of each pixel constituting the image. I was fiddling with the structure - IplImage to start with but couldn't make much progress.
Please let me know of any means to do this.
cheers
Have you tried OpenCV 2.0? They have a new C++ interface which makes things much easier. You can use their new Mat class to load images, access pixels efficiently, etc. It's much cleaner than IplImage fun. I use \doc\opencv.pdf as my reference to anything I need. It's got tutorials, and examples with the new C++ interface, etc. - enough and more to get you started.
If you have anymore specific OpenCV questions, please feel free to ask.
Here's some demo code to get you started: (I've used the cv namespace):
// Load the image (looks like MATLAB :) ? )
Mat M = imread("h:\\lena.bmp");
// Display
namedWindow("Lena",CV_WINDOW_AUTOSIZE);
imshow("Lena",M);
waitKey();
// Crop out rectangle from (100,100) of size (200,200) of the red channel
const int offset[2] = {100,100};
const int dims[2] = {200,200};
Mat Red(dims[0],dims[1],CV_8UC1);
// Read it from M into Red
uchar* lena = M.data;
for(int i=0;i<dims[0];++i)
for(int j=0;j<dims[0];++j)
{
// P = i*rows*channels + j*channels + c
Red.at<uchar>(i,j) = *(lena + (i+offset[0])*M.rows*M.channels() + (j+offset[1])*M.channels()+0);
}
//Display
namedWindow("RedRect",CV_WINDOW_AUTOSIZE);
imshow("RedRect",Red);
waitKey();