I am trying to use an HSV threshold to find a red cup in an image. One of my implementations uses the cvVideoCamera delegate and gets a mat straight from the camera. My other implementation lets the user record the video then we extract the frames using AVFoundation for processing.
When I threshold the image from AVFoundation I get nothing back besides a black image.
Here is my code:
inRange(gray, Scalar(114, 135, 135), Scalar(142, 255, 255), dst);
The first image is an example of an image that works properly and the second is an image from AVFoundation which does not threshold how I expect, it produces an all black image.
Does anyone have an idea why the second image produces different results when the color of the cup looks quite similar?
An image pulled from a video is reconstructed in a manner that depends on the video's encoding and compression codec. Short version is that unless you happen to pick out a keyframe (which you generally won't have an api to do, so trying to do this isn't viable), you're getting a image that is reconstructed from the video.
So a image taken from the video from same time that you took a straight-up image from the camera (assuming you could do both at the same time) would be different. After any sort processing you'll, of course, get different result.
Before applying your threshold, get the raw image for before approaches. Look at them (or a delta of them) and you'll see that just aren't the same image. The second approach will likely have introduced artifacts from being encoded into video from multiple frames, then reconstructed into a single frame image.
To overcome this issue, use OpenCV Native conversion functions.
#import <opencv2/imgcodecs/ios.h>
UIImage* MatToUIImage(const cv::Mat& image);
void UIImageToMat(const UIImage* image,
cv::Mat& m, bool alphaExist = false);
Related
ARKit runs at 60 frames/sec, which equates to 16.6ms per frame.
My current code to convert the CVPixelBufferRef (kCVPixelFormatType_420YpCbCr8BiPlanarFullRange format) to a cv::Mat (YCrCb) runs in 30ms, which causes ARKit to stall and everything to lag.
Does anyone have any ideas on how to to a quicker conversion or do I need to drop the frame rate?
There is a suggestion by Apple to use Metal, but I'm not sure how to do that.
Also I could just take the grayscale plane, which is the first channel, which runs in <1ms, but ideally I need the colour information as well.
In order to process an image in a pixel buffer using Metal, you need to do following.
Call CVMetalTextureCacheCreateTextureFromImage to create CVMetalTexture object on top of the pixel buffer.
Call CVMetalTextureGetTexture to create a MTLTexture object, which Metal code (GPU) can read and write.
Write some Metal code to convert the color format.
I have an open source project (https://github.com/snakajima/vs-metal), which processes pixel buffers (from camera, not ARKit) using Metal. Feel free to copy any code from this project.
I tried to convert Ycbcr to RGB, do image processing in RGB mat and convert it back to Ycbcr, it worked very slowly. I suggest only do that with a static image. For realtime processing, we should process directly in cv::Mat. ARFrame.capturedImage is Ycbcr buffer. So, the solution is
Sperate the buffer to 2 cv::Mat (yPlane and cbcrPlane). Keep in mind, we do not clone memory, we create 2 cv::Mat with base addresses is yPlane address and cbcrPlane address.
Do image process on yPlane and cbcrPlane, size(cbcrPlane) = size(yPlane) / 2.
You can check out my code here: https://gist.github.com/ttruongatl/bb6c69659c48bac67826be7368560216
I have two webcams (both are Logitech C615). I want to adjust the webcams in a way that they make nearly the same picture in same environment. (the reason is that I want to render this images onto an occulus rift).
I'm using OpenCV to connect the cameras. My first innocent try was to get all the CV_CAP_PROPs from the one cam and set the values to the other cam. That doesn't work very well.
Is there may already a function I could use or can you give me another approach?
--- EDIT: histogram equalization ---
That's the result:
The result is better than before but as you can see the hue is different.
Try to convert them to YCrCb and equalize just the Y channel (and convert them back to BGR if you need to).
This should equalize the brightness of both images.
Snippet:
cv::cvtColor( frame, frame, CV_BGR2YCrCb);
cv::split( frame, channels);
cv::equalizeHist( channels[0], channels[0] );
cv::merge( channels, 3, frame );
cv::cvtColor( frame, frame, CV_YCrCb2BGR );
I want to do following thing within in my iOS app:
user can draw something on white background paper.
my app allows user to capture the drawn image. Here the image will capture with background white color.
finally from the captured image i need to mask the white background color and just get the image alone into UIImage object.
I completed the steps 1 and 2. But i do not have any idea how to do the last step. Is there any openCV library that i can use it with my iOS app?.
Any help that might be really appreciated.
Well, since OpenCV itself is THE library, I guess that you are looking for a way to do that with OpenCV.
First, convert the input image to Mat, which is the data type OpenCV uses to represent an image;
Then, assuming the background is white, threshold the Mat to separate the background from whatever the user draw. According to the example below, the result of this operation makes the background black, and every pixel that is not black will represent something the user has draw:
Finally, convert the resulting Mat to UIImage: for this, iterate on the Mat and copy every pixel that is not black to the UIImage to have an UIImage that contains only what the user draw.
A better idea is to iterate on the thresholded Mat, figure out which pixel is not black, and instead of copying it directly to the new UIImage, copy that pixel (x,y) from the original UIImage, so you have a colored pixel at the end, which gives a more realistic result.
So I am very new to OpenCV (2.1), so please keep that in mind.
So I managed to calibrate my cheap web camera that I am using (with a wide angle attachment), using the checkerboard calibration method to produce the intrinsic and distortion coefficients.
I then have no trouble feeding these values back in and producing image maps, which I then apply to a video feed to correct the incoming images.
I run into an issue however. I know when it is warping/correcting the image, it creates several skewed sections, and then formats the image to crop out any black areas. My question then is can I view the complete warped image, including some regions that have black areas? Below is an example of the black regions with skewed sections I was trying to convey if my terminology was off:
An image better conveying the regions I am talking about can be found here! This image was discovered in this post.
Currently: The cvRemap() returns basically the yellow box in the image linked above, but I want to see the whole image as there is relevant data I am looking to get out of it.
What I've tried: Applying a scale conversion to the image map to fit the complete image (including stretched parts) into frame
CvMat *intrinsic = (CvMat*)cvLoad( "Intrinsics.xml" );
CvMat *distortion = (CvMat*)cvLoad( "Distortion.xml" );
cvInitUndistortMap( intrinsic, distortion, mapx, mapy );
cvConvertScale(mapx, mapx, 1.25, -shift_x); // Some sort of scale conversion
cvConvertScale(mapy, mapy, 1.25, -shift_y); // applied to the image map
cvRemap(distorted,undistorted,mapx,mapy);
The cvConvertScale, when I think I have aligned the x/y shift correctly (guess/checking), is somehow distorting the image map making the correction useless. There might be some math involved here I am not correctly following/understanding.
Does anyone have any other suggestions to solve this problem, or what I might be doing wrong? I've also tried trying to write my own code to fix distortion issues, but lets just say OpenCV knows already how to do it well.
From memory, you need to use InitUndistortRectifyMap(cameraMatrix,distCoeffs,R,newCameraMatrix,map1,map2), of which InitUndistortMap is a simplified version.
cvInitUndistortMap( intrinsic, distort, map1, map2 )
is equivalent to:
cvInitUndistortRectifyMap( intrinsic, distort, Identity matrix, intrinsic,
map1, map2 )
The new parameters are R and newCameraMatrix. R species an additional transformation (e.g. rotation) to perform (just set it to the identity matrix).
The parameter of interest to you is newCameraMatrix. In InitUndistortMap this is the same as the original camera matrix, but you can use it to get that scaling effect you're talking about.
You get the new camera matrix with GetOptimalNewCameraMatrix(cameraMat, distCoeffs, imageSize, alpha,...). You basically feed in intrinsic, distort, your original image size, and a parameter alpha (along with containers to hold the result matrix, see documentation). The parameter alpha will achieve what you want.
I quote from the documentation:
The function computes the optimal new camera matrix based on the free
scaling parameter. By varying this parameter the user may retrieve
only sensible pixels alpha=0, keep all the original image pixels if
there is valuable information in the corners alpha=1, or get something
in between. When alpha>0, the undistortion result will likely have
some black pixels corresponding to “virtual” pixels outside of the
captured distorted image. The original camera matrix, distortion
coefficients, the computed new camera matrix and the newImageSize
should be passed to InitUndistortRectifyMap to produce the maps for
Remap.
So for the extreme example with all the black bits showing you want alpha=1.
In summary:
call cvGetOptimalNewCameraMatrix with alpha=1 to obtain newCameraMatrix.
use cvInitUndistortRectifymap with R being identity matrix and newCameraMatrix set to the one you just calculated
feed the new maps into cvRemap.
I have a method that does some processing on an IplImage and the method works as it should if the image is 640x480 pixels. But if it is 480x640 pixels, it does't not... because the image needs to be rotated to become 640x480 again, but then I need to rotate it back to 480x640 or translate ther coordinates taken from cvHaarDetectObjects to 480x640.
Can anybody tell me how can I do this?
thanks!!
Try transpose followed by flip. The flip is needed because transpose leaves an mirrored image when compared to the results of a rotation. If the algorithm can work with the mirrored image directly, I would recommend simply flipping the coordinate values of the detection result, rather than flipping the input image.
(Disclaimer: I haven't tried transpose or flip on multi-channel images)