Cocos2d: pixel perfect collision for batched sprites - ios

I found a pixel perfect collision algorithm developed by Daniel Vilchez and included in a project shared in this cocos2d-iphone.org forum topic.
Below there is the part of the algorithm I am interested. I am trying to modify this because whenever I used CCRenderTexture, as originally in the code, the App crashed.
I am thinking of alternative methods based on circle collision but those are "not pixel perfect" and in the case my bullet is a wave with this shape it wouldn't work well.
**I am wondering how can I get the algorithm working with sprites batched in a CCSpriteBatchNode? And if so does this strictly include the usage of CCRenderTexture? **
To be precise, this question is partially related to this other question of mine, on creating an instance of CCRenderTexture that causes my App to crash. I post two different ones because here I am asking about the algorithm, in the other one I just ask why CCRenderTexture causes my App to crash (without using Daniel's pixel perfect algorithm, but just creating an instance of CCRenderTexture).
Adapted CODE (here is missing CCRenderTexture because it made my app crashing, so I commented out the usage of _rt - instance of CCRenderTexture). The code does not work properly, so I guess I need CCRenderTexture and hence I asked the question:
-(BOOL) isPixelPerfectCollisionBetweenSpriteA:(CCSprite*)spr1 spriteB:(CCSprite*) spr2
{
BOOL isCollision = NO;
CGRect intersection = CGRectIntersection([spr1 boundingBox], [spr2 boundingBox]);
// Look for simple bounding box collision
if (!CGRectIsEmpty(intersection))
{
// Get intersection info
unsigned int x = intersection.origin.x;
unsigned int y = intersection.origin.y;
unsigned int w = intersection.size.width;
unsigned int h = intersection.size.height;
unsigned int numPixels = w * h;
//NSLog(#"\nintersection = (%u,%u,%u,%u), area = %u",x,y,w,h,numPixels);
// Draw into the RenderTexture
//[_rt beginWithClear:0 g:0 b:0 a:0];
// Render both sprites: first one in RED and second one in GREEN
glColorMask(1, 0, 0, 1);
[spr1 visit];
glColorMask(0, 1, 0, 1);
[spr2 visit];
glColorMask(1, 1, 1, 1);
// Get color values of intersection area
ccColor4B *buffer = malloc( sizeof(ccColor4B) * numPixels );
glReadPixels(x, y, w, h, GL_RGBA, GL_UNSIGNED_BYTE, buffer);
//[_rt end];
// Read buffer
unsigned int step = 1;
for(unsigned int i=0; i<numPixels; i+=step)
{
ccColor4B color = buffer[i];
if (color.r > 0 && color.g > 0)
{
isCollision = YES;
break;
}
}
// Free buffer memory
free(buffer);
}
return isCollision;
EDIT: I found also KKPixelMaskSprite but it doesn't seem to work for high resolution sprites batched in CCSpriteBatchNodes (see comment here).

Related

iOS Tesseract: bad results

I just started to get my hands dirty with the Tesseract library, but the results are really really bad.
I followed the instructions in the Git repository ( https://github.com/gali8/Tesseract-OCR-iOS ). My ViewController uses the following method to start recognizing:
Tesseract *t = [[Tesseract alloc] initWithLanguage:#"deu"];
t.delegate = self;
[t setVariableValue:#"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" forKey:#"tessedit_char_whitelist"];
[t setImage:img];
[t recognize];
NSLog( #"Recognized text: %#", [t recognizedText] );
labelRecognizedText.text = [t recognizedText];
t = nil;
The sample image from the project tempalte
works well (which tells me that the project itself is setup correctly), but whenever I try to use other images, the recognized text is a complete mess. For example, I tried to take a picture of my finder displaying the sample image:
https://dl.dropboxusercontent.com/u/607872/tesseract.jpg (1,5 MB)
But Tesseract recognizes:
Recognized text: s f l TO if v Ysssifss f
ssqxizg ss sfzzlj z
s N T IYIOGY Z I l EY s s
k Es ETL ZHE s UEY
z xhks Fsjs Es z VIII c
s I XFTZT c s h V Ijzs
L s sk sisijk J
s f s ssj Jss sssHss H VI
s s H
i s H st xzs
s s k 4 is x2 IV
Illlsiqss sssnsiisfjlisszxiij s
K
Even when the character whitelist only contains numbers, I don't get a result even close to what the image looks like:
Recognized text: 3 74 211
1
1 1 1
3 53 379 1
3 1 33 5 3 2
3 9 73
1 61 2 2
3 1 6 5 212 7
1
4 9 4
1 17
111 11 1 1 11 1 1 1 1
I assume there's something wrong with the way fotos are taken from the iPad mini's camera I currently use, but I can't figure out what and why.
Any hints?
Update #1
In response to Tomas:
I followed the tutorial in your post but encountered several errors along the way...
the UIImage+OpenCV category cannot be used in my ARC project
I cannot import <opencv2/...> in my controllers, auto-completion does not offer it (and therefore [UIImage CVMat] is not defined)
I think there's something wrong with my integration of OpenCV, even though I followed the Hello-tutorial and added the framework. Am I required to build OpenCV on my Mac as well or is it sufficient to just include the framework in my Xcode project?
Since I don't really know what you might consider as "important" at this point (I've already read several posts and tutorials and tried different steps), feel free to ask :)
Update #2
#Tomas: thanks, the ARC-part was essential. My ViewController already has been renamed to .mm. Forget the part about "cannot import opencv2/" since I already included it in my TestApp-Prefix.pch (as stated in the Hello-tutorial).
On to the next challenge ;)
I noticed that when I use images taken with the camera, the bounds for the roi object aren't calculated successfully. I played around with the device orientation and put a UIImage in my view to see the image processing steps, but sometimes (even when the image is correctly aligned) the values are negative because the if-condition in the bounds.size()-for-loop isn't met. The worst case I had: minX/Y and maxX/Y were never touched. Long story short: the line starting with Mat roi = inranged(cv::Rect( throws an exception (assertion failed because the values were < 0 ). I don't know if the number of contours matter, but I assume so because the bigger the images, the more likely the assertion exception is.
To be perfectly honest: I haven't had the time to read OpenCV's documentation and understand what your code does, but as of now, I don't think there's a way around. Seems like, unfortunately for me, my initial task (scan receipt, run OCR, show items in a table) requires more resources (=time) than I thought.
There's nothing wrong in the way your taking the pictures from your iPad per se. But you just can't throw in such a complex image and expect Tesseract to magically determine which text to extract. Take a closer look to the image and you'll notice it has no uniform lightning, it's extremely noisy so it may not be the best sample to start playing with.
In such scenarios it is mandatory to pre process the image in order to provide the tesseract library with something simpler to recognise.
Below find a very naive pre processing example that uses OpenCV (http://www.opencv.org), a popular image processing framework. It should give you and idea to get you started.
#import <TesseractOCR/TesseractOCR.h>
#import <opencv2/opencv.hpp>
#import "UIImage+OpenCV.h"
using namespace cv;
...
// load source image
UIImage *img = [UIImage imageNamed:#"tesseract.jpg"];
Mat mat = [img CVMat];
Mat hsv;
// convert to HSV (better than RGB for this task)
cvtColor(mat, hsv, CV_RGB2HSV_FULL);
// blur is slightly to reduce noise impact
const int blurRadius = img.size.width / 250;
blur(hsv, hsv, cv::Size(blurRadius, blurRadius));
// in range = extract pixels within a specified range
// here we work only on the V channel extracting pixels with 0 < V < 120
Mat inranged;
inRange(hsv, cv::Scalar(0, 0, 0), cv::Scalar(255, 255, 120), inranged);
Mat inrangedforcontours;
inranged.copyTo(inrangedforcontours); // findContours alters src mat
// now find contours to find where characters are approximately located
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
findContours(inrangedforcontours, contours, hierarchy, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0));
int minX = INT_MAX;
int minY = INT_MAX;
int maxX = 0;
int maxY = 0;
// find all contours that match expected character size
for (size_t i = 0; i < contours.size(); i++)
{
cv::Rect brect = cv::boundingRect(contours[i]);
float ratio = (float)brect.height / brect.width;
if (brect.height > 250 && ratio > 1.2 && ratio < 2.0)
{
minX = MIN(minX, brect.x);
minY = MIN(minY, brect.y);
maxX = MAX(maxX, brect.x + brect.width);
maxY = MAX(maxY, brect.y + brect.height);
}
}
// Now we know where our characters are located
// extract relevant part of the image adding a margin that enlarges area
const int margin = img.size.width / 50;
Mat roi = inranged(cv::Rect(minX - margin, minY - margin, maxX - minX + 2 * margin, maxY - minY + 2 * margin));
cvtColor(roi, roi, CV_GRAY2BGRA);
img = [UIImage imageWithCVMat:roi];
Tesseract *t = [[Tesseract alloc] initWithLanguage:#"eng"];
[t setVariableValue:#"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" forKey:#"tessedit_char_whitelist"];
[t setImage:img];
[t recognize];
NSString *recognizedText = [[t recognizedText] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];
if ([recognizedText isEqualToString:#"1234567890"])
NSLog(#"Yeah!");
else
NSLog(#"Epic fail...");
Notes
The UIImage+OpenCV category can be found here. If you're under ARC check this.
Take a look at this to get you started with OpenCV in Xcode. Note that OpenCV is a C++ framework which can't be imported in plain C (or Objective-C) source files. The easiest workaround is to rename your view controller from .m to .mm (Objective-C++) and reimport it in your project.
There is different behavior of tesseract result.
It requires good quality of picture means good texture visibility.
Large size picture take much time to process its also good to resize it into small before processing.
It will good to perform some color effect on image before sending it to tesseract. Use effects which could enhance the visibility of image.
There is sometime different behavior of processing photo by using Camera or by Camera Album.
In case of taking photo directly from Camera try below function.
- (UIImage *) getImageForTexture:(UIImage *)src_img{
CGColorSpaceRef d_colorSpace = CGColorSpaceCreateDeviceRGB();
/*
* Note we specify 4 bytes per pixel here even though we ignore the
* alpha value; you can't specify 3 bytes per-pixel.
*/
size_t d_bytesPerRow = src_img.size.width * 4;
unsigned char * imgData = (unsigned char*)malloc(src_img.size.height*d_bytesPerRow);
CGContextRef context = CGBitmapContextCreate(imgData, src_img.size.width,
src_img.size.height,
8, d_bytesPerRow,
d_colorSpace,
kCGImageAlphaNoneSkipFirst);
UIGraphicsPushContext(context);
// These next two lines 'flip' the drawing so it doesn't appear upside-down.
CGContextTranslateCTM(context, 0.0, src_img.size.height);
CGContextScaleCTM(context, 1.0, -1.0);
// Use UIImage's drawInRect: instead of the CGContextDrawImage function, otherwise you'll have issues when the source image is in portrait orientation.
[src_img drawInRect:CGRectMake(0.0, 0.0, src_img.size.width, src_img.size.height)];
UIGraphicsPopContext();
/*
* At this point, we have the raw ARGB pixel data in the imgData buffer, so
* we can perform whatever image processing here.
*/
// After we've processed the raw data, turn it back into a UIImage instance.
CGImageRef new_img = CGBitmapContextCreateImage(context);
UIImage * convertedImage = [[UIImage alloc] initWithCGImage:
new_img];
CGImageRelease(new_img);
CGContextRelease(context);
CGColorSpaceRelease(d_colorSpace);
free(imgData);
return convertedImage;
}
I have been struggling with Tesseract character recognition for weeks. Here are two things I learned to get it to work better...
If you know what font you will be reading, clear the training and retrain it for only that font. Multiple fonts slows the OCR processing down and also increases the ambiguity in the Tesseract decision process. This will lead to greater accuracy and speed.
After OCR processing is really needed. You will end up with a matrix of characters that Tesseract recognizes. You will need to further process the characters to narrow down on what you are trying to read. So for instance, if your application is reading food labels, knowing the rules for the words and sentences that make up the food label will help recognize a series of characters that make up that label.
Convert your UIImage from srgb to rgb format .
if you are using IOS 5.0 and above use
use #import <Accelerate/Accelerate.h>
else uncomment //IOS 3.0-5.0
-(UIImage *) createARGBImageFromRGBAImage: (UIImage*)image
{ //CGSize size = CGSizeMake(320, 480);
CGSize dimensions = CGSizeMake(320, 480);
NSUInteger bytesPerPixel = 4;
NSUInteger bytesPerRow = bytesPerPixel * dimensions.width;
NSUInteger bitsPerComponent = 8;
unsigned char *rgba = malloc(bytesPerPixel * dimensions.width * dimensions.height);
unsigned char *argb = malloc(bytesPerPixel * dimensions.width * dimensions.height);
CGColorSpaceRef colorSpace = NULL;
CGContextRef context = NULL;
colorSpace = CGColorSpaceCreateDeviceRGB();
context = CGBitmapContextCreate(rgba, dimensions.width, dimensions.height, bitsPerComponent, bytesPerRow, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrderDefault); // kCGBitmapByteOrder32Big
CGContextDrawImage(context, CGRectMake(0, 0, dimensions.width, dimensions.height), [image CGImage]);
CGContextRelease(context);
CGColorSpaceRelease(colorSpace);
const vImage_Buffer src = { rgba, dimensions.height, dimensions.width, bytesPerRow };
const vImage_Buffer dis = { rgba, dimensions.height, dimensions.width, bytesPerRow };
const uint8_t map[4] = {3,0,1,2};
vImagePermuteChannels_ARGB8888(&src, &dis, map, kvImageNoFlags);
//IOS 3.0-5.0
/*for (int x = 0; x < dimensions.width; x++) {
for (int y = 0; y < dimensions.height; y++) {
NSUInteger offset = ((dimensions.width * y) + x) * bytesPerPixel;
argb[offset + 0] = rgba[offset + 3];
argb[offset + 1] = rgba[offset + 0];
argb[offset + 2] = rgba[offset + 1];
argb[offset + 3] = rgba[offset + 2];
}
}*/
colorSpace = CGColorSpaceCreateDeviceRGB();
context = CGBitmapContextCreate(dis.data, dimensions.width, dimensions.height, bitsPerComponent, bytesPerRow, colorSpace, kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrderDefault); // kCGBitmapByteOrder32Big
CGImageRef imageRef = CGBitmapContextCreateImage(context);
image = [UIImage imageWithCGImage: imageRef];
CGImageRelease(imageRef);
CGContextRelease(context);
CGColorSpaceRelease(colorSpace);
free(rgba);
free(argb);
return image;
}
Tesseract *t = [[Tesseract alloc] initWithLanguage:#"eng"];
[t setVariableValue:#"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" forKey:#"tessedit_char_whitelist"];
[t setImage:[self createARGBImageFromRGBAImage:img]];
[t recognize];
The swift equivalent of #FARAZ's answer
func getImageForTexture(srcImage: UIImage) -> UIImage{
let d_colorSpace = CGColorSpaceCreateDeviceRGB()
let d_bytesPerRow: size_t = Int(srcImage.size.width) * 4
/*
* Note we specify 4 bytes per pixel here even though we ignore the
* alpha value; you can't specify 3 bytes per-pixel.
*/
let imgData = malloc(Int(srcImage.size.height) * Int(d_bytesPerRow))
let context = CGBitmapContextCreate(imgData, Int(srcImage.size.width), Int(srcImage.size.height), 8, Int(d_bytesPerRow), d_colorSpace,CGImageAlphaInfo.NoneSkipFirst.rawValue)
UIGraphicsPushContext(context!)
// These next two lines 'flip' the drawing so it doesn't appear upside-down.
CGContextTranslateCTM(context, 0.0, srcImage.size.height)
CGContextScaleCTM(context, 1.0, -1.0)
// Use UIImage's drawInRect: instead of the CGContextDrawImage function, otherwise you'll
srcImage.drawInRect(CGRectMake(0.0, 0.0, srcImage.size.width, srcImage.size.height))
UIGraphicsPopContext()
/*
* At this point, we have the raw ARGB pixel data in the imgData buffer, so
* we can perform whatever image processing here.
*/
// After we've processed the raw data, turn it back into a UIImage instance.
let new_img = CGBitmapContextCreateImage(context)
let convertedImage = UIImage(CGImage: new_img!)
return convertedImage
}

How to draw many gradient lines quickly

I'm developing an app that has to draw 320 vertical gradient lines on a portrait iPhone screen where each gradient line is either 1px or 2px wide (non-retina vs retina). Each gradient line has 1000 positions, with each position able to have a unique color. These 1000 colors (floats) sit in a C-style 2D array (an array of arrays, 320 arrays of 1000 colors)
Currently, the gradient lines are drawn in a For Loop inside the drawRect method of a custom UIView. The problem I'm having is that it takes longer than ONE second to cycle through the For Loop and draw all 320 lines. Within that ONE second, I have another thread that's updating the color arrays and but since it takes longer than ONE second to draw, I don't see every update. I see every second or third update.
I'm using the exact same procedure in my Android code, which has no problems drawing 640 gradient lines (double the amount) multiple times in a second using a SurfaceView. My Android app never misses an update.
If you look at the Android code, it actually draws gradient lines to TWO separate canvases. The array size is dynamic and can be up to half the landscape resolution width of an Android phone (ex 1280 width = 1280/2 = 640 lines). Since the Android app is fast enough, I allow landscape mode. Even with the double the data as an iPhone and drawing to two separate canvases, the Android code runs multiple times a second. The iPhone code with half the number of lines and only drawing to a single context can not draw in under a second.
Is there a faster way to draw 320 vertical gradient lines (each with 1000 positions) on an iPhone?
Is there a hardware accelerated SurfaceView equivalent for iOS that can draw many gradients really fast?
//IPHONE - drawRect method
int totalNumberOfColors = 1000;
int i;
CGFloat *locations = malloc(totalNumberOfColors * sizeof locations[0]);
for (i = 0; i < totalNumberOfColors; i++) {
float division = (float)1 / (float)(totalNumberOfColors - 1);
locations[i] = i * division;
}
CGContextRef ctx = UIGraphicsGetCurrentContext();
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
for (int k = 0; k < 320; k++) {
CGFloat * colorComponents = arrayOfFloatArrays[k];
CGGradientRef gradient = CGGradientCreateWithColorComponents(
colorSpace,
colorComponents,
locations,
(size_t)(totalNumberOfColors));
CGRect newRect;
if (currentPositionOffset >=320) {
newRect = CGRectMake(0, 0, 1, CGRectGetMaxY(rect));
} else {
newRect = CGRectMake(319 - (k * 1), 0, 1, CGRectGetMaxY(rect));
}
CGContextSaveGState(ctx);
//NO CLIPPING STATE
CGContextAddRect(ctx, newRect);
CGContextClip(ctx);
//CLIPPING STATE
CGContextDrawLinearGradient(
ctx,
gradient,
CGPointMake(0, 0),
CGPointMake(0, CGRectGetMaxY(rect)),
(CGGradientDrawingOptions)NULL);
CGContextRestoreGState(ctx);
//RESTORE TO NO CLIPPING STATE
CGGradientRelease(gradient);
}
//ANDROID - public void run() method on SurfaceView
for (i = 0; i < sonarData.arrayOfColorIntColumns.size() - currentPositionOffset; i++) {
Paint paint = new Paint();
int[] currentColors = sonarData.arrayOfColorIntColumns.get(currentPositionOffset + i);
//Log.d("currentColors.toString()",currentColors.toString());
LinearGradient linearGradient;
if (currentScaleFactor > 1.0) {
int numberOfColorsToUse = (int)(1000.0/currentScaleFactor);
int tmpTopOffset = currentTopOffset;
if (currentTopOffset + numberOfColorsToUse > 1000) {
//shift tmpTopOffset
tmpTopOffset = 1000 - numberOfColorsToUse - 1;
}
int[] subsetOfCurrentColors = new int[numberOfColorsToUse];
System.arraycopy(currentColors, tmpTopOffset, subsetOfCurrentColors, 0, numberOfColorsToUse);
linearGradient = new LinearGradient(0, tmpTopOffset, 0, getHeight(), subsetOfCurrentColors, null, Shader.TileMode.MIRROR);
//Log.d("getHeight()","" + getHeight());
//Log.d("subsetOfCurrentColors.length","" + subsetOfCurrentColors.length);
} else {
//use all colors
linearGradient = new LinearGradient(0, 0, 0, getHeight(), currentColors, null, Shader.TileMode.MIRROR);
//Log.d("getHeight()","" + getHeight());
//Log.d("currentColors.length","" + currentColors.length);
}
paint.setShader(linearGradient);
sonarData.checkAndAddPaint(paint);
numberOfColumnsToDraw = i + 1;
}
//Log.d(TAG,"numberOfColumnsToDraw " + numberOfColumnsToDraw);
currentPositionOffset = currentPositionOffset + i;
if (currentPositionOffset >= sonarData.getMaxNumberOfColumns()) {
currentPositionOffset = sonarData.getMaxNumberOfColumns() - 1;
}
if (numberOfColumnsToDraw > 0) {
Canvas canvas = surfaceHolder.lockCanvas();
if (AppInstanceData.sonarBackgroundImage != null && canvas != null) {
canvas.drawBitmap(AppInstanceData.sonarBackgroundImage, 0, getHeight()- AppInstanceData.sonarBackgroundImage.getHeight(), null);
if (cacheCanvas != null) {
cacheCanvas.drawBitmap(AppInstanceData.sonarBackgroundImage, 0, getHeight()- AppInstanceData.sonarBackgroundImage.getHeight(), null);
}
}
for (i = drawOffset; i < sizeToDraw + drawOffset; i++) {
Paint p = sonarData.paintArray.get(i - dataStartOffset);
p.setStrokeWidth(2);
//Log.d("drawGradientLines", "canvas.getHeight() " + canvas.getHeight());
canvas.drawLine(getWidth() - (i - drawOffset) * 2, 0, getWidth() - (i - drawOffset) * 2, canvas.getHeight(), p);
if (cacheCanvas != null) {
cacheCanvas.drawLine(getWidth() - (i - drawOffset) * 2, 0, getWidth() - (i - drawOffset) * 2, canvas.getHeight(), p);
}
}
surfaceHolder.unlockCanvasAndPost(canvas);
}
No comment on the CG code — it's been a while since I've drawn any gradients — but a couple of notes:
You shouldn't be doing that in drawRect because it's called a lot. Draw into an image and display it.
There's no matching free for the malloc, so you're leaking memory like crazy.
It'll have a learning curve, but implement this using OpenGL ES 2.0. I previously took something that was drawing a large number of gradients as well, and reimplemented it using OpenGL ES 2.0 and custom vertex and fragment shaders. It is way faster than the equivalent drawing done using Core Graphics, so you will probably see a big speed boost as well.
If you don't know any OpenGL yet, I would suggest finding some tutorials for working with OpenGL ES 2.0 (has to be 2.0 because that's what offers the ability to write custom shaders) on iOS, to learn the basics. Once you do that, you should be able to significantly increase the performance of your drawing, way above that of the Android version, and maybe would be incentive to make the Android version use OpenGL as well.

Detect basket ball Hoops and ball tracking

Detect the hoop(basket).To see the samples of "hoop".
Count the no of successful attempts(shoot) and the failure attempts.
I am using opencv.
Input:
Camera position will be static.
The Portrait mode videos from any mobile device.
ref:
What have i tried:
Able to track the basket ball. Still, seeking for a better solution.
results:
My code:
int main () {
VideoCapture vid(path);
if (!vid.isOpened())
exit(-1);
int i_frame_height = vid.get(CV_CAP_PROP_FRAME_HEIGHT);
i_height_basketball = i_height_basketball * I_HEIGHT / i_frame_height;
int fps = vid.get(CV_CAP_PROP_FPS);
Mat mat_black(640, 480, CV_8UC3, Scalar(0, 0, 0));
vector <Mat> vec_frames;
for (int i_push = 0; i_push < I_NO_FRAMES_STORE; i_push++)
vec_frames.push_back(mat_black);
vector <Mat> vec_mat_result;
for (int i_push = 0; i_push < I_RESULT_STORE; i_push++)
vec_mat_result.push_back(mat_black);
int count_frame = 0;
while (true) {
int clk_start = clock();
Mat image, result;
vid >> image;
if (image.empty())
break;
resize(image, image, Size(I_WIDTH, I_HEIGHT));
image.copyTo(vec_mat_result[count_frame % I_RESULT_STORE]);
if (count_frame >= 1)
vec_mat_result[(count_frame - 1) % I_RESULT_STORE].copyTo(result);
GaussianBlur(image, image, Size(9, 9), 2, 2);
image.copyTo(vec_frames[count_frame % I_NO_FRAMES_STORE]);
if (count_frame >= I_NO_FRAMES_STORE - 1) {
Mat mat_diff_temp(I_HEIGHT, I_WIDTH, CV_32S, Scalar(0));
for (int i_diff = 0; i_diff < I_NO_FRAMES_STORE; i_diff++) {
Mat mat_rgb_diff_temp = abs(vec_frames[ (count_frame - 1) % I_NO_FRAMES_STORE ] - vec_frames[ (count_frame - i_diff) % I_NO_FRAMES_STORE ]);
cvtColor(mat_rgb_diff_temp, mat_rgb_diff_temp, CV_BGR2GRAY);
mat_rgb_diff_temp = mat_rgb_diff_temp > I_THRESHOLD;
mat_rgb_diff_temp.convertTo(mat_rgb_diff_temp, CV_32S);
mat_diff_temp = mat_diff_temp + mat_rgb_diff_temp;
}
mat_diff_temp = mat_diff_temp > I_THRESHOLD_2;
// mat_diff_temp.convertTo(mat_diff_temp, CV_8U);
Mat mat_roi = mat_diff_temp.rowRange(0, i_height_basketball);
// imshow("ROI", mat_roi);
Moments mm = cv::moments(mat_roi, true);
Point p_center = Point(mm.m10 / mm.m00, mm.m01 / mm.m00);
circle(result, p_center, 3, CV_RGB(0, 255, 0), -1);
line(result, Point(0, i_height_basketball), Point(result.cols, i_height_basketball), Scalar(225, 0, 0), 1);
}
count_frame = count_frame + 1;
int clk_processing_time = (clock() - clk_start);
if (count_frame > 1)
imshow("image", result);
// waitKey(0);
int delay = (1000 / fps) - clk_processing_time;
if (delay <= 0)
delay = 2;
if (waitKey(delay) >= 27)
break;
}
vid.release();
return 0;
}
Questions:
How to detect the hoop? I thought of doing with Square detection to detect the square regions around the hoop.
What is the best way of counting the successful shoots? Or How to count ?
I have what I suspect will be a fairly strong baseline: once the ball has commenced its downward arc, if the ball demonstrates significant upward movement again, its a miss. Otherwise, its a basket. This won't catch airballs, but I suspect they're relatively few anyway.
I think you could get a whole lot of mileage out of learning the ball trajectory of a successful shot and not worry too much about the hoop. Furthermore, didn't you say the camera was fixed-position? Doesn't that mean the hoop's always in the same place, and so you could just specify its location?
EDIT:
If you absolutely did have to find the hoop, I'd look for an object (sub-region of the image) of about the same size as the ball (which you say you can track) that's orange. More generally, you could learn a classifier for the hoop based on the training images you linked to, and apply it at a mixture of locations and scales, searching for the best match. You should know its approximate location, i.e. that it's in the upper portion of the image and likely to be to one side or the other. Then you could use proximity features to this identified region in addition to trajectory features to build a classifier for whether the shot succeeded or not.

How to Change CCTexture2D Color

I have a polygon that I fill using a texture and glDrawArray (using the method described in this tutorial: http://www.raywenderlich.com/32954/how-to-create-a-game-like-tiny-wings-with-cocos2d-2-x-part-1).
I want to be able to fill my polygon using a solid color, which is generated at random during gameplay. To do this using the technique from the tutorial, I need to dynamically create a texture that is just a solid color (for example, I might want to generate a 1x1 red square and use that to fill my polygons).
Is there a way to change the color of a texture in cocos2d, similar to how you would change the color of a sprite using [mySprite changeColor:ccRed]? So if I had my initial texture, say a 1x1 white square, is there a way I can change that texture to a 1x1 red square?
I have already tried using CCRenderTexture (as described in this tutorial: http://www.raywenderlich.com/33266/how-to-create-dynamic-textures-with-ccrendertexture-in-cocos2d-2-x) but, as I will be filling numerous polygons, this method proves to be quite slow.
I have also tried using the following code to create my texture:
// fill with solid red
GLubyte buffer[3] = {255, 0, 0};
CCTexture2D *texture = [[CCTexture2D alloc] initWithData:buffer pixelFormat:kCCTexture2DPixelFormat_RGB888 pixelsWide:1 pixelsHigh:1 contentSize:m];
While the above works fairly well, it is still slower than just grabbing the texture from a CCSprite. Basically, I am looking for a way to generate a dynamic texture as efficiently as possible.
Here is the code I am using to fill my polygons:
GLubyte buffer[3] = {arc4random()%256,arc4random()%256,arc4random()%256};
CGSize size;
size.width = 2; size.height = 2;
CCTexture2D *texture = [[CCTexture2D alloc] initWithData:buffer pixelFormat:kCCTexture2DPixelFormat_RGB888 pixelsWide:1 pixelsHigh:1 contentSize:size];
ccTexParams params = {GL_LINEAR, GL_LINEAR, GL_REPEAT, GL_REPEAT};
[texture setTexParameters:&params];
ccGLBindTexture2D([texture name]);
glVertexAttribPointer(kCCVertexAttrib_Position, 2, GL_FLOAT, GL_FALSE, 0, array); //where array is an array of points defining a polygon
glVertexAttribPointer(kCCVertexAttrib_TexCoords, 2, GL_FLOAT, GL_FALSE, 0, array);
glDrawArrays(GL_TRIANGLE_STRIP, 0, (GLsizei)4);
[texture dealloc];
Any help is appreciated.
Maybe what you are looking for is a mutable texture?
Here is a great blog post which utilizes CCMutableTextures http://www.cocos2d-iphone.org/pixel-based-destructible-ground-with-cocos2d/
Here is my open source project https://github.com/crebstar/PWNDestructibleTerrain
This is an open source project I've been working on over the Summer to create destructible terrain environments. The repo I just posted is without physics (soon to come), but provides an interface that wraps around mutable textures for sprites. It is fairly primitive as I started working on it a month ago, but it demonstrates how to use the CCMutableTexture class.
Around two or so years ago, Lam Hoang Pham released the CCMutableTexture class as open source. I built upon and around his library to provide more drawing utility and various other small features. The one caveat with using the CCMutableTexture class is you cannot use PVR's and must use a UIImage to provide the texture. I haven't noticed many performance issues with this method. The main problem would be you can't use a spritesheet.
Anyways here are some examples of how it is used:
// FROM THE GAME LAYER
[destTerrainSystem drawCircle:ccp(300,100) withRadius:30.0f withColor:ccc4(0, 0, 0, 0)];
[destTerrainSystem drawSquare:ccp(500,100) withRadius:30.0f withColor:ccc4(0, 0, 0, 0)];
// IN DESTTERRAIN
-(void) drawCircle:(CGPoint)circleOrigin withRadius:(float)radius withColor:(ccColor4B)color {
int localXOrigin = circleOrigin.x - self.position.x;
int localYOrigin = self.contentSize.height - (circleOrigin.y - self.position.y);
CCMutableTexture2D * terrainTexture = (CCMutableTexture2D *) [self texture];
[terrainTexture drawCircle:ccp(localXOrigin, localYOrigin) withRadius:radius withColor:color];
if ([delegate shouldApplyAfterEachDraw] || self.applyAfterDraw) [terrainTexture apply];
} // end drawCircle
-(void) drawSquare:(CGPoint)squareOrigin withRadius:(float)radius withColor:(ccColor4B)color {
int localXOrigin = squareOrigin.x - self.position.x;
int localYOrigin = self.contentSize.height - (squareOrigin.y - self.position.y);
CCMutableTexture2D * terrainTexture = (CCMutableTexture2D *) [self texture];
[terrainTexture drawSquare:ccp(localXOrigin, localYOrigin) withRadius:radius withColor:color];
if ([delegate shouldApplyAfterEachDraw] || self.applyAfterDraw)
[terrainTexture apply];
} // end drawSquare
// IN CCMUTABLETEXTURE
-(void) drawCircle:(CGPoint)circleOrigin withRadius:(float)radius withColor:(ccColor4B)color {
/*
Draws a circle. There is some overlap here but it is fairly efficient
*/
int x = radius;
int y = 0;
int radiusError = 1 - x;
while (x >= y) {
// Bottom half
[self drawHorizontalLine:(x + circleOrigin.x) :(circleOrigin.x - x) :(y + circleOrigin.y) withColor:color];
// Top half
[self drawHorizontalLine:(x + circleOrigin.x) :(circleOrigin.x - x) :(circleOrigin.y - y) withColor:color];
// left side
[self drawVerticalLine:(x + circleOrigin.y) endY:(circleOrigin.y - x) atX:(-y + circleOrigin.x) withColor:color];
// right side
[self drawVerticalLine:(x + circleOrigin.y) endY:(circleOrigin.y - x) atX:(y + circleOrigin.x) withColor:color];
y++;
if (radiusError < 0) {
radiusError = radiusError + ((2 * y) +1);
} else {
x--; // Comment this out to draw a square
radiusError = radiusError + (2 * (y - x + 1));
} // end if
} // end while
// Cache the altered col values
for (int col = circleOrigin.x - radius; col <= circleOrigin.x + radius; col++) {
if (col < 0 || col >= size_.width) continue;
[alteredColumns addObject:[NSNumber numberWithInt:col]];
} // end for
} // end draw circle
The CCMutableTexture maintains a model of the texture in an array of pixels (row major storage). You can then access, change, and poll for the property of each pixel. After you have modified the array, you can then apply the changes by calling apply. This allows for some flexibility and performance tweaking as apply can be an expensive call.
There is a lot more you can do... But this should be a good starting point. Both links have example code on how to use the CCMutableTexture.
Hope this helps

How to convert an 8-bit OpenCV IplImage* to a 32-bit IplImage*?

I need to convert an 8-bit IplImage to a 32-bits IplImage. Using documentation from all over the web I've tried the following things:
// general code
img2 = cvCreateImage(cvSize(img->width, img->height), 32, 3);
int height = img->height;
int width = img->width;
int channels = img->nChannels;
int step1 = img->widthStep;
int step2 = img2->widthStep;
int depth1 = img->depth;
int depth2 = img2->depth;
uchar *data1 = (uchar *)img->imageData;
uchar *data2 = (uchar *)img2->imageData;
for(h=0;h<height;h++) for(w=0;w<width;w++) for(c=0;c<channels;c++) {
// attempt code...
}
// attempt one
// result: white image, two red spots which appear in the original image too.
// this is the closest result, what's going wrong?!
// see: http://files.dazjorz.com/cache/conversion.png
((float*)data2+h*step2+w*channels+c)[0] = data1[h*step1+w*channels+c];
// attempt two
// when I change float to unsigned long in both previous examples, I get a black screen.
// attempt three
// result: seemingly random data to the top of the screen.
data2[h*step2+w*channels*3+c] = data1[h*step1+w*channels+c];
data2[h*step2+w*channels*3+c+1] = 0x00;
data2[h*step2+w*channels*3+c+2] = 0x00;
// and then some other things. Nothing did what I wanted. I couldn't get an output
// image which looked the same as the input image.
As you see I don't really know what I'm doing. I'd love to find out, but I'd love it more if I could get this done correctly.
Thanks for any help I get!
The function you are looking for is cvConvertScale(). It automagically does any type conversion for you. You just have to specify that you want to scale by a factor of 1/255 (which maps the range [0...255] to [0...1]).
Example:
IplImage *im8 = cvLoadImage(argv[1]);
IplImage *im32 = cvCreateImage(cvSize(im8->width, im8->height), 32, 3);
cvConvertScale(im8, im32, 1/255.);
Note the dot in 1/255. - to force a double division. Without it you get a scale of 0.
Perhaps this link can help you?
Edit In response to the second edit of the OP and the comment
Have you tried
float value = 0.5
instead of
float value = 0x0000001;
I thought the range for a float color value goes from 0.0 to 1.0, where 1.0 is white.
Floating point colors go from 0.0 to 1.0, and uchars go from 0 to 255. The following code fixes it:
// h is height, w is width, c is current channel (0 to 2)
int b = ((uchar *)(img->imageData + h*img->widthStep))[w*img->nChannels + c];
((float *)(img2->imageData + h*img2->widthStep))[w*img2->nChannels + c] = ((float)b) / 255.0;
Many, many thanks to Stefan Schmidt for helping me fix this!
If you do not put the dot (.), some compilers will understand is as an int division, giving you a int result (zero in this case).
You can create an IplImage wrapper using boost::shared_ptr and template-metaprogramming. I have done that, and I get automatic garbage collection, together with automatic image conversions from one depth to another, or from one-channel to multi-channel images.
I have called the API blImageAPI and it can be found here:
http://www.barbato.us/2010/10/14/image-data-structure-based-shared_ptr-iplimage/
It is very fast, and make code very readable, (good for maintaining algorithms)
It is also can be used instead of IplImage in opencv algorithms without changing anything.
Good luck and have fun writing algorithms!!!
IplImage *img8,*img32;
img8 =cvLoadImage("a.jpg",1);
cvNamedWindow("Convert",1);
img32 = cvCreateImage(cvGetSize(img8),IPL_DEPTH_32F,3);
cvConvertScale(img8,img32,1.0/255.0,0.0);
//For Confirmation Check the pixel values (between 0 - 1)
for(int row = 0; row < img32->height; row++ ){
float* pt = (float*) (img32->imageData + row * img32->widthStep);
for ( int col = 0; col < width; col++ )
printf("\n %3.3f , %3.3f , %3.3f ",pt[3*col],pt[3*col+1],pt[3*col+2]);
}
cvShowImage("Convert",img32);
cvWaitKey(0);
cvReleaseImage(&img8);
cvReleaseImage(&img32);
cvDestroyWindow("Convert");

Resources