How to minimize face-detection error - opencv

Question

here is the code now I can both detect face and mouth together, and able to roughly measure the distance of its bounding box <--
the problem is the mouth detection seems to detects everything they defines as mouth even it is not
and I want to use the "face" bounding box as a mouth detection region to minimize its error, I don't know if Forloop stacked would work? by put mouth loop inside face loop?? I'm fairly new to coding any help would be appreciated
import gab.opencv.*;
import java.awt.Rectangle;
import processing.video.*;
Capture video;
OpenCV f;
OpenCV m;
void setup() {
size(800, 600);
video = new Capture(this, 800/2, 600/2);
f = new OpenCV(this, 800/2, 600/2);
m = new OpenCV(this, 800/2, 600/2);
video.start();
}
void draw() {
scale(2);
f.loadImage(video);
m.loadImage(video);
f.loadCascade(OpenCV.CASCADE_FRONTALFACE);
m.loadCascade(OpenCV.CASCADE_MOUTH);
image(video, 0, 0 );
noFill();
stroke(0, 255, 0);
strokeWeight(3);
Rectangle[] mouth = m.detect();
Rectangle[] face = f.detect();
println(mouth.length);
strokeWeight(3);
for (int i = 0; i < face.length; i++) {
println(face[i].x + "," + face[i].y);
rect(face[i].x, face[i].y, face[i].width, face[i].height);
}
for (int i = 0; i < mouth.length; i++) {
println(mouth[i].x + "," + mouth[i].y);
rect(mouth[i].x, mouth[i].y, mouth[i].width, mouth[i].height);
}
for (int i = 0; i < mouth.length; i++) {
fill(255, 0, 0);
noStroke();
ellipse((mouth[i].x)+(mouth[i].width/2), mouth[i].y, 5, 5);
ellipse((mouth[i].x)+(mouth[i].width/2), (mouth[i].y)+ (mouth[i].height), 5, 5);
}
for (int i = 0; i < mouth.length; i++) {
int px = (mouth[i].x)+(mouth[i].width/2);
int py = (mouth[i].y)+(mouth[i].height);
int mOpen = int (dist(px, mouth[i].y, px, py));
println(mOpen);
}
}
void captureEvent(Capture d) {
d.read();
}

Answer 1

There are a couple issues:
You shouldn't be loading OpenCV cascades multiple times a second in draw(). You should do it once in setup() and just call detect() in draw()
OpenCV for Processing seems to override the cascade loaded in the second instance with a cascade loaded in the first instance
If accuracy isn't a huge issue, you can get away with a single cascade: the mouth one. Note that there are options/hints you can use for the detect function which may help the detection. For example you can tell the detector to detect largest object only, give it a hint of the smallest and largest bounding boxes the mouth would have with your setup and how much should the results filtered out.
Here's a code sample for the above:
import gab.opencv.*;
import java.awt.Rectangle;
import org.opencv.objdetect.Objdetect;
import processing.video.*;
Capture video;
OpenCV opencv;
//cascade detections parameters - explanations from Mastering OpenCV with Practical Computer Vision Projects
int flags = Objdetect.CASCADE_FIND_BIGGEST_OBJECT;
// Smallest object size.
int minFeatureSize = 20;
int maxFeatureSize = 80;
// How detailed should the search be. Must be larger than 1.0.
float searchScaleFactor = 1.1f;
// How much the detections should be filtered out. This should depend on how bad false detections are to your system.
// minNeighbors=2 means lots of good+bad detections, and minNeighbors=6 means good detections are given but some are missed.
int minNeighbors = 6;
void setup() {
size(320, 240);
noFill();
stroke(0, 192, 0);
strokeWeight(3);
video = new Capture(this,width,height);
video.start();
opencv = new OpenCV(this,320,240);
opencv.loadCascade(OpenCV.CASCADE_MOUTH);
}
void draw() {
//feed cam image to OpenCV, it turns it to grayscale
opencv.loadImage(video);
opencv.equalizeHistogram();
image(opencv.getOutput(), 0, 0 );
Rectangle[] mouths = opencv.detect(searchScaleFactor,minNeighbors,flags,minFeatureSize, maxFeatureSize);
for (int i = 0; i < mouths.length; i++) {
text(mouths[i].x + "," + mouths[i].y + "," + mouths[i].width + "," + mouths[i].height,mouths[i].x, mouths[i].y);
rect(mouths[i].x, mouths[i].y, mouths[i].width, mouths[i].height);
}
}
void captureEvent(Capture c) {
c.read();
}
Note that facial hair can cause false positives.
I have provided more in depth notes in an answer to your previous related question. I recommend focusing on the FaceOSC part as it will be more accurate.