CascadeClassifier in java does not find face with webcam - opencv

I am trying to translate the OpenCV CascadeClassifier tutorial from C++ to Java. Working good in C++. Also this java tutorial is working fine.
But the translation is simply not detecting the face. I don't get explicit errors. I can see the processing of the video input from the webcam (grey/histogram...) and the video display. Cascade load doesn't give error. But the CascadeClassifier call just doesn't return any faces... So, you probably can skip all the code and just go to my CascadeClassifier call, down to public Mat detect(Mat inputframe). As I am new to Java and OpenCV, I paste the rest (I removed anything I felt may not be significant), just in case, but don't mean for you to debug that...
I have also tried this call (and other portions) in many different ways and nothing... running out of ideas...
Thank you!!
import java.awt.*;
import java.awt.image.BufferedImage;
import javax.swing.*;
import org.opencv.core.Mat;
import org.opencv.core.MatOfRect;
import org.opencv.highgui.VideoCapture;
import org.opencv.imgproc.Imgproc;
import org.opencv.objdetect.CascadeClassifier;
class My_Panel extends JPanel{
private static final long serialVersionUID = 1L;
private BufferedImage image;
private CascadeClassifier face_cascade;
// Create a constructor method
public My_Panel(){
super();
String face_cascade_name = "/haarcascade_frontalface_alt.xml";
//String face_cascade_name = "/lbpcascade_frontalface.xml";
//-- 1. Load the cascades
String str;
str = getClass().getResource(face_cascade_name).getPath();
str = str.replace("/C:","C:");
face_cascade_name=str;
face_cascade=new CascadeClassifier(face_cascade_name);
if( !face_cascade.empty())
{
System.out.println("--(!)Error loading A\n");
return;
}
else
{
System.out.println("Face classifier loooaaaaaded up");
}
}
private BufferedImage getimage(){
return image;
}
public void setimage(BufferedImage newimage){
image=newimage;
return;
}
/**
* Converts/writes a Mat into a BufferedImage.
*
* #param matrix Mat of type CV_8UC3 or CV_8UC1
* #return BufferedImage of type TYPE_3BYTE_BGR or TYPE_BYTE_GRAY
*/
public BufferedImage matToBufferedImage(Mat matrix) {
int cols = matrix.cols();
int rows = matrix.rows();
int elemSize = (int)matrix.elemSize();
byte[] data = new byte[cols * rows * elemSize];
int type;
matrix.get(0, 0, data);
switch (matrix.channels()) {
case 1:
type = BufferedImage.TYPE_BYTE_GRAY;
break;
case 3:
type = BufferedImage.TYPE_3BYTE_BGR;
// bgr to rgb
byte b;
for(int i=0; i<data.length; i=i+3) {
b = data[i];
data[i] = data[i+2];
data[i+2] = b;
}
break;
default:
return null;
}
BufferedImage image2 = new BufferedImage(cols, rows, type);
image2.getRaster().setDataElements(0, 0, cols, rows, data);
return image2;
}
public void paintComponent(Graphics g){
BufferedImage temp=getimage();
g.drawImage(temp,10,10,temp.getWidth(),temp.getHeight(), this);
}
public Mat detect(Mat inputframe){
Mat mRgba=new Mat();
Mat mGrey=new Mat();
MatOfRect faces = new MatOfRect();
//MatOfRect eyes = new MatOfRect();
inputframe.copyTo(mRgba);
inputframe.copyTo(mGrey);
Imgproc.cvtColor( mRgba, mGrey, Imgproc.COLOR_BGR2GRAY);
Imgproc.equalizeHist( mGrey, mGrey );
face_cascade.detectMultiScale(mGrey, faces);
//face_cascade.detectMultiScale(mGrey, faces, 1.1, 2, 0|Objdetect.CASCADE_SCALE_IMAGE, new Size(30, 30), new Size(200,200) );
//face_cascade.detectMultiScale(mGrey, faces, 1.1, 2, 2//CV_HAAR_SCALE_IMAGE,
// ,new Size(30, 30), new Size(200,200) );
System.out.println(String.format("Detected %s faces", faces.toArray().length));
return mGrey;
}
}
public class window {
public static void main(String arg[]){
// Load the native library.
System.loadLibrary("opencv_java245");
String window_name = "Capture - Face detection";
JFrame frame = new JFrame(window_name);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.setSize(400,400);
My_Panel my_panel = new My_Panel();
frame.setContentPane(my_panel);
frame.setVisible(true);
//-- 2. Read the video stream
BufferedImage temp;
Mat webcam_image=new Mat();
VideoCapture capture =new VideoCapture(0);
if( capture.isOpened())
{
while( true )
{
capture.read(webcam_image);
if( !webcam_image.empty() )
{
frame.setSize(webcam_image.width()+40,webcam_image.height()+60);
//-- 3. Apply the classifier to the captured image
// At this point I was wondering where this should be done.
// I put it within the panel class, but maybe one could actually
// create a processor object...
webcam_image=my_panel.detect(webcam_image);
//-- 4. Display the image
temp=my_panel.matToBufferedImage(webcam_image);
my_panel.setimage(temp);
my_panel.repaint();
}
else
{
System.out.println(" --(!) No captured frame -- Break!");
break;
}
}
}
return;
}
}
PS.: Other info, just in case:
mGrey is: Mat [ 480*640*CV_8UC1, isCont=true, isSubmat=false, nativeObj=0x19d9af48, dataAddr=0x19dc3430 ]
face is: Mat [ 0*0*CV_8UC1, isCont=false, isSubmat=false, nativeObj=0x194bb048, dataAddr=0x0 ]

I have tried your code and it works fine! You have only one issue with haarcascade_frontalface_alt.xml file location. Try to use a full path to the file:
face_cascade= new CascadeClassifier("D:/HelloCV/src/haarcascade_frontalface_alt.xml");

Related

proper video streaming with rxjava

To handle a video stream from a webcam (delivered by opencv) i am considering to use RxJava.
I am hoping to achieve the following:
being able to control the frames per second to be delivered
to be able to handle different inputs - e.g. a life webcam, a video or even a still picture
being able to switch to a picture-by-picture handling under the control of a gui
I have been experimenting a bit with RxJava but i am confused about the debounce, throttleFirst and async operators
Examples like https://stackoverflow.com/a/48723331/1497139 show some code but I am missing more detailed explanation.
Where could I find a decent example for video processing or something similar along the needs mentioned above?
The code below does some non async logic at this time - please let me know if i could build on it:
ImageFetcher
import org.opencv.core.Mat;
import org.opencv.videoio.VideoCapture;
import rx.Observable;
import rx.functions.Action1;
import rx.functions.Func0;
import rx.functions.Func1;
/**
* fetcher for Images
*
*/
public class ImageFetcher {
// OpenCV video capture
private VideoCapture capture = new VideoCapture();
private String source;
protected int frameIndex;
public int getFrameIndex() {
return frameIndex;
}
/**
* fetch from the given source
*
* #param source
* - the source to fetch from
*/
public ImageFetcher(String source) {
this.source = source;
}
/**
* try opening my source
*
* #return true if successful
*/
public boolean open() {
boolean ret = this.capture.open(source);
frameIndex=0;
return ret;
}
/**
* fetch an image Matrix
*
* #return - the image fetched
*/
public Mat fetch() {
if (!this.capture.isOpened()) {
boolean ret = this.open();
if (!ret) {
String msg = String.format(
"Trying to fetch image from unopened VideoCapture and open %s failed",
source);
throw new IllegalStateException(msg);
}
}
final Mat frame = new Mat();
this.capture.read(frame);
frameIndex++;
return !frame.empty() ? frame : null;
}
#Override
protected void finalize() throws Throwable {
super.finalize();
}
/**
* convert me to an observable
* #return a Mat emitting Observable
*/
public Observable<Mat> toObservable() {
// Resource creation.
Func0<VideoCapture> resourceFactory = () -> {
VideoCapture capture = new VideoCapture();
capture.open(source);
return capture;
};
// Convert to observable.
Func1<VideoCapture, Observable<Mat>> observableFactory = capture -> Observable
.<Mat> create(subscriber -> {
boolean hasNext = true;
while (hasNext) {
final Mat frame = this.fetch();
hasNext = frame!=null && frame.rows()>0 && frame.cols()>0;
if (hasNext) {
String msg = String.format("->%6d:%4dx%d", frameIndex, frame.cols(), frame.rows());
System.out.println(msg);
subscriber.onNext(frame);
}
}
subscriber.onCompleted();
});
// Disposal function.
Action1<VideoCapture> dispose = VideoCapture::release;
return Observable.using(resourceFactory, observableFactory, dispose);
}
}
ImageSubscriber
import org.opencv.core.Mat;
import rx.Subscriber;
public class ImageSubscriber extends Subscriber<Mat> {
public Throwable error;
public int cols = 0;
public int rows=0;
public int frameIndex=0;
public boolean completed = false;
public boolean debug = false;
#Override
public void onCompleted() {
completed = true;
}
#Override
public void onError(Throwable e) {
error = e;
}
#Override
public void onNext(Mat mat) {
cols = mat.cols();
rows = mat.rows();
frameIndex++;
if (cols==0 || rows==0)
System.err.println("invalid frame "+frameIndex);
if (debug) {
String msg = String.format("%6d:%4dx%d", frameIndex, cols, rows);
System.out.println(msg);
}
}
};

Leptonica OpenCV Java convert Mat to Pix and vise versa

I use the following lept4j and OpenCV Maven dependencies:
<!-- Leptonica -->
<dependency>
<groupId>net.sourceforge.lept4j</groupId>
<artifactId>lept4j</artifactId>
<version>1.9.0</version>
</dependency>
<!-- OpenCV -->
<dependency>
<groupId>org.openpnp</groupId>
<artifactId>opencv</artifactId>
<version>3.2.0-1</version>
</dependency>
I'd like to use OpenCV and Leptonica functions together. In order to do this, I need to be able to convert Mat to Pix and Pix to Mat.
This is what I have for now:
public static Pix matToGrayscalePix(Mat mat) {
if (mat == null) {
throw new IllegalArgumentException("Recycled matrix");
}
final byte[] bytes = new byte[(int) mat.total()];
mat.get(0, 0, bytes);
ByteBuffer buff = ByteBuffer.wrap(bytes);
return Leptonica1.pixReadMem(buff, new NativeSize(buff.capacity()));
}
public static Mat pixToGrayscaleMat(Pix pix) {
if (pix == null) {
throw new IllegalArgumentException("Recycled matrix");
}
PointerByReference pdata = new PointerByReference();
NativeSizeByReference psize = new NativeSizeByReference();
int format = net.sourceforge.lept4j.ILeptonica.IFF_TIFF;
Leptonica1.pixWriteMem(pdata, psize, pix, format);
byte[] b = pdata.getValue().getByteArray(0, psize.getValue().intValue());
return new MatOfByte(b).reshape(0, pix.h);
}
But these functions doesn't work right now. What am I doing wrong ?
Try the following:
public static Pix convertMatToPix(Mat mat) {
MatOfByte bytes = new MatOfByte();
Imgcodecs.imencode(".tif", mat, bytes);
ByteBuffer buff = ByteBuffer.wrap(bytes.toArray());
return Leptonica1.pixReadMem(buff, new NativeSize(buff.capacity()));
}
public static Mat convertPixToMat(Pix pix) {
PointerByReference pdata = new PointerByReference();
NativeSizeByReference psize = new NativeSizeByReference();
Leptonica1.pixWriteMem(pdata, psize, pix, ILeptonica.IFF_TIFF);
byte[] b = pdata.getValue().getByteArray(0, psize.getValue().intValue());
Leptonica1.lept_free(pdata.getValue());
return Imgcodecs.imdecode(new MatOfByte(b), Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
}

strange cvSaveImage behavior

i have recently started using opencv and this one has baffled me.
void saveImageSnippet(char *imageName, int height, int width, void* data, int nChannels) //data is char[height * width]
{
char fName[200]="c:\\testimg\\";
FILE *fptr;
IplImage *img;
sprintf(fName,"%s%s.bmp",fName,imageName);
img = cvCreateImageHeader(cvSize(width, height),8/*depth*/,nChannels);
img->imageData=(unsigned char*)data;
cvSaveImage(fName, img); //Unhandled exception
cvReleaseImage(&img);
}
At cvSaveImage : Unhandled exception at 0x6e8e871d in vc2008_1x.exe: 0xC0000005: Access violation reading location 0x745c3a63.
Is there anything i am not doing right?
now an interesting part,
if i add a couple of unused variables, cvSaveImage works just fine
void saveImageSnippet(char *imageName, int height, int width, void* data, int nChannels)
{
int var1, var2; //unused variables
char fName[200]="c:\\testimg\\";
FILE *fptr;
IplImage *img;
sprintf(fName,"%s%s.bmp",fName,imageName);
img = cvCreateImageHeader(cvSize(width, height),8/*depth*/,nChannels);
img->imageData=(unsigned char*)data;
cvSaveImage(fName, img); //now it works fine
cvReleaseImage(&img);
}
please use opencv's c++ api,
avoid anything, that has iplimages in it (especially here on SO !)
#include "opencv2/highgui.hpp" // no, *not* highgui.h !
using namespace cv;
int main()
{
string fName = format("c:\\testimg\\%s.bmp",imageName);
Mat img = imread( fName );
if ( img.empty() )
return -1;
// your image was not loaded
imsave(fName);
return 0;
}

Handling a vector of cv::Mat with a Method

I want to give a method a vector to fill it with a different number of Mat Objects, something like this:
int main()
{
vector<cv::Mat> areas;
doSomething(areas);
return 0;
}
doSomething(vector<cv::Mat> &areas)
{
cv::Mat first = zeros(...
cv::Mat second = ....
cv::Mat third = ...
areas.push_back(first);
areas.push_back(second);
areas.push_back(third);
}
Of course that doesn't work, because there is not enough memory allocated at the begin and the memory is only on the stack!
A second idea I had is to make pointers:
int main()
{
vector<cv::Mat> *areas = new vector<cv::Mat>();
doSomething(&areas);
return 0;
}
doSomething(vector<cv::Mat> *areas)
{
cv::Mat first = zeros(...
cv::Mat second = ....
cv::Mat third = ...
areas->push_back(first);
areas->push_back(second);
areas->push_back(third);
}
But again the problem is, that I have no idea at the beginning how much space is to allocate.
The third idea was to return the vector at the "normal" way:
int main()
{
vector<cv::Mat> areas;
areas = doSomething();
return 0;
}
vector<cv::Mat> doSomething()
{
vector<cv::Mat> areas;
cv::Mat first = zeros(...
cv::Mat second = ....
cv::Mat third = ...
areas->push_back(first);
areas->push_back(second);
areas->push_back(third);
return areas;
}
In this case of course only the headers of the vector was copied and not the Mat Objects in the vector
Do you have any idea how i am able to solve this problem?
Thanks in advance!

Face Detection in JavaCV

I have the following code to detect faces.
public class FaceDetection {
public static final String XML_FILE =
" C:/opencv/data/haarcascades/haarcascade_frontalface_default.xml";
public static void main(String[] args){
IplImage img = cvLoadImage("C:/Users/Ioanna/Desktop/lena.png");
detect(img);
}
public static void detect(IplImage src){
CvHaarClassifierCascade cascade = new
CvHaarClassifierCascade(cvLoad(XML_FILE));
CvMemStorage storage = CvMemStorage.create();
CvSeq sign = cvHaarDetectObjects(
src,
cascade,
storage,
1.5,
3,
CV_HAAR_DO_CANNY_PRUNING);
cvClearMemStorage(storage);
int total_Faces = sign.total();
for(int i = 0; i < total_Faces; i++){
CvRect r = new CvRect(cvGetSeqElem(sign, i));
cvRectangle (
src,
cvPoint(r.x(), r.y()),
cvPoint(r.width() + r.x(), r.height() + r.y()),
CvScalar.RED,
2,
CV_AA,
0);
}
cvShowImage("Result", src);
cvWaitKey(0);
}
}
When it runs return the following error
OpenCV Error: Null pointer (Invalid classifier cascade) in unknown function, file ......\src\opencv\modules\objdetect\src\haar.cpp, line 1514
Exception in thread "main" java.lang.RuntimeException: ......\src\opencv\modules\objdetect\src\haar.cpp:1514: error: (-27) Invalid classifier cascade
at com.googlecode.javacv.cpp.opencv_objdetect.cvHaarDetectObjects(Native Method)
at com.googlecode.javacv.cpp.opencv_objdetect.cvHaarDetectObjects(opencv_objdetect.java:238)
at FaceDetection.FaceDetection.detect(FaceDetection.java:32)
at FaceDetection.FaceDetection.main(FaceDetection.java:24)
Do anyone know what is the problem?
Thanks
Copy the link C:/Users/Ioanna/Desktop/lena.png
and paste it in the file of the class and Copy the link C:/opencv/data/haarcascades/haarcascade_frontalface_default.xml
and paste it in the file of the class
public static final String XML_FILE = "haarcascade_frontalface_default.xml";
IplImage img = cvLoadImage("lena.jpg");

Resources