I am trying to extract mouth(lips) from images. What I did is this. I first extract faces from images and tried to detect mouth using haarcascade_mcs_mouth.xml. However, It keeps detecting other parts instead detecting mouth from the face. Is there any other way to detect mouth from face images?
You can use some facial landmark detector such as Flandmark detector:
http://cmp.felk.cvut.cz/~uricamic/flandmark/
Or STASM:
http://www.milbo.users.sonic.net/stasm/
To correctly detect mouth regions, you can try more advanced methods like the following one:
X. Zhu, D. Ramanan. Face Detection, Pose Estimation and Landmark Localization in the Wild Computer Vision and Pattern Recognition (CVPR) Providence, Rhode Island, June 2012. (Project page), (PDF Online available), (Source code)
Several real examples can be seen from here.
Related
In such tasks, I tend to you Mediapipe or Dlib to detect the landmarks of the face and get the specific coordinates I'm interested to work with.
But in the case of the human face taken from a Profil view, Dlib can't detect anything and Mediapipe shows me a standard 3D face-mesh superimposed on top of the 2D image which provides false coordinates.
I was wondering if anyone with Computer Vision (Image Processing) knowledge can guide me on how to detect the A & B points coordinates from this image
PS: The color of the background changes & also the face location is not standard.
Thanks in advance strong text
Your question seems a little unclear. If you just want (x,y) screen coordinates you can use this answer to convert the (x,y,z) that mediapipe gives you to just (x,y). If this doesn't
doesnt work for you I would recommend this repo or this one which both only work with 68 facial landmarks, but this should be sufficient for your use case.
If all of this fails I would recommend retraining hrnet on a dataset with profile views. I believe either 300-W dataset or the 300-VW dataset provides some data with heads at extreme angles.
If you wish to get the 3D coordinates in camera coordinates (X,Y,Z) you're going to have to use solvePNP. This will require getting calibration info and model mesh points for whatever facial landmark detector you use. You can find a some examples for some of this here
I'm playing with Eye Gaze estimation using a IR Camera. So far i have detected the two Pupil Center points as follows:
Detect the Face by using Haar Face cascade & Set the ROI to Face.
Detect the Eyes by using Haar Eye cascade & Name it as Left & Right Eye Respectively.
Detect the Pupil Center by thresholding the Eye region & found the pupil center.
So far I've tried to find the gaze direction by using the Haar Eye Boundary region. But this Haar Eye rect is not always showing the Eye Corner points. So the results was poor.
Then I've tried to tried to detect Eye Corner points using GFTT, Harriscorners & FAST but since I'm using NIR Camera the Eye Corner points are not clearly visible & so i cant able to get the exact corner positions.So I'm stuck here!
What else is the best feature that can be tracked easily from face? I heard about Flandmark but i think that is also will not work in IR captured images.
Is there any feature that can be extracted easily from the face images? Here I've attached my sample output image.
I would suggest flandmark, even if your intuition is the opposite - I've used it in my master thesis (which was about head pose estimation, a related topic). And if the question is whether it will work with the example image you've provided, I think it might detect features properly - even on a gray scaled image. I think in the flandmark they probably convert to image to grayscale before applying a detector (like the haar detector works). Moreover, It surprisingly works with the low resolution images what is an advantage too (especially when you're saying eye corners are not clearly visible). And flandmark can detect both eye corners, mouth corners and nose tip (actually I will not rely on the last one, from my experience detecting nose tip on the single image is quite noisy, however works fine with an image sequence with some filtering e.g. average, Kalman). If you decide to use that technique and it works, please let us know!
I'm making system for emotion detection on Android mobile phone. I'm using OpenCV's Cascades (LBP's or Haars) to find face, eyes, mouth areas etc. What I have observed till now that accuracy isn't stable. There are situations where I can't find eye or I have "additional faces" in the background due to very slight change of light. What I wanted to ask is:
1) Is Haar Cascade more Accurate than LBP?
2) Is there any good method for increasing accuracy of detection? Like find face/eyes etc on binarized image, or use some edge detection filter, saturation, anything else?
you can try Microsoft API for face emotion detection ..i am trying in my project so..result is best..try this link
https://www.microsoft.com/cognitive-services/en-us/emotion-api
sometimes HAAR or LBP will not get a good enough result for a face detection system. if you want to get more good acc. i think you can try to using STASM
it base on opencv and using Haar to detect face and landmark. something others you can also try YOLO Face detection
if you want to build your own face detection system just base on Haar or LBP and make them get a good result, maybe you will need to using the LBP to find out the face faster and train a CNN model to get the last good result, it can make your system to detect faces in realtime. as i know, the SEETAFACE is using this way to make a realtime faces detection.
I'm working on an application which needs to detect the location of a face in a video stream, using a web cam placed at desk height (and slightly off to the side of the user).
I've already implemented a version of OpenCV (using their Haar detection) and it works ok... the problem is that it tends to lose the position of the face if the user turns their head to the side (or looks up).
Since the webcam is sitting on the desk, it is tilted up at a 30 degree angle. The OpenCV detection algorithm is trained using fully frontal images, but not up-angle images like the ones I'm using. I know OpenCV also has a profile Haar file that can be used.. but from my research it seems that the results are quite mixed on profile detection. In addition, I don't really have control over the background or lighting of the image... so this sometimes also effects the efficacy of the OpenCV detection algorithm.
So, I guess what I'm asking is... are there other face detection algorithms (that are hopefully free, as this is part of my university research) that are better for detecting faces for this type of setup? It seems like some of the built-in webcams (for Macs and PCs) actually have fairly robust algorithms for detecting faces (and then overlaying cheesy cartoon images over the faces)... but they seem to work well regardless of background or lighting. Do you have any recommendations?
Thanks.
For research purposes, you can use the Haar cascades in OpenCV, things are different if you want to go commercial (in which case you need to consider LBP cascades instead). Just be sure to quote the Viola-Jones paper in your references.
To improve the results of face detection, you have several paths:
individual image detection: you can send rotated images to a frontal cascade to account for some variability without training your own cascade
individual image detection but more work) : train your own cascade in operating conditions closer to the ones of your app
stability in video streams (as in webcams & co.) : this is achieved by adding a layer of tracking around the face detection. Depending on your knowledge about this topic, you can use your own filter, have fun with OpenCV's particle or Kalman filter, implement a simple first or second order low pass filter on the face position or a PID tracker on the detected face...
Any of these tracking filters will enhance a lot your results when processing video streams.
Use CLM-framework for accurate realtime face detection and face landmark detection.
Example of the system in action: http://youtu.be/V7rV0uy7heQ
You may find it useful.
How can I detect irises in a face with opencv?
Have a look at this forum thread. There's some source code there to get you started, but be careful about using it directly -- the original author seemed to have problems compiling it.
Start with detecting circles - see cvHoughCircles - hint, eyes have a series of concentric circles.
OpenCV has Face Detection module which uses Haar Cascade. You can use the same method to detect Iris. You collect some iris images and make it as positive set and non iris images as negative set. The use the Haar Training module to train it.
Quick and dirty would be making an eye detection first with Haar filter, there are good model xml files shipped with opencv 2.4.2. Then you do some skin detection (in the HSV space rather than the rgb space) to identify the area of the eye in the middle, or circle search.
Also, projections, histogram-based decisions can be used once the eye area is cropped.