The aim of this article is to describe the features and issues related to Android face detection, in order to create a Demo application that allows realtime human face tracking. Since its first release (API Level 1) Android provides a native Face Detector available in android.media.FaceDetector. It works in a very simple way, just invoking findFaces method on the target bitmap bitmap of interest.
Another way to achieve face detection is to use OpenCV, a library of programming functions for real time computer vision. In particular, OpenCV offers an object detector based on haar-like features, trained with a few hundreds of positive and negative examples. By choosing a suitable classifier, like haarcascade_frontalface_alt2.xml, OpenCV allows to detect human faces. The OpenCV libraries are written in C/C++. Android platform is supported, but it requires to write C/C++ code and to use Native Development Kit (NDK) and Java Native Interface (JNI). This makes OpenCV quite hard to configure and implement for Android OS.
In order to use OpenCV features without dealing with the above process, Java programmers can use JavaCV, a wrapper for the most commonly used libraries by researchers in the field of computer vision. From December 2010, JavaCV is compatible with Android API Level 8. JavaCV provides precompiled native libraries, mapped to Java methods.
With JavaCV, the setup of an Android application is much easier than OpenCV, but writing code follows exactly structures, objects and methods of C/C++ libraries. In addition JavaCV will increases by about 15Mb the size of the application and the compatibility of Android since API Level 8 excludes a large proportion of devices (about 48%). In order to understand the performance differences between Android Native Face Detector (NFD) and JavaCV Face Detector (JcvFD), we performed some tests using the Face Detection Camera Compare Test application (a little tool we built for this specific purpose).
This application allows you to select a picture from your devices (or take a new one) and uses NFD and JcvFD to detect faces, comparing timing and results. 15 different images have been used for test, in particular five with a single face to be recognized, five with couples and the last five with groups. For each test we checked if the response was correct (true positive), if the face was not recognized (true negative) or if a part of the photo was mistakenly recognized as the face (false positive). It also keeps track of the processing time required to obtain a result for each implementation.
Before analyzing the results, it is appropriate to highlight the implementation’s differences between the two solutions. As previously mentioned, Android Native Face Detector use findFace method to find faces:
FaceDetector.Face faces[] = new FaceDetector.Face[NUM_FACES]; FaceDetector arrayFaces = new FaceDetector(width, height, NUM_FACES); arrayFaces.findFaces(image, faces); for(int i=0; i<faces.length && !stop; i++){ face = faces[i]; … }
Instead JavaCV requires many more lines of code to obtain the faces detected:
new JavaCvErrorCallback().redirectError(); // Load classifier xml File classifierFile = extractResource(CURR_CASCADE); if (classifierFile == null || classifierFile.length() <= 0) throw new FileNotFoundException("Could not load the classifier file."); // Load OpenCV JNI library System.loadLibrary("cv"); cascade = new CvHaarClassifierCascade(cxcore.cvLoad(classifierFile.getAbsolutePath())); storage = CvMemStorage.create(); IplImage grayImage = highgui.cvLoadImage(imgPath, highgui.CV_LOAD_IMAGE_GRAYSCALE); cv.cvEqualizeHist(grayImage, grayImage); CvSize minSize = new CvSize(grayImage.width/16, grayImage.height/16); CvSeq faces = cvHaarDetectObjects(grayImage, cascade, storage,Haar.SCALE_FACTOR, Haar.NEIGHBOURS_NR, Haar.FLAGS, minSize.byValue()); int nFaces = 0; for (int i = 0; (i < faceSel.length && i < faces.total); i++) { CvRect r = new CvRect(cvGetSeqElem(faces, i)); … } cvClearMemStorage(storage);
The results are summarized in the following chart:
From a total of 40 faces, both solutions have correctly recognized 33 faces. JcvFD was more accurate, especially on noisy images (i.e. portraits with sunglasses, like Test 3). In contrast, the high accuracy allowed JcvFD to recognize five fake faces (false positive, Tests 4a, 4b, 7, 8, 11). For both solutions, a face tilted to one side is a common cause of mistaken recognition, instead of the same face in right position (false positive in Test 4a and 4b, 8, 15). In all tests, the maximum number of faces to recognize was limited to 5. This is why in Test 12 and 13 there aren’t errors also with two different results.
The main reasons which let us prefer the Native Face Detector compared to JavaCV are without any doubt the much lower processing time and a wider compatibility. As shown in the chart below, in all tests regardless of the correct recognitions of one or more faces, NFD returns a result in about 2 – 2,5 seconds, while JavaCV takes an average of 7 seconds.
These results, along with the features mentioned earlier in this article, makes the NativeFace Detector the right choice for implementing the Live Face Tracker (video below). In the next article we will discuss how we realized the Live Face Tracker!
Please let us know any comment or correction you may have! Thanks for reading. Francesco 
[...] This post was mentioned on Twitter by Marcelo Queiroz, Small Screen Design. Small Screen Design said: Small Screen Design » About Face Detection on Android – Part 1 http://t.co/1nvoXYa via @SmallScrnDesign [...]
[...] the previous article we talked about face detection and the reason why Native Face Detector is the right choice for our [...]
[...] capabilities investigating various solutions. Francesco Florio (Small Screen Design) wrote a very interesting article about our findings and the features and issues related to Android face detection, in order to [...]
hello, its me. thanks for the quick reply!
i was given a task to use OpenCV/JavaCV for face detection and recognition, problem is, that i can’t seem to setup fully JavaCV on my development, i don’t know if i missed out something on the instructions.
thanks again!
Hi Konqueror7,
thanks for reading my articles.
Fortunately both OpenCV and JavaCV projects are very active, so things change fast. The best way to configure properly OpenCV/JavaCV is to follow their official README always present in each release.
Best,
Francesco
hello, i followed the instructions, placing OpenCV files into my PATH in Windows, and also adding the JARs to the classpath and library in eclipse, but get an exception always.
http://pastie.org/pastes/1615943/text
my JavaCV code was only “System.loadLibrary(“cv”);”
thanks!
Hi konqueror7,
OpenCV files are used by JavaCV JAR files. If you change folder paths it doesn’t work. In your Eclipse project, make a libs folder and puts JavaCV files into it (*.jar and armeabi folder with library files), then add Jar files as project libraries.
I hope this suggestions are useful for you
Best,
Francesco
thanks again for helping me out here!
http://db.tt/Nf7oXce
that is my project structure, i don’t know if you could compare it with your if you still got the source code.
oh, i don’t quite understand the part of JavaCV’s instructions on “First, put javacpp.jar, javacv.jar, and javacv-*.jar somewhere in your classpath, and make sure that the library files of OpenCV can be found either in their default installation directory or in the system PATH, which includes the current directory under Windows. Here are some more specific instructions for common cases: ”
maybe there lies my problem,,,,
thanks!
Sorry,
for my articles I used the previous JavaCV release that uses JNA instead of JAVACPP, futhermore I don’t use Windows.
Maybe you can find more help in the JavaCV discussion forum.
Good luck,
Francesco
ah, then i’ll try the previous release then,,,thanks!
and oh, i got a linux box too, i just booted in windows because maybe it was something being a 64-bit version of linux…
thanks!
Hi guys…
Im putting together an android app using openCV…. and im still stumped on what i need and should use… im learning the code as i go along and iv put some android apps together with eclipse… now im having some problems with openCV and linking the libraries…..
im new to openCV and i dont know what the hell is going on… i want to though!!!! lol… Iv downloaded your facedetection app and i just wana make something like that… well put it together as a learning experience.
btw im an IT student.
any help with how to get openCV, Cygwin, eclipse and android to work together is much appriciated…………..
Wow! Thank you! I always wanted to write in my site something like that. Can I take part of your post to my blog?
I delight in, lead to I discovered exactly what I was looking for. You have ended my four day lengthy hunt! God Bless you man. Have a great day. Bye
That’s a slick answer to a chalelngnig question
This is sooo great, thank you.