Face Normalization & Analysis

The goal of this finished project was the research and development of state of the art computer vision and object recognition algorithms to analyze face portrait images according to the ICAO (International Civil Aviation Organization) standards and specifications. Therefore a close cooperation with Siemens IT Solutions and Services Biometric Center in Graz existed, where the Biometry group was developing a software solution for this purpose. Current passports issued in the European Union contain biometric data like e.g. digital photographs and fingerprints in order to uniquely identify its owner. To be able to read passports all over the world, the ICAO has specified a number of guidelines and requirements for the structure of these biometric features. In case of face portrait images, examples for these requirements are neutral appearance, eyes opened, mouth closed, frontal pose, straight-looking eyes, properly-sitting eye-glasses, or uncovered faces. Since these analysis steps have to be performed in an automatic fashion, each of these requirements imposes certain computer vision research challenges which are tackled in this research project. Examples for the topics involved in these analysis steps are model-based segmentation using active shape and active appearance models, fast and robust AdaBoost based machine learning algorithms for face and face component detection, or classification of facial expressions using multi-classifier fusion approaches.

People

Martin Urschler, see also personal ICG website.

Markus Storer was a research assistant at the Institute for Computer Graphics and Vision at Graz University of Technology. He finished his PhD-Thesis focused on face normalization and pose analysis in 2011.

Matjaz Divjak was a post doc at the Institute for Computer Graphics and Vision at Graz University of Technology. He was employed for 2 years at ICG under a Marie Curie fellowship (PREWORK).

Related master student works:

Martin Hirzer did his master's thesis on the fully automatic segmentation of normalized face portrait images using a hybrid segmentation framework combining prior knowledge, a color distribution model and a total variation based boundary delineation.
Paul Urthaler did his master's thesis on the segmentation of different types of eyeglasses in the context of ICAO face analysis. Additionally the accordance of different eyeglasses to the ICAO specification was evaluated.
Matthias Straka did his master's thesis in the context of head pose estimation from facial images. He was working on the analysis of SIFT and HOG descriptors in the context of a support vector regression for pose estimation.

Funding Source

Record not found: ResearchProject

Project Related Publications

Supervised Local Subspace Learning for Continuous Head Pose Estimation Head pose estimation from images has recently attracted much attention in computer vision due to its diverse applications in face recognition, driver monitoring and human computer interaction. This paper presents Supervised Local Subspace Learning (SL2), a method that learns a local linear model from a sparse and non-uniformly sampled training set. SL2 learns a mixture of local tangent spaces that is robust to under-sampled regions, and due to its regularization properties it is also robust to over-fitting.
Huang D, Storer M, De La Torre F, Bischof H: Presented at IEEE CVPR 2011, Colorado Springs, USA. PDF

Efficient Robust Active Appearance Model Fitting
The Active Appearance Model (AAM) is a widely used approach for model based vision showing excellent results. But one major drawback is that the method is not robust against occlusions. Thus, if parts of the image are occluded the method converges to local minima and the obtained results are unreliable. To overcome this problem we propose a robust AAM fitting strategy. Storer M, Roth P, Urschler M, Bischof H, Birchbauer J: Published as chapter in Communications in Computer and Information Science (CCIS) vol. 68, Springer, 2010. PDF,Video1,Video2

Occlusion Detection for ICAO Compliant Facial Photographs In this work we present a novel algorithm for occlusion detection and evaluate its performance on several databases. Storer M, Urschler M, Bischof H: Presented at IEEE Computer Society Workshop on Biometrics, associated with IEEE CVPR 2010, San Francisco, USA. PDF

Intensity-Based Congealing for Unsupervised Joint Image Alignment
We present an approach for unsupervised alignment of an ensemble of images called congealing. Storer M, Urschler M, Bischof H: Presented at 20th International Conference on Pattern Recognition (ICPR) 2010, Istanbul, Turkey. PDF

Person Independent Head Pose Estimation by Non-Linear Regression and Manifold Embedding This paper describes an approach to head pose estimation in passport type images with an emphasis on high accuracy for near-frontal poses as well as person independence. Straka M, Urschler M, Storer M, Bischof H, Birchbauer J: Presented at 34th Workshop of the Austrian Association for Pattern Recognition (OAGM) 2010. PDF

3D-MAM: 3D Morphable Appearance Model for Efficient Fine Head Pose Estimation from Still Images This paper describes an approach to head pose estimation in passport type images with an emphasis on high accuracy for near-frontal poses as well as person independence. Storer M, Urschler M, Bischof H: Presented at International Conference on Computer Vision Workshop on Subspace Methods 2009, Kyoto, Japan. PDF

Fast-Robust PCA Principal Component Analysis (PCA) is not robust against outliers. Hence, we propose a novel fast and robust PCA (FR-PCA) algorithm. Storer M, Roth P, Urschler M, Bischof H: Presented at 16h Scandinavian Conference on Image Analysis 2009, Oslo, Norway. PDF

An Automatic Hybrid Segmentation Approach for Aligned Face Portrait Images Principal Component Analysis (PCA) is not robust against outliers. Hence, we propose a novel fast and robust PCA (FR-PCA) algorithm. Hirzer M, Urschler M, Bischof H, Birchbauer J: Presented at 33rd Workshop of the Austrian Association for Pattern Recognition (OAGM) 2009, Stainz, Austria. PDF

Robust Facial Component Detection for Face Alignment Applications In this work we present a robust facial component detection framework for identifying face and eyes in arbitrary image in order to perform ICAO compliant face normalization. Urschler M, Storer M, Bischof H, Birchbauer J: Presented at 33rd Workshop of the Austrian Association for Pattern Recognition (OAGM) 2009, Stainz, Austria. PDF

Eye blink based fatigue detection for prevention of Computer Vision Syndrome Computer Vision Syndrome (CVS) affects a large number of people. We present a prototype of a computer vision system for real-time eye blink based detection of eye fatigue, which is a common symptom leading towards CVS. Experimental results show that the proposed system is capable of detecting common cases of fatigued behaviour linked with long-term computer use. Divjak M, Bischof H: Presented at IAPR Conference on Machine Vision Applications (MVA) 2009, Tokyo, Japan. PDF

Classifier Fusion for Robust ICAO Compliant Face Analysis In this work we present a robust facial component detection framework for identifying face and eyes in arbitrary image in order to perform ICAO compliant face normalization. Storer M, Urschler M, Bischof H, Birchbauer J: Presented at IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2008, Amsterdam, Netherlands. PDF

On Combining Classifiers for Assessing Portrait Image Compliance with ICAO/ISO Standards
Evaluation of several classifier combination schemes for checking facial images against parts of the International Civil Aviation Organization (ICAO) criteria. Storer M, Urschler M, Bischof H, Birchbauer J: Presented at Biometrics and Electronic Signatures (BIOSIG) 2008, Darmstadt, Germany. PDF

Face Image Normalization and Expression/Pose Validation for the Analysis of Machine Readable Travel Documents
We present a face normalization and analysis system implementing several parts of the International Civil Aviation Organization (ICAO) standard. The goal is to prepare automated recognition from travel document photos. Storer M, Urschler M, Bischof H, Birchbauer J: Presented at 32nd Workshop of the Austrian Association for Pattern Recognition (OAGM) 2008, Linz, Austria. PDF

Real-time video-based eye blink analysis for detection of low blink-rate during computer use
This paper presents an approach for real-time remote detection of eyeblink parameters. First, a combination of boosted classifiers and LucasKanade tracking is used to follow the movement of face and eyes. Then, detailed eye movement is described by normal flow. Finally, a discrete finite state machine is used to detect eye-blinks. Divjak M, Bischof H: Presented at First International Workshop on Tracking Humans for the Evaluation of their Motion in Image Sequences (THEMIS) 2008, Leeds, UK. PDF