The FERET Evaluation Methodology for Face-Recognition Algorithms.
AbstractTwo of the most critical requirements in support of producing reliable face-recognition systems are a large database of facial images and a testing procedure to evaluate systems. The Face Recognition Technology (FERET) program has addressed both issues through the FERET database of facial images and the establishment of the FERET tests. To date, 14,126 images from 1,199 individuals are included in the FERET database, which is divided into development and sequestered portions of the database. In September 1996, the FERET program administered the third in a series of FERET face-recognition tests. The primary objectives of the third test were to 1) assess the state of the art, 2) identify future areas of research, and
Introduction
Over the last decade, face recognition has become an active area of research in computer
vision, neuroscience, and psychology. Progress has advanced to the point that
face-recognition systems are being demonstrated in real-world settings [5]. The rapid development
of face recognition is due to a combination of factors: active development of
The work reported here is part of the Face Recognition Technology (FERET) program, which is
sponsored by the U.S. Department of Defense Counterdrug Technology Development Program. Portions
of this work was done while Jonathon Phillips was at the U.S. Army Research Laboratory (ARL). Jonathon
Phillips acknowledges the support of the National Institute of Justice.
algorithms, the availability of a large database of facial images, and a method for evaluating
the performance of face-recognition algorithms. The FERET database and evaluation
methodology address the latter two points and are de facto standards. There have been
three FERET evaluations with the most recent being the Sep96 FERET test.
The Sep96 FERET test provides a comprehensive picture of the state-of-the-art in face
recognition from still images. This was accomplished by evaluating algorithms' ability on
different scenarios, categories of images, and versions of algorithms. Performance was
computed for identification and verification scenarios. In an identification application, an
algorithm is presented with a face that it must identify the face; whereas, in a verification
application, an algorithm is presented with a face and a claimed identity, and the algorithm
must accept or reject the claim. In this paper, we describe the FERET database, the
evaluation protocol, and present identification results. Verification results
are presented in Rizvi et al. [8].
To obtain a robust assessment of performance, algorithms are evaluated against different
categories of images. The categories are broken out by lighting changes, people
wearing glasses, and the time between the acquisition date of the database image and the
image presented to the algorithm. By breaking out performance into these categories, a
better understanding of the face recognition field in general as well as the strengths and
weakness of individual algorithms is obtained. This detailed analysis helps to assess which
applications can be successfully addressed.
All face recognition algorithms known to the authors consist of two parts: (1) face
detection and normalization and (2) face identification. Algorithms that consist of both
parts are referred to as fully automatic algorithms, and those that consist of only the
second part are partially automatic algorithms. The Sep96 test evaluated both fully and
partially automatic algorithms. Partially automatic algorithms are given a facial image
and the coordinates of the center of the eyes. Fully automatic algorithms are only given
facial images.
The availability of the FERET database and evaluation methodology has made a
significant difference in the progress of development of face-recognition algorithms. Before
the FERET database was created, a large number of papers reported outstanding
recognition results (usually ? 95% correct recognition) on limited-size databases (usually
(In fact, this is still true.) Only a few of these algorithms reported
results on images utilizing a common database, let alone met the desirable goal of being
evaluated on a standard testing protocol that included separate training and testing sets.
As a consequence, there was no method to make informed comparisons among various
algorithms.
The FERET database has made it possible for researchers to develop algorithms on
a common database and to report results in the literature using this database. Results
reported in the literature do not provide a direct comparison among algorithms because
each researcher reported results using different assumptions, scoring methods, and images.
The independently administered FERET test allows for a direct quantitative assessment
of the relative strengths and weaknesses of different approaches.
More importantly, the FERET database and tests clarify the current state of the art
in face recognition and point out general directions for future research. The FERET
tests allow the computer vision community to assess overall strengths and weaknesses
in the field, not only on the basis of the performance of an individual algorithm, but
in addition on the aggregate performance of all algorithms tested. Through this type
of assessment, the community learns in an unbiased and open manner of the important
technical problems to be addressed, and how the community is progressing toward solving
these problems.
Background
The first FERET tests took place in August 1994 and March 1995 (for details of these tests
and the FERET database and program, see Phillips et al [5, 6] and Rauss et al [7]); The
FERET database collection began in September 1993 along with the FERET program.
The August 1994 test established, for the first time, a performance baseline for face-recognition
algorithms. This test was designed to measure performance on algorithms
that could automatically locate, normalize, and identify faces from a database. The test
consisted of three subtests, each with a different gallery and probe set. The gallery contains
the set of known individuals. An image of an unknown face presented to the algorithm
is called a probe, and the collection of probes is called the probe set. The first subtest
examined the ability of algorithms to recognize faces from a gallery of 316 individuals.
The second was the false-alarm test, which measured how well an algorithm rejects faces
not in the gallery. The third baselined the effects of pose changes on performance.
The second FERET test, that took place in March 1995, measured progress since
August 1994 and evaluated algorithms on larger galleries. The March 1995 evaluation
consisted of a single test with a gallery of 817 known individuals. One emphasis of the
test was on probe sets that contained duplicate images. A duplicate is defined as an image
of a person whose corresponding gallery image was taken on a different date.
The FERET database is designed to advance the state of the art in face recognition,
with the images collected directly supporting both algorithm development and the FERET
evaluation tests. The database is divided into a development set, provided to researchers,
and a set of sequestered images for testing. The images in the development set are
representative of the sequestered images.
The facial images were collected in 15 sessions between August 1993 and July 1996.
Collection sessions lasted one or two days. In an effort to maintain a degree of consistency
throughout the database, the same physical setup and location was used in each photography
session. However, because the equipment had to be reassembled for each session,
there was variation from session to session (figure 1).
Images of an individual were acquired in sets of 5 to 11 images, collected under relatively
unconstrained conditions. Two frontal views were taken (fa and fb); a different
facial expression was requested for the second frontal image. For 200 sets of images, a
third frontal image was taken with a different camera and different lighting (this is referred
to as the fc image). The remaining images were collected at various aspects between right
and left profile. To add simple variations to the database, photographers sometimes took
a second set of images, for which the subjects were asked to put on their glasses and/or
pull their hair back. Sometimes a second set of images of a person was taken on a later
date; such a set of images is referred to as a duplicate set. Such duplicates sets result in
variations in scale, pose, expression, and illumination of the face.
By July 1996, 1564 sets of images were in the database, consisting of 14,126 total
images. The database contains 1199 individuals and 365 duplicate sets of images. For
duplicate I fc duplicate II

Figure

1: Examples of different categories of probes (image). The duplicate I image was
taken within one year of the fa image and the duplicate II and fa images were taken at
least one year apart.
some people, over two years elapsed between their first and most recent sittings, with some
subjects being photographed multiple times (figure 1). The development portion of the
database consisted of 503 sets of images, and was released to researchers. The remaining
images were sequestered by the Government.
3 Test Design
3.1 Test Design Principles
The FERET Sep96 evaluation protocol was designed to assess the state of the art, advance
the state of the art, and point to future directions of research. To succeed at this, the test
design must solve the three bears problem. The test cannot be neither too hard nor too
easy. If the test is too easy, the testing process becomes an exercise in "tuning" existing
algorithms. If the test is too hard, the test is beyond the ability of existing algorithmic
techniques. The results from the test are poor and do not allow for an accurate assessment
of algorithmic capabilities.
The solution to the three bears problem is through the selection of images in the
test set and the testing protocol. Tests are administered using a testing protocol that
states the mechanics of the tests and the manner in which the test will be scored. In face
recognition, the protocol states the number of images of each person in the test, how the
output from the algorithm is recorded, and how the performance results are reported.
The characteristics and quality of the images are major factors in determining the
difficulty of the problem being evaluated. For example, if faces are in a predetermined
position in the images, the problem is different from that for images in which the faces can
be located anywhere in the image. In the FERET database, variability was introduced
by the inclusion of images taken at different dates and locations (see section 2). This
resulted in changes in lighting, scale, and background.
The testing protocol is based on a set of design principles. Stating the design principle
allows one to assess how appropriate the FERET test is for a particular face recognition
algorithm. Also, design principles assist in determining if an evaluation methodology
for testing algorithm(s) for a particular application is appropriate. Before discussing the
design principles, we state the evaluation protocol.
In the testing protocol, an algorithm is given two sets of images: the target set and the
query set. We introduce this terminology to distinguish these sets from the gallery and
probe sets that are used in computing performance statistics. The target set is given to
the algorithm as the set of known facial images. The images in the query set consists of
unknown facial images to be identified. For each image q i in the query set Q, an algorithm
reports a similarity s i (k) between q i and each image t k in the target set T . The testing
protocol is designed so that each algorithm can use a different similarity measure and we
do not compare similarity measures from different algorithms. The key property of the
new protocol, which allows for greater flexibility in scoring, is that for any two images q i
and t k , we know s i (k).
This flexibility allows the evaluation methodology to be robust and comprehensive;
it is achieved by computing scores for virtual galleries and probe sets. A gallery G is a
virtual gallery if G is a subset of the target set, i.e., G ae T . Similarly, P is a virtual probe
set if P ae Q. For a given gallery G and probe set P, the performance scores are computed
by examination of similarity measures s i (k) such that q G.
The virtual gallery and probe set technique allows us to characterize algorithm performance
by different categories of images. The different categories include (1) rotated
images, (2) duplicates taken within a week of the gallery image, (3) duplicates where the
time between the images is at least one year, (4) galleries containing one image per person,
and (5) galleries containing duplicate images of the same person. We can create a gallery of
100 people and estimate an algorithm's performance by recognizing people in this gallery.
Using this as a starting point, we can then create virtual galleries of 200;
people and determine how performance changes as the size of the gallery increases. Another
avenue of investigation is to create n different galleries of size 100, and calculate the
variation in algorithm performance with the different galleries.
To take full advantage of virtual galleries and probe sets, we selected multiple images
of the same person and placed them into the target and query sets. If such images were
marked as the same person, the algorithms being tested could use the information in the
evaluation process. To prevent this from happenning, we require that each image in the
target set be treated as an unique face. (In practice, this condition is enforced by giving
every image in the target and query set a unique random identification.) This is the first
design principle.
The second design principle is that training is completed prior to the start of the test.
This forces each algorithm to have a general representation for faces, not a representation
tuned to a specific gallery. Without this condition, virtual galleries would not be possible.
For algorithms to have a general representation for faces, they must be gallery (class)
insensitive. Examples are algorithms based on normalized correlation or principal component
analysis (PCA). An algorithm is class sensitive if the representation is tuned to
a specific gallery. Examples are straight forward implementation of Fisher discriminant
analysis [1, 9]. Fisher discriminant algorithms were adapted to class insensitive testing
methodologies by Zhao et al [13, 14], with performance results of these extensions being
reported in this paper.
The third design rule is that all algorithms tested compute a similarity measure between
two facial images; this similarity measure was computed for all pairs of images
between the target and query sets. Knowing the similarity score between all pairs of
Face Recognition
Algorithm
(run at Testee's site)
Output
File
Scoring
Code
Government
(run at
Results
Probe
Image
Name
Gallery
Image
Name
(One Image/Person)
Gallery images
List of Probes

Figure

2: Schematic of the FERET testing procedure
images from the target and query sets allows for the construction of virtual galleries and
probe sets.
3.2 Test Details
In the Sep96 FERET test, the target set contained 3323 images and the query set 3816
images. All the images in the target set were frontal images. The query set consisted
of all the images in the target set plus rotated images and digitally modified images.
We designed the digitally modified images to test the effects of illumination and scale.
(Results from the rotated and digitally modified images are not reported here.) For each
query image q i , an algorithm outputs the similarity measure s i (k) for all images t k in the
target set. For a given query image q i , the target images t k are sorted by the similarity
scores s i (\Delta). Since the target set is a subset of the query set, the test output contains the
similarity score between all images in the target set.
There were two versions of the Sep96 test. The target and query sets were the same for
each version. The first version tested partially automatic algorithms by providing them
with a list of images in the target and query sets, and the coordinates of the center of
the eyes for images in the target and query sets. In the second version of the test, the
coordinates of the eyes were not provided. By comparing the performance between the
two versions, we estimate performance of the face-locating portion of a fully automatic
algorithm at the system level.
The test was administered at each group's site under the supervision of one of the au-
thors. Each group had three days to complete the test on less than 10 UNIX workstations
(this limit was not reached). We did not record the time or number of workstations because
execution times can vary according to the type of machines used, machine and network
configuration, and the amount of time that the developers spent optimizing their code (we
wanted to encourage algorithm development, not code optimization). (We imposed the
time limit to encourage the development of algorithms that could be incorporated into
operational, fieldable systems.)
The images contained in the gallery and probe sets consisted of images from both
the developmental and sequestered portions of the FERET database. Only images from
the FERET database were included in the test; however, algorithm developers were not
prohibited from using images outside the FERET database to develop or tune parameters
in their algorithms.
The FERET test is designed to measure laboratory performance. The test is not
concerned with speed of the implementation, real-time implementation issues, and speed
and accuracy trade-offs. These issues and others, need to be addressed in an operational,
fielded system, were beyond the scope of the Sep96 FERET test.

Figure

presents a schematic of the testing procedure. To ensure that matching was
not done by file name, we gave the images random names. The nominal pose of each face
was provided to the testee.
4 Decision Theory and Performance Evaluation
The basic models for evaluating the performance of an algorithm are the closed and open
universes. In the closed universe, every probe is in the gallery. In an open universe,
some probes are not in the gallery. Both models reflect different and important aspects of
face-recognition algorithms and report different performance statistics. The open universe
models verification applications. The FERET scoring procedures for verification is given
in Rizvi et al [8].
The closed-universe model allows one to ask how good an algorithm is at identifying
a probe image; the question is not always "is the top match correct?" but "is the correct
answer in the top n matches?" This lets one know how many images have to be examined
to get a desired level of performance. The performance statistics are reported as cumulative
match scores. The rank is plotted along the horizontal axis, and the vertical axis is
the percentage of correct matches. The cumulative match score can be calculated for any
subset of the probe set. We calculated this score to evaluate an algorithm's performance
on different categories of probes, i.e., rotated or scaled probes.
The computation of an identification score is quite simple. Let P be a probe set and
jPj the size of P. We score probe set P against gallery G, where and
by comparing the similarity scores s i (\Delta) such that G. For
each probe image G. We assume that
a smaller similarity score implies a closer match. If g k and p i are the same image, then
The function id(i) gives the index of the gallery image of the person in probe
is an image of the person in g id(i) . A probe p i is correctly identified if s i (id(i))
is the smallest scores for G. A probe p i is in the top k if s i (id(i)) is one of the k-th
smallest score s i (\Delta) for gallery G. Let R k denote the number of probes in the top k. We
reported R k =jPj, the fraction of probes in the top k. As an example, let
and 100. Based on the formula, the performance score for R 5 is
In reporting identification performance results, we state the size of the gallery and the
number of probes scored. The size of the gallery is the number of different faces (people)
contained in the images that are in the gallery. For all results that we report, there is one
image per person in the gallery, thus, the size of the gallery is also the number of images
in the gallery. The number of probes scored (also, size of the probe set) is jPj. The probe
set may contain more than one image of a person and the probe set may not contain an
image of everyone in the gallery. Every image in the probe set has a corresponding image
in the gallery.
5 Latest Test Results
The Sep96 FERET test was designed to measure algorithm performance for identification
and verification tasks. Both tasks are evaluated on the same sets of images. We report
the results for 12 algorithms that includes 10 partially automatic algorithms and 2 fully
automatic algorithms. The test was administered in September 1996 and March 1997
(see table 1 for details of when the test was administered to which groups and which
version of the test was taken). Two of these algorithms were developed at the MIT
Media Laboratory. The first was the same algorithm that was tested in March 1995.
This algorithm was retested so that improvement since March 1995 could be measured.
The second algorithm was based on more recent work [2, 3]. Algorithms were also tested
from Excalibur Corp. (Carlsbad, CA), Michigan State University (MSU) [9, 14], Rutgers
University [11], University of Southern California (USC) [12], and two from University of
Maryland (UMD) [1, 13, 14]. The first algorithm from UMD was tested in September 1996
and a second version of the algorithm was tested in March 1997. For the fully automatic
version of test, algorithms from MIT and USC were evaluated.
The final two algorithms were our implementation of normalized correlation and a
principal components analysis (PCA) based algorithm [4, 10]. These algorithms provide
a performance baseline. In our implementation of the PCA-based algorithm, all images
were (1) translated, rotated, and scaled so that the center of the eyes were placed on
specific pixels, (2) faces were masked to remove background and hair, and (3) the non-masked
facial pixels were processed by a histogram equalization algorithm. The training
set consisted of 500 faces. Faces were represented by their projection onto the first 200
eigenvectors and were identified by a nearest neighbor classifier using the L 1 metric. For
normalized correlation, the images were (1) translated, rotated, and scaled so that the
center of the eyes were placed on specific pixels and (2) faces were masked to remove
background and hair.
5.1 Partially automatic algorithms
We report identification scores for four categories of probes. The first probe category was
the FB probes (fig 3). For each set of images, there were two frontal images. One of the
images was randomly placed in the gallery, and the other image was placed in the FB
probe set. (This category is denoted by FB to differentiate it from the fb images in the
FERET database.) The second probe category contained all duplicate frontal images in
the FERET database for the gallery images. We refer to this category as the duplicate I
probes. The third category was the fc (images taken the same day, but with a different
camera and lighting). The fourth consisted of duplicates where there is at least one year
between the acquisition of the probe image and corresponding gallery image. We refer
to this category as the duplicate II probes. For this category, the gallery images were
acquired before January 1995 and the probe images were acquired after January 1996.

Table

1: List of groups that took the Sept96 test broken out by versions taken and dates
administered. (The 2 by MIT indicates that two algorithms were tested.)
Test
September March
Version of test Group 1996 1997 Baseline
Fully Automatic MIT Media Lab [2, 3] ffl
U. of So. California (USC) [12] ffl
Eye Coordinates Given Baseline PCA [4, 10] ffl
Baseline Correlation ffl
Excalibur
Michigan State U. [9, 14] ffl
Rutgers U. [11] ffl
U Maryland [1, 13, 14] ffl ffl
The gallery for the FB, duplicate I, and fc probes was the same and consisted of 1196
frontal images with one image person in the gallery (thus the gallery contained 1196
individuals). Also, none of the faces in the gallery images wore glasses. The gallery for
duplicate II probes was a subset of 864 images from the gallery for the other categories.
The results for identification are reported as cumulative match scores. Table 2 shows
the categories corresponding to the figures presenting the results, type of results, and size
of the gallery and probe sets (figs 3 to 6).
In figures 7 and 8, we compare the difficulty of different probe sets. Whereas, figure 4
reports identification performance for each algorithm, figure 7 shows a single curve that
is an average of the identification performance of all algorithms for each probe category.
For example, the first ranked score for duplicate I probe sets is computed from an average
of the first ranked score for all algorithms in figure 4. In figure 8, we presented current
upper bound for performance on partially automatic algorithms for each probe category.
For each category of probe, figure 8 plots the algorithm with the highest top rank score
(R 1 ).

Figures

7 and 8 reports performance of four categories of probes, FB, duplicate I,
fc, duplicate II.

Table

2: Figures reporting results for partially automatic algorithms. Performance is
broken out by probe category.
Figure no. Probe Category Gallery size Probe set size
4 duplicate I 1196 722
6 duplicate II 864 234
Cumulative
match
score
Rank
MSU
UMD 96
MIT 95
Baseline
Baseline EF
Excalibur
Rutgers
(a)0.80.91
Cumulative
match
score
Rank
UMD 97
USC
UMD 96
Baseline
Baseline EF
(b)

Figure

3: performance against FB probes. (a) Partially automatic algo-0.95
Cumulative
match
score
Rank
Excalibur
Baseline EF
Baseline
MIT 95
MSU
UMD 96
Rutgers
(a)0.40.60.81
Cumulative
match
score
Rank
USC
UMD 97
Baseline EF
Baseline
UMD 96
(b)

Figure

4: performance against all duplicate I probes. (a) Partially automatic
Cumulative
match
score
Rank
UMD 96
MSU
Excalibur
Baseline EF
Rutgers
MIT 95
Baseline
(a)0.20.61
Cumulative
match
score
Rank
USC
UMD 97
UMD 96
Baseline EF
Baseline
(b)

Figure

5: performance against fc probes. (a) Partially automatic algorithms
Cumulative
match
score
Rank
Baseline EF
Excalibur
Rutgers
MIT 95
Baseline
UMD 96
MSU
(a)0.20.61
Cumulative
match
score
Rank
USC
Baseline EF
UMD 97
Baseline
UMD 96
(b)

Figure

performance against duplicate II probes. (a) Partially automatic
Cumulative
Match
Score
Rank
FB probes
Duplicate I probes
fc probes
Duplicate II probes

Figure

7: Average identification performance of partially automatic algorithms on each
probe category.0.20.61
Cumulative
Match
Score
Rank
FB probes
fc probes
Duplicate I probes
Duplicate II probes

Figure

8: Current upper bound identification performance of partially automatic algorithm
for each probe category.
Cumulative
Match
Score
Rank
USC partially automatic
MIT partially automatic
USC fully automatic
MIT fully automatic

Figure

9: Identification performance of fully automatic algorithms against partially automatic
algorithms for FB
Cumulative
Match
Score
Rank
USC partially automatic
USC fully automatic
MIT partially automatic
MIT fully automatic

Figure

10: Identification performance of fully automatic algorithms against partially automatic
algorithms for duplicate I probes.
5.2 Fully Automatic Performance
In this subsection, we report performance for the fully automatic algorithms of the MIT
Media Lab and USC. To allow for a comparison between the partially and fully automatic
algorithms, we plot the results for the partially and fully automatic algorithms. Figure 9
shows performance for FB probes and figure 10 shows performance for duplicate I probes.
(The gallery and probe sets are the same as in subsection 5.1.)
5.3 Variation in Performance
From a statistical point of view, a face-recognition algorithm estimates the identity of
a face. Consistent with this view, we can ask about the variance in performance of
an algorithm: "For a given category of images, how does performance change if the
algorithm is given a different gallery and probe set?" In tables 3 and 4, we show how
algorithm performance varies if the people in the galleries change. For this experiment, we
constructed six galleries of approximately 200 individuals, in which an individual was in
only one gallery (the number of people contained within each gallery versus the number of
probes scored is given in tables 3 and 4). Results are reported for the partially automatic
algorithms. For the results in this section, we order algorithms by their top rank score
on each gallery; for example, in table 3, the UMD Mar97 algorithm scored highest on
gallery 1 and the baseline PCA and correlation tied for 9th place. Also included in this
table is average performance for all algorithms. Table 3 reports results for FB probes.

Table

4 is organized in the same manner as table 3, except that duplicate I probes are
scored. Tables 3 and 4 report results for the same gallery. The galleries were constructed
by placing images within the galleries by chronological order in which the images were
collected (the first gallery contains the first images collected and the 6th gallery contains
the most recent images collected). In table 4, mean age refers to the average time between
collection of images contained in the gallery and the corresponding duplicate probes. No
scores are reported in table 4 for gallery 6 because there are no duplicates for this gallery.
6 Discussion and Conclusion
In this paper we presented the Sep96 FERET evaluation protocol for face recognition
algorithms. The protocol makes it possible to independently evaluate algorithms. The
protocol was designed to evaluate algorithms on different galleries and probe sets for different
scenarios. Using this protocol, we computed performance on identification and
verification tasks. The verification results are presented in Rizvi et al. [8], and all verification
results mentioned in this section are from that paper. In this paper we presented
detailed identification results. Because of the Sep96 FERET evaluation protocol's ability
to test algorithms performance on different tasks for multiple galleries and probe sets, it
is the de facto standard for measuring performance of face recognition algorithms. These
results show that factors effecting performance include scenario, date tested, and probe
category.
The Sep96 test was the latest FERET test (the others were the Aug94 and Mar95
tests [6]). One of the main goals of the FERET tests has been to improve the performance
of face recognition algorithms, and is seen in the Sep96 FERET test. The first case is
the improvement in performance of the MIT Media Lab September 1996 algorithm over

Table

3: Variations in identification performance on six different galleries on FB probes.
Images in each gallery do not overlap. Ranks range from 1-10.
Algorithm Ranking by Top Match
Gallery Size / Scored Probes
200/200 200/200 200/200 200/200 200/199 196/196
Algorithm gallery 1 gallery 2 gallery 3 gallery 4 gallery 5 gallery 6
Baseline
Baseline correlation 9 9 9
Excalibur
Michigan State Univ. 3 4 5 8 4 4
Rutgers Univ. 7 8 9 6 7 9
Average Score 0.935 0.857 0.904 0.918 0.843 0.804

Table

4: Variations in identification performance on five different galleries on duplicate
probes. Images in each of the gallery does not overlap. Ranks range from 1-10.
Algorithm Ranking by Top Match
Gallery Size / Scored Probes
200/143 200/64 200/194 200/277 200/44
Mean Age of Probes (months) 9.87 3.56 5.40 10.70 3.45
Algorithm gallery 1 gallery 2 gallery 3 gallery 4 gallery 5
Baseline
Baseline correlation
Excalibur
Michigan State Univ. 9
Rutgers Univ.
Average Score 0.238 0.620 0.645 0.523 0.687
the March 1995 algorithm; the second is the improvement of the UMD algorithm between
September 1996 and March 1997.
By looking at progress over the series of FERET tests, one sees that substantial
progress has been made in face recognition. The most direct method is to compare the
performance of fully automatic algorithms on fb probes (the two earlier FERET tests
only evaluated fully automatic algorithms. The best top rank score for fb probes on the
Aug94 test was 78% on a gallery of 317 individuals, and for Mar95, the top score was
93% on a gallery of 831 individuals [6]. This compares to 87% in September 1996 and
95% in March 1997 (gallery of 1196 individuals). This method shows that over the course
of the FERET tests, the absolute scores increased as the size of the database increased.
The March 1995 score was from one of the MIT Media Lab algorithms, and represents an
increase from 76% in March 1995.
On duplicate I probes, MIT Media Lab improved from 39% (March 1995) to 51%
(September 1996); USC's performance remained approximately the same at 57-58% between
March 1995 and March 1997. This improvement in performance was achieved while
the gallery size increased and the number of duplicate I probes increased from 463 to 722.
While increasing the number of probes does not necessarily increase the difficulty of identification
tasks, we argue that the Sep96 duplicate I probe set was more difficult to process
then the Mar95 set. The Sep96 duplicate I probe set contained the duplicate II probes
and the Mar95 duplicate I probe set did not contain a similar class of probes. Overall,
the duplicate II probe set was the most difficult probe set.
Another goal of the FERET tests is to identify areas of strengths and weaknesses
in the field of face recognition. We addressed this issue by computing algorithm performance
for multiple galleries and probe sets. From this evaluation, we concluded that
algorithm performance is dependent on the gallery and probe sets. We observed variation
in performance due to changing the gallery and probe set within a probe category, and
by changing probe categories. The effect of changing the gallery while keeping the probe
category constant is shown in tables 3 and 4. For fb probes, the range for performance is
80% to 94%; for duplicate I probes, the range is 24% to 69%. Equally important, tables 3
and 4 shows the variability in relative performance levels. For example, in table 4, UMD
duplicate performance varies between number three and nine. Similar results were
found in Moon and Phillips [4] in their study of principal component analysis-based face
recognition algorithms. This shows that an area of future research could measure the effect
of changing galleries and probe sets, and statistical measures that characterize these
variations.

Figures

7 and 8 shows probe categories characterized by difficulty. These figures show
that fb probes are the easiest and duplicate II probes are the most difficult. On average,
duplicate I probes are easier to identify than fc probes. However, the best performance on
fc probes is significantly better than the best performance on duplicate I and II probes.
This comparative analysis shows that future areas of research could address processing of
duplicate II probes and developing methods to compensate for changes in illumination.
The scenario being tested contributes to algorithm performance. For identification,
the MIT Media Lab algorithm was clearly the best algorithm tested in September 1996.
However, for verification, there was not an algorithm that was a top performer for all probe
categories. Also, for the algorithms tested in March 1997, the USC algorithm performed
overall better than the UMD algorithm for identification; however, for verification, UMD
overall performed better. This shows that performance on one task is not predictive of
performance on another task.
The September 1996 FERET test shows that definite progress is being made in face
recognition, and that the upper bound in performance has not been reached. The improvement
in performance documented in this paper shows directly that the FERET series
of tests have made a significant contribution to face recognition. This conclusion is indirectly
supported by (1) the improvement in performance between the algorithms tested
in September 1996 and March 1997, (2) the number of papers that use FERET images
and report experimental results using FERET images, and (3) the number of groups that
participated in the Sep96 test.



--R

Discriminant analysis for recognition of human face images.
Bayesian face recognition using deformable intensity surfaces.
Probabilistic visual learning for object detection.
Analysis of PCA-based face recognition algorithms
The face recognition technology (FERET) program.
The FERET database and evaluation procedure for face-recognition algorithms
The FERET (Face Recognition Technology) program.
The feret verification testing protocol for face recognition algorithms.
Using discriminant eigenfeatures for image retrieval.
Eigenfaces for recognition.
Face recognition using transform coding of gray scale projection projections and the neural tree network.

Discriminant analysis of principal components for face recognition.
Discriminant analysis of principal components for face recognition.
--TR

--CTR
Alice J. O'Toole , Joshua Harms , Sarah L. Snow , Dawn R. Hurst , Matthew R. Pappas , Janet H. Ayyad , Herve Abdi, A Video Database of Moving Faces and People, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.5, p.812-816, May 2005
J. Shi , A. Samal , D. Marx, How effective are landmarks and their geometry for face recognition?, Computer Vision and Image Understanding, v.102 n.2, p.117-133, May 2006
Zhang , Longbin Chen , Mingjing Li , Hongjiang Zhang, Automated annotation of human faces in family albums, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA
Linlin Shen , Li Bai, Information theory for Gabor feature selection for face recognition, EURASIP Journal on Applied Signal Processing, v.2006 n.1, p.8-8, 01 January
Xiaogang Wang , Xiaoou Tang, Bayesian face recognition using Gabor features, Proceedings of the ACM SIGMM workshop on Biometrics methods and applications, November 08, 2003, Berkley, California
Florent Perronnin , Jean-Luc Dugelay , Kenneth Rose, A Probabilistic Model of Face Mapping with Local Transformations and Its Application to Person Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.7, p.1157-1171, July 2005
Ayoub K. Al-Hamadi , Robert Niese , Axel Panning , Bernd Michaelis, Toward robust face analysis method of non-cooperative persons in stereo color image sequences, Machine Graphics & Vision International Journal, v.15 n.3, p.245-254, January 2006
Julia Vogel , Bernt Schiele, Performance evaluation and optimization for content-based image retrieval, Pattern Recognition, v.39 n.5, p.897-909, May, 2006
Haitao Zhao , Shaoyuan Sun , Zhongliang Jing , Jingyu Yang, Rapid and brief communication: Local structure based supervised feature extraction, Pattern Recognition, v.39 n.8, p.1546-1550, August, 2006
Kyong Chang , Kevin W. Bowyer , Sudeep Sarkar , Barnabas Victor, Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.9, p.1160-1165, September
Hochul Shin , Seong-Dae Kim , Hae-Chul Choi, Generalized elastic graph matching for face recognition, Pattern Recognition Letters, v.28 n.9, p.1077-1082, July, 2007
Wang , Xiao Wang , Jufu Feng, Subspace distance analysis with application to adaptive Bayesian algorithm for face recognition, Pattern Recognition, v.39 n.3, p.456-464, March, 2006
Xiaoyang Tan , Jun Liu , Songcan Chen, Letters: Sub-intrapersonal space analysis for face recognition, Neurocomputing, v.69 n.13-15, p.1796-1801, August, 2006
Shaohua Zhou , Volker Krueger , Rama Chellappa, Probabilistic recognition of human faces from video, Computer Vision and Image Understanding, v.91 n.1-2, p.214-245, July
LinLin Shen , Li Bai , Michael Fairhurst, Gabor wavelets and General Discriminant Analysis for face identification and verification, Image and Vision Computing, v.25 n.5, p.553-563, May, 2007
Xiaoxun Zhang , Yunde Jia, Face recognition with local steerable phase feature, Pattern Recognition Letters, v.27 n.16, p.1927-1933, December 2006
Xiaoxun Zhang , Yunde Jia, Face recognition with local steerable phase feature, Pattern Recognition Letters, v.27 n.16, p.1927-1933, December, 2006
Zhihong Pan , Glenn Healey , Manish Prasad , Bruce Tromberg, Face Recognition in Hyperspectral Images, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.12, p.1552-1560, December
Kyong I. Chang , Kevin W. Bowyer , Patrick J. Flynn, An Evaluation of Multimodal 2D+3D Face Biometrics, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.4, p.619-624, April 2005
Shumeet Baluja , Henry A. Rowley, Boosting Sex Identification Performance, International Journal of Computer Vision, v.71 n.1, p.111-119, January   2007
Ming-Hsuan Yang, Extended isomap for pattern classification, Eighteenth national conference on Artificial intelligence, p.224-229, July 28-August 01, 2002, Edmonton, Alberta, Canada
Weilong Chen , Meng Joo Er , Shiqian Wu, PCA and LDA in DCT domain, Pattern Recognition Letters, v.26 n.15, p.2474-2482, November 2005
Zhang Xiaoxun , Jia Yunde, Symmetrical null space LDA for face and ear recognition, Neurocomputing, v.70 n.4-6, p.842-848, January, 2007
Wangmeng Zuo , Kuanquan Wang , David Zhang , Hongzhi Zhang, Combination of two novel LDA-based methods for face recognition, Neurocomputing, v.70 n.4-6, p.735-742, January, 2007
QingShan Liu , Rui Huang , HanQing Lu , SongDe Ma, Kernel-based nonlinear discriminant analysis for face recognition, Journal of Computer Science and Technology, v.18 n.6, p.788-795, November
Xiaogang Wang , Xiaoou Tang, A Unified Framework for Subspace Face Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.9, p.1222-1228, September 2004
Philip de Chazal , John Flynn , Richard B. Reilly, Automated Processing of Shoeprint Images Based on the Fourier Transform for Use in Forensic Science, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.3, p.341-350, March 2005
Wang , Xiao Wang , Xuerong Zhang , Jufu Feng, The equivalence of two-dimensional PCA to line-based PCA, Pattern Recognition Letters, v.26 n.1, p.57-60, 1 January 2005
T. Martiriggiano , M. Leo , T. D'Orazio , A. Distante, Face recognition by Kernel independent component analysis, Proceedings of the 18th international conference on Innovations in Applied Artificial Intelligence, p.55-58, June 22-24, 2005, Bari, Italy
Xiaoyan Mu , Mehmet Artiklar , Paul Watta , Mohamad H. Hassoun, An RCE-based Associative Memory with Application to Human Face Recognition, Neural Processing Letters, v.23 n.3, p.257-271, June      2006
Jiatao Song , Zheru Chi , Jilin Liu, A robust eye detection method using combined binary edge and intensity information, Pattern Recognition, v.39 n.6, p.1110-1125, June, 2006
A. N. Rajagopalan , K. Srinivasa Rao , Y. Anoop Kumar, Face recognition using multiple facial features, Pattern Recognition Letters, v.28 n.3, p.335-341, February, 2007
Simon Lucey , Iain Matthews, Face refinement through a gradient descent alignment approach, Proceedings of the HCSNet workshop on Use of vision in human-computer interaction, p.43-49, November 01-01, 2006, Canberra, Australia
Ofer Melnik , Yehuda Vardi , Cun-Hui Zhang, Concave Learners for Rankboost, The Journal of Machine Learning Research, 8, p.791-812, 5/1/2007
Jianguo Lee , Jingdong Wang , Changshui Zhang , Zhaoqi Bian, Probabilistic tangent subspace: a unified view, Proceedings of the twenty-first international conference on Machine learning, p.67, July 04-08, 2004, Banff, Alberta, Canada
Zhong , Irek Defe, DCT Histogram optimization for image database retrieval, Pattern Recognition Letters, v.26 n.14, p.2272-2281, 15 October 2005
Zhong Jin , Zhen Lou , Jingyu Yang , Quansen Sun, Face detection using template matching and skin-color information, Neurocomputing, v.70 n.4-6, p.794-800, January, 2007
Nikolaos V. Boulgouris , Zhiwei X. Chi, Human gait recognition based on matching of body components, Pattern Recognition, v.40 n.6, p.1763-1770, June, 2007
Wang , Yan Zhang , Jufu Feng, On the Euclidean Distance of Images, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.8, p.1334-1339, August 2005
John Daugman, Statistical Richness of Visual Phase Information: Update on Recognizing Persons by Iris Patterns, International Journal of Computer Vision, v.45 n.1, p.25-38, October 2001
Elaine M. Newton , Latanya Sweeney , Bradley Malin, Preserving Privacy by De-Identifying Face Images, IEEE Transactions on Knowledge and Data Engineering, v.17 n.2, p.232-243, February 2005
Xin Chen , Patrick J. Flynn , Kevin W. Bowyer, IR and visible light face recognition, Computer Vision and Image Understanding, v.99 n.3, p.332-358, September 2005
Simon Lucey , Tsuhan Chen, Integrating monolithic and free-parts representations for improved face verification in the presence of pose mismatch, Pattern Recognition Letters, v.28 n.8, p.895-903, June, 2007
Loris Nanni , Dario Maio, Weighted Sub-Gabor for face recognition, Pattern Recognition Letters, v.28 n.4, p.487-492, March, 2007
Todd A. Stephenson , Tsuhan Chen, Adaptive Markov random fields for example-based super-resolution of faces, EURASIP Journal on Applied Signal Processing, v.2006 n.1, p.225-225, 01 January
Dario Maio , Davide Maltoni , Raffaele Cappelli , J. L. Wayman , Anil K. Jain, FVC2000: Fingerprint Verification Competition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.3, p.402-412, March 2002
Jian-Gang Wang , Hui Kong , Eric Sung , Wei-Yun Yau , Eam Khwang Teoh, Fusion of appearance image and passive stereo depth map for face recognition based on the bilateral 2DLDA, Journal on Image and Video Processing, v.2007 n.2, p.6-6, August 2007
Xiaoxun Zhang , Yunde Jia, A linear discriminant analysis framework based on random subspace for face recognition, Pattern Recognition, v.40 n.9, p.2585-2591, September, 2007
Juwei Lu , K. N. Plataniotis , A. N. Venetsanopoulos, Regularized discriminant analysis for the small sample size problem in face recognition, Pattern Recognition Letters, v.24 n.16, p.3079-3087, December
Rein-Lien Hsu , Anil K. Jain, Generating Discriminating Cartoon Faces Using Interacting Snakes, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.11, p.1388-1398, November
Linlin Shen , Li Bai, MutualBoost learning for selecting Gabor features for face recognition, Pattern Recognition Letters, v.27 n.15, p.1758-1767, November 2006
Nikolaos V. Boulgouris , Konstantinos N. Plataniotis , Dimitrios Hatzinakos, Gait recognition using linear time normalization, Pattern Recognition, v.39 n.5, p.969-979, May, 2006
Linlin Shen , Li Bai, MutualBoost learning for selecting Gabor features for face recognition, Pattern Recognition Letters, v.27 n.15, p.1758-1767, November, 2006
Yong Ma , Shihong Lao , Erina Takikawa , Masato Kawade, Discriminant analysis in correlation similarity measure space, Proceedings of the 24th international conference on Machine learning, p.577-584, June 20-24, 2007, Corvalis, Oregon
Alice J. O'toole , Fang Jiang , Herv Abdi , James V. Haxby, Partially Distributed Representations of Objects and Faces in Ventral Temporal Cortex, Journal of Cognitive Neuroscience, v.17 n.4, p.580-590, April 2005
K. Srinivasa Rao , A. N. Rajagopalan, A probabilistic fusion methodology for face recognition, EURASIP Journal on Applied Signal Processing, v.2005 n.1, p.2772-2787, 1 January 2005
Juwei Lu , K. N. Plataniotis , A. N. Venetsanopoulos, Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition, Pattern Recognition Letters, v.26 n.2, p.181-191, 15 January 2005
Xiao-Yuan Jing , Hau-San Wong , David Zhang, Face recognition based on discriminant fractional Fourier feature extraction, Pattern Recognition Letters, v.27 n.13, p.1465-1471, 1 October 2006
Afzel Noore , Richa Singh , Mayank Vatsa, Robust memory-efficient data level information fusion of multi-modal biometric images, Information Fusion, v.8 n.4, p.337-346, October, 2007
Jaepil Ko , Hyeran Byun, N-division output coding method applied to face recognition, Pattern Recognition Letters, v.24 n.16, p.3115-3123, December
Damon L. Woodard , Patrick J. Flynn, Finger surface as a biometric identifier, Computer Vision and Image Understanding, v.100 n.3, p.357-384, December 2005
Ethan Meyers , Lior Wolf, Using Biologically Inspired Features for Face Processing, International Journal of Computer Vision, v.76 n.1, p.93-104, January   2008
Jian Huang , Pong C. Yuen , J. H. Lai , Chun-hung Li, Face recognition using local and global features, EURASIP Journal on Applied Signal Processing, v.2004 n.1, p.530-541, 1 January 2004
Zhang , Xilin Chen , Chunli Wang , Yiqiang Chen , Wen Gao, Recognition of sign language subwords based on boosted hidden Markov models, Proceedings of the 7th international conference on Multimodal interfaces, October 04-06, 2005, Torento, Italy
Creed F. Jones, III , A. Lynn Abbott, Optimization of color conversion for face recognition, EURASIP Journal on Applied Signal Processing, v.2004 n.1, p.522-529, 1 January 2004
Martin D. Levine , Ajit Rajwade, Three-dimensional view-invariant face recognition using a hierarchical pose-normalization strategy, Machine Vision and Applications, v.17 n.5, p.309-325, September 2006
Ralph Gross , Iain Matthews , Simon Baker, Appearance-Based Face Recognition and Light-Fields, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.4, p.449-465, April 2004
nsen Toygar , Adnan Acan, Overlapping on partitioned facial images, Proceedings of the 6th WSEAS International Conference on Signal, Speech and Image Processing, p.197-202, September 22-24, 2006, Lisbon, Portugal
Chunghoon Kim , Chong-Ho Choi, Image covariance-based subspace method for face recognition, Pattern Recognition, v.40 n.5, p.1592-1604, May, 2007
Peng Wang , Qiang Ji, Multi-view face and eye detection using discriminant features, Computer Vision and Image Understanding, v.105 n.2, p.99-111, February, 2007
Vytautas Perlibakas, Distance measures for PCA-based face recognition, Pattern Recognition Letters, v.25 n.6, p.711-724, 19 April 2004
Aleix M. Martnez , Ming-Hsuan Yang , David J. Kriegman, Special issue on face recognition, Computer Vision and Image Understanding, v.91 n.1-2, p.1-5, July
Liang Wang , Tieniu Tan , Huazhong Ning , Weiming Hu, Silhouette Analysis-Based Gait Recognition for Human Transactions on Pattern Analysis and Machine Intelligence, v.25 n.12, p.1505-1518, December
Taha I. El-Arief , Khaled A. Nagaty , Ahmed S. El-Sayed, Eigenface vs. Spectroface: a comparison on the face recognition problems, Proceedings of the Fourth conference on IASTED International Conference: Signal Processing, Pattern Recognition, and Applications, p.321-327, February 14-16, 2007, Innsbruck, Austria
Ruud M. Bolle , Nalini K. Ratha , Sharath Pankanti, Error analysis of pattern recognition systems: the subsets bootstrap, Computer Vision and Image Understanding, v.93 n.1, p.1-33, January 2004
Yong Zhang , Dmitry B. Goldgof , Sudeep Sarkar , Leonid V. Tsap, A sensitivity analysis method and its application in physics-based nonrigid motion modeling, Image and Vision Computing, v.25 n.3, p.262-273, March, 2007
Patrick Courtney , Neil A. Thacker, Performance characterisation in computer vision: statistics in testing and design, Imaging and vision systems: theory, assessment and applications, Nova Science Publishers, Inc., Commack, NY, 2001
Tommy W. Chow , M. K. Rahman, Face Matching in Large Database by Self-Organizing Maps, Neural Processing Letters, v.23 n.3, p.305-323, June      2006
Chao-Chun Liu , Dao-Qing Dai , Hong Yan, Local Discriminant Wavelet Packet Coordinates for Face Recognition, The Journal of Machine Learning Research, 8, p.1165-1195, 5/1/2007
Dit-Yan Yeung , Hong Chang , Guang Dai, Learning the kernel matrix by maximizing a KFD-based class separability criterion, Pattern Recognition, v.40 n.7, p.2021-2028, July, 2007
Andrew Senior , Rein-Lien Hsu , Mohamed Abdel Mottaleb , Anil K. Jain, Face Detection in Color Images, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.5, p.696-706, May 2002
Mario E. Munich , Pietro Perona, Visual Identification by Signature Tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.2, p.200-217, February
Benjamin J. Balas , Pawan Sinha, Region-based representations for face recognition, ACM Transactions on Applied Perception (TAP), v.3 n.4, p.354-375, October 2006
Song Mao , Tapas Kanungo, Empirical Performance Evaluation Methodology and Its Application to Page Segmentation Algorithms, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.3, p.242-256, March 2001
Outdoor recognition at a distance by fusing gait and face, Image and Vision Computing, v.25 n.6, p.817-832, June, 2007
Guang Dai , Dit-Yan Yeung , Yun-Tao Qian, Face recognition using a kernel fractional-step discriminant analysis algorithm, Pattern Recognition, v.40 n.1, p.229-243, January, 2007
Bruce A. Draper , Kyungim Baek , Marian Stewart Bartlett , J. Ross Beveridge, Recognizing faces with PCA and ICA, Computer Vision and Image Understanding, v.91 n.1-2, p.115-137, July
Sudeep Sarkar , P. Jonathon Phillips , Zongyi Liu , Isidro Robledo Vega , Patrick Grother , Kevin W. Bowyer, The HumanID Gait Challenge Problem: Data Sets, Performance, and Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.2, p.162-177, February 2005
Jian Yang , Alejandro F. Frangi , Jing-yu Yang , David Zhang , Zhong Jin, KPCA Plus LDA: A Complete Kernel Fisher Discriminant Framework for Feature Extraction and Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.2, p.230-244, February 2005
Junmei Zhu , Christoph von der Malsburg, Maplets for correspondence-based object recognition, Neural Networks, v.17 n.8-9, p.1311-1326, October/November 2004
P. Nicholl , A. Amira , D. Bouchaffra , R. H. Perrott, A statistical multiresolution approach for face recognition using structural hidden Markov models, EURASIP Journal on Advances in Signal Processing, v.2008 n.1, p.1-10, January 2008
Richa Singh , Mayank Vatsa , Afzel Noore, Improving verification accuracy by synthesis of locally enhanced biometric images and deformable model, Signal Processing, v.87 n.11, p.2746-2764, November, 2007
Christian Eckes , Jochen Triesch , Christoph von der Malsburg, Analysis of cluttered scenes using an elastic matching approach for stereo images, Neural Computation, v.18 n.6, p.1441-1471, June 2006
Jie Wang , K. N. Plataniotis , Juwei Lu , A. N. Venetsanopoulos, On solving the face recognition problem with one training sample per subject, Pattern Recognition, v.39 n.9, p.1746-1762, September, 2006
Conrad Sanderson , Samy Bengio , Yongsheng Gao, On transforming statistical models for non-frontal face verification, Pattern Recognition, v.39 n.2, p.288-302, February, 2006
Seong G. Kong , Jingu Heo , Faysal Boughorbel , Yue Zheng , Besma R. Abidi , Andreas Koschan , Mingzhong Yi , Mongi A. Abidi, Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition, International Journal of Computer Vision, v.71 n.2, p.215-233, February  2007
Kevin W. Bowyer , Kyong Chang , Patrick Flynn, A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition, Computer Vision and Image Understanding, v.101 n.1, p.1-15, January 2006
Rodrigo de Luis-Garca , Carlos Alberola-Lpez , Otman Aghzout , Juan Ruiz-Alzola, Biometric identification systems, Signal Processing, v.83 n.12, p.2539-2557, December
Xiaoyang Tan , Songcan Chen , Zhi-Hua Zhou , Fuyan Zhang, Face recognition from a single image per person: A survey, Pattern Recognition, v.39 n.9, p.1725-1745, September, 2006
Seong G. Kong , Jingu Heo , Besma R. Abidi , Joonki Paik , Mongi A. Abidi, Recent advances in visual and infrared face recognition: a review, Computer Vision and Image Understanding, v.97 n.1, p.103-135, January 2005
Ming-Hsuan Yang , David J. Kriegman , Narendra Ahuja, Detecting Faces in Images: A Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.1, p.34-58, January 2002
W. Zhao , R. Chellappa , P. J. Phillips , A. Rosenfeld, Face recognition: A literature survey, ACM Computing Surveys (CSUR), v.35 n.4, p.399-458, December
Axel Pinz, Object categorization, Foundations and Trends in Computer Graphics and Vision, v.1 n.4, p.255-353, December 2005
face recognition;algorithm evaluation;FERET database