Efficient Region Tracking With Parametric Models of Geometry and Illumination.
AbstractAs an object moves through the field of view of a camera, the images of the object may change dramatically. This is not simply due to the translation of the object across the image plane. Rather, complications arise due to the fact that the object undergoes changes in pose relative to the viewing camera, changes in illumination relative to light sources, and may even become partially or fully occluded. In this paper, we develop an efficient, general framework for object trackingone which addresses each of these complications. We first develop a computationally efficient method for handling the geometric distortions produced by changes in pose. We then combine geometry and illumination into an algorithm that tracks large image regions using no more computation than would be required to track with no accommodation for illumination changes. Finally, we augment these methods with techniques from robust statistics and treat occluded regions on the object as statistical outliers. Throughout, we present experimental results performed on live video sequences demonstrating the effectiveness and efficiency of our methods.
Introduction
Visual tracking has emerged as an important component of systems in several application
areas including vision-based control [1, 32, 38, 15], human-computer interfaces [10, 14, 20],
surveillance [30, 29, 19], agricultural automation [27, 41], medical imaging [12, 4, 45] and
visual reconstruction [11, 42, 48]. The central challenge in visual tracking is to determine the
image position of a target region (or features) of an object as it moves through a camera's field
of view. This is done by solving what is known as the temporal correspondence problem: the
problem of matching the target region in successive frames of a sequence of images taken at
closely-spaced time intervals. The correspondence problem for visual tracking has, of course,
much in common with the correspondence problems which arise in stereopsis and optical flow.
It differs, however, in that the goal is not to determine the exact correspondence for every
image location in a pair of images, but rather to determine, in a gross sense, the movement of
an entire target region over a long sequence of images. What makes tracking difficult is the
extreme variability often present in the images of an object over time. This variability arises
from three principle sources: variation in object pose, variation in illumination, and partial
or full occlusion of the target. When ignored, any one of these three sources of variability is
enough to cause a tracking algorithm to lose its target.
In this paper, we develop a general framework for region tracking which includes models
for image changes due to motion, illumination, and partial occlusion. In the case of motion,
all points in the target region are presumed to be part of the same object allowing us the
luxury - at least for most applications - of assuming that these points move coherently
in space. This permits us to develop low-order parametric models for the image motion of
points within a target region-models that can be used to predict the movement of the points
and track the target through an image sequence. In the case of illumination, we exploit the
observations of [25, 17, 5] to model image variation due to changing illumination by low-dimensional
linear subspaces. The motion and illumination models are then woven together
in an efficient algorithm which establishes temporal correspondence of the target region
by simultaneously determining motion and illumination parameters. These parameters not
only shift and deform image coordinates, but also adjust brightness values within the target
region to provide the best match to a fixed reference image. Finally, in the case of partial
occlusion, we apply results from robust statistics [16] to show that this matching algorithm is
easily extended to include automatic rejection of outlier pixels in a computationally efficient
manner.
The approach to matching described in this paper is based on comparing the so-called
sum-of-squared differences (SSD) between two regions, an idea that has been explored in
a variety of contexts including stereo matching [35], optical flow computation [2], hand-eye
coordination [38], and visual motion analysis [44]. Much of the previous work using SSD
matching for tracking has modeled the motion of the target region as pure translation in
the image plane [48, 38], which implicitly assumes that the underlying object is translating
parallel to the image plane and is being viewed orthographically. For inter-frame calculations
such as those required for optical flow or motion analysis, pure translation is typically
adequate. However, for tracking applications in which the correspondence for a finite size
image patch must be computed over a long time span, the pure translation assumption is
soon violated [44]. In such cases, both geometric image distortions such as rotation, scaling,
shear, and illumination changes introduce significant changes in the appearance of the target
region and, hence, must be accounted for in order to achieve reliable matching.
Attempts have been made to include more elaborate models for image change in region
tracking algorithms, but with sizable increases in the computational effort required to establish
correspondence. For example, Rehg and Witkin [40] describe energy-based algorithms
for tracking deforming image regions, and Rehg and Kanade [39] consider articulated objects
undergoing self-occlusion. More recently, Black and Yacoob [8] describe an algorithm
for recognizing facial expressions using motion models which include both affine and simple
polynomial deformations of the face and its features. Black and Jepson [7] develop a robust
algorithm for tracking a target undergoing changes in pose or appearance by combining a
simple parametric motion model with an image subspace method [37]. These algorithms
require from several seconds to several minutes per frame to compute, and most do not
address the problems of changes in appearance due to illumination.
In contrast, we develop a mathematical framework for the region tracking problem that
naturally incorporates models for geometric distortions and varying illumination. Using this
framework, we show that the computations needed to perform temporal matching can be
factored to greatly improve algorithm efficiency. The result is a family of region-tracking
algorithms which can easily track large image regions (for example the face of a user at a
workstation, at a using no special hardware other than a standard digitizer.
To date, most tracking algorithms achieving frame-rate performance track only a sparse
collection of features (or contours). For example, Blake et al. [9] and Isaard and Blake [33]
describe a variety of novel methods for incorporating both spatial and temporal constraints
on feature evolution for snake-like contour tracking. Lowe [34] and Gennery [21] describe
edge-based tracking methods using rigid three-dimensional geometric models. Earlier work
by Ayache [3] and Crowley [13] use incrementally constructed rigid models to constrain image
matching.
In practice, feature-based and region-based methods can be viewed as complementary
techniques. In edge-rich environments such as a manufacturing floor, working with sparse
features such as edges has the advantage of computational simplicity - only a small area
of the image contributes to the tracking process, and the operations performed in that
region are usually very simple. Furthermore, edge-based methods use local derivatives and,
hence, tend to be insensitive to global changes in the intensity and/or composition of the
incident illumination. However, in less structured situations strong edges are often sparsely
distributed in an image, and are difficult to detect and match robustly without a strong
predictive model [33]. In such cases, the fact that region-based methods make direct and
complete use of all available image intensity information eliminates the need to identify and
model a special set of features to track. By incorporating illumination models and robust
estimation methods and by making the correspondence algorithm efficient, the robustness
and performance of our region tracking algorithms closely rivals that achieved by edge-based
methods.
The remainder of this article is organized as follows. Section 2 establishes a framework for
posing the problem of region tracking for parametric motion models and describes conditions
under which an efficient tracking algorithm can be developed. Section 3 then shows how
models of illumination can be incorporated with no loss of computational efficiency. Section
4 details modifications for handling partial target occlusion via robust estimation techniques.
Section 5 presents experimental results from an implementation of the algorithms. Finally,
Section 6 presents a short discussion of performance improving extensions to our tracking
algorithm.
Tracking Moving Objects
In this section, we describe a framework for the efficient tracking of a target region through
an image sequence. We first write down a general parametric model for the set of allowable
image motions and deformations of the target region. We then pose the tracking problem as
the problem of finding the best (in a least squares sense) set of parameter values describing
the motions and deformations of the target through the sequence. Finally, we describe how
the best set of parameters can be efficiently computed.
2.1 On Recovering Structured Motion
We first consider the problem of describing the motion of a target region of an object through
a sequence of images. Points on the surface of the object, including those in the target region,
are projected down into the image plane. As the object moves through space, the projected
points move in the image plane. If the 3-D structure of the object is known in advance,
then we could exactly determine the set of possible motions of the points in the images. In
general, this information is not known in advance. Therefore, we approximate the set of
possible motions by a parametric model for image motions.
Let I(x; t) denote the brightness value at the location in an image acquired at
time t and let r x
I(x; t) denote the spatial gradient at that location and time. The symbol
denotes an identified "initial" time and we refer to the image at time t 0 as the reference
image. Let the set be a set of N image locations which define a target
region. We refer to the brightness values of the target region in the reference image as the
reference template.
Over time, the relative motion between the target object and the camera causes the image
of the target to shift and to deform. Let us model the image motion of the target region
of the object by a parametric motion model f(x; -) parameterized by -
with We assume that f is differentiable in both - and x: We call
- the motion parameter vector. We consider recovering the motion parameter vector for
each image in the tracking sequence as the equivalent to "tracking the object." We write
-   (t) to denote the ground truth values of these parameters at time t, and -(t) to denote
the corresponding estimate. The argument t will be suppressed when it is obvious from its
context.
Suppose that a reference template is acquired at time t 0 and that initially -
us assume for now that the only changes in subsequent images of the target
are completely described by f ; i.e. there are no changes in the illumination of the target. It
follows that for any time t ? t 0 there is a parameter vector -   (t) such that
This a generalization of the so-called image constancy assumption [28]. The motion parameter
vector of the target region can be estimated at time t by minimizing the following least
squares objective function
For later developments, it is convenient to rewrite this optimization problem in vector
notation. To this end, let us consider images of the target region as vectors in an N dimensional
space. The image of the target region at time t, under the change of coordinates f
with parameters -; is written as
This vector is subsequently referred to as the rectified image at time t with parameters -:
We also make use of the partial derivatives of I with respect to the components of - and
the time parameter t: These are written as
I - i
I - i
I - i
I - i
and
I t (-;
@t
I
I t (f(x
I t (f(x N ; -); t)
Using this vector notation, the image constancy assumption (1) can be rewritten as
I(-   (t);
and (2) becomes
In general, (6) is a non-convex objective function. Thus, in the absence of a good starting
point, this problem will usually require some type of costly global optimization procedure to
solve [6].
In the case of visual tracking, the continuity of motion provides such a starting point.
Suppose that, at some arbitrary time the geometry of the target region is described
by -(t): We recast the tracking problem as one of determining a vector of offsets, ffi- such
that -(t ffi- from an image acquired at t + -: Incorporating this modification
into (6), we redefine the objective function as a function on ffi-
If the magnitude of the components of ffi- are small, then it is possible to apply continuous
optimization procedures to a linearized version of the problem [7, 28, 35, 47, 44]. The
linearization is carried out by expanding I(- in a Taylor series about - and t;
where h:o:t denotes higher order terms of the expansion, and M is the Jacobian matrix of I
with respect to -; i.e. the N \Theta n matrix of partial derivatives which can be written in column
form as
While the notation above explicitly indicates that the values of the partial derivatives are
a function of the evaluation point (-; t); these arguments will be suppressed when obvious
from their context.
By substituting (8) into (7) and ignoring the higher order terms, we have
With the additional approximation -I t (-;
O(ffi-
Solving the set of equations yields the solution
provided the matrix M is full rank. When this is not the case, we are faced with a generalization
of the aperture problem, i.e. the target region does not have sufficient structure to
determine all of the elements of - uniquely.
In subsequent developments, it will be convenient to define the error vector
Incorporating this definition into (12), we see that the solution of
(6) at time t a solution at time t is
where M is evaluated at (-; t):
2.2 An Efficient Tracking Algorithm
From (13), we see that to track the target region through the image sequence, we must
compute the Jacobian matrix M(-; t): Each element of this matrix is given by
where r f I is the gradient of I with respect to the components of the vector f . Recall that
the Jacobian matrix of the transformation f regarded as a function of - is the 2 \Theta n matrix
By making use of (15), M can be written compactly in row form as
Because M depends on time-varying quantities, it may appear that it must be completely
recomputed at each time step-a computationally expensive procedure involving the calculation
of the image gradient vector, the calculation of a 2 \Theta n Jacobian matrix, and n 2 \Theta 1
vector inner products for each of the N pixels of the target region. However, we now show
that it is possible to reduce this computation by both eliminating the need to recompute
image gradients and by factoring M:
First, we eliminate the need to compute image gradients. To do so, let us assume that
our estimate is exact, i.e. differentiating both sides of (1) we obtain
is the 2 \Theta 2 Jacobian matrix of f treated as a function of
@x
@y
Combining (17) with (16), we see that M can be written as
r x I(x
It follows that for any choice of image deformations, the image spatial gradients need only
be calculated once on the reference template. This is not surprising given that the target at
only a distortion of the target at time t 0 ; and so its image gradients are also
a distortion of those at t 0 : This transformation also allows us to drop the time argument of
M and regard it solely as a function of -:
The remaining non-constant factor in M is a consequence of the fact that, in general, f x
and f - involve components of - and, hence, implicitly vary with time. However, suppose
that we choose f so that f \Gamma1
can be factored into the product of a 2 \Theta k matrix \Gamma which
depends only on image coordinates, and a k \Theta n matrix \Sigma which depends only on - as
For example, as discussed in more detail below, one family of such factorizations results
when f is a linear function of the image coordinate vector x:
Combining (19) with (20), we have
r x
r x I(x
As a result, we have shown that M can be written as a product of an constant N \Theta k matrix
M 0 and a time-varying k \Theta n matrix \Sigma:
We can now exploit this factoring to define an efficient tracking algorithm which operates
as follows:
offline:
ffl Define the target region.
ffl Acquire and store the reference template.
ffl Compute and store M 0
and
online:
ffl Use the most recent motion parameter estimate -(t) to rectify the target region
in the current image.
by taking the difference between the rectified image and the
reference template.
ffl Solve the system \Sigma T  \Sigma
is evaluated at
The online computation performed by this algorithm is quite small, and consists of two n \Theta k
matrix multiplies, k N-vector inner products, n k-vector inner products, and an n \Theta n linear
system solution, where k and n are typically far smaller than N:
We note that the computation can be further reduced if \Sigma is invertible. In this case, the
solution to the linear system can be expressed as
where \Sigma is evaluated at -(t): The factor (M 0
T can be computed
offline, so the online computation is reduced to n N-vector inner products and n n-vector
inner products.
2.3 Some Examples
2.3.1 Linear Models
Let us assume that f(x; -) is linear in x: Then we have
and, hence, f x
It follows that f \Gamma1
x
f - is linear in the components of x and the factoring
defined in (20) applies. We now present three examples illustrating these concepts.
Pure Translation: In the case of pure translation, the allowed image motions are parameterized
by the vector
It follows immediately that f x and f - are both the 2 \Theta 2 identity matrix, and therefore
and \Sigma is the 2 \Theta 2 identity matrix.
The resulting linear system is nonsingular if the image gradients in the template region
are not all collinear, in which case the solution at each time step is just
Note that in this case
constant matrix which can be computed
offline.
Translation, Rotation and Scale: Objects which are viewed under scaled orthography
and which do not undergo out-of-plane rotation can be modeled using a four parameter model
consisting an image-plane rotation through an angle '; a scaling by s; and a translation by
u: The change of coordinates is given by
where R(') is a 2 \Theta 2 rotation matrix. After some minor algebraic manipulations, we obtain
and
From this M 0 can be computed using (21) and, since \Sigma is invertible, the solution to the
linear system becomes
This result can be explained as follows. The matrix M 0
is the linearization of the system
about the target has orientation '(t) and s(t): Image rectification
effectively rotates the target by \Gamma' and scales by 1
s
so the displacements of the target are
computed in the original target coordinate system. \Sigma \GammaT then applies a change of coordinates
to rotate and scale the computed displacements from the original target coordinate system
back to the actual target coordinates.
Affine Motion: The image distortions of planar objects viewed under orthographic projection
are described by a six-parameter linear change of coordinates. Suppose that we
define
a c
b d
After some minor algebraic manipulations, we obtain
and
Note that \Sigma is once again invertible which allows for additional computational savings as
before.
2.3.2 Nonlinear Motion Models
The separability property needed for factoring does not hold for any type of nonlinear motion.
However, consider a motion model of the form
Intuitively, this model performs a quadratic distortion of the image
according to the equation example, a polynomial model of this form was
used in [8] to model the motions of lips and eyebrows on a face. Again, after several algebraic
steps we arrive at
and
Note this general result holds for any distortion which can be expressed exclusively as either
adding more freedom to the motion model, for example
combining affine and polynomial distortion, often makes factoring impossible. One possibility
in such cases is to use a cascaded model in which the image is first rectified using an affine
distortion model, and then the resulting rectified image is further rectified for polynomial
distortion.
2.4 On the Structure of Image Change
The Jacobian matrix M plays a central role in the algorithms described above, so it is informative
to digress briefly on its structure. If we consider the rectified image as a continuous
time-varying quantity, then its total derivative with respect to time is
dI
dt
dt
I t or -
Note that this is simply a differential form of (8). Due to the image constancy assumption
(1), it follows that -
This is, of course, a parameterized version of Horn's
optical flow constraint equation [28].
In this form, it is clear that the role of M is to relate variations in motion parameters
to variations in brightness values in the target region. The solution given in (13) effectively
reverses this relationship and provides a method for interpreting observed changes in brightness
as motion. In this sense, we can think of the algorithm as performing correlation on
temporal changes (as opposed to spatial structure) to compute motion.
To better understand the structure of M; recall that in column form, it can be written
in terms of the partial derivatives of the rectified image:
Thus, the model states that the temporal variation in image brightness in the target region
is a weighted combination of the vectors I - i
: We can think of each of these columns (which
have an entry for every pixel in the target region) as a "motion template" which directly
represents the changes in brightness induced by the motion represented by the corresponding
motion parameter. For example, in the top row of Figure 1, we have shown these templates
for several canonical motions of an image of a black square on a white background. Below,
we show the corresponding templates for a human face.
The development in this section has assumed that we start with a given parametric
motion model from which these templates are derived. Based on that model, the structure
of each entry of M is given by (15) which states that
The image gradient r f I defines, at each point in the image, the direction of strongest
intensity change. The vector f - j
evaluated at x i is the instantaneous direction and magnitude
of motion of that image location captured by the parameter The collection of the latter
for all pixels in the region represents the motion field defined by the motion parameter
Target Image X Translation Y Translation Rotation Scale
Target Image X Translation Y Translation Rotation Scale

Figure

1: Above, the reference template for a bright square on a dark background the motion
template for four canonical motions. Below, the same motion templates for a human face.
Thus, the change in the brightness of the image location x i due to the motion parameter - j
is the projection of the image gradient onto the motion vector.
This suggests how our techniques can be used to perform structured motion estimation
without an explicit parametric motion model. First, if the changes in images due to motion
can be observed directly (for example, by computing the differences of images taken before
and after small reference motions are performed), then these can be used as the motion
templates which comprise M: Second, if a one or more motion fields can be observed (for
example, by tracking a set of fiducial points in a series of training images), then projecting
each element of the motion field onto the corresponding image gradient yields motion templates
for those motion fields. The linear estimation process described above can be used to
time-varying images in terms of those basis motions.
Illumination-Insensitive Tracking
The systems described above are inherently sensitive to changes in illumination of the target
region. This is not surprising, as the incremental estimation step is effectively computing a
structured optical flow, and optical flow methods are well-known to be sensitive to illumination
changes [28]. Thus, shadowing or shading changes of the target object over time lead
to bias, or, in the worst case, complete loss of the target.
Recently, it has been shown that a relatively small number of "basis" images can often
be used to account for large changes in illumination [5, 17, 22, 24, 43]. Briefly, the reason for
this is as follows. Consider a point p on a Lambertian surface and a collimated light source
characterized by a vector s 2 IR 3 , such that the direction of s gives the direction of the light
rays and ksk gives the intensity of the light source. The irradiance at the point p is given by
where n is the unit inwards normal vector to the surface at p and a is the non-negative absorption
coefficient (albedo) of the surface at the point p [28]. This shows that the irradiance
at the point p, and hence the gray level seen by a camera, is linear on s 2 IR 3 .
Therefore, in the absence of self-shadowing, given three images of a Lambertian surface
from the same viewpoint taken under three known, linearly independent light source di-
rections, the albedo and surface normal can be recovered; this is the well-known method
of photometric stereo [50, 46]. Alternatively, one can reconstruct the image of the surface
under a novel lighting direction by a linear combination of the three original images [43]. In
other words, if the surface is purely Lambertian and there is no shadowing, then all images
under varying illumination lie within a 3-D linear subspace of IR N , the space of all possible
images (where N is the number of pixels in the images).
A complication comes when handling shadowing: all images are no longer guaranteed to
lie in a linear subspace [5]. Nevertheless, as done in [24], we can still use a linear model as
an approximation: a small set of basis images can account for much of the shading changes
that occur on patches of non-specular surfaces. Naturally, we need more than three images
(we use between 8 and 15) and a higher than three dimensional linear subspace (we use 4 or
if we hope to provide good approximation to these effects.
Returning to the problem of region tracking, suppose now that we have a basis of image
vectors where the ith element of each of the basis vectors corresponds to
the image location x us choose the first basis vector to be the template image,
To model brightness changes, let us choose the second basis vector to be
a column of ones, i.e. us choose the remaining basis vectors by
In practice, choosing a value close to the mean of the brightness of the image produces a more numerically
performing SVD (singular value decomposition) on a set of training images of the target,
taken under varying illumination. We denote the collection of basis vectors by the matrix
Suppose now that so that the template image and the current target region
are registered geometrically at time t: The remaining difference between them is due to
illumination. From the above discussion, it follows that interframe changes in the current
target region can be approximated by the template image plus a linear combination of the
basis vectors B, i.e.
where the vector - . Note that because the template image and an image
of ones are included in the basis B, we implicitly handle both variation due to contrast
changes and variation due to brightness changes. The remaining basis vectors are used to
handle more subtle variation - variation that depends both on the geometry of the target
object and on the nature of the light sources.
Using the vector-space formulation for motion recovery established in the previous sec-
tion, it is clear that illumination and geometry can be recovered in one global optimization
step solved via linear methods. Incorporating illumination into (7) we have the following
modified optimization:
Substituting (42) into (43) and performing the same simplifications and approximations
as before, we arrive at
Solving
ffi-
In most tracking applications, we are only interested in the motion parameters. We can
eliminate explicit computation of these parameters by first optimizing over - in (44). Upon
stable linear system.
substituting the resulting solution back into (44) and then solving for ffi- we arrive at
Note that if the columns of B are orthogonal vectors, B T B is the identity matrix.
It is easy to show that in both equations, factoring M into time-invariant and time-varying
components as described above leads to significant computational savings. Since
the illumination basis is time-invariant, the dimensionality of the time-varying portion of
the computation depends only on the number of motion fields to be computed, not on the
illumination model. Hence, we have shown how to compute image motion while accounting
for variations in illumination using no more online computation than would be required to
compute pure motion.
Making Tracking Resistant to Occlusion
As a system tracks objects over a large space, it is not uncommon that other objects "intrude"
into the picture. For example, the system may be in the process of tracking a target region
which is the side of a building when, due to observer motion, a parked car begins to occlude
a portion of that region. Similarly the target object may rotate, causing the tracked region
to "slide off" and pick up a portion of the background. Such intrusions will bias the motion
parameter estimates and, in the long term can potentially cause mistracking. In this section,
we describe how to avoid such problems. For the sake of simplicity, we develop a solution for
the case where we are only recovering motion parameters; the modifications for combined
motion and illumination models are straightforward.
A common approach to this problem is to assume that occlusions create large image
differences which can be viewed as "outliers" by the estimation process [7]. The error metric
is then modified to reduce sensitivity to "outliers" by solving a robust optimization problem
of the form
where ae is one of a variety of "robust" regression metrics [31].
It is well-known that optimization of (47) is closely related to another approach to robust
estimation-iteratively reweighted least squares (IRLS). We have chosen to implement the
optimization using a somewhat unusual form of IRLS due to Dutter and Huber [16]. In
order to formulate the algorithm, we introduce the notation of an "inner iteration" which
is performed one or more times at each time step. We will use a superscript to denote this
iteration.
Let ffi- i denote the value of ffi- computed by the ith inner iteration with ffi-
the vector of residuals in the ith iteration r i as
We introduce a diagonal weighting matrix W which has entries
The inner iteration cycle at time t + - is consists of performing an estimation step by
solving the linear system
where \Sigma is evaluated at -(t)) and r i and W i are given by (48) and (49), respectively. This
process is repeated for k iterations.
This form of IRLS is particularly efficient for our problem. It does not require recomputation
of   or \Sigma and, since the weighting matrix is diagonal, does not add significantly to
the overall computation time needed to solve the linear system. In addition, the error vector
e is fixed over all inner iterations, so these iterations do not require the additional overhead
of acquiring and warping images.
As discussed in [16], on linear problems this procedure is guaranteed to converge to a
unique global minimum for a large variety of choices of ae: In this article, ae is taken to be a
so-called "windsorizing" function [31] which is of the form
where r is normalized to have unit variance. The parameter - is a user-defined threshold
which places a limit on the variations of the residuals before they are considered outliers.
This function has the advantage of guaranteeing global convergence of the IRLS method
while being cheap to compute. The updating function for matrix entries is
As stated, the weighting matrix is computed anew at each iteration, a process which can
require several inner iterations. However, given that tracking is a continuous process, it is
natural to start with an initial weighting matrix that is closely related to that computed at
the end of the previous estimation step. In doing so, two issues arise. First, the fact that
the linear system we are solving is a local linearization of a nonlinear system means that, in
cases when inter-frame motion is large, the effect of higher-order terms of the Taylor series
expansion will cause areas of the image to masquerade as outliers. Second, if we assume that
areas of the image with low weights correspond to intruders, it makes sense to add a "buffer
zone" around those areas for the next iteration to pro-actively cancel the effects of intruder
motion.
Both of these problems can be dealt with by noting that the diagonal elements of W
themselves form an image where "dark areas" (those locations with low value ) are areas of
occlusion or intrusion, while "bright areas" (those with value 1) are the expected target. Let
Q(x) to be the pixel values in the eight-neighborhood of the image coordinate x plus the
value at x itself. We use two common morphological operators [26]
and
When applied to a weighting matrix image, close has the effect of removing small areas of
outlier pixels, while open increases their size. Between frames of the sequence we propagate
the weighting matrix forward after applying one step of close to remove small areas of outliers
followed by two or three steps of open to buffer detected intruders.
5 Implementation and Experiments
This section illustrates the performance of the tracking algorithm under a variety of circum-
stances, noting particularly the effects of image warping, illumination compensation, and
outlier detection. All experiments were performed on live video sequences by an SGI Indy
equipped with a 175Mhz R4400 SC processor and VINO image acquisition system.
Rotation Scale Aspect Ratio Shear

Figure

2: The columns of the motion Jacobian matrix for the planar target and their geometric
interpretations.
5.1 Implementation
We have implemented the methods described above within the X Vision environment [23].
The implemented system incorporates all of the linear motion models described in Section
2, non-orthonormal illumination bases as described in Section 3, and outlier rejection using
the algorithm described in Section 4.
The image warping required to support the algorithm is implemented by factoring linear
transformations into a rotation matrix and a positive-definite upper-diagonal matrix. This
factoring allows image warping to be implemented in two stages. In the first stage, an image
region surrounding the target is acquired and rotated using a variant on standard Bresenham
line-drawing algorithms [18]. The acquired image is then scaled and sheared using a bilinear
interpolation. The resolution of the region is then reduced by averaging neighboring pixels.
Spatial and temporal derivatives are computed by applying Prewitt operators on the reduced
scale images. More details on this level of the implementation can be found in [23].
Timings of the algorithm 2 indicate that it can perform frame rate (30 Hz) tracking of
image regions of up to 100 \Theta 100 pixels at one-half resolution undergoing affine distortions
and illumination changes. Similar performance has been achieved on a 120Mhz Pentium
processor and 70 Mhz Sun SparcStation. Higher performance is achieved for smaller regions,
lower resolutions, or fewer parameters. For example, tracking the same size region while
computing just translation at one-fourth resolution takes just 4 milliseconds per cycle.
5.2 Planar Tracking
As a baseline, we first consider tracking a non-specular planar object-the cover of a book.
Affine warping augmented with brightness and contrast compensation is the best possible
linear approximation to this case (it is exact for an orthographic camera model and purely
Lambertian surface). As a point of comparison, recent work by Black and Jepson [7] used
the rigid motion plus scaling model for SSD-based region tracking. Their reduced model is
more efficient and may be more stable since fewer parameters must be computed, but it does
ignore the effects of changing aspect ratio and shear.
We tested both the rigid motion plus scale (RM+S) and full affine motion models on
the same live video sequence of the book cover in motion. Figure 2 shows the set of motion
templates (the columns of the motion matrix) for an 81 \Theta 72 region of a book cover tracked
at one third resolution. Figure 3 shows the results of tracking. The upper series of images
shows several images of the object with the region tracked indicated with a black frame (the
RM+S algorithm) and a white frame (the FA algorithm). The middle row of images shows
the output of the warping operator from the RM+S algorithm. If the computed parameters
were error-free, these images would be identical. However, because of the inability to correct
for aspect ratio and skew, the best fit leads to a skewed image. The bottom row shows
the output of the warping operator for the FA algorithm. Here we see that the full affine
warping is much better at accommodating the full range of image distortions. The graph at
the bottom of the figure shows the least squares residual (in squared gray-values per pixel).
Here, the difference between the two geometric models is clearly evident.
5.3 Human Face Tracking
There has been a great deal of recent interest in face tracking in the computer vision literature
[8, 14, 36]. Although faces can produce images with significant variation due to
empirical results suggest that a small number of basis images of a face gathered
under different illuminations is sufficient to accurately account for most gross shading
and illumination effects [24]. At the same time, the depth variations exhibited by facial features
are small enough to be well-approximated by an affine warping model. The following
2 Because of additional data collection overhead, the tracking performance in the experiments presented
here is slower than the stated figures.
Frame 0 Frame 50 Frame 70 Frame 120 Frame 150 Frame 230
Residuals: Planar Test
FA
RM+S
Gray values
Frames10.0020.0030.0040.00

Figure

3: Top, several images of a planar region and the corresponding warped image computed
by a tracker computing position, orientation and scale (RM+S), and one computing
a full affine deformation (FA). The image at the left is the initial reference image. Bottom,
the graph of the SSD residuals for both algorithms.
experiments demonstrate the ability of our algorithm to track a face as it undergoes changes
in pose and illumination, and under partial occlusion. Throughout, we assume the subject is
roughly looking toward the camera, so we use the rigid motion plus scaling (RM+S) motion
model.

Figure

1 on page 14 shows the columns of the motion matrix for this model.
5.3.1 Geometry
We first performed a test to determine the accuracy of the computed motion parameters
for the face and to investigate the effect of the illumination basis on the sensitivity of those
estimates. During this test, we simultaneously executed two tracking algorithms: one using
the rigid motion plus scale model (RM+S) and one which additionally included an illumination
model for the face (RM+S+I). The algorithms were executed on a sequence which
did not contain large changes in the illumination of the target. The top row of Figure 4
shows images excerpted from the video sequence. In each image, the black frames denote
the region selected as the best match by RM+S and the white frames correspond to the
best match computed by RM+S+I. For this test, we would expect both algorithms to be
quite accurate and to exhibit similar performance unless the illumination basis significantly
affected the sensitivity of the computation. As is apparent from the figures, the computed
motion parameters of both algorithms are extremely similar for the entire run - so close
that in many cases one frame is obscured by the other.
In order to demonstrate the absolute accuracy of the tracking solution, below each live
image in Figure 4 we have included the corresponding rectified image computed by RM+S+I.
The rectified image at time 0 is the reference template. If the motion of the target fit the
RM+S motion model, and the computed parameters were exact, then we would expect each
subsequent rectified image to be identical to the reference template. Despite the fact that
the face is non-planar and we are using a reduced motion model, we see that the algorithm
is quite effective at computing an accurate geometric match.
Finally, the graph in Figure 4 shows the residuals of the linearized SSD computation at
each time step. As is apparent from the figures, the residuals of both algorithms are also
extremely similar for the entire run. From this experiment we conclude that, in the absence
of illumination changes, the performance of both algorithms is quite similar - including
illumination models does not appear to reduce accuracy.
Frame 0 Frame
Residuals: Face with No Lighting Changes
RM+S+I
RM+S
Gray values
Frames20.00

Figure

4: Top row, excerpts from a sequence of tracked images of a face. The black frames
represent the region tracked by an SSD algorithm using no illumination model (RM+S) and
the white frames represent the regions tracked by an algorithm which includes an illumination
model (RM+S+I). In some cases the estimates are so close that only one box is visible.
Middle row, the region within the frame warped by the current motion estimate. Bottom
row, the residuals of the algorithms expressed in gray-scale units per pixel as a function of
time.

Figure

5: The illumination basis for the face (B). The left two images are included to
compensate for brightness and contrast, respectively, while the remaining four images compensate
for changes in lighting direction.
5.3.2 Illumination
In a second set of experiments, we kept the face nearly motionless and varied the illumination.
We used an illumination basis of four orthogonal image vectors. This basis was computed
offline by acquiring ten images of the face under various lighting conditions. A singular value
decomposition (SVD) was applied to the resulting image vectors and the vectors with the
maximum singular values were chosen to be included in the basis. The illumination basis is
shown in Figure 5.

Figure

6 shows the effects of illumination compensation for the illumination situations
depicted in the first row. As with warping, if the compensation were perfect, the images of
the bottom row would appear to be identical up to brightness and contrast. In particular,
note how the strong shading effects of frames 70 through 150 have been "corrected" by the
illumination basis.
5.3.3 Combining Illumination and Geometry
Next, we present a set of experiments illustrating the interaction of geometry and illumi-
nation. In these experiments we again executed two algorithms again labeled RM+S and
RM+S+I. As the algorithms were operating, a light was periodically switched on and off
and the face moved slightly. The results appear in Figure 7. In the residual graph, we see
that the illumination basis clearly "accounts" for the shading on the face quite well, leading
to a much lower fluctuation of the residuals. The sequence of images shows an excerpt near
the middle of the sequence where the RM+S algorithm (which could not compensate for il-
0Figure

The first row of images shows excerpts of a tracking sequence. The second row is
a magnified view of the region in the white frame. The third row contains the images in the
second row after adjustment for illumination using the illumination basis shown in Figure 5
(for sake of comparison we have not adjusted for brightness and contrast).
Frame 90 Frame 100 Frame 110 Frame 120 Frame 130 Frame 140 Frame 150
Residuals: Face with Illumination Changes
RM+S+I
RM+S
Gray values
Frames20.000.00 50.00 100.00 150.00 200.00 250.00 300.00

Figure

7: Top, an excerpt from a tracking sequence containing changes in both geometry and
illumination. The black frame corresponds to the algorithm without illumination (RM+S)
and the write frame corresponds to the algorithm with an illumination basis (RM+S+I).
Note that the algorithm which does not use illumination completely looses the target until
the original lighting is restored. Bottom, the residuals, in gray scale units per pixel, of the
two algorithms as a light is turned on and off.
lumination changes) completely lost the target for several frames, only regaining it after the
original lighting was restored. Since the target was effectively motionless during this period,
this can be completely attributed to biases due to illumination effects. Similar sequences
with larger target motions often cause the purely geometric algorithm to loose the target
completely.
5.3.4 Tracking With Outliers
Finally, we illustrate the performance of the method when the image of the target becomes
partially occluded. We again track a face. The motion and illumination basis are the same
as before. In the weighting matrix calculations the pixel variance was set to 5 and the outlier
threshold was set to 5 variance units.
The sequence is an "office" sequence which includes several "intrusions" including the
background, a piece of paper, a telephone, a soda can, and a hand. As before we executed two
versions of the tracker, the non-robust algorithm from the previous experiment (RM+S+I)
and a robust version (RM+S+I+O). Figure 8 shows the results. The upper series of images
shows the region acquired by both algorithms (the black frame corresponds to RM+S+I, the
white to RM+S+I+O). As is clear from the sequence, the non-robust algorithm is disturbed
significantly by the occlusion, whereas the robust algorithm is much more stable. In fact, a
slight motion of the head while the soda can is in the image caused the non-robust algorithm
to mistrack completely. The middle series of images shows the output of the warping operation
for the robust algorithm. The lower row of images depicts the weighting values attached
to each pixel in the warped image. Dark areas correspond to "outliers." Note that, although
the occluded region is clearly identified by the algorithm, there are some small regions away
from the occlusion which received a slightly reduced weight. This is due to the fact that
the robust metric used introduces some small bias into the computed parameters. In areas
where the spatial gradient is large (e.g. near the eyes and mouth), this introduces some false
rejection of pixels.
It is also important to note that the dynamical performance of the tracker is significantly
reduced by including outliers. Large, fast motions tend to cause the algorithm to "turn
off" areas of the image where there are large gradients, slowing convergence. At the same
time, performing outlier rejection is more computationally intensive as it requires explicit
computation of both the motion and illumination parameter to calculate the residual values.
6 Discussion and Conclusions
We have shown a straightforward and efficient solution to the problem of tracking regions
undergoing geometric distortion, changing illumination, and partial occlusion. The method
is simple, yet robust, and it builds on an already popular method for solving spatial and
temporal correspondence problems.
Although the focus in this article has been on parameter estimation techniques for tracking
using image rectification, the same estimation methods can be used for directly controlling
devices. For example, instead of computing a parameter estimate -; the incremental solutions
ffi- can be used to control the position and orientation of a camera so to stabilize the
target image by active motion. Hybrid combinations of camera control and image warping
are also possible.
Frame 0 Frame
Residuals: Face with Partial Occlusion
RM+S+I+O
RM+S+I
gray values

Figure

8: The first row of images shows excerpts of a tracking sequence with occurrences
of partial occlusion. The black frame corresponds to the algorithm without outlier rejection
(RM+S+I) and the write frame corresponds to the algorithm with outlier rejection
(RM+S+I+O). The second row is a magnified view of the region in the white frame. The
third row contains the corresponding outlier images where darker areas mark outliers. The
graph at the bottom compares the residual values for both algorithms.
One possible objection to the methods is the requirement that the change from frame
to frame is small, limiting the speed at which objects can move. Luckily, there are several
means for improving the dynamical performance of the algorithms. One possibility is to
include a model for the motion of the underlying object and to incorporate prediction into
the tracking algorithm. Likewise, if a model of the noise characteristics of images is available,
the updating method can modified to incorporate this model. In fact, the linear form of the
solution makes it straightforward to incorporate the estimation algorithm into a Kalman
filter or similar iterative estimation procedure.
Performance can also be improved by operating the tracking algorithm at multiple levels
of resolution. One possibility, as is used by many authors [7, 44], is to perform a complete
coarse to fine progression of estimation steps on each image in the sequence. Another possi-
bility, which we have used successfully in prior work [23], is to dynamically adapt resolution
based on the motion of the target. That is, when the target moves quickly estimation is performed
at a coarse resolution, and when it moves slowly the algorithm changes to a higher
resolution. The advantage of this approach is that it not only increases the range over which
the linearized problem is valid, but it also reduces the computation time required on each
image when motion is fast.
We are actively continuing to evaluate the performance of these methods, and to extend
their theoretical underpinnings. One area that still needs attention is the problem of determining
an illumination basis online, i.e. while tracking the object. Initial experiments in this
direction have shown that online determination of the illumination basis can be achieved,
although we have not included such results in this paper. As in [7], we are also exploring
the use of basis images to handle changes of view or aspect not well addressed by warping.
We are also looking at the problem of extending the method to utilize shape information
on the target when such information is available. In particular, it is well known [49] that
under orthographic projection, the image deformations of a surface due to motion can be
described with a linear motion model. This suggests that our methods can be extended
to handle such models. Furthermore, as with the illumination basis, it may be possible
to estimate the deformation models online, thereby making it possible to efficiently track
arbitrary objects under changes in illumination, pose, and partial occlusion.

Acknowledgments

This research was supported by ARPA grant N00014-93-1-1235, Army DURIP grant DAAH04-95-
1-0058, National Science Foundation grant IRI-9420982, Army Research Office grant DAAH04-95-
1-0494, and by funds provided by Yale University. The second author would like to thank David
Mumford, Alan Yuille, David Kriegman, and Peter Hallinan for discussions contributing the ideas
in this paper.



--R


A computational framework and an algorithm for the measurement of structure from motion.
Maintaining representations of the environment of a mobile robot.
Tracking medical 3D data with a deformable parametric model.
What is the set of images of an object under all possible lighting conditions.
Fast object recognition in noisy images using simulated annealing.
Robust matching and tracking of articulated objects using a view-based representation
Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion
A framework for spatio-temporal control in the tracking of visual contour
A state-based technique for the summarization of recognition of gesture
Object models from contour sequences.

Measurement and integration of 3-D structures by tracking edge lines
Active face tracking and pose estimation in an interactive room.
Dynamic monocular machine vision.
Numerical methods for the nonlinear robust regression problem.

Computer Graphics.
Tracking of occluded vehicles in traffic scenes.
Tracking humans in action: A 3D model-based approach
Visual tracking of known three-dimensional objects

A portable substrate for real-time vision applications

A Deformable Model for Face Recognition Under Arbitrary Lighting Condi- tions
Computer and Robot Vision.
A fruit-tracking system for robotic harvesting
Computer Vision.
Visual surveillance monitoring and watching.
Using stereo vision to pursue moving agents with a mobile robot.
Robust Statistics.
A tutorial introduction to visual servo control.
Contour tracking by stochastic propagation of conditional density.
Robust model-based motion tracking through the integration of search and estimation
An iterative image registration technique with an application to stereo vision.
Face tracking and pose representation.
Visual learning and recognition of 3-D objects from appearence
Visual tracking of a moving target by a camera mounted on a robot: A combination of control and vision.
Visual tracking of high DOF articulated structures: An application to human hand tracking.
Visual tracking with deformation models.
Learning dynamics of complex motions from image sequences.
Affine Analysis of Image Sequences.
Geometry and Photometry in 3D Visual Recognition.
Good features to track.
A model-based integrated approach to track myocardial deformation using displacement and velocity constraints
Determining Shape and Reflectance Using Multiple Images.
Image mosaicing for tele-reality applications
Shape and motion from image streams under orthography: A factorization method.
Recognition by a linear combination of models.
Analysing images of curved surfaces.
--TR

--CTR
Guopu Zhu , Qingshuang Zeng , Changhong Wang, Rapid and brief communication: Efficient edge-based object tracking, Pattern Recognition, v.39 n.11, p.2223-2226, November, 2006
Qinghong Guo , Mrinal K. Mandal , Micheal Y. Li, Efficient Hodge-Helmholtz decomposition of motion fields, Pattern Recognition Letters, v.26 n.4, p.493-501, March 2005
Simon Lucey , Iain Matthews, Face refinement through a gradient descent alignment approach, Proceedings of the HCSNet workshop on Use of vision in human-computer interaction, p.43-49, November 01-01, 2006, Canberra, Australia
Kuang-Chih Lee , Jeffrey Ho , Ming-Hsuan Yang , David Kriegman, Visual tracking and recognition using probabilistic appearance manifolds, Computer Vision and Image Understanding, v.99 n.3, p.303-331, September 2005
Alastair Reid , John Peterson , Greg Hager , Paul Hudak, Prototyping real-time vision systems: an experiment in DSL design, Proceedings of the 21st international conference on Software engineering, p.484-493, May 16-22, 1999, Los Angeles, California, United States
Iain Matthews , Takahiro Ishikawa , Simon Baker, The Template Update Problem, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.6, p.810-815, June 2004
Frdric Jurie , Michel Dhome, Hyperplane Approximation for Template Matching, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.7, p.996-1000, July 2002
David Hasler , Luciano Sbaiz , Sabine Ssstrunk , Martin Vetterli, Outlier Modeling in Image Matching, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.3, p.301-315, March
Timothy F. Cootes , Gareth J. Edwards , Christopher J. Taylor, Active Appearance Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.6, p.681-685, June 2001
A. Del Bue , F. Smeraldi , L. Agapito, Non-rigid structure from motion using ranklet-based tracking and non-linear optimization, Image and Vision Computing, v.25 n.3, p.297-310, March, 2007
Jaime Ortegn-Aguilar , Eduardo Bayro-Corrochano, Lie Algebra and System Identification Techniques for 3D Rigid Motion Estimation and Monocular Tracking, Journal of Mathematical Imaging and Vision, v.25 n.2, p.173-185, September 2006
Louis-Philippe Morency , Trevor Darrell, From conversational tooltips to grounded discourse: head poseTracking in interactive dialog systems, Proceedings of the 6th international conference on Multimodal interfaces, October 13-15, 2004, State College, PA, USA
Masao Shimizu , Masatoshi Okutomi, Multi-Parameter Simultaneous Estimation on Area-Based Matching, International Journal of Computer Vision, v.67 n.3, p.327-342, May       2006
Oliver Williams , Andrew Blake , Roberto Cipolla, Sparse Bayesian Learning for Efficient Visual Tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.8, p.1292-1304, August 2005
David Schreiber, Robust template tracking with drift correction, Pattern Recognition Letters, v.28 n.12, p.1483-1491, September, 2007
Eduardo Bayro-Corrochano , Jaime Ortegn-Aguilar, Lie algebra approach for tracking and 3D motion estimation using monocular vision, Image and Vision Computing, v.25 n.6, p.907-921, June, 2007
Simon Baker , Iain Matthews, Lucas-Kanade 20 Years On: A Unifying Framework, International Journal of Computer Vision, v.56 n.3, p.221-255, February-March 2004
Iain Matthews , Jing Xiao , Simon Baker, 2D vs. 3D Deformable Face Models: Representational Power, Construction, and Real-Time Fitting, International Journal of Computer Vision, v.75 n.1, p.93-113, October   2007
Stephen Benoit , Frank P. Ferrie, Towards direct recovery of shape and motion parameters from image sequences, Computer Vision and Image Understanding, v.105 n.2, p.145-165, February, 2007
Jie Wei , Izidor Gertner, MRF-MAP-MFT visual object segmentation based on motion boundary field, Pattern Recognition Letters, v.24 n.16, p.3125-3139, December
Charles S. Wiles , Atsuto Maki , Natsuko Matsuda, Hyperpatches for 3D Model Acquisition and Tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.12, p.1391-1403, December 2001
S. Benhimane , E. Malis, Homography-based 2D Visual Tracking and Servoing, International Journal of Robotics Research, v.26 n.7, p.661-676, July      2007
Horst W. Haussecker , David J. Fleet, Computing Optical Flow with Physical Models of Brightness Variation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.6, p.661-673, June 2001
Hayman , Torfi Thrhallsson , David Murray, Tracking While Zooming Using Affine Transfer and Multifocal Tensors, International Journal of Computer Vision, v.51 n.1, p.37-62, January
Luca Vacchetti , Vincent Lepetit , Pascal Fua, Stable Real-Time 3D Tracking Using Online and Offline Information, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.10, p.1385-1391, October 2004
Marco La Cascia , Stan Sclaroff , Vassilis Athitsos, Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.4, p.322-336, April 2000
Christopher Rasmussen , Gregory D. Hager, Probabilistic Data Association Methods for Tracking Complex Visual Objects, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.6, p.560-576, June 2001
King Yuen Wong , Minas E. Spetsakis, Tracking based motion segmentation under relaxed statistical assumptions, Computer Vision and Image Understanding, v.101 n.1, p.45-64, January 2006
Iain Matthews , Simon Baker, Active Appearance Models Revisited, International Journal of Computer Vision, v.60 n.2, p.135-164, November 2004
Rmi Coudarcher , Florent Duculty , Jocelyn Serot , Frdric Jurie , Jean-Pierre Derutin , Michel Dhome, Managing algorithmic skeleton nesting requirements in realistic image processing applications: the case of the SKiPPER-II parallel programming environment's operating model, EURASIP Journal on Applied Signal Processing, v.2005 n.1, p.1005-1023, 1 January 2005
Gang Hua , Ying Wu, Sequential mean field variational analysis of structured deformable shapes, Computer Vision and Image Understanding, v.101 n.2, p.87-99, February 2006
Peter N. Belhumeur , James S. Duncan , Gregory D. Hager , Drew V. Mcdermott , A. Stephen Morse , Steven W. Zucker, Computational Vision at Yale, International Journal of Computer Vision, v.35 n.1, p.5-12, Nov. 1999
Kentaro Toyama , Gregory D. Hager, Incremental Focus of Attention for Robust Vision-Based Tracking, International Journal of Computer Vision, v.35 n.1, p.45-63, Nov. 1999
Markus Quaritsch , Markus Kreuzthaler , Bernhard Rinner , Horst Bischof , Bernhard Strobl, Autonomous multicamera tracking on embedded smart cameras, EURASIP Journal on Embedded Systems, v.2007 n.1, p.35-35, January 2007
Muriel Pressigout , Eric Marchand, Real-time Hybrid Tracking using Edge and Texture Information, International Journal of Robotics Research, v.26 n.7, p.689-713, July      2007
Stan Sclaroff , John Isidoro, Active blobs: region-based, deformable appearance models, Computer Vision and Image Understanding, v.89 n.2-3, p.197-225, February
Pedro M. Q. Aguiar , Jos M. F. Moura, Rank 1 Weighted Factorization for 3D Structure Recovery: Algorithms and Performance Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.9, p.1134-1049, September
David J. Fleet , Michael J. Black , Oscar Nestares, Bayesian inference of visual motion boundaries, Exploring artificial intelligence in the new millennium, Morgan Kaufmann Publishers Inc., San Francisco, CA,
Fernando De la Torre , Michael J. Black, Robust parameterized component analysis: theory and applications to 2D facial appearance models, Computer Vision and Image Understanding, v.91 n.1-2, p.53-71, July
V. Javier Traver , Filiberto Pla, Similarity motion estimation and active tracking through spatial-domain projections on log-polar images, Computer Vision and Image Understanding, v.97 n.2, p.209-241, February 2005
Jos M. Buenaposada , Luis Baumela, A computer vision based human-robot interface, Autonomous robotic systems: soft computing and hard computing methodologies and applications, Physica-Verlag GmbH, Heidelberg, Germany,
W. Zhao , R. Chellappa , P. J. Phillips , A. Rosenfeld, Face recognition: A literature survey, ACM Computing Surveys (CSUR), v.35 n.4, p.399-458, December
Richard Szeliski, Image alignment and stitching: a tutorial, Foundations and Trends in Computer Graphics and Vision, v.2 n.1, p.1-104, January 2006
Vincent Lepetit , Pascal Fua, Monocular model-based 3D tracking of rigid objects, Foundations and Trends in Computer Graphics and Vision, v.1 n.1, p.1-89, September 2006
motion estimation;real-time vision;visual tracking;illumination;robust statistics