On the Fourier Properties of Discontinuous Motion.
Retinal image motion and optical flow as its approximation are fundamental concepts in the field of vision, perceptual and computational. However, the computation of optical flow remains a challenging problem as image motion includes discontinuities and multiple values mostly due to scene geometry, surface translucency and various photometric effects such as reflectance. In this contribution, we analyze image motion in the frequency space with respect to motion discontinuities and translucence. We derive the frequency structure of motion discontinuities due to occlusion and we demonstrate its various geometrical properties. The aperture problem is investigated and we show that the information content of an occlusion almost always disambiguates the velocity of an occluding signal suffering from the aperture problem. In addition, the theoretical framework can describe the exact frequency structure of Non-Fourier motion and bridges the gap between Non-Fourier visual phenomena and their understanding in the frequency domain.
Introduction
A fundamental problem in processing sequences of
images is the computation of optical
ow, an approximation
to image motion dened as the projection
of velocities of 3D surface points onto the
imaging plane of a visual sensor. The importance
of motion in visual processing cannot be under-
stated: in particular, approximations to image
motion may be used to estimate 3D scene properties
and motion parameters from a moving visual
sensor [21, 30, 31, 42, 51, 50, 1, 5, 38, 22, 54, 56,
34, 20, 16, 23], to perform motion segmentation
[7, 40, 45, 36, 47, 14, 25, 8, 2, 46, 15], to compute
the focus of expansion and time-to-collision [44,
41, 48, 24, 49, 9], to perform motion-compensated
image encoding [10, 13, 35, 37, 39, 55], to compute
stereo disparity [3, 12, 26, 28], to measure blood
ow and heart-wall motion in medical imagery
[43], and, recently, to measure minute amounts
of growth in corn seedlings [6, 29].
1.1. Organization of Paper
This contribution addresses the problem of multiple
image motions arising from occlusion and
translucency phenomena. We present a theoretical
framework for discontinuous optical
ow in
the Fourier domain. The concept of image velocity
as a geometric function is described in Section
1.
S. S. Beauchemin and J. L. Barron
Section 2 is an analysis of occlusion in Fourier
space with a constant model of velocity. Our
approach focuses on the frequency structure of
occluding surfaces and the theoretical results are
constructed incrementally. For instance, a simple
model of velocity is used to develop the structure
of occlusion with sinusoidal signals which are
then generalized to arbitrary signals. These theoretical
results demonstrate that occlusion may be
dierentiated from translucency and the motions
associated with both the occluding and occluded
surfaces can be discriminated.
Section 3 is an investigation of the aperture
problem and degenerate 1 signals, as they appear
in the theoretical framework. For example, it is
shown that the full velocity of a degenerate signal
is almost always computable at the occlusion.
Section 4 is a study of related issues such
as translucency phenomena, Non-Fourier motion,
generalized occlusion boundaries and phase shifts.
Numerical experiments supporting the framework
are presented. Results obtained with sets of sinusoidal
signals created synthetically are compared
with their corresponding theoretical predictions.
Section 5 summarizes our results.
1.2. Contribution
The motivation for the theoretical framework emanates
from the observation that occlusion and
translucency in the context of computing optical
ow constitute di-cult challenges and threatens
its precise computation. The theoretical results
cast light on the exact structure of occlusion and
translucency in the frequency domain.
The results are essentially theoretical and stated
in the form of Theorems and Corollaries. Relevant
numerical experiments which support the
theoretical results are presented. In addition, this
contribution bridges what is seen as an important
gap between Non-Fourier models of visual stimuli
and optical
ow methods in Computer Vision.
In fact, Non-Fourier visual stimuli, to which belong
translucency and occlusion eects, have been
studied mainly with respect to the motion percept
these stimuli elicit among human subjects
[11, 52, 53]. However, more recently, it has been
conjectured that a viable computational analysis
of Non-Fourier motion could be carried out with
Fourier analysis, since many Non-Fourier stimuli
turn out to have simple frequency characterizations
[19]. The results presented herein extend the
concept of Non-Fourier stimuli such as occlusion
and translucency from being not at all explained
by its Fourier characteristics to the establishment
of exact frequency models of visual stimuli exhibiting
occlusions and translucencies.
As a rst attempt to understand occlusion, the
simplest set of controllable parameters were used,
such as the structure of occlusion boundaries and
the number of distinct frequencies for representing
the occluding and occluded surfaces. A constant
model of velocity was also used and no signal deformations
(such as those created by perspective
projection) were permitted. These preliminary results
are extended to image signals composed of an
arbitrary number of discrete frequencies. Dirichlet
conditions are hypothesized for each signal thus
allowing to expand them as complex exponential
series.
The potential use of the information-content
of an occlusion boundary is outlined. Occluding
boundaries contain a wealth of information that is
not exploited by conventional optical
ow frame-
works, due to a theoretical void. It is shown that
a degenerate occluding signal exhibiting a linear
spectrum is supplemented by the linear orientation
of its occluding boundary. These two spectra
almost always yield the full velocity of an occluding
signal suering from the aperture problem.
The structure of occlusion when both signals are
degenerate is also shown. It is demonstrated that
this particular case collapses to a one-dimensional
structure.
The Corollaries show that additive translucency
phenomena may be understood as a special case of
the theoretical framework. In addition, the velocities
associated with both the occluding and occluded
signals may be identied as such, without
the need of scenic information such as depth.
1.3. Image Motion
Image motion is expressed in terms of the 3D
motion parameters of the visual sensor and the
3D environmental points of the scene: let P
(X; Y; Z) be an environmental
point,
x
y
z be the visu-
On the Fourier Properties of Discontinuous Visual Motion 3
Y
Z
y
x
Image plane
Environmental surface
z
y
y
x
x
Fig. 1. The geometry of the visual
sensor.
x
y
z are the instantaneous rotation and
translation of the visual sensor. p(t) is the perspective projection of P(t) onto the imaging surface.
al sensor's respective instantaneous rate of change
in rotation and translation and
P^ z the perspective
projection of P onto the imaging surface
(the focal length of the sensor is assumed to be 1),
z is a normalized vector along the line-of-
sight axis Z. The setup is shown in Figure 1. The
instantaneous 3D velocity of P is given by
P: (1)
The relationship between the 3D motion parameters
and 2D velocity that results from the projection
of V onto the image plane can be obtained
by temporally dierentiating p:
_
Z
Z
_
Y
Z
Y _
Z
Using
x
z Y; T
y
z X
x
Y
y X) for
substitution in (2), one obtains the image velocity
equation [31]:
xy
y
z
x
xy
y
x
z
Hence, image motion is a purely geometric quantity
and, consequently, for optical
ow to be exactly
image motion, a number of conditions have
to be satised. These are: a) uniform illumina-
surface re
ectance and c) pure
translation parallel to the image plane. Realisti-
cally, these conditions are never entirely satised
in scenery. Instead, it is assumed that these conditions
hold locally in the scene and therefore locally
on the image plane. The degree to which these
conditions are satised partly determines the accuracy
with which optical
ow approximates image
motion.
1.4. Multiple Motions
Given an arbitrary environment and a moving visual
sensor, the motion eld generated onto the
imaging plane by a 3D scene within the visual
eld is represented as function (3) of the motion
parameters of the visual sensor. Discontinuities
in image motion are then introduced in (3) whenever
the depth Z is other than single-valued and
dierentiable 2 . The occurrence of occlusion causes
the depth function to exhibit a discontinuity,
whereas translucency leads to a multiple-valued
depth function.
4 S. S. Beauchemin and J. L. Barron
1.5. Models of Optical Flow
Generally, the optical
ow function may be expressed
as a polynomial in some local coordinate
system of the image space of the visual sensor. It
is assumed that the center of the neighborhood
coincides with the origin of the local coordinate
system. In this case, we may write the Taylor series
expansion of a i th velocity about the origin
as:
r
x;t=0
However, we simply adopt
in what follows Fleet and Jepson's [18] constant
model of optical
ow denoted as:
where a i is now the velocity vector. Hence, a
2D intensity prole I 0 translating with velocity
yields the following spatiotemporal image intensity
We use a negative translational rate in (5) without
loss of generality and for mere mathematical
convenience.
1.6. Signal Translation in the Frequency Domain
Consider a signal I i (x) translating at a constant
t). For this signal, the Fourier transform
of the optical
ow constraint equation is obtained
with the dierentiation property as:
F
where i is the imaginary number, ^ I i (k) is the
Fourier transform of I i (x) and -(k T a
a Dirac delta function. Expression (7) yields
a constraint on velocity. Simi-
larly, the Fourier transform of a translating image
signal I i (x; t) is obtained with the shift property
as:
Z Z
I
Z Z
I
e i!t dt
which also yields the constraint k T a
Hence, (7) and (8) demonstrate that the frequency
analysis of image motion is in accordance with
the motion constraint equation [18]. It is also observed
that k T a in the frequency
domain, an oriented plane passing through
the origin, with normal vector a i descriptive of full
velocity, onto which the Fourier spectrum of I i (x)
lies.
1.7. Related Literature
Traditionally, motion perception has been equated
with orientation of power in the frequency domain.
The many optical
ow methods use what Chubb
and Sperling term the Motion-From-Fourier-
Components (MFFC) principle [11] in which the
orientation of the plane or line through the origin
of the frequency space that contains most of the
spectral power gives the rate of image translation.
The MFFC principle states that for a moving
stimulus, its Fourier transform has substantial
power over some regions of the frequency domain
whose points spatiotemporally correspond to sinusoidal
gratings with drift direction consonant with
the perceived motion [11]. In addition, current
models of human perception involve some frequency
analysis of the imagery, such as band-pass l-
tering and similar processes. However, some classes
of moving stimuli which elicit a strong percept
in subjects fail to show a coherent spatiotemporal
frequency distribution of their power and cannot
be understood in terms of the MFFC principle.
Examples include drift-balanced visual stimuli
[11], Fourier and Non-Fourier plaid superpositions
[52], amplitude envelopes, sinusoidal beats
and various multiplicative phenomena [19]. By
drift-balanced it is meant that a visual stimulus
with two (leftward and rightward, for example) or
more dierent motions shows identical contents of
Fourier power for each motion and therefore, according
to the MFFC principle, should not elicit
a coherent motion percept. However, some classes
of drift-balanced stimuli dened by Chubb and
Sperling do elicit strong coherent motion percept-
On the Fourier Properties of Discontinuous Visual Motion 5
s, contrary to the predictions of the usual MFFC
model.
Sources of Non-Fourier motion also include the
motion of texture boundaries and the motion of
motion boundaries. For instance, transparency as
considered by Fleet and Langley [19] is an example
of Non-Fourier motion, as transparency causes
the relative scattering of Fourier components
away from the spectrum of the moving stimuli. In
addition, occlusion, modeled as in (13), is another
example of Non-Fourier motion which is closely
related to the Theta motion stimuli of Zanker [53],
where the occlusion window moves independently
from both the foreground and the background,
thus involving three independent velocities.
It has been observed by Fleet and Langley that
many Non-Fourier motion stimuli have simple
characterizations in the frequency domain, namely
power distributions located along lines or planes
which do not contain the origin of the frequency
space, as required by the MFFC idealization
[19]. Occlusion and translucency being among
those Non-Fourier visual stimuli, we develop their
exact frequency representations, state their properties
with respect to image motion (or optical
ow), consider the aperture problem and include
additive translucency phenomena within the theoretical
framework.
1.8. Methodology
To analyze the frequency structure of image signals
while preserving representations that are as
general as possible, an eort is made to only pose
those hypotheses that would preserve the generality
of the analysis to follow. We describe the
assumptions and the proof techniques with which
the theoretical results were obtained.
Image Signals The geometry of visual scenes
under perspective projection generally yields
complex image signals. Conceptually, assumptions
concerning scene structure should
not be made, as they constrain the geometry
of observable scenes. In addition, any measured
physical signal, such as image intensi-
ties, satises Dirichlet conditions. Such signals
admit a nite number of nite discon-
tinuities, are absolutely integrable and may
be expanded into complex exponential series.
Dirichlet conditions constitute the sum of assumptions
made on image signals.
Velocity On a local basis, constant models of
signal translation may be adequate to describe
velocity. However, linear models admit an increased
number of deformations, such as signal
dilation. Hence, the extent used for signal
analysis may be larger with linear models. We
considered a constant model of velocity, leaving
deformations of higher order for further
analysis.
Occluding Boundaries Object frontiers and
their projection onto the imaging plane are
typically unconstrained in shape and are dicult
to model on a large spatial scale. Simpler,
local models appear to be more appropriate.
The framework includes occlusion boundaries
as locally straight edges, represented with step
functions. This hypothesis only approximates
reality and limits the analysis to local image
regions. However, we outline in which way
this hypothesis can be relaxed to include occlusion
boundaries of any shape.
Proof Techniques The Theorems and their
Corollaries established in this analysis emanate
from a general approach to modeling
visual scenes exhibiting occlusion discontinuities
or translucency. An equation which describes
the spatio-temporal pattern of the superposition
of a background and an occluding
signal is established [17], in which a characteristic
function describing the position of an
occluding signal within the imaging space of
the visual sensor is dened:
within the occluding signal
and two image signals I 1 (x) and I 2 (x), corresponding
to the occluding and occluded signals
respectively, are dened to form the over-all
signal pattern:
velocity. Note that
the characteristic function describing the object
has the same velocity as its corresponding
6 S. S. Beauchemin and J. L. Barron
intensity pattern I 1 (x). In (10) are inserted
the hypotheses made on its various components
and the structure of occlusion in the frequency
domain is developed. That is to say,
signal structures are expanded into complex
exponential series, such as:
I
n= ~c in e ix T Nk
where I i (x) is the i th intensity pattern, c in
are complex coe-cients, k i are fundamental
are integers
and I . Occlusion boundaries become
locally straight edges, represented with step
functions such as:
where n 1 is a vector normal to the tangent of
the occluding boundary. In addition, degenerate
image signals under occlusion are inves-
tigated, thus describing the aperture problem
in the context of the framework. Whenever
technically possible, the theoretical results
were compared with numerical experiments
using Fast Fourier Transforms operating on
synthetically generated image sequences.
Relevance of Fourier Analysis Many algorithms
operating in the Fourier domain for
which a claim of multiple motions capability is
made have been developed [27]. However, this
is performed without a complete knowledge
of the frequency structure of occlusion phe-
nomena. In addition, Non-Fourier spectra,
including occlusion and translucency eects
have been conjectured to have mathematically
simple characterizations in Fourier space [19].
Consequently, the use of Fourier analysis as
a local tool is justied as long as one realizes
that it constitutes a global idealization of local
phenomena. In that sense, Fourier analysis is
used as a local tool whenever Gabor lters,
wavelets or local Discrete Fourier Transforms
are employed for signal analysis.
Experimental Technique Given the theoretical
nature of this contribution, the purpose
of the numerical experiments is to verify the
validity of the theoretical results. In order to
accomplish this, the frequency content of the
image signals used in the experiments must
be entirely known to the experimenter, thus
forbidding the use of natural image sequences.
In addition, image signals with single frequency
components are used in order to facilitate
the interpretation of experiments involving
3D Fast Fourier transforms. The use of
more complex signals impedes a careful examination
of the numerical results and do not
extend the understanding of the phenomena
under study in any particular way.
2. Spectral Structure of Occlusion
The analysis begins with the consideration of a
simple case of occlusion consisting of two translating
sinusoidal signals. These preliminary results
are then generalized to arbitrary signals and the
aperture problem is examined.
2.1. Sinusoidal Image Signals
The case in which two sinusoidals play the role of
the object and the background is rst considered.
Let I i (x) be an image signal translating with velocity
Its
Fourier transform is
Let I 1 (x) be occluding another image signal I 2 (x),
with respective velocities v 1 (x; t) and v 2 (x; t).
The resulting occlusion scene can then be expressed
as:
where U(x) is (12). The Fourier transform of (13)
is:
U(k) is the Fourier transform of a step function
U(x) written as
On the Fourier Properties of Discontinuous Visual Motion 7
THEOREM 1. Let I 1 (x) and I 2 (x) be cosine functions with respective angular frequencies k T
t)). The frequency spectrum of
the occlusion is:
a)
where a = a 1 a 2 and n 1 is a normal vector perpendicular to the occluding boundary.
with n 1 as a vector normal to the occlusion boundary
1 as its negative reciprocal ( n y
Theorem 1 is derived to examine occlusion with
the simplest set of parameters, such as the form of
occlusion boundaries, the number of distinct frequencies
required to represent both the occluding
and occluded image signals, and a constant model
of velocity. Even with this constrained domain of
derivation, a number of fundamental observations
are made, such as: the occlusion in frequency space
is formed of the Fourier transform of a step
function convolved with every existing frequency
of both the occluding and occluded sinusoidal signals
and, the power content of the distortion term
is entirely imaginary, forming lines of decreasing
power which do not contain the origin, around
the frequencies of both the occluding and occluded
signals. Their orientation is parallel to the spectrum
of the occluding signal, and the detection of
their orientation allows to identify the occluding
velocity, leaving the occluded velocity to be interpreted
as such.
We performed a series of experiments to graphically
demonstrate the composition of a simple occlusion
scene. To simplify the interpretation of
the experiments, we used 1D sinusoidal signals
composed of single frequencies. In addition, the
signals are Gaussian-windowed in order to avoid
the Gibbs phenomenon when computing their Fast
Fourier Transforms (FFTs). Figure 2a), b) and c)
show the components of a simple occlusion scene,
pictured in 2d). Figure 2a) is the occluding signal
with spatial frequency 2
and velocity 1:0, such
that
I 1 (x;
and in 2b) is the occluded signal with spatial frequency8 and velocity 1:0, yielding
I 2 (x;
The occluding boundary in Figure 2c) is a 1D step
function, written as
and translates with a velocity identical to that of
I 1 .
The resulting occlusion scene in Figure 2d) is
constructed with the following 1D version of (13):
I
where I 1 is (17), I 2 is (18) and  is (19). Figures
2e) through h) show the amplitude spectra
of gures 2a) through d) respectively, where it is
easily observed the the spectrum of the step function
(19) is convolved with each frequency of both
sinusoidals. Further, Theorem 1 predicts Fourier
spectra such as 2h) in their entirety as is demonstrated
by the experiments in section 2.3.
2.2. Generalized Image Signals
For this analysis to gain generality, we need to
nd a suitable set of mathematical functions to
represent physical quantities such as image signals
that lend themselves to the analysis to follow and
which do not impose unnecessary hypotheses on
the structure of those signals.
8 S. S. Beauchemin and J. L. Barron
d)
Amplitude Spectrum of Sinusoid 1 Amplitude Spectrum of Sinusoid 2 Amplitude Spectrum of Step Function Amplitude Spectrum of Occlusion
Fig. 2. (top): The composition of a simple 1D occlusion scene. a) The occluding sinusoidal signal with frequency 2and
velocity 1:0. b) The occluded sinusoidal signal with frequency 2and velocity 1:0. c) The translating step function used
to create the occlusion scene. d) The occlusion as a combination of a), b) and c). (center): Image plots of amplitude
spectra and (bottom): amplitude spectra as 3D graphs.
For this purpose we hypothesize that image signals
satisfy Dirichlet conditions in the sense that
for any interval x 1  x  x 2 , the function f(x)
representing the signal must be single-valued, have
a nite number of maxima and minima and a -
nite number of nite discontinuities. Finally, f(x)
should be absolutely integrable in such a way that,
within the interval, we obtain
Z x2
In addition, any function representing a physical
quantity satises Dirichlet conditions. Hence,
those conditions can be assumed for visual signals
without loss of generality and, in this context, the
complex exponential series expansion, or Fourier
series 1
converges uniformly to f(x).
Theorem 2 generalizes Theorem 1 from sinusoidal
to arbitrary signals. Theorems 1 and 2
introduce the approximation of occluding boundaries
with step functions and, as surfaces of any
shape may be imaged, the forms of their boundaries
are typically unconstrained. On a local basis,
however, as long as the spatial extent of analy-
On the Fourier Properties of Discontinuous Visual Motion 9
THEOREM 2. Let I 1 (x) and I 2 (x) be 2D functions satisfying Dirichlet conditions such that they may
be expressed as complex exponential series expansions:
I 1
are integers, x are spatial coordinates, k
are fundamental frequencies and c 1n and c 2n are complex coe-cients. Let I 1 (x;
I and the occluding boundary be represented by:
where n 1 is a vector normal to the occluding boundary. The frequency spectrum of the occlusion is:
~
where a = a 1 a 2 .
sis remains su-ciently small, the approximation
of occluding boundaries as straight-edged lines is
su-cient and greatly simplies the derivation of
the results. Also for simplicity, a constant model
of velocity is adopted, which is thought of as a
valid local approximation of reality [4, 32]. How-
ever, the constraint on the shape of the occluding
boundary may be removed while preserving the
validity of most of the theoretical results, as we
later demonstrate. As expected, the sum of properties
identied in Theorem 1 hold for Theorem
2. For instance, it is found that the Fourier spectrum
of the occluding boundary is convolved with
every existing frequency of both the occluding and
occluded signals in a manner consonant with its
velocity. That is to say, its spectral orientation is
descriptive of the motion of the occluding signal.
Hence we state the following corollary:
COROLLARY 1. Under an occlusion phe-
nomenon, the velocities of the occluding and occluded
signals can always be identied as such.
Under occlusion, the spectral orientation of the
occluding boundary is parallel to the plane descriptive
of the occluding velocity and detecting
the spectral orientation of the boundary amounts
to identify the occluding velocity, leaving the occluded
velocity to be considered as such.

Figure

3 demonstrates the composition of a simple
2D occlusion scene and the Fourier spectra of
its components. Figure 3a) is the occluding signal
with spatial frequency ( 2
1:0; 1:0) such that
I 1 (x;
and Figure 3b) is the occluded signal with spatial
frequency
8 ) and velocity (1:0; 1:0), yielding
I 2 (x;
S. S. Beauchemin and J. L. Barron
y
d)
Fig. 3. (top): The composition of a simple 2D occlusion scene. a) The occluding sinusoidal signal with frequency ( 2; 2)
and velocity ( 1:0; 1:0). b) The occluded sinusoidal signal with frequency ( 2; 2) and velocity (1:0; 1:0). c) The step
function used to create the occlusion scene with normal vector (
d) The occlusion as a combination of a), b) and
c). (bottom) e) through h): Image plots of corresponding amplitude spectra.
The occluding boundary in Figure 3c) is a 2D
step function identical to (12) and translates with
a velocity which equals that of I 1 , the occluding
signal. The resulting occlusion scene in Figure
3d) is constructed with (13). Figures 3e) through
show the 3D amplitude spectra of Figures 3a)
through d), respectively.
In the experiments with 2D signals depicted
in

Figure

4, the spatial frequencies of the occluding
and occluded signals are k T
Only the velocities
and the orientation of the occlusion boundary
vary. The velocities of the occluding and occluded
signals and the occlusion boundary normal
vectors, from left to right in Figure 3, are
a) a T
and n T
p2 ); c) a T
and n T
As per Theorem 1, the spectral extrema located
at
the spatiotemporal frequencies of both signals
and t the constraint planes k T a
The oblique spectra intersecting
the peaks are the convolutions of the spectrum of
the step function with the frequencies of both signals
and t lines described by the intersection of
planes
These spectra are parallel to the constraint plane
of the occluding signal and are consonant with its
velocity.
Theorem 2 is the generalization of Theorem 1
from sinusoidal to arbitrary signals and its geometric
interpretation is similar. For instance,
frequencies
t the constraint planes of the occluding and occluded
signals, dened as k T a
In the distortion term, the Dirac
- function with arguments (k
1 and
k T a 1 +! a T Nk 2 represent a set of lines parallel
to the constraint plane of the occluding signal
and, for every discrete frequency
exhibited by both signals, there is
a frequency spectrum tting the lines given by the
intersection of planes k T a
On the Fourier Properties of Discontinuous Visual Motion 11
y
d)
@
@
@

a 1
a 2
@
@
@

a 2
a 1

a 1 a 2
@
@
@

a 1
a 2
Fig. 4. Four cases of predicted and computed Fourier spectra of occlusion scenes. In all cases the frequencies of the
occluding and the occluded signals are ( 2; 2) and ( 2; 2). (top): a) Occluding and occluded velocities a
and a Occluding and occluded velocities a Occluding and occluded
velocities boundary normal (1:0; 0). d) Occluding and occluded velocities
Computed FFTs of corresponding occlusion scenes. (bottom) i)
through l): Fourier spectra predicted by theoretical results.
and
The magnitudes of
these spectra are determined by their corresponding
scaling functions c 1n [(k
Theorem 2 reveals useful
constraint planes, as the power spectra of both
signals peak within planes k T a
and the constraint planes arising
from the distortion are parallel to the spectrum of
the occluding signal I 1 (x; t).
3. The Aperture Problem: Degenerate
Cases
In the Fourier domain, the power spectrum of a
degenerate signal is concentrated along a linear
rather than a planar structure. To see this, consider
a 1D signal moving with a constant model
of velocity in a 2D space, in the direction of the
gradient normal n i and with speed s
S. S. Beauchemin and J. L. Barron
The Fourier transform of this signal is given by
i is the negative reciprocal of n i . The
their intersection forms a linear constraint onto
which the spectrum of the degenerate signal re-
sides. Therefore, the planar orientation describing
full velocity is undetermined. However, the presence
of an occlusion boundary disambiguates the
measurement of a degenerate occluding signal in
most cases as a straight-edged occlusion boundary
provides one constraint on normal velocity and so
does its corresponding degenerate occluding sig-
nal. Since these structures have an identical full
velocity, these constraints should be consistent
with it, allowing to form a system of equations
to obtain full velocity. For instance, consider
the Fourier transform of a translating occluding
degenerate signal expressed as its complex exponential
series expansion:
Z
I 1
c in -(k nk 1
is the normal of the signal, s 1 is its speed
is the fundamental frequency. Ad-
ditionally, consider the Fourier transform of the
occluding boundary with normal vector n 2 and
speed
The convolution of (4) and (29) yields the following
spectrum:
c 1n -(k nk 1
c 1n
Expression (30) allows to derive two directional
vectors, tting the spectra of the degenerate occluding
signal and boundary respectively, which
are d T
cross product yields a vector a 1 normal to the
planar structure containing both spectra, which
is the full velocity of the degenerate occluding sig-
nal. The constraints on normal velocities form the
following system of equations
a T
a T
and its solution, obtained by dividing d 1 d 2 with
its third component, is
which is full velocity when a constant model is
used. This system has a unique solution if and
only if n 1
s 2 and (31) has no unique solution. Thus, we state
the following Theorem:
THEOREM 3. The full velocity of a degenerate
occluding signal is obtainable from the structure
of the Fourier spectrum if and only if its normal
is dierent from the normal of the occlusion
boundary.
We performed experiments with degenerate signals
as shown in Figure 5. An occluding degenerate
sinusoidal pattern with spatial orientation
translating with normal velocity
depicted in Figure 5a). The
pattern was generated according to
I 1 (x;
As can be seen from its Fourier transform 5e), the
frequency content is composed of two - functions
from which only a normal velocity estimate can be
obtained by computing the orientation of the line
passing through the spectral peaks and the origin
of the frequency space.
Figure 5b) shows the occluding signal and the
occlusion edge combination. The normal vector
to the edge is 1:0). The Fourier transform
is shown in 5f), where the spectrum of the
edge is convolved with the peaks of the signal.
In this case, the full velocity of the degenerate
signal is obtained by computing the normal vector
to the plane containing the entire spectrum
On the Fourier Properties of Discontinuous Visual Motion 13
y
d)
Fig. 5. Cases of degenerate occluding signals. (top): a) Occluding signal with normal Occluding
signal and boundary with normal Occluded signal with normal d) Complete occlusion
scene. (bottom) e) through h): Corresponding frequency spectra.
and the origin of the frequency space. Figure 5c)
shows the occluded signal with spatial orientation
translating with normal velocity
This pattern was generated according
to
I 2 (x;
and its frequency content appears in 5g).
The complete occlusion scene is shown in 5d)
and the corresponding frequency content is depicted
in 5h). To disambiguate the normal velocity of
the occluding signal, it is rst necessary to identify
the occluding velocity. This is accomplished by
nding a line that is parallel to the spectral orientation
of the Fourier transform of the occluding
edge and that also contains the frequency content
of one signal. In this case, this signal is said to be
occluding, and the normal to the plane containing
its frequency spectrum, including the spectrum of
the occluding edge convolved with its discrete fre-
quencies, yields a full velocity measurement.
4. Related Considerations
In this section we consider the relationship between
additive translucency and the theoretical
framework, the eects of occluding edges away
from the origin of the spatiotemporal domain, occluding
boundaries of various shapes and the relevancy
of the theoretical model with respect to
Non-Fourier motions such as Zanker's Theta motions
[53].
4.1. Translucency
Transmission of light through translucent material
may cause multiple motions to arise within an
image region. Generally, this eect is depicted on
the image plane as
function of the density of the
translucent material [17]. Under the local assumption
of spatially constant f( 1 ) with translucency
factor ', (35) is reformulated as a weighted super-
14 S. S. Beauchemin and J. L. Barron
x
a) b) c) d)
Fig. 6. The composition of an additive transparency scene. a): First sinusoidal signal with frequency k
sinusoidal signal with frequency k Transparency created with the
superposition of rst and second sinusoidal signal. d): Frequency spectrum of transparency.
position of intensity proles, written as
where I 1 (v 1 (x; t)) is the intensity prole of the
translucent material and I 2 (v 2 (x; t)) is the intensity
prole of the background. With I 1 (v 1 (x; t))
and I 2 (v 2 (x; t)) satisfying Dirichlet conditions,
the frequency spectrum of (36) is written as:
Hence, with respect to its frequency structure,
translucency may be reduced to a special case of
occlusion for which the distortion terms vanish.

Figure

6 shows the Fourier transform of an additive
translucency composed of two sinusoidals.
4.2. Phase Shifts
For reasons of simplicity and clarity, in each Theorem
and numerical result, the occluding boundary
contained the origin of the coordinate system. We
generalize this by describing the occlusion boundary
as
where y 0 is the y-axis intercept. The Fourier spectrum
of such a boundary includes a phase shift and
is written as:
Equation (39) can be further simplied as:
The Fourier spectrum of the boundary is to be
convolved with the complex exponential series expansions
of the occluding and occluded signals and
subsequently with the Fourier transform of the
Gaussian window. In the case of the occluding
signal, the convolution with the the shifted occlusion
boundary can be written as:
and, similarly for the occluded signal:
These convolutions are combined together as before
to obtain the Fourier spectrum of occlusion
On the Fourier Properties of Discontinuous Visual Motion 15
d)
Fig. 7. Phase shifts from occluding edge. (top): a) y
through h): Corresponding frequency spectra. The relative magnitude between the occluding and occluded signals depend
on their respective visible areas under the Gaussian envelope. For instance, The frequencies of the occluding signal dominate
over those of the occluded signal in e), and vice versa in h).
with an occluding boundary not containing the origin
of the space.
We conducted experiments with 1D image signals
and shifted the occlusion point with dieren-
t values of y 0 in (38). As observed in Figure 7,
these phase shifts do not alter the structure of occlusion
in frequency space. The variations in the
amplitude spectra are due to the Gaussian windowing
of the occlusion scene. For instance, the
frequency peaks of the occluding signal in Figure
7e) show more power than those of the occluded
signal, owing to the fact that the signal is dominant
within the Gaussian window. The contrary is
observed when the occluded signal occupies most
of the window, as shown in Figure 7h).
4.3. Generalized Occluding Boundaries
Typically, occlusion boundaries are unconstrained
in shape, yielding a variety of occluding situations.
Under the hypothesis that the motion of the occluding
boundary is rigid on the image plane, we
can derive the frequency structure of such occlusion
events. For instance, consider a generalized
occlusion boundary represented by the characteristic
function (x) in the coordinates of the image
plane and the Fourier transforms of the complex
exponential series expansions of both the occluding
and occluded signals I 1 and I 2 . Substituting
these terms into (13) yields the following Fourier
spectrum
from which it is easily observed that the spectrum
of the occluding boundary is repeated at every
non-zero frequency of both signals. The spectrum
S. S. Beauchemin and J. L. Barron
y
d)
Fig. 8. Generalized occluding boundaries. a), b) and c): Images from a sequence in which the occluding pattern moves
with velocity a T= ( 1:0; 1:0). Spatial frequency of the sinusoidal texture within the circular boundary is k T= ( 2; 2).
d): The frequency spectrum of the sequence, where the plane contains the spectrum of the boundary convolved with the
frequency of the texture.
occupies a plane descriptive of full velocity and
can be used to perform such measurements.

Figure

8 shows an experiment where the occluding
signal is within a circular occlusion boundary.
The signal and boundary are moving at a constant
velocity a T
and the occluded
signal is a background of constant intensity. Figures
8a) through c) show the motion of the occluding
region while Figure 8d) is the frequency spectrum
of the sequence, from which we observe the
spectrum of the circular boundary and the peaks
representing the frequencies of the occluding sinusoidal
texture are conned to a planar region fully
descriptive of the image motion.
4.4. Non-Fourier Motion
Non-Fourier motion is characterized by its inability
to be explained by the MFFC principle. In other
terms, such motions generate power distributions
that are inconsistent with translational mo-
tion. Sources of Non-Fourier motion include such
phenomena as translucency and occlusion and, in
particular, Zanker's Theta motion stimuli involving
occlusion [53]. This category of motion is described
by an occlusion window that translates
with a velocity that is uncorrelated with the velocities
of the occluding and occluded signals. For
1D image signals, such an occlusion scene can be
expressed as:
As Zanker and Fleet [53, 19], we model the occlusion
window with a rectangle function in the
spatial coordinate as
x x0
x x0
x x0
Such a function has a non-zero value in the interval
2 ] and zero otherwise. We then
write the Fourier transform of the occlusion scene
as:
KX
KX
c 2n -(k nk
and the phase shift from x 0 in (45)
. The spectra -(kv 3
and are consonant with the
motion of the occluding window and represent a
On the Fourier Properties of Discontinuous Visual Motion 17
x
x
a) b) c) d)
Fig. 9. Examples of Theta Motion. a): Velocities of occlusion window, occluding and occluded signals are v
Frequency spectrum of a). c): Velocities of occlusion window, occluding and
occluded signals are Frequency spectrum of c).
case of Non-Fourier motion, as they do not contain
the origin.
We performed two experiments with Theta motions
as pictured in Figure 9. It is easily observed
that the spectrum of the sinc function is convolved
with each frequency of both signals and that its
orientation is descriptive of the velocity of the win-
dow. As expected, the visible peaks represent the
motions of both signals in the MFFC sense.
5. Conclusion
Retinal image motion and optical
ow as its approximation
are fundamental concepts in the eld
of vision. The computation of optical
ow is
a challenging problem as image motion includes
discontinuities and multiple values mostly due to
scene geometry, surface translucency and various
photometric eects such as surface re
ectance. In
this contribution, we analyzed image motion in
frequency space with respect to motion discontinuities
and surface translucence. The motivation
for such a study emanated from the observation
that the frequency structure of occlusion, translucency
and Non-Fourier motion in frequency space
was not known. The results cast light on the exact
structure of occlusion, translucency, Theta mo-
tion, the aperture problem and signal degeneracy
for a constant model of image motion in the frequency
domain with related geometrical properties


Appendix


Proof Method of Theorem 2
The Fourier transform of the complex exponential
series expansion of a 2D signal is:
I
Z ~X
n= ~c in e ix T Nk i e ik T x dx
n= ~c in -(k Nk i )
and the Fourier transform of 2D step function under
constant velocity is:
Z
where n i is a vector normal to the occlusion
boundary. Introducing (A1) and (A2) into the
Fourier transform of (13) under constant velocity
and solving the convolutions leads to Theorem 2.
S. S. Beauchemin and J. L. Barron
Notes
1. Signals that are termed as degenerate have a spatially
constant intensity gradient or, in other words, a unique
texture orientation. This phenomenon is generally referred
to as the aperture problem which arises when
the Fourier spectrum of I i (x) is concentrated on a line
rather than on a plane [18, 33]. Spatiotemporally, this
depicts the situation in which I i (x; t) exhibits a single
orientation. In this case, one only obtains the speed and
direction of motion normal to the orientation, noted as
v?i (x; t). If many normal velocities are found in a single
neighborhood, their respective spectra t the plane
0 from which full velocity may be obtained.
2. This assertion assumes dierentiable sensor motion.



--R

Determining three-dimensional motion and structure from optical ow generated by several moving objects
A fast obstacle detection method based on optical ow.
Disparity analysis of images.
Performance of optical ow techniques.
The feasibility of motion and structure from noisy time-varying image velocity information
Optic ow to measure minute increments in plant growth.
A model for the detection of motion over time.
Motion segmentation and qualitative dynamic scene analysis from an image sequence.

A split-merge parallel block-matching algorithm for video displacement estimation

Stereo correspondence from optical ow.
The sampling and reconstruction of time-varying imagery with application in video systems
On the detection of motion and the computation of optical ow.
Obstacle detection by evaluation of optical ow

Measurement of Image Velocity.
Computation of component image velocity from local phase information.
Computational analysis of non-fourier motion
The use of optical ow for autonomous navigation.
Optical motions and space perception: An extension of gibson's analysis.
Subspace methods for recovering rigid motion 2: Algorithm and implemen- tation
Recovery of ego-motion using image stabilization
Direct computation of the focus of expan- sion
Segmentation of frame sequences obtained by a moving observer.

Mixture models for optica ow computation.
Vertical and horizontal disparities from phase.

A computer algorithm for reconstructing a scene from two projections.
The interpretation of a moving retinal image.
An iterative image- registration technique with an application to stereo vision
Directional selectivity and its use in early visual processing.
The accuracy of the computation of optical ow and of the recovery of motion parameters.
A video encoding system using conditional picture-element replenishment
Scene segmentation from visual motion using global optimization.
Advances in picture coding.
Motion recovery from image sequences using only
Motion compensated television coding: Part 1.


Motion and structure from optical ow.
Motion estimation from tagged mr image sequences.
How do we avoid confounding the direction we are looking and the direction we are moving.
Movement detectors of the correlation type provide sucient information for local computation of 2d velocity
multiple motions from optical ow.
Edge detection and motion detection.
Bounds on time-to-collision and rotational component from rst-order derivatives of image ow
A fast method to estimate sensor translation.
Uniqueness and estimation of three-dimensional motion parameters of rigid objects with curved surfaces
Estimating three-dimensional motion parameters of a rigid planar patch 2: Singular value decomposition
Coherence and transparency of moving plaids composed of fourier and non-fourier gratings
Theta motion: A paradoxical stimulus to explore higher-order motion extraction

An error-weighted regularization algorithm for image motion- eld estima- tion
Automatic feature point extraction and tracking in image sequences for unknown image motion.
--TR
Edge detection and motion detection
Scene segmentation from visual motion using global optimization
Bounds on time-to-collision and rotational component from first-order derivatives of image flow
Obstacle detection by evaluation of optical flow fields
Stereo correspondence from optic flow
Computation of component image velocity from local phase information
The feasibility of motion and structure from noisy time-varying image velocity information
Techniques for disparity measurement
On the Detection of Motion and the Computation of Optical Flow
Three-dimensional motion computation and object segmentation in a long sequence of stereo frames
Subspace methods for recovering rigid motion I
Motion recovery from image sequences using only first order optical flow information
Motion segmentation and qualitative dynamic scene analysis from an image sequence
Performance of optical flow techniques
The use of optical flow for the autonomous navigation
Measurement of Image Velocity
The Accuracy of the Computation of Optical Flow and of the Recovery of Motion Parameters
A Fast Method to Estimate Sensor Translation
multiple motions from optical flow
A Fast Obstacle Detection Method based on Optical Flow

--CTR
Weichuan Yu , Gerald Sommer , Steven Beauchemin , Kostas Daniilidis, Oriented Structure of the Occlusion Distortion: Is It Reliable?, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.9, p.1286-1290, September 2002
Abhijit S. Ogale , Yiannis Aloimonos, A Roadmap to the Integration of Early Visual Modules, International Journal of Computer Vision, v.72 n.1, p.9-25, April     2007
Weichuan Yu , Gerald Sommer , Kostas Daniilidis, Multiple motion analysis: in spatial or in spectral domain?, Computer Vision and Image Understanding, v.90 n.2, p.129-152, May
image motion;aperture problem;occlusion;optical flow;non-Fourier motion