Exploiting Discontinuities in Optical Flow.
Most optical flow estimation techniques have substantial difficulties dealing with flow discontinuities. Methods which simultaneously detect flow boundaries and use the detected boundaries to aid in flow estimation can produce significantly improved results. Current approaches to implementing these methods still have important limitations, however. We demonstrate three such problems: errors due to the mixture of image properties across boundaries, an intrinsic ambiguity in boundary location when only short sequences are considered, and difficulties insuring that the motion of a boundary aids in flow estimation for the surface to which it is attached without corrupting the flow estimates for the occluded surface on the other side. The first problem can be fixed by basing flow estimation only on image changes at edges. The second requires an analysis of longer time intervals. The third can be aided by using a boundary detection mechanism which classifies the sides of boundaries as occluding and occluded at the same time as the boundaries are detected.
Introduction
Discontinuities in optical flow are normally viewed
as a serious impediment to producing accurate estimates
of the flow. Due to the ambiguous nature
of flow within small space-time neighborhoods of
an image sequence, all methods for estimating flow
either explicitly or implicitly assume some sort of
spatial and/or temporal continuity. This assumption
is often violated at surface boundaries. This
paper points out important shortcomings in one
class of methods for estimating possibly discontinuous
optical flow and indicates how improvements
can be made.
Two general approaches to dealing with flow
discontinuities are found in the literature. The
first explicitly allows for a mixed distribution near
boundaries. Marr and Poggio (1977) and Barnard
and Thompson (1980) did this by basing their
analyses on prominent modes of possible disparities
over local neighborhoods. Using currently
popular terminology, the modes acted as a robust
estimator for the dominant disparity in the neigh-
borhood, though neither paper explained this ef-
fect. Some years later, Prazdny (1985) made the
idea explicit. Scott's principal component approach
(Scott, 1988) and Schunck's constraint line
clustering (Schunck, 1989) utilized the same con-
cept. More recently, Black and Anadan (1990)
explicitly used robust estimation in determining
optical flow near boundaries. Bergen et al. (1990)
used a more specialized approach in which one
dominant motion is found first and then used
to produce a new image sequence in which the
found motion is nulled, allowing a second motion
Thompson
to be more easily estimated. Once flow is estimated
using any of the above approaches, it is
relatively straightforward to find the flow discontinuities
(Thompson, 1985).
The second approach is to make discontinuities
explicit, thus avoiding the need to deal with
mixed distributions. All of these methods follow
the basic structure of Geman and Geman
(1984) or Blake and Zisserman (1987). The most
common approach utilizes a Markov random field
(MRF) formulation with explicit line processes
(Murray and Buxton, 1987; Gamble and Poggio
1987; Hutchinson et al., 1988; Koch et al., 1989;
Konrad and Dubois, 1992; Heitz and Bouthemy,
1993). Interacting region and line processes are
involved. The region processes combine information
over local neighborhoods to reduce ambiguity
and improve reliability of individual flow es-
timates. The line processes estimate locations of
discontinuities and act to keep points on opposite
sides from interacting in the region processes.
The estimation of discontinuities can be greatly
assisted if independent information about possible
surface boundaries is available. In particular,
most of the papers referenced above allow the assertion
of a boundary element only at locations
likely to correspond to contrast edges.
This paper addresses optical flow estimation using
explicit discontinuity detection. We provide
evidence for three claims:
ffl Line processes, as usually implemented, do
not completely remove undesirable interactions
between surfaces on either side of a motion
boundary.
ffl There is an intrinsic ambiguity in the localization
of motion boundaries which can only
be resolved over multiple frame pairs.
ffl Distinguishing between the occluding and occluded
sides of an optical flow boundary can
aid the flow estimation process.
Ways of exploiting these observations are
demonstrated using a simple flow estimation pro-
cedure. They can easily be incorporated into more
complete flow estimation systems.
2. Estimating Optical Flow in the Presence
of Discontinuities
Existing flow estimation techniques which involve
explicit motion boundary detection suffer from
two deficiencies when applied to image sequences
with textured backgrounds. Interactions between
surfaces on either side of the boundary can still oc-
cur, degrading the accuracy of the estimated flow.
Furthermore, motion boundary detection itself is
subject to a localization ambiguity when only a
single frame pair is analyzed at a time.
2.1. Mixed image properties near the boundary
Clearly, optical flow cannot be determined based
only on a single space/time image point. All
methods for estimating optical flow use image
properties that must be computed over neighbor-
hoods. Gradient-based techniques (e.g, Fennema
and Thompson, 1979; Horn and Schunck, 1981)
require the estimation of spatial and temporal
derivatives using some form of finite difference.
Region-based methods (e.g., Anandan, 1989) compare
local neighborhoods around selected points
in each frame. Space-time filtering methods (e.g.,
Adelson and Bergen, 1986; Heeger, 1988; Fleet
and Jepson, 1990) are implemented using FIR filters
with finite spatial and temporal extent. Image
properties are often computed after some form of
pre-filtering (Kearney et al., 1987; Barron et al.,
1994) also involving operations with finite spatial
and/or temporal extent. In addition, many
flow estimation methods impose explicit flow continuity
constraints (e.g., Fennema and Thompson
(1979), Barnard and Thompson (1980), Horn
and Schunck (1981), and the various MRF algo-
rithms).
The line processes used to deal with flow discontinuities
in MRF-like methods in fact only affect
the imposition of explicit flow continuity con-
straints. Image properties associated with points
to one side of an asserted boundary can still be effected
by the intensity patterns on the other side
of the boundary. Derivatives estimated using finite
differences and pre-filtering will have substantial
errors in the vicinity of motion boundaries,
particularly if the surfaces to one of both sides are
highly textured. A similar effect occurs with spa-
Exploiting Discontinuities in Optical Flow 165
tiotemporal filters. While the line processes stop
direct interactions between estimated flow to either
side of a surface boundary, the flow estimates
themselves can still be highly inaccurate.
If done in a systematic manner, the use of adaptive
window sizes can reduce the effects of mixed
image properties at boundaries (Kanade and Oku-
tomi, 1994; Nagel, 1995). An alternate and more
easily implemented solution is to base flow estimates
on correctly chosen sparse image features.
The features must be lines or points that necessarily
occur either well away from surface boundaries
or occur exactly at the boundary and for
which localization is not significantly effected by
image structure to either side of the boundary.
The Moravec "interest operator" (Barnard and
Thompson, 1980), extrema in the difference of
Gaussians (Mutch and Thompson, 1983), corner
detectors (Ranagrajan et al., 1988), and methods
based on contrast edges (Hildreth, 1983; Waxman
et al., 1988) satisfy this property.
2.2. Ambiguity of surface boundary location
Many of the methods for detecting flow discontinuities
are based on looking for large magnitude
values of the flow gradient. Unless the location of
the flow discontinuity is already known or methods
which allow for multiple motions near boundaries
are used, estimated flow will be smoothed
over near the boundary. Figure 1 illustrates the
problem. Estimated flow magnitude is plotted
against position along an axis perpendicular to
a surface boundary. Away from the boundary,
flow estimates will be reasonably accurate. They
should also be accurate on the occluding surface
nearby
texture
actual
boundary
Position
Flow
Maximum rate of
change in flow
Fig. 1. Optical flow gradients are not sufficient to accurately
locate flow boundaries.
near the boundary, since the boundary itself will
have sufficient structure to allow flow estimation
and is moving with the occluding surface. On the
occluded side, however, flow estimates won't be
accurate until observed sufficiently far from the
boundary to allow the moving texture of the occluded
surface to dominate any effects associated
with the surface boundary, which is moving with
a different optical flow. The maximum rate of
change in flow will typically be noticeably offset
from the actual boundary.
Whatever flow-based boundary detection method
is used, there is an intrinsic localization ambiguity
associated with instantaneous flow that cannot
be overcome without either non-motion cues
to the boundary location or analysis over longer
time intervals (Thompson and Barnard, 1981). 1
Consider the pattern shown on the left in Figure
2. Three bars, ff, fi, and fl, are moving to the
left. Three other bars, a, b, and c, are moving
to the right. Assuming that occlusion boundaries
are likely to have associated contrast edges, then
the edge of the occluding surface is either along
the right side of bar fl or the left side of bar c.
additional information, there is no way
to resolve this ambiguity. Not until the next time
step, shown on the right in the figure, is it apparent
that the lighter colored bars on the left are
occluding the darker colored bars to the right and
that the true occlusion boundary is at the edge
of bar fl. This problem affects not only boundary
detection based on flow gradients, but also
methods which are based on the appearance and
disappearance of surface texture (Kaplan, 1969;
Mutch and Thompson, 1985), methods which allow
for multiple motions within a local neighbor-
hood, and methods which attempt to recognize
occlusion boundaries using properties of similarity
surfaces (e.g., Anandan, 1989). A method for
deciding which possibility - bar fl, bar c, or the
newly appeared bar d - really corresponds to the
surface boundary is described in section 3.2.1.
3. An Algorithm for Flow Estimation in
the Presence of Discontinuities
This section outlines an optical flow estimation algorithm
which effectively addresses the problems
discussed in section 2. It has similarities to the
Thompson
b b a
a c
occlusion boundary?
c
d
occlusion boundary?
Fig. 2. Intrinsic ambiguity in boundary location at one instant in time.
method presented in Heitz and Bouthemy (1993),
though differs in a number of important regards:
the algorithm correctly deals with mixed image
properties near surface boundaries, it uses a more
complete classification of boundary types, and it
is able to resolve boundary ambiguities when textured
backgrounds are present. The algorithm
utilizes explicit line processes to avoid undesirable
interactions across motion discontinuities in a
manner analogous to the MRF methods, but using
a much simpler formulation. The important
features of this algorithm can be added to most
existing flow estimation approaches.
In the remainder of this paper, our analysis excludes
motions consisting exclusively of rotations
of revolute objects around an axis perpendicular
to the line of sight and corresponding to the axis of
symmetry of the object. This particular situation
introduces additional ambiguities, the resolution
of which is still largely an open question (Thomp-
son et al., 1992; Thompson and Painter, 1992).
3.1. Dealing with discontinuities
At flow discontinuities, effective flow estimation
methods must avoid smoothing the flow across the
boundary and must also keep image properties associated
with a surface on one side of the boundary
from corrupting the flow estimation for the
surface on the other side. We deal with this second
problem by basing flow estimation solely on
image features which are relatively unaffected by
this mixture of image properties. In particular,
we base flow calculations only on image properties
at edges. As long as no temporal pre-filtering
is performed, spatial contrast edges will either be
close approximations to surface boundaries or far
enough away from surface boundaries so that local
image properties near the edge are largely unaffected
by image regions that are part of other
surfaces. This has the additional advantage of
minimizing difficulties due to apparent misregis-
trations between contrast and motion edges due
to localization errors when separate image primitives
are used. The standard Horn and Schunck
(1981) method is utilized, modified so that spatial
and temporal image gradients are only used
at spatial edges. For other image locations, flow
is estimated using the interpolation properties intrinsic
to the method.
Given information about the location of surface
boundaries, it is straightforward to avoid flow
computation interactions across boundaries. The
Gauss-Seidel method used in the basic Horn and
Schunck algorithm bases flow estimates for a given
pixel at a particular iteration step on the average
values at neighboring pixels in the previous itera-
tion. It is only necessary to make sure this average
does not include any pixels on the other side of a
motion discontinuity.
Discontinuity detection is based on an analysis
of flow differences across contrast edges which potentially
signal surface boundaries. When possi-
ble, the surfaces to either side of a detected boundary
are classified as occluding or occluded using
the boundary flow constraint, which states that the
flow associated with the occluding surface immediately
adjacent to the boundary will be equal to
the flow of the boundary (Thompson et al., 1985).
Violations of the boundary flow constraint can be
used to identify occluded or disoccluded surfaces,
Exploiting Discontinuities in Optical Flow 167
Fig. 3. The surface to the left has an optical flow different
from that of the boundary and hence is an occluded surface.
Since it is moving away from the boundary, it is in fact
being disoccluded.
as shown in Figure 3, without any need to know
the camera or object motions involved. This relationship
is only useful when there are differences
across the boundary in the component of flow normal
to the boundary. If the flows to either side of a
boundary are parallel to the boundary itself, then
it is not possible to determine which side is occluding
the other without additional information
about camera motion. Such sheer boundaries still
indicate occlusion, however.
To detect and classify motion boundaries, the
flow associated with every point on a contrast edge
is compared to the flow to either side. The flows to
either side are separated into components normal
to the edge orientation, f ?and f ?and parallel to
the edge, f kand f k. Detection and classification
proceeds by first checking to see if one surface is
progressively occluding or disoccluding the other.
If not, sheering surfaces are checked for:
if max(d ?; d ?) - T 1
if d ?? d ?side 1 is occluded, side 2 is
occluding
else
side 1 is occluding, side 2 is
occluded.
else if j f
sides 1 and 2 are sheer
This method, while less elegant that the detection
and classification scheme described in
Thompson et al. (1985) is more reliable. Note in
particular that f ?and f ?are separately compared
to the normal flow of the edge, rather than
just taking their difference. This is important, as
it increases the sensitivity of the edge detection.
Because of the effects described in section 2.2, the
difference in flow normal to the edge attributable
to one surface moving relative to another will be
concentrated in d ?or d ?. Additional differences
in flow normal to the edge will arise due to smooth
variations in flow on the occluding side of the edge
and from various noise effects. As a result, basing
detection on j f ?\Gamma f ?j will increase the classification
error rate.
Flow estimation at a boundary pixel neither affects
nor is affected by the flow at pixels to the
occluding side of the boundary. Interactions to
the occluding side, however, are allowed, since the
boundary is part of that surface. A conservative
approach is taken for sheer boundaries. The flow
estimation at the boundary is isolated from the
flow values to either side, but does interact with
the estimation of other values along the bound-
ary. The complete algorithm interleaves flow estimation
and boundary detection. Enough Gauss-Seidel
iterations are done to get a reasonable estimate
of flow values. Boundaries are then detected
and classified. Better flow estimates are obtained
by performing additional iterations using this in-
formation. The process is repeated as necessary.
3.2. Improving boundary localization over time
Optical flow estimation over sequences longer than
a single frame pair can improve the efficiency of
iterative algorithms by providing reasonably accurate
initial values and can improve the accuracy
of many methods by using some form of temporal
coherence to reduce the effects of noise. It
turns out that longer sequences are also the key
to resolving the localization ambiguity described
in section 2.2.
3.2.1. Boundary projection Surface boundaries
persist over time. Once these boundaries are
found at one time step, their position can be easily
predicted for the next time step. This is done by
taking each point on a detected occlusion boundary
and looking for a contrast edge in the future
frame at the corresponding location, offset by the
flow of the boundary point. Substantial efficiency
is achieved by starting the iterations at the next
time step using these predicted boundary loca-
tions. Even more importantly, over time actual
Thompson
surface boundaries will be maintained while detected
boundaries that in fact correspond to surface
texture near true boundaries will disappear.
To see why this is so, consider Figure 2. At
time step 0, the interleaved iterate and classify
algorithm will end up finding potential occlusion
boundaries to the right of bar fl and to the left
of bar c. The flow estimates assigned to the region
between the bars is likely to be a muddle.
At time step 1, bar d has appeared. The flow
of this new "texture element" will initially propagate
both left and right, since there is no surface
boundary indication associated with the bar. As
the flow propagation approaches bar c, the differential
flow across the (false) boundary on the
left side of c is reduced, leading to a reclassification
of the edge as non-occluding. The sides of
d never get classified as occluding, since the only
flow not consistent with the edge motion is blocked
by the boundary at fl from propagating towards d.
The situation shown in Figure 2 is extreme in that
a full texture element appears over a single time
step. In practice, texture element spacing is usually
much larger than inter-frame motions and the
effect described above takes place more gradually.
3.2.2. Flow projection Horn and Schunck suggest
improving the efficiency of their algorithm by
using the results obtained for one time step to initialize
the iterations at the next time step. Simply
using the flow obtained for each pixel at one time
step to initialize the estimates for the same pixel
at the next time step is adequate over smooth sur-
faces, but can actually lead to worse results at occlusion
boundaries than if a default of zero flow
is used in the initialization. This is because the
flow associated with a disoccluded surface region
will typically be very different from the flow of the
surface which was previously occluding it. Some
improvement can be obtained by projecting the
flow estimates at one time step to pixels in the
next time step in a manner that takes into account
the flow value itself. This fails, however, to provide
initial estimates for flow in occluded regions
where multiple flow values project to the same
point and in disocclusion regions where no flow
values from the previous time step will project
(Black and Anandan, 1990).
Effective projection of flow into occluded and
disoccluded regions in the next time step is pos-
disoccluded
region region
flow
zero
occluded
occlusion boundary
occlusion boundary
zero
flow
Fig. 4. Projecting flow into occlusion and disocclusion regions

sible if occlusion boundaries have been detected
and classified in the current time step. Figure 4
shows a simplified situation in which an occlusion
boundary is stationary, the occluding surface is
to the left, the occluded surface is to the right,
and the motion of the occluded surface is normal
to the boundary. Two possibilities exist: the occluded
surface is moving either towards or away
from the boundary. (In the case of pure sheer
motion, no occlusion or disocclusion regions ex-
ist.) Movement away from the boundary causes
disocclusion. The gray area on the left of Figure
4 indicates the region in the next time step
that will correspond to surface visible for the first
time and which will have flow close to that on
the occluded side of the boundary. Movement towards
the boundary causes occlusion. The gray
area on the right in Figure 4 indicates the region
in the next time step where flow vectors from two
different surfaces will both project. The best estimate
of the actual flow will be the flow of the
boundary itself. In either case, the region of oc-
clusion/disocclusion runs from the boundary to a
line found by adding the flow vector of the occluded
surface point nearest each boundary point
to the boundary point location. For the more general
case of moving boundaries, the region of oc-
clusion/disocclusion runs from the projection of
the boundary into the next time step to the line
found by adding the flow vector of the occluded
surface point nearest each boundary point to the
projected boundary point location. The flow to
be filled into these regions is either the flow of the
boundary or the flow of the nearest occluded surface
point, depending on the classification of the
boundary.
Exploiting Discontinuities in Optical Flow 169
Fig. 5. Original image sequence, frames
Note that the principal goal here is to improve
efficiency, not implement some sort of temporal
continuity constraint. Temporal consistency can
also be used to improve accuracy in flow esti-
mation, though this constraint can lead to problems
in occlusion/disocclusion regions (Murray
and Buxton, 1987). Thus, the approach described
in the preceding paragraph might be extended
to better exploit temporal continuity at and near
boundaries.
4. Experimental Results
This section presents results from controlled experiments
run on synthetic data. Implementation
details and results on real imagery are presented
in Thompson (1995). While the limitations of using
synthetic data are well known, it is the only
way to do fair quantitative comparisons between
alternate approaches. The accuracy of estimated
flow was measured using the inter-frame angular
deviation described in Barron, 1994. While this
measure is useful in evaluating the results of the
controlled experiments described below, the absolute
magnitudes of flow errors should not be compared
with those obtained in other circumstances.
In the tests reported here, most of the flow estimation
error occurs at flow discontinuities, which
correspond to a substantial number of the pixels
in the test sequences. This is not the case with
other image sequences, such as those used in Bar-
ron, 1994. Error magnitude comparisons between
dissimilar image sequences will be dominated by
the number of pixels at or near discontinuities at
least as much as by the quality of the flow estimation
algorithms applied.
In evaluating the results, it is also important
to note that the performance of the methods described
here depends on both the particular motion
analysis algorithm used and the quality of
the (static) edges on which it is based. In order
to provide a grounds for comparing algorithms, a
simple edge detector was used and manipulations
of the test sequences were structured to avoid perturbations
of edge detector performance. For ex-
ample, sub-pixel motions were avoided, since edge
detector results would be affected by the anti-aliasing
scheme used to generate synthetic images
with non-integral motions. To get the best possible
performance, as would be desirable if quantitative
comparisons were been done with competing
methods, a more sophisticated edge detector
should be used.

Figure

5 shows the first four images from an
120 by 160 pixel synthetically generated sequence
with a textured rectangle moving across a textured
background. The rectangle is moving four
pixels to the right and two pixel down per frame.
The background is moving two pixel down and
sheer boundaries
occluding sides
side
side
occluded
occluded
Fig. 6. "True" boundary classifications for moving rectangle
test sequence.
Thompson
Fig. 7. Flow estimation using standard Horn and Schunck method (first four frame pairs).
Fig. 8. Flow estimation using Horn and Schunck with explicit boundary detection (first four frame pairs).
to the left per frame. Figure 6 shows the true
occluded/occluding and sheer boundaries for this
sequence. In this and subsequent figures, sheer
boundaries are marked by a light gray pattern on
either side of a dark line while occluded/occluding
boundaries are marked by a darker gray pattern
on the occluded side of the edge.

Figure

7 shows the optical flow estimated by the
standard Horn and Schunck algorithm, applied to
the first four frame pairs of the moving rectangle
sequence. The errors in flow estimation, particularly
at and near the boundaries of the moving
rectangle, are obvious. Figure 8 demonstrates
the results of adding explicit boundary analysis
to the basic Horn and Schunck algorithm. The
final flow estimates and boundary classifications
at each time step are shown. While the flow estimates
are improved, substantial error remains
due to the mixing of image properties across the
boundaries, which seriously distorts the gradient
computations. Figure 9 uses the same algorithm
as

Figure

8, except that only image properties at
contrast edges are used in the flow computations.
This minimizes the effects of image property mixing
between surfaces. Results for four frame pairs
are shown. Flow estimates are further improved.
The ambiguity in detecting and classifying boundaries
when background texture appears near occluding
contours, which is manifested in the figure
as missing and extraneous boundary segments, is
still affecting accuracy, however. Figure 10 shows
the effect of using boundary projection to initialize
processing at the next time step. The boundary
classification errors have largely been eliminated
and the flow estimates correspondingly improved.

Table

1 lists the quantitative average errors associated
with each of these algorithms.
Exploiting Discontinuities in Optical Flow 171
Fig. 9. Edge-based flow estimation using explicit discontinuity detection.
Fig. 10. Flow estimation using explicit discontinuity detection and projection.

Table

1. Flow errors for different estimation algorithms.
frame
Basic Horn and Schunck 13.38
Horn and Schunck with boundary
detection 11.31
Edge-based flow estimation with
boundary detection 7.55
Edge-based flow est. with boundary
detection, projection 7.55
The next set of experiments tests the value of
classifying surface boundaries by labeling the sides
as occluding or occluded. A test sequence consisting
of a uniform intensity disk moving over a tex-
Thompson
Fig. 11. Moving disk image sequence.

Table

2. Flow errors using symmetric and asymmetric boundary classification.
frame
Symmetric boundary classification 6.75
Asymmetric boundary classification 5.55

Table

3. Flow errors when total iterations are reduced.
frame
Boundary projection, no flow projection 21.99
copied flow 21.99
projected flow 21.99
Boundary projection, copied flow. 21.99
Boundary projection, projected flow. 21.99
tured background was used, with foreground and
background motions as in the previous tests (Fig-
ure 11). To insure convergence given the large,
uniform brightness area, 600 iterations were run
at every time step. Occluding/occluded boundary
classification should allow the motion of the
disk boundary itself to propagate into the otherwise
featureless interior of the disk. As is clear in

Table

2, accuracy is improved.
The final set of experiments address the issue of
how well projecting the estimated flow from one
time step to initialize computations at the next
time step helps in flow determination. The image
sequence used was that shown in Figure 5. A
total of iterations were run at each time step,
as compared to 150/step for the results shown in

Figures

7-10. Five cases were examined, with the
resulting errors shown in Table 3. For the first
case, the algorithm employing boundary classification
and boundary projection was used as in

Figure

10, only with the reduced number of itera-
tions. In the second case, no boundary projection
was done, but flow in each time step after the first
was initialized copying the values obtained in the
previous time step. (While boundaries were not
projected from frame to frame, boundary detection
and classification was used within each time
Exploiting Discontinuities in Optical Flow 173
step.) With reduced iteration, initializing flow estimates
is clearly more valuable than initializing
boundary estimates. In the third test, there was
no boundary projection and flow estimates were
initialized using the projection method described
in section 3.2.2. There is a substantial improvement
in accuracy. The last two cases use boundary
projection and flow that is initialized by either
copying or projecting values from the previous
time step. Again, flow projection is clearly
superior to simple copying.
5.

Summary

This paper has examined a number of important
issues affecting the accuracy of optical flow estimates
made by algorithms which explicitly take
into account flow discontinuities. Theoretical and
experimental evidence has been presented showing
that the line processes typically used to keep flow
smoothness constraints from corrupting estimates
across boundaries can still allow image properties
associated with a surface to one side of a boundary
to improperly influence flow computations on
the other side of the boundary. A second problem
affecting flow estimation is the intrinsic ambiguity
of boundary localization in textured environments
when only two frames are considered at a
time. This effect was clearly demonstrated using
one particular flow estimation algorithm, but occurs
independent of the algorithm used. Experimentation
has shown the value of distinguishing
between occluding and occluded sides of a surface
corresponding to a flow discontinuity. This information
can be used to improve flow estimates and
is important in projecting flow estimates to future
time steps so as to reduce computational require-
ments. Finally, we have shown how the line processes
used in MRF formulations for estimating
optical flow can also be used in the conceptually
simpler Horn and Schunck algorithm. While the
techniques we have described for improving flow
estimation at boundaries are most easily implemented
in algorithms such as Horn and Schunck
and MRF methods, they can be applied across
the spectrum of approaches to optical flow esti-
mation. For example, the area correlation algorithm
described in Smitley and Bajcsy (1981) can
be viewed as utilizing something akin to a line
process when matching image regions.

Acknowledgements

This work was supported by National Science
Foundation grant IRI-9112267.
Notes
1. There is evidence that the human vision system resolves
this ambiguity by favoring whichever possible boundary
has the strongest non-motion edge properties (Yonas,
1990).



--R

The extracton of spatio-temporal energy in human and machine vision
A computational framework and an algorithm for the measurement of visual motion.
Disparity analysis of images.
Performance of optical flow techniques.
Computing two motions from three frames.
A model for the detection of motion over time.
Visual Reconstruction.
Velocity determination in scenes containing several moving objects.
Computation of component image velocity from local phase information.
Visual integration and detection of discontinuities: The key role of intensity edges.
Stochastic relaxation
Optical flow estimation using spatiotemporal filters.

IEEE Trans.
The Measurement of Visual Motion.
Determining optical flow.

Computing motion using analog and binary resistive networks.

A stereo matching algorithm with an adaptive window: Theory and experiment.
Kinetic disruption of optical texture: The perception of depth at an edge.
Optical flow estimation: An error analysis of gradient-based methods with local optimization

Computing optical flow in resistive networks and in the primate visual system.
Bayesian estimation of motion vector fields.
Cooperative computation of stereo disparity.
Scene segmentation from visual motion using global optimization.
Hierarchical estimation of spatial properties from motion.
Analysis of accretion and deletion at boundaries in dynamic scenes.
Optical flow estimation and the interaction between measurement errors at adjacent pixel positions.
International Journal of Computer Vision
Detection of binocular disparities.
Optimal corner detection.
Image flow segmentation and estimation by constraint line clustering.
Local and Global Interpretation of Moving Images.
Stereo processing of aerial
Exploiting discontinuities in optical flow.



Dynamic occlusion analysis in optical flow fields.
Qualitative constraints for structure-from-motion
Convected activation profiles and the measurement of visual motion.


Abstracts in Investigative Ophthalmology and Visual Sci- ence
--TR
visual motion;discontinuities;optical flow;edges