Filters, Random Fields and Maximum Entropy (FRAME).
This article presents a statistical theory for texture modeling. This theory combines filtering theory and Markov random field modeling through the maximum entropy principle, and interprets and clarifies many previous concepts and methods for texture analysis and synthesis from a unified point of view. Our theory characterizes the ensemble of images I with the same texture appearance by a probability distribution f(I) on a random field, and the objective of texture modeling is to make inference about f(I), given a set of observed texture examples.In our theory, texture modeling consists of two steps. (1) A set of filters is selected from a general filter bank to capture features of the texture, these filters are applied to observed texture images, and the histograms of the filtered images are extracted. These histograms are estimates of the marginal distributions of f( I). This step is called feature extraction. (2) The maximum entropy principle is employed to derive a distribution p(I), which is restricted to have the same marginal distributions as those in (1). This p(I) is considered as an estimate of f( I). This step is called feature fusion. A stepwise algorithm is proposed to choose filters from a general filter bank. The resulting model, called fields And Maximum Entropy), is a Markov random field (MRF) model, but with a much enriched vocabulary and hence much stronger descriptive ability than the previous MRF models used for texture modeling. Gibbs sampler is adopted to synthesize texture images by drawing typical samples from p(I), thus the model is verified by seeing whether the synthesized texture images have similar visual appearances to the texture images being modeled. Experiments on a variety of 1D and 2D textures are described to illustrate our theory and to show the performance of our algorithms. These experiments demonstrate that many textures which are previously considered as from different categories can be modeled and synthesized in a common framework.
Introduction
Texture is an important characteristic of the appearance of objects in natural scenes,
and is a powerful cue in visual perception. It plays an important role in computer vision,
graphics, and image encoding. Understanding texture is an essential part of understanding
human vision.
Texture analysis and synthesis has been an active research area during the past three
decades, and a large number of methods have been proposed, with different objectives
or assumptions about the underlying texture formation processes. For example, in computer
graphics, reaction-diffusion equations (Witkin and Kass 1991) have been adopted
to simulate some chemical processes that may generate textures on skin of animals. In
computer vision and psychology, however, instead of modeling specific texture formation
process, the goal is to search for a general model which should be able to describe a wide
variety of textures in a common framework, and which should also be consistent with the
psychophysical and physiological understanding of human texture perception.
The first general texture model was proposed by Julesz in the 1960's. Julesz suggested
that texture perception might be explained by extracting the so-called 'k-th order' statis-
tics, i.e., the co-occurrence statistics for intensities at k-tuples of pixels (Julesz 1962).
Indeed, early works on texture modeling were mainly driven by this conjecture (Haralick
1979). A key drawback for this model is that the amount of data contained in the k-th
order statistics is gigantic and thus very hard to handle when k ? 2. On the other hand,
psychophysical experiments show that the human visual system does extract at least some
statistics of order higher than two (Diaconis and Freeman 1981).
More recent work on texture mainly focus on the following two well-established areas.
One is filtering theory, which was inspired by the multi-channel filtering mechanism
discovered and generally accepted in neurophysiology (Silverman et al. 1989). This mechanism
suggests that visual system decomposes the retinal image into a set of sub-bands,
which are computed by convolving the image with a bank of linear filters followed by
some nonlinear procedures. The filtering theory developed along this direction includes
the Gabor filters (Gabor 1946, Daugman 1985) and wavelet pyramids (Mallat 1989, Simoncelli
etc. 1992, Coifman and Wickerhauser 1992, Donoho and Johnstone 1994). The
filtering methods show excellent performance in classification and segmentation (Jain and
Farrokhsia 1991).
The second area is statistical modeling, which characterizes texture images as arising
from probability distributions on random fields. These include time series models (Mc-
Cormick and Jayaramamurthy 1974), Markov chain models (Qian and Terrington 1991),
and Markov random field (MRF) models (Cross and Jain 1983, Mao and Jain 1992, Yuan
and Rao 1993). These modeling approaches involve only a small number of parameters,
thus provide concise representation for textures. More importantly, they pose texture
analysis as a well-defined statistical inference problem. The statistical theories enable
us not only to make inference about the parameters of the underlying probability models
based on observed texture images, but also to synthesize texture images by sampling
from these probability models. Therefore, it provides a rigorous way to test the model by
checking whether the synthesized images have similar visual appearances to the textures
being modeled (Cross and Jain 1983). But usually these models are of very limited forms,
hence suffer from the lack of expressive power.
This paper proposes a modeling methodology which is built on and directly combines
the above two important themes for texture modeling. Our theory characterizes the
ensemble of images I with the same texture appearances by a probability distribution
f(I) on a random field. Then given a set of observed texture examples, our goal is to infer
f(I). The derivation of our model consists of two steps.
(I) A set of filters is selected from a general filter bank to capture features of the
texture. The filters are designed to capture whatever features might be thought to be
characteristic of the given texture. They can be of any size, linear or nonlinear. These
filters are applied to the observed texture images, and histograms of the filtered images
are extracted. These histograms estimate the marginal distributions of f(I). This step is
called feature extraction.
(II) Then a maximum entropy distribution p(I) is constructed, which is restricted to
match the marginal distributions of f(I) estimated in step (I). This step is called feature
fusion.
A stepwise algorithm is proposed to select filters from a general filter bank, and at
each step k it chooses a filter F (k) so that the marginal distributions of f(I) and p(I)
with respect to F (k) have the biggest distance in terms of L 1 norm. The resulting model,
called fields And Maximum Entropy), is a Markov random
field (MRF) model 1 , but with a much more enriched vocabulary and hence much stronger
descriptive power compared with previous MRF models. The Gibbs sampler is adopted
to synthesize texture images by drawing samples from p(I), thus the model is tested by
synthesizing textures in both 1D and 2D experiments.
Our theory is motivated by two aspects. Firstly, a theorem proven in section (3.2)
shows that a distribution f(I) is uniquely determined by its marginals. Therefore if
a model p(I) matches all the marginals of f(I), then recent
psychophysical research on human texture perception suggests that two 'homogeneous'
textures are often difficult to discriminate when they have similar marginal distributions
from a bank of filters (Bergen and Adelson 1991, Chubb and Landy 1991). Our method
is inspired by and bears some similarities to Heeger and Bergen's (1995) recent work
on texture synthesis, where many natural looking texture images were synthesized by
matching the histograms of filter responses organized in the form of a pyramid.
This paper is arranged as follows. First we set the scene by discussing filtering methods
and Markov random field models in section (2), where both the advantages and disadvantages
of these approaches are addressed. Then in section (3), we derive our FRAME
model and propose a feature matching algorithm for probability inference and stochastic
simulation. Section (4) is dedicated to the design and selection of filters. The texture
modeling experiments are divided into three parts. Firstly section (5) illustrates important
concepts of the FRAME model by designing three experiments for one dimensional
texture synthesis. Secondly a variety of 2D textures are studied in section (6). Then
section (7) discusses a special diffusion strategy for modeling some typical texton images.
Finally section (8) concludes with a discussion and the future work.
Among statisticians, MRF usually refers to those models where the Markov neighborhood is very
small, e.g. 2 or 3 pixels away. Here we use it for any size of neighborhood.
Filtering and MRF Modeling
2.1 Filtering theory
In the various stages along the visual pathway, from retina, to V1, to extra-striate cortex,
cells with increasing sophistication and abstraction have been discovered: center-surround
isotropic retinal ganglion cells, frequency and orientation selective simple cells, and complex
cells that perform nonlinear operations. Motivated by such physiological discoveries,
the filtering theory proposes that the visual system decomposes a retinal image into a set
of sub-band images by convolving it with a bank of frequency and orientation selective
linear filters. This linear filtering process is then followed by some nonlinear operations.
In the design of various filters, Gaussian function plays an important role due to its nice
low-pass frequency property. To fix notation, we define an elongated two-dimensional
Gaussian function as:
with location parameters parameters (oe x ; oe y ).
A simple model for the radially symmetric center-surround ganglion cells is the Laplacian
of Gaussian with oe
Similarly, a model for the simple cells is the Gabor filter (Daugman 1985), which is a
pair of cosine and sine waves with frequency ! and amplitude modulated by the Gaussian
By carefully choosing the frequency ! and rotating the filter in the x-y coordinate system,
we obtain a bank of filters which cover the entire frequency domain. Such filters are used
for image analysis and synthesis successfully by (Jain and Farrokhsia 199, Lee 1992 ).
Other filter banks have also been designed for image processing by (Simoncelli etc. 1992).
The filters mentioned above are linear. Some functions are further applied to these
linear filters to model the nonlinear functions of the complex cell. One way to model the
complex cell is to use the power of each pair of Gabor filter j In fact,
is the local spectrum S(!) of I at (x; y) smoothed by a Gaussian function.
Thus it serves as a spectrum analyzer.
Although these filters are very efficient in capturing local spatial features, some problems
are not well understood. For example, i) given a bank of filters, how to choose the
best set of filters? especially when some of the filters are linear while others are nonlinear,
or the filters are highly correlated to each other, ii) after selecting the filters, how to
fuse the features captured by them into a single texture model? These questions will be
answered in the rest of the paper.
2.2 MRF modeling
MRF models were popularized by Besag (Besag 1973) for modeling spatial interactions
on lattice systems and were used by (Cross and Jain 1983) for texture modeling. An
important characteristic of MRF modeling is that the global patterns are formed via
stochastic propagation of local interactions, which is particularly appropriate for modeling
textures since they are characterized by global but not predictable repetitions of similar
local structures.
In MRF models, a texture is considered as a realization from a random field I defined
over a spatial configuration D, for example, D can be an array or a lattice. We denote I(~v)
as the random variable at a location ~v 2 D, and let Dg be a neighborhood
system of D, which is a collection of subsets of D satisfying 1)
. The pixels in N ~v are called neighbors of ~v. A subset C of D is a clique if every
pair of distinct pixels in C are neighbors of each other; C denotes the set of all cliques.
Definition. p(I) is an MRF distribution with respect to N if p(I(~v) j
denotes the values of all pixels other than ~v, and for A ae D,
denotes the values of all pixels in A.
Definition. p(I) is a Gibbs distribution with respect to N if
Z
where Z is the normalizing constant (or partition function), and -C () is a function of
intensities of pixels in clique C (called potential of C). Some constraints can be imposed
on -C for them to be uniquely determined.
The Hammersley-Clifford theorem establishes the equivalence between MRF and the
Gibbs distribution (Besag
Theorem 1 For a given N , p(I) is an MRF distribution () p(I) is a Gibbs distribution.
This equivalence provides a general method for specifying an MRF on D, i.e., first
choose an N , and then specify -C . The MRF is stationary if for every C 2 C, -C depends
only on the relative positions of its pixels. This is often assumed in texture modeling.
Existing MRF models for texture modeling are mostly auto-models (Besag 1973) with
potentials, i.e. -C j 0 if jCj ? 2, and p(I) has the following form
Z
expf
where and ~v are neighbors.
The above MRF model is usually specified through conditional distributions,
where the neighborhood is usually of order less than or equal to three pixels, and some
further restrictions are usually imposed on g for p(I(~v) j I(\Gamma~v)) to be a linear regression
or the generalized linear model.
Two commonly used auto-models are the auto-binomial model and the auto-normal
model. The auto-binomial model is used for images with finite grey levels I(~v) 2 f0;
1g (Cross and Jain 1983), the conditional distribution is a logistic regression,
where
log p ~v
It can be shown that the joint distribution is of the form
Z expf
G
When 2, the auto-binomial model reduces to the auto-logistic model (i.e., the Ising
model), which is used to model binary images.
The auto-normal model is used for images with continuous grey levels (Yuan and Rao
1993). It is evident that if an MRF p(I) is a multivariate normal distribution, then p(I)
must be auto-normal, so the auto-normal model is also called a Gaussian MRF or GMRF.
The conditional distribution is a normal regression,
and p(I) is of the form
i.e., the multivariate normal distribution N(-; oe the diagonal elements of B
are unity and the off-diagonal (~u; ~v) element of it is \Gammafi ~u\Gamma~v .
Another MRF model for texture is the OE-model (Geman and Graffigne 1986):
Z expf\Gamma
where OE is a known even symmetric function, and the OE-model can be viewed as extended
from the Potts model (Winkler 1995).
The advantage of the auto-models is that the parameters in the models can be easily
inferred by auto-regression, but they are severely limited in the following two aspects: i)
the cliques are too small to capture features of texture, ii) the statistics on the cliques
specifies only the first-order and second order moments (e.g. means and covariances for
GMRF). However, many textures has local structures much larger than three or four
pixels, and the covariance information or equivalently spectrum can not adequately characterize
textures, as suggested the existence of distinguishable texture pairs with identical
second-order or even third-order moments, as well as indistinguishable texture pairs with
different second-order moments (Diaconis and Freeman 1981). Moreover many textures
are strongly non-Gaussian, regardless of neighborhood size.
The underlying cause of these limitations is that equation 3 involves too many parameters
if we increase the neighborhood size or the order of the statistics, even for the
simplest auto-models. This suggests that we need carefully designed functional forms
for -C () to efficiently characterize local interactions as well as the statistics on the local
interactions.
3 From maximum entropy to FRAME model
3.1 The basics of maximum entropy
Maximum entropy (ME) is an important principle in statistics for constructing a probability
distributions p on a set of random variables X (Jaynes 1957). Suppose the available
information is the expectations of some known functions OE
R
Let\Omega be the set of all probability distribution p(x)
which satisfy the constraints, i.e.,
The ME principle suggests that a good choice of the probability distribution is the one
that has the maximum entropy, i.e.,
Z
log p(x)dxg; (11)
subject to E p [OE n
R
and
R
By Lagrange multipliers, the solution for p(x) is:
where  is the Lagrange parameter, and Z (
R expf\Gamma
is the partition function that depends on   and it has the following properties:
@ log Z
Z
In equation (12), (- 1 ; :::; -N ) is determined by the constraints in equation (11). But a
closed form solution for not available in general, especially when OE n (\Delta) gets
complicated, so instead we seek numerical solutions by solving the following equations
iteratively.
dt
The second property of the partition function Z ( ) tells us that the Hessian matrix
of log Z ( ) is the covariance matrix of vector (OE 1 (x); OE 2 (x); :::; OE N (x)) which is definitely
positive 2 , and log Z ( ) is strictly concave with respect to (- 1 ; :::; -N ), so is log p(x;  ).
Therefore given a set of consistent constraints, there is a unique solution for
in equation (13).
3.2 Deriving the FRAME model
Let image I be defined on a discrete domain D, D can be a N \Theta N lattice. For each pixel
and L is an interval of R or L ae Z. For each texture, we assume that
there exists a "true" joint probability density f(I) over the image space L jDj , and f(I)
should concentrate on a subspace of L jDj which corresponds to texture images that have
perceptually similar texture appearances. Before we derive the FRAME model, we first
fix the notation as below.
Given an image I and a filter F (ff) with being an index of filter, we let
I (ff) I(~v) be the filter response at location ~v, and I (ff) the filtered image. The
marginal empirical distribution (histogram) of I (ff) is
where ffi() is the Dirac delta function. The marginal distribution of f(I) with respect to
F (ff) at location ~v is denoted by
f (ff)
Z Z
I
(ff) (~v)=z
At first thought, it seems an intractable problem to estimate f(I) due to the overwhelming
dimensionality of image I. To reduce dimensions, we first introduce the following
theorem.
Here, it is reasonable to assume that OE n (x) is independent of OE j (x) if i
Theorem 2 Let f(I) be the j D j-dimensional continuous probability distribution of a
texture, then f(I) is a linear combination of f (-) , the latter are the marginal distributions
on the linear filter response F (-)   I.
[Proof ]. By inverse Fourier transform, we have
Z
Z
f (-) is the characteristic function of f(I), and
Z
Z
e \Gamma2- i!-; I? f(I)dI
Z
e \Gamma2- iz dz
Z
Z
f(I)dI
Z
e \Gamma2- iz dz
Z
Z
Z
e \Gamma2- iz f (-) (z)dz
is the inner product, and by definition f (-)
R
R
is the marginal distribution of F (-)   I, and we define F (-) as a linear filter. 2
Theorem 2 transforms f(I) into a linear combination of its one dimensional marginal
distributions. 3 Thus it motivates a new method for inferring f(I): construct a distribution
p(I) so that p(I) has the same marginal distributions f (-) . If p(I) matches all marginal
distributions of f(I), then f(I). But this method will involve uncountable number
of filters and each filter F (-) is as big as image I.
Our second motivation comes from recent psychophysical research on human texture
perception, and the latter suggests that two homogeneous textures are often difficult to
discriminate when they produce similar marginal distributions for responses from a bank
of filters (Bergen and Adelson 1991, Chubb and Landy 1991). This means that it is
plausible to ignore some statistical properties of f(I) which are not important for human
texture discrimination.
To make texture modeling a tractable problem, in the rest of this paper we make
the following assumptions to limit the number of filters and the window size of each
3 It may help understand the spirit of this theorem by comparing it to the slice-reconstruction of 3D
volume in tomography.
filter for computational reason, though these assumptions are not necessary conditions
for our theory to hold true. 1). We limit our model to homogeneous textures, thus f(I) is
stationary with respect to location ~v. 4 2). For a given texture, all features which concern
texture perception can be captured by "locally" supported filters. In other words, the
sizes of filters should be smaller than the size of the image. For example, the size of image
is 256 \Theta 256 pixels, and the sizes of filters we used are limited to be less than 33 \Theta 33
pixels. These filters can be linear or non-linear as we discussed in section (2.1). 3). Only
a finite set of filters are used to estimate f(I)
Assumptions 1) and 2) are made because we often have access to only one observed
(training) texture image. For a given observed image I obs and a filter F (ff) , we let I obs(ff)
denote the filtered image, and H obs(ff) (z) the histogram of I obs(ff) . According to assumption
1), f (ff)
is independent of ~v. By ergodicity, H obs(ff) (z) makes a consistent
estimator to f (ff) (z). Assumption 2) ensures that the image size is lager relative to the
support of filters, so that ergodicity takes effect for H obs(ff) (z) to be an accurate estimate
of f (ff) (z).
Now for a specific texture, let Kg be a finite set of well selected
filters, and f (ff) (z); are the corresponding marginal distributions of f(I). We
denote the probability distribution p(I) which matches these marginal distributions as a
set,
is the marginal distribution of p(I) with respect to filter F (ff) at
location ~v. Thus according to assumption 3), any p(I)
2\Omega is perceptually a good enough
model for the texture, provided that we have enough well selected filters. Then we choose
from\Omega a distribution p(I) which has the maximum entropy,
Z
p(I) log p(I)dIg; (15)
subject to E
and
R
4 Throughout this paper, we use circulant boundary conditions.
The reason for us to choose the maximum entropy (ME) distribution is that while
p(I) satisfies the constraints along some dimensions, it is made as random as possible
in other unconstrained dimensions, since entropy is a measure of randomness. In other
words, p(I) should represent information no more than that is available. Therefore an ME
distribution gives the simplest explanation for the constraints and thus the purest fusion
of the extracted features.
The constraints on equation (15) differ from the ones given in section (3.1) in that z
takes continuous real values, hence there are uncountable number of constraints, therefore,
the Lagrange parameter - takes the form as a function of z. Also since the constraints are
the same for all locations ~v 2 D, - should be independent of ~v. Solving this maximization
problem gives the ME distribution:
Z
is a set of selected filters, and
is the Lagrange parameter. Note that in equation (17), for each filter F (ff) , - (ff) () takes
the form as a continuous function of the filter response I (ff) (~v).
To proceed further, let's derive a discrete form of equation (17). Assume that the filter
response I (ff) (~v) is quantitized into L discrete grey levels, therefore z takes values from
set fz (ff)
L g. In general, the width of these bins do not have to be equal, and
the number of grey levels L for each filter response may vary. As a result, the marginal
distributions and histograms are approximated by piecewisely constant functions of L
bins, and we denote these piecewise functions as vectors. H
L ) is
the histogram of I (ff) , H obs(ff) denotes the histogram of I obs(ff) , and the potential function
- (ff) () is approximated by vector -
equation (16) is rewritten as:
by changing the order of summations:
The virtue of equation (18) is that it provides us with a simple parametric model for
the probability distribution on I, and this model has the following properties,
specified by
ffl Given an image I, its histograms H (1) are sufficient statistics, i.e.
p(I;   K ; SK ) is a function of (H
We plug equation (18) into the constraints of the ME distribution, and solve for
iteratively by the following equations,
d- (ff)
dt
In equation (19), we have substituted H obs(ff) for f (ff) , and E p(I;  K ;S K ) (H (ff) ) is the expected
histogram of the filtered image I (ff) where I follows p(I;   K ; SK ) with the current
K . Equation (19) converges to the unique solution at
K as we discussed in
section (3.1), and -
K is called the ME-estimator.
It is worth mentioning that this ME-estimator is equivalent to the maximum likelihood
estimator (MLE),
log p(I obs ;   K
log Z (  K
By gradient accent, maximizing the log-likelihood gives equation (19), following property
i) of the partition function Z (  K ).
In Equation (19), at each step, given   K and hence p(I;   K ; SK ), the analytic form of
available, instead we propose the following method to estimate it
as we did for f (ff) before. We draw a typical sample from p(I;   K ; SK ), and thus synthesize
a texture image I syn . Then we use the histogram H syn(ff) of I syn(ff) to approximate
This requires that the size of I syn that we are synthesizing should be
large enough. 5
To draw a typical sample image from p(I;   K ; SK ), we use the Gibbs sampler (Geman
and Geman 1984) which simulates a Markov chain in the image space L jDj . The Markov
chain starts from any random image, for example, a white noise image, and it converges
to a stationary process with distribution p(I;   K ; SK ). Thus when the Gibbs sampler
converges, the images synthesized follow distribution p(I;   K ; SK ).
In summary, we propose the following algorithm for inferring the underlying probability
model p(I;   K ; SK ) and for synthesizing the texture according to p(I;   K ; SK ). The
algorithm stops when the subband histograms of the synthesized texture closely match
the corresponding histograms of the observed images. 6
5 Empirically, 128 \Theta 128 or 256 \Theta 256 seems to give a good estimation.
6 We assume the histogram of each subband I
(ff) is normalized such that
all the
computed in this algorithm have one extra degree of freedom for each ff, i.e., we can
increase f- (ff)
by a constant without changing p(I;   K ; SK ). This constant will be absorbed
by the partition function Z (  K ).
Algorithm 1. The FRAME algorithm
Input a texture image I obs .
Select a group of K filters g.
Compute Kg.
Initialize - (ff)
Initialize I syn as a uniform white noise texture. 7
Repeat
Calculate H syn(ff) :::; K from I syn , use it for E p(I;  K ;SK
Update - (ff)
Apply Gibbs sampler to flip I syn for w sweeps under p(I;   K
Until 1P L
Algorithm 2. The Gibbs Sampler for w sweeps
Given image I(~v), flip counter/ 0
Repeat
Randomly pick a location ~v under the uniform distribution.
For being the number of grey levels of I
Calculate
Randomly flip I(~v) / val under p(val j I(\Gamma~v)).
flip counter / flip counter
Until flip counter=w \Theta M \Theta N .
In algorithm 2, to compute I(\Gamma~v)), we set I(~v) to val, due to Markov
property, we only need to compute the changes of I (ff) at the neighborhood of ~v. The size
of the neighborhood is determined by the size of filter F (ff) . With the updated I (ff) , we
calculate H (ff) , and the probability is normalized such that
1.
In the Gibbs sampler, flipping a pixel is a step of the Markov chain, and we define
flipping jDj pixels as a sweep, where jDj is the size of the synthesized image. Then the
overall iterative process becomes an inhomogeneous Markov chain. At the beginning of
the process, p(I;   K ; SK ) is a "hot" uniform distribution. By updating the parameters,
the process get closer and closer to the target distribution, which is much colder. So the
algorithm is very much like a simulated annealing algorithm (Geyer and Thompson 1995),
7 Note that the white noise image with uniform distribution are the samples from p(I;   K ; SK ) with
which is helpful for getting around local modes of the target distribution. We refer to
(Winkler 1995) for discussion of alternative sampling methods.
The computational complexity of the above algorithm is notoriously large: O(U \Theta w \Theta
jDj \Theta G \Theta K \Theta FH \Theta FW ) with U the number of updating steps for   K , w the number
of sweeps each time we update   K , D the size of image I syn , G the number of grey levels
of I, K the number of filters, and FH;FW are the average window sizes of the filters. To
synthesize a 128 \Theta 128 texture, the typical complexity is about 50 \Theta 4 \Theta 128 \Theta 128 \Theta 8 \Theta 4 \Theta
takes about one day to run on a Sun-20.
Therefore it is very important to choose a small set of filter which can best capture the
features of the texture. We discuss how to choose filters in section (4).
3.3 A general framework
The probability distribution we derived in the last section is of the form
This model is significant in the following aspects.
First, the model is directly built on the features I (ff) (~v) extracted by a set of filters
F (ff) . By choosing the filters, it can easily capture the properties of the texture at multiple
scales and orientations, either linear or nonlinear. Hence it is much more expressive than
the cliques used in the traditional MRF models.
Second, instead of characterizing only the first and second order moments of the
marginal distributions as the auto-regression MRF models did, the FRAME model includes
the whole marginal distribution. Indeed, in a simplified case, if the constraints
on the probability distribution are given in the form of kth-order moments instead of
marginal distributions, then the functions - (ff) (\Delta) in equation (21) become polynomials
of order m. In such case, the complexity of the FRAME model is measured by the following
two aspects: 1) the number of filters and the size of the filter, 2) the order of
the moments-m. As we will see in later experiments, equation (21) enable us to model
strongly non-Gaussian textures.
It is also clear to us that all existing MRF texture models can be shown as special
cases of FRAME models.
Finally, if we relax the homogeneous assumption, i.e., let the marginal distribution
of I (ff) (~v) depend on ~v, then by specifying these marginal distributions, denoted by f (ff)
p(I) will have the form
Z expf\Gamma
This distribution is relevant in texture segmentation where - (ff)
~v are assumed piecewise
consistent with respect to ~v, and in shape inference when - (ff)
~v changes systematically with
respect to ~v, resulting in a slowly varying texture. We shall not study non-stationary
textures in this paper.
In summary, the FRAME model incorporates and generalizes the attractive properties
of the filtering theory and the random fields models, and it interprets many previous
methods for texture modeling in a unified view of point.
4 Filter selection
In the last section, we build a probability model for a given texture based on a set of
filters SK . For computational reasons SK should be chosen as small as possible, and a
key factor for successful texture modeling is to choose the right set of filters that best
characterizes the features of the texture being modeled. In this section, we propose a
novel method for filter selection.
4.1 Design of the filter bank
To describe a wide variety of textures, we first need to design a filter bank B. B can
include all previously designed multi-scale filters (Daugman 1985, Simoncelli etc 1992) or
wavelets (Mallat 1989, Donoho and Johnstone 1994, Coifman and Wickerhauser 1992).
In this paper, we should not discuss the optimal criterion for constructing a filter bank.
Throughout the experiments in this paper, we use five kinds of filters.
1. The intensity filter ffi(), and it captures the DC component.
2. The isotropic center-surround filters, i.e., the Laplacian of Gaussian filters. Here
we rewrite equation (1) as:
const
2oe stands for the scale of the filter. We choose eight scales with
these filters by LG(T ).
3. The Gabor filters with both sine and cosine components. We choose a simple case
from equation (2):
const
We choose 6 scales
We can see that these filters are not even approximately orthogonal to each other. We
denote by Gcos(T ; ') and Gsin(T ; ') the cosine and sine components of the Gabor filters.
4. The spectrum analyzers denoted by SP (T ; '), whose responses are the power of
the Gabor pairs: j (Gabor   I)(x; y) j 2 .
5. Some specially designed filters for one dimensional textures and the textons, see
section (5) and (7).
4.2 A stepwise algorithm for filter selection
For a fixed model complexity K, one way to choose SK from B is to search for all possible
combinations of K filters in B and compute the corresponding p(I;   K ; SK ). Then by
comparing the synthesized texture images following each p(I;   K ; SK ), we can see which
set of filters is the best. Such a brute force search is computationally infeasible, and for a
specific texture we often do not know what K is. Instead, we propose a stepwise greedy
strategy. We start from S an uniform distribution, and then
sequentially introduce one filter at a time.
Suppose that at the k-th step we have chosen S obtained
a maximum entropy distribution
so that E p(I;  k ;S k ) [H (ff) k. Then at the (k 1)-th step, for each
filter F (fi) 2 B=S k , we denote by the distance between
are respectively the marginal distributions of p(I;   k
and f(I) with respect to filter F (fi) . Intuitively, the bigger d(fi) is, the more information
F (fi) carries, since it reports a big difference between p(I;   k ; S k ) and f(I). Therefore we
should choose the filter which has the maximal distance, i.e.,
Empirically we choose to measure the distance d(fi) in terms of L p -norm, i.e.,
In the experiments of this paper, we choose
To estimate f (fi) and E p(I;  k ;S k ) [H (fi) ], we applied F (fi) to the observed image I obs and
the synthesized image I syn sampled from the p(I;   k ; S k ), and substitute the histograms
of the filtered images for f (fi) and E p(I;  k ;S k ) [H (fi) ], i.e.,
The filter selection procedure stops when the d(fi) for all filters F (fi) in the filter bank
are smaller than a constant ffl. Due to fluctuation, various patches of the same observed
texture image often have a certain amount of histogram variance, and we use such a
variance for ffl.
In summary, we propose the following algorithm for filter selection.
Algorithm 3. Filter Selection
Let B be a bank of filters, S the set of selected filters, I obs the observed texture image, and
I syn the synthesized texture image.
dist. I syn / uniform noise.
For do
Compute I obs(ff) by applying F (ff) to I obs .
Compute of I obs(ff) .
Repeat
For each F (fi) 2 B=S do
Compute I syn(fi) by applying F (fi) to I syn
Compute of I syn(fi)
Choose F (k+1) so that d(k
S
Starting from p(I) and I syn , run algorithm 1 to compute new p   (I) and I syn  .
p(I) / p   (I) and I syn / I syn  .
Until d (fi) ! ffl
Before we conclude this section, we would like to mention that the above criterion
for filter selection is related to the minimax entropy principle studied in (Zhu, Wu, and
Mumford 1996). The minimax entropy principle suggests that the optimal set of filters
SK should be chosen to minimize the Kullback-Leibler distance between p(I;   K ; SK ) and
f(I), and the latter is measured by the entropy of the model p(I;   K ; SK ) up to a constant.
Thus at each step k filter is selected so that it minimizes the entropy of p(I;   k
by gradient descent, i.e.,
and   + is the new Lagrange parameter. A brief derivation is
given in the appendix.
5 Experiments in one dimension
In this section we illustrate some important concepts of the FRAME model by studying a
few typical examples for 1D texture modeling. In these experiments, the filters are chosen
by hand.
For one-dimensional texture the domain is a discrete array and a pixel
is indexed by x instead of ~v. For any x 2 [0; 255], I(x) is discretized into G grey levels,
with experiment I and III, and experiment II.
Experiment I. This experiment is designed to show the analogy between filters in
texture modeling and vocabulary in language description, and to demonstrate how a
both histograms are normalized to have We note this measure is
robust with respect to the choice of the bin number L (e.q. we can take as well as the
normalization of the filters.
a

Figure

1 The observed and synthesized pulse textures. a) the observed; b) synthesized using only the
intensity filter; c) intensity filter plus rectangular filter with d) Gabor filter with
Gabor filter plus intensity filter.
texture can be specified by the marginal distributions of a few well selected filters.
The observed texture, as shown in figure (1.a), is a periodic pulse signal with period
once every 8 pixels and all the other pixels. First
we choose an intensity filter, and the filter response is the signal itself. The synthesized
texture by FRAME is displayed in figure (1.b). Obviously it has almost the same number
of pulses as the observed one, and so has approximately the same marginal distribution for
intensity. Unlike the observed texture, however, these pulses are not arranged periodically.
To capture the period of the signal, we add one more special filter, an 8 \Theta 1 rectangular
1], and the synthesized signal is shown in figure (1.c), which has
almost the same appearance as in Figure (1.a). We can say that the probability p(I)
specified by these two filters models the properties of the input signal very well.

Figure

(1.d) is the synthesized texture using a nonlinear filter which is an 1D spectrum
analyzer SP (T ) with 8. Since the original periodic signal has flat power spectrum,
and the Gabor filters only extract information in one frequency band, therefore the texture
synthesized under p(I) has power spectrum near frequency 2-but are totally free at
other bands. Due to the maximum entropy principle, the FRAME model allows for the
unconstrained frequency bands to be as noisy as possible. This explains why figure (1.d)
is noise like while having roughly a period of 8. If we add the intensity filter, then
probability p(I) captures the observed signal again, and a synthesized texture is shown in
figure (1.e).
This experiment shows that the more filters we use, the closer we can match the
synthesized images to the observed. But there are some disadvantages for using too many
filters. Firstly it is computationally expensive, and secondly, since we have few observed
examples, it may overly constrain the probability p(I), i.e. it may make p(I) 'colder' than
it should be.
Experiment II. In this second experiment we compare FRAME against Gaussian
MRF models by showing the inadequacy of the GMRF model to express high order
statistics.
To begin with, we choose a gradient filter r with impulse response [\Gamma1; 1] for com-
parison, and the filtered image is denoted by rI.
The GMRF models are concerned with only the mean and variance of the filter re-
sponses. As an example, we put the following two constraints on distribution p(I),
Since we use a circulant boundary, the first constraint always holds, and the resulting
maximum entropy probability is
Z
expf\Gamma-
The numeric solution given by the FRAME algorithm is and two synthesized
texture images are shown in figure (3.b) and (3.c). Figure (3.a) is a white noise texture
for comparison.
x
x
I
x
x
I
x
x
.
.
. 94
a b

Figure

a) The designed marginal distribution of rI; and b) The designed marginal distribution of \DeltaI.
As a comparison, we now ask rI(x) to follow the distribution shown in figure (2.a).
Clearly in this case E p [rI(x)] is a non-Gaussian distribution with first and second moments
as before, i.e., synthesized textures are
displayed in figure (3.d and e). The textures in figure (3.d and e) possess the same first
and second order moments as in figure (3.b and c), but figure (3.d and e) have specific
higher order statistics and looks more specific than in figure (3.b and c). It demonstrates
that the FRAME model has more expressive power than the GMRF model.
Now we add a Laplacian filter \Delta with impulse response [0:5; \Gamma1:0; 0:5], and we ask
\DeltaI(x) to follow the distribution shown in figure (2.b). Clearly the number of peaks and
valleys in I(x) are specified by the two short peaks in figure (2.b), the synthesized texture
c d

Figure

3 a. The uniform white noise texture, b.c. the texture of GMRF, d,e, the texture with higher
order statistics, f. the texture specified with one more filter.
is displayed in figure (3.f). This experiment also shows the analogy between filters and
vocabulary.
Experiment III. This experiment is designed to demonstrate how a single non-linear
Gabor filter is capable of forming global periodic textures. The observed texture
is a perfect sine wave with period T hence it has a single Fourier component.
We choose the spectrum analyzer SP (T ) with period 16. The synthesized texture
is in figure (4.a). The same is done for another sine wave that has period T
and correspondingly the result is shown in figure (4.b). Figure (4) show clear globally
periodic signals formed by single local filters. The noise is due to the frequency resolution
of the filters. Since the input textures are exactly periodic, the optimal resolution will
requires the Gabor filters to be as long as the input signal, which is computationally more
expensive.

Figure

4 The observed textures are the pure sine waves with period T=16, and 32 respectively. Periodic
texture synthesized by a pair of Gabor filters, a. T=16, b. T=32.
6 Experiments in two dimensions
In this section, we discuss texture modeling experiments in two dimensions. We first take
one texture as an example to show in detail the procedure of algorithm 3, then we will
apply algorithm 3 to other textures.

Figure

(5.a) is the observed image of animal fur. We start from the uniform noise
image in figure (5.b). The first filter picked by the algorithm is a Laplacian of Gaussian
filter LG(1:0) and its window size is 5 \Theta 5. It has the largest error
all the filters in the filters bank. Then we synthesize texture as shown in figure (5.c),
which has almost the same histogram at the subband of this filter (the error d(fi) drops
to 0:035).
Comparing figure (5.c) with figure (5.b), we notice that this filter captures local
smoothness feature of the observed texture. Then the algorithm sequentially picks 5 more
filters. They are 2) Gcos(6:0; 120
each of which captures features at various scales and orientations.
The sequential conditional errors for these filters are respectively 0:424; 0:207; 0:132; 0:157; 0:059
and the texture images synthesized using filters are shown in figure (5.d,e,f).
Obviously, with more filters added, the synthesized texture gets closer to the observed
one.
To show more details, we display the subband images of the 6 filters in figure (6), the
histograms of these subbands H (ff) and the corresponding estimated parameters - (ff) are
plotted in figure (7) and figure (8) respectively.

Figure

5 Synthesis of the fur texture. a is the observed texture, b,c,d,e,f are the synthesized textures
using filters respectively. See text for interpretation.

Figure

6 The subband images by applying the 6 filters to the fur image: a Laplacian of Gaussian
a b c

Figure

7 a,b,c,d,e,f are respectively the histograms H (ff) for
lambda i
lambda i
a b c
lambda i
lambda i
3 41.53.5i
lambda i

Figure

8 a,b,c,d,e,f are respectively the - (ff) for
In figure (7), the histograms are approximately Gaussian functions, and correspon-
dently, the estimated - (ff) in figure (8) are close to quadratic functions. Hence in this
example, the high order moments seemly do not play a major role, and the probability
model can be made simpler. But this will not be always true for other textures. In figure
(8), we also notice that the computed - (ff) becomes smaller and smaller when ff gets
bigger, which suggests that the filters chosen in later steps make less and less contribution
to p(I), and thus confirms our early assumption that the marginal distributions of
a small number of filtered images are good enough to capture the underlying probability
distribution f(I).

Figure

(9.a) is the scene of the mud ground with footprints of animals, these footprints
are filled with water and get brighter. This is a case of sparse features. Figure (9.b) is
the synthesized texture using 5 filters chosen by algorithm 3.

Figure

(10.a) is an image taken from the skin of cheetah. the synthesized texture using
6 filters is displayed in figure (10.b). We notice that in figure (10.a) the texture is not
homogeneous, the shapes of the blobs vary with spatial locations and the left upper corner
is darker than the right lower one. The synthesized texture, shown in figure (10.b), also
has elongated blobs introduced by different filters, but we notice that the bright pixels

Figure

9 a. the observed texture-mud, b, the synthesized one using 5 filters
a b

Figure

a. the observed texture-cheetah blob, b, the synthesized one using 6 filters
c d

Figure

11 a. the input image of fabric, b. the synthesized image with two spectrum analyzers plus
the Laplacian of Gaussian filter. c,d The filter response of the two spectrum analyzers for the fabric
texture
spread uniformly across the image.
Finally we show a texture of fabric in figure (11.a), which has clear periods along both
horizontal and vertical directions. We want to use this texture to test the use of non-linear
filters, so we choose two spectrum analyzers to capture the first two salient periods,
one in the horizontal direction, the other in the vertical direction. The filter responses
I (ff) are the sum of squares of the sine and cosine component responses. The
filter responses are shown in figure (11c, d), and are almost constant. We also use the
intensity filter and the Laplacian of Gaussian filter LG(
(with window size 3 \Theta
to take care of the intensity histogram and the smoothness. The synthesized texture
is displayed in figure (11.b). If we carefully look at figure (11.b), we can see that this
synthesized texture has mis-arranged lines at two places, which may indicate that the
sampling process was trapped in a local maximum of p(I).
7 The Sampling strategy for textons
In this section, we study a special class of textures formed from identical textons, which
psychophysicists studied extensively. Such texton images are considered as rising from
a different mechanism from other textures in both psychology perception and previous
texture modeling, and the purpose of this section is to demonstrate that they can still
be modeled by the FRAME model, and to show an annealing strategy for computing

Figure

(12.a,b) are two binary (\Gamma1; +1 for black and white pixels) texton images with
circle and cross as the primitives. These two image are simply generated by sequentially
superimposing 128 15\Theta15 masks on a 256\Theta256 lattice using uniform distribution, provided
that the dropping of one mask does not destroy the existing primitives. At the center of
the mask is a circle (or a cross).
For these textures, choosing filters seems easy: we simply select the above 15 \Theta 15
mask as the linear filter. Take the circle texton as an example. By applying the filter to
the circle image and a uniform noise image, we obtain the histograms H obs (solid curve)
and H(x) (dotted curve) plotted in figure (13.a). We observe that there are many isolated
a b

Figure

Two typical texton images. a circle, b cross
peaks in H obs , which set up " potential wells" so that it becomes extremely unlikely to
change a filter response at a certain location from one peak to another by flipping one
pixel at a time.
-22Filter response
lambda
a b

Figure

a. The solid curve is the histogram of the circle image, and the dotted curve is the histogram
of the noise image; b. the estimated -() function in the probability model for the image of circles
To facilitate the matching process, we propose the following heuristics. We smooth
H obs with a Gaussian window G oe , or equivalently run the "heat" diffusion equation on
H obs (x; t) within the interval [x are respectively the minimal and
maximal filter response.
dH obs (x; t)
dt
@x
@x
The boundary conditions help to preserve the total "heat". Obviously, the larger t is, the
smoother the H obs (x; t) will be. Therefore we start from matching H(x) to H obs (x; t) with
a large t (see figure (14.a)), then gradually decrease t and match H(x) to the histograms
shown in figure (14.b,c,d,e,f) sequentially. This process is similar to the simulated annealing
method. The intuitive idea is to set up "bridges" between the peaks in the original
histogram, which encourages the filter response change to the two ends, where the texton
forms, then we gradually destruct these "bridges".
At the end of the process, the estimated - function for the circle texton is shown in
figure (13.b), and the synthesized images are shown in figure (15). We notice that the
cross texton is more difficult to deal with because it has slightly more complex structures
than the circle, and may need more carefully designed filters.
Histogram
Histogram
Histogram
a b c
Histogram
Histogram
Histogram

Figure

14 The diffused histogram H obs smaller and smaller from a to f.

Figure

Two synthesized texton images.
Although there is a close relationship between FRAME and the previous MRF models,
the underlying philosophies are quite different. Traditional MRF approaches favor the
specification of conditional distributions (Besag 1973). For auto-models, p(I(~v) j I(\Gamma~v))
are linear regressions or logistic regressions, so the modeling, inference, and interpretation
can be done in a traditional way. While it is computationally efficient for estimating the
fi coefficients, this method actually limits our imagination for building a general model.
Since the only way to generalize auto-models in the conditional distribution framework
is to either increase neighborhood size, and thus introduce more explanatory variables in
these auto-regressions, or introduce interaction terms (i.e., high order product terms of
the explanatory variables). However, even with a modest neighborhood (e.g., 13 \Theta 13),
the parameter size will be too large for any sensible inference.
Our FRAME model, on the contrary, favors the specification of the joint distribution
and characterizes local interactions by introducing non-linear functions of filter responses.
This is not restricted by the neighborhood size since every filter introduces the same
number of parameters regardless of its size, which enables us to explore structures at large
scales (e.g., 33 \Theta 33 for the fabric texture). Moreover, FRAME can easily incorporate
local interactions at different scales and orientations.
It is also helpful to appreciate the difference between FRAME and the Gibbs distribution
although both focus on the joint distributions. The Gibbs distribution is specified
via potentials of various cliques, and the fact that most physical systems only have pair
potentials (i.e., no potentials from the cliques with more than two pixels) is another reason
why most MRF models for textures are restricted to auto-models. FRAME , on
the other hand, builds potentials from finite-support filters and emphasizes the marginal
distributions of filter responses.
Although it may take a large number of filters to model a wide variety of textures, when
it comes to modeling a certain texture, only a parsimonious set of the most meaningful
filters needs to be selected. This selectivity greatly reduces the parameter size, thus
allows accurate inference and modest computing. So FRAME is like a language: it has an
efficient vocabulary (of filters) capable of describing most entities (textures), and when
it comes to a specific entity, a few of the most meaningful words (filters) can be selected
from the vocabulary for description. This is similar to the visual coding theory (Barlow
et al 1989, Field 1989) which suggests that the sparse coding scheme has advantages over
the compact coding scheme. The former assumes non-Gaussian distributions for f(I),
whereas the latter assumes Gaussian distributions.
Compared to the filtering method, FRAME has the following advantages: 1) solid
statistical modeling, 2) it does not rely on the reversibility or reconstruction of I from
fI (ff) g, and thus the filters can be designed freely. For example, we can use both linear
and nonlinear filters, and the filters can be highly correlated to each other, whereas in the
filtering method, a major concern is whether the filters form a tight frame (Daubechies
1992).
There are various classifications for textures with respect to various attributes, such as
Fourier and non-Fourier corresponding to whether the textures show periodic appearance;
deterministic and stochastic corresponding to whether the textures can be characterized
by some primitives and placement rules; and macro- and micro-textures in relation to
the scales of local structures. FRAME erases these artificial boundaries and characterizes
them in a unified model with different filters and parameter values. It has been well
recognized that the traditional MRF models, as special cases of FRAME, can be used
to model stochastic, non-Fourier micro-textures. From the textures we synthesized, it
is evident that FRAME is also capable of modeling periodic and deterministic textures
(fabric and pulses), textures with large scale elements (fur and cheetah blob), and textures
with distinguishable textons (circles and cross bars), thus it realizes the full potential of
MRF models.
But the FRAME model is computationally very expensive. The computational complexity
of the FRAME model comes from two major aspects. I). When bigger filters are
adopted to characterize low resolution features, the computational cost will increase proportionally
with the size of the filter window. II). The marginal distributions
are estimated from sampled images, which requires long iterations for high accuracy of
estimation. One promising way to reduce the computational cost is to combine the pyramid
representation with the pseudo-likelihood estimation (Besag 1977). The former cuts
the size of low resolution filters by putting them at the high levels of the pyramid as did
in (Popat and Picard 1993), and the latter approximates E p [H (ff) ] by pseudo-likelihood
and thus avoid the sampling process. But this method shall not be studied in this paper.
No doubt many textures will not be easy to model, for example some human synthesized
textures, such as textures on oriental rugs and clothes. It seems that the synthesis
of such textures requires far more sophisticated or high-level features than those we used
in this paper, and these high-level features may correspond to high-level visual process.
At the same time, many theoretical issues remain yet to be fully understood, for example,
the convergence properties of the sampling process and the definition of the best sampling
procedures; the relationship between the sampling process and the physical process which
forms the textures of nature and so on; and how to apply this texture model to the image
segmentation problem (Zhu and Yuille 1996). It is our hope that this work will simulate
future research efforts in this direction.


Appendix

. Filter pursuit and minimax entropy.
This appendix briefly demonstrates the relationship between the filter pursuit method and the
principle (Zhu, Wu and Mumford 1996).
Let p(I;   K ; SK ) be the maximum entropy distribution obtained at step k (see equation (18)),
since our goal is to estimate the underlying distribution f(I), the goodness of p(I;   K ; SK ) can be
measured by the Kullback-Leibler distance between p(I;   K ; SK ) and f(I) (Kullback and Leibler
Z
f(I) log f(I)
it can be shown that
As entropy(f(I)) is fixed, to minimize KL(f; p(I;   K ; SK )) we need to choose SK such that
has the minimum entropy, while given the selected filter set SK , p(I;   K ; SK ) is
computed by maximizing entropy(p(I)). In other words, for a fixed filter number K, the best set
of filters is chosen by
p2\Omega K
where\Omega K is defined as equation (14). We call equation (29) the minimax entropy principle (Zhu,
Wu, Mumford 1996).
A stepwise greedy algorithm to minimize the entropy proceeds as the following. At step k
suppose we choose F (fi) , and obtain the ME distribution p(I;
f (ff) for fi. Then the goodness of F (fi) is measured by the decrease of the Kullback-Leibler
distance KL(f(I); p(I;   k It can be shown that
where M is a covariance matrix of H (fi) , for details see (Zhu,Wu, Mumford 1996). Equation (31)
measures a distance between f (fi) and E p(I;  k;Sk ) [H (fi) ] in terms of variance, and therefore suggests
a new form for the distance D(E p(I;  k ;Sk equation (26), and this new form
emphasizes the tails of the marginal distribution where important texture features lies, but the
computational complexity is higher than the L 1 -norm distance. So far we have shown the filter
selection in algorithm 3 is closely related to a minimax entropy principle.

Acknowledgments

This work was supported by the NSF grant DMS-91-21266 to David Mumford. The
second author was supported by a grant to D.B. Rubin.



--R

"Finding minimum entropy codes."
"Theories of visual texture perception."
"Spatial interaction and the statistical analysis of lattice systems (with discussion)."
"Efficiency of pseudolikelihood estimation for simple Gaussian fields."
"Orthogonal distribution analysis: a new approach to the study of texture perception."
"Entropy based algorithms for best basis selection."
"Markov random field texture models."
Ten lectures on wavelets.

"On the Statistics of Vision: the Julesz Conjecture"
"Ideal de-noising in an orthonormal basis chosen from a libary of bases. "
"Relations between the statistics of natural images and the response properties of cortical cells"
"Theory of communication."
"Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images."
"Markov random field image models and their applications to computer vision."
"Annealing Markov chain Monto Carlo with applications to ancestral inference."
"Statistics and structural approach to texture."
"Pyramid-based texture analysis/synthesis."
"Unsupervised texture segmentation using Gabor filters."
"Information theory and statistical mechanics"
"Visual pattern discrimination."
"On information and sufficiency"
Image representation using 2D Gabor wavelets.
"Mumtiresolution approximations and wavelet orthonormal bases of L 2 (R)."
Texture classification and segmentation using multiresolution simultaneous autoregressive models.
"Time series models for texture synthesis."
"Novel cluster-based probability model for texture synthesis, classification, and compression."
"Multidimensional Markov chain models for image tex- tures."
"Spatial-frequency organization in primate striate cortex."

"Object and texture classification using higher order statistics."
Image Analysis
"Reaction-diffusion textures."
"Spectral estimation for random fields with applications to Markov modeling and texture Classification."
"Region Competition: unifying snakes, region growing, and Bayes/MDL for multi-band image segmentation"
"Minimax Entropy Principle and its applications"
--TR

--CTR
Dmitri Bitouk , Michael I. Miller , Laurent Younes, Clutter Invariant ATR, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.5, p.817-821, May 2005
Katy Streso , Francesco Lagona, Hidden Markov random field and frame modelling for TCA image analysis, Proceedings of the 24th IASTED international conference on Signal processing, pattern recognition, and applications, p.310-315, February 15-17, 2006, Innsbruck, Austria
Krishnamoorthy Sivakumar , John Goutsias, Morphologically Constrained GRFs: Applications to Texture Synthesis and Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.21 n.2, p.99-113, February 1999
Fei Wu , Changshui Zhang , Jingrui He, An evolutionary system for near-regular texture synthesis, Pattern Recognition, v.40 n.8, p.2271-2282, August, 2007
Alexey Zalesny , Dominik Auf der Maur , Luc Van Gool, Composite textures: emulating building materials and vegetation for 3D models, Proceedings of the 2001 conference on Virtual reality, archeology, and cultural heritage, November 28-30, 2001, Glyfada, Greece
Julian J. McAuley , Tibrio S. Caetano , Alex J. Smola , Matthias O. Franz, Learning high-order MRF priors of color images, Proceedings of the 23rd international conference on Machine learning, p.617-624, June 25-29, 2006, Pittsburgh, Pennsylvania
O. Stahlhut, Extending natural textures with multi-scale synthesis, Graphical Models, v.67 n.6, p.496-517, November 2005
Wen , Xiao-Qing Ding, Visual similarity based document layout analysis, Journal of Computer Science and Technology, v.21 n.3, p.459-465, May 2006
Jeremy S. De Bonet, Multiresolution sampling procedure for analysis and synthesis of texture images, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.361-368, August 1997
Kwang In Kim , Keechul Jung , Se Hyun Park , Hang Joon Kim, Support Vector Machines for Texture Classification, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.11, p.1542-1550, November 2002
Uri Lipowezky, Groves decipherment from space photos using prototype matching, Pattern Recognition Letters, v.25 n.13, p.1479-1489, 1 October 2004
R. Fablet , P. Bouthemy, Motion Recognition Using Nonparametric Image Motion Models Estimated from Temporal and Multiscale Cooccurrence Statistics, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.12, p.1619-1624, December
Chung-Ming Chen , Henry Horng-Shing Lu , Yao-Lin Chen, A discrete region competition approach incorporating weak edge enhancement for ultrasound image segmentation, Pattern Recognition Letters, v.24 n.4-5, p.693-704, February
Feng Dong , Gordon Clapworthy , Hai Lin, Technical Section: Pseudo surface-texture synthesis, Computers and Graphics, v.31 n.2, p.252-261, April, 2007
Joshua Gluckman, Visually Distinct Patterns with Matching Subband Statistics, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.2, p.252-264, February 2005
Qiang Chen , Jian Luo , Pheng Ann Heng , Xia De-shen, Fast and active texture segmentation based on orientation and local variance, Journal of Visual Communication and Image Representation, v.18 n.2, p.119-129, April, 2007
Alexei A. Efros , William T. Freeman, Image quilting for texture synthesis and transfer, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.341-346, August 2001
K. -O. Cheng , N. -F. Law , W. -C. Siu, Multiscale directional filter bank with applications to structured and random texture retrieval, Pattern Recognition, v.40 n.4, p.1182-1194, April, 2007
Luminita A. Vese , Stanley J. Osher, Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing, Journal of Scientific Computing, v.19 n.1-3, p.553-572, December
Fast texture synthesis using tree-structured vector quantization, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.479-488, July 2000
Xin Li, Image resolution enhancement via data-driven parametric models in the wavelet space, Journal on Image and Video Processing, v.2007 n.1, p.12-12, January 2007
Alan L. Yuille , James M. Coughlan, Fundamental Limits of Bayesian Inference: Order Parameters and Phase Transitions for Road Tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.2, p.160-173, February 2000
Aaron D. Lanterman , Ulf Grenander , Michael I. Miller, Bayesian Segmentation via Asymptotic Partition Functions, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.4, p.337-347, April 2000
Zhuowen Tu , Song-Chun Zhu, Image Segmentation by Data-Driven Markov Chain Monte Carlo, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.5, p.657-673, May 2002
Jun-Hong Cui , Michalis Faloutsos , Dario Maggiorini , Mario Gerla , Khaled Boussetta, Measuring and modelling the group mmbership in the internet, Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement, October 27-29, 2003, Miami Beach, FL, USA
Andrew Nealen , Marc Alexa, Hybrid texture synthesis, Proceedings of the 14th Eurographics workshop on Rendering, June 25-27, 2003, Leuven, Belgium
Lin Liang , Ce Liu , Ying-Qing Xu , Baining Guo , Heung-Yeung Shum, Real-time texture synthesis by patch-based sampling, ACM Transactions on Graphics (TOG), v.20 n.3, p.127-150, July 2001
Jingdan Zhang , Kun Zhou , Luiz Velho , Baining Guo , Heung-Yeung Shum, Synthesis of progressively-variant textures on arbitrary surfaces, ACM Transactions on Graphics (TOG), v.22 n.3, July
Ann B. Lee , David Mumford , Jinggang Huang, Occlusion Models for Natural Images: A Statistical Study of a Scale-Invariant Dead Leaves Model, International Journal of Computer Vision, v.41 n.1-2, p.35-59, January-February 2001
Ronan Fablet , Patrick Bouthemy, Non-Parametric Motion Activity Analysis for Statistical Retrieval with Partial Query, Journal of Mathematical Imaging and Vision, v.14 n.3, p.257-270, May 2001
Song Chun Zhu , Xiu Wen Liu , Ying Nian Wu, Exploring Texture Ensembles by Efficient Markov Chain Monte Carlo-Toward a 'Trichromacy' Theory of Texture, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.6, p.554-569, June 2000
James Coughlan , Huiying Shen, Dynamic quantization for belief propagation in sparse spaces, Computer Vision and Image Understanding, v.106 n.1, p.47-58, April, 2007
Simon Osindero , Max Welling , Geoffrey E. Hinton, Topographic Product Models Applied to Natural Scene Statistics, Neural Computation, v.18 n.2, p.381-414, February 2006
John MacCormick , Andrew Blake, A Probabilistic Exclusion Principle for Tracking Multiple Objects, International Journal of Computer Vision, v.39 n.1, p.57-71, Aug. 2000
Xuejie Qin , Yee-Hong Yang, Estimating parameters for procedural texturing by genetic algorithms, Graphical Models, v.64 n.1, p.19-39, January 2002
Wojciech Matusik , Matthias Zwicker , Frdo Durand, Texture design using a simplicial complex of morphable textures, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
Antonio Torralba , Aude Oliva, Depth Estimation from Image Structure, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.9, p.1226-1238, September 2002
Ziv Bar-Joseph , Ran El-Yaniv , Dani Lischinski , Michael Werman, Texture Mixing and Texture Movie Synthesis Using Statistical Learning, IEEE Transactions on Visualization and Computer Graphics, v.7 n.2, p.120-135, April 2001
Joachim Hornegger , Heinrich Niemann, Probabilistic Modeling and Recognition of 3-D Objects, International Journal of Computer Vision, v.39 n.3, p.229-251, Sept./Oct. 2000
Luminita A. Vese , Stanley J. Osher, Image Denoising and Decomposition with Total Variation Minimization and Oscillatory Functions, Journal of Mathematical Imaging and Vision, v.20 n.1-2, p.7-18, January-March 2004
J. Sullivan , A. Blake , M. Isard , J. MacCormick, Bayesian Object Localisation in Images, International Journal of Computer Vision, v.44 n.2, p.111-135, September 2001
Gabriele Gorla , Victoria Interrante , Guillermo Sapiro, Texture Synthesis for 3D Shape Representation, IEEE Transactions on Visualization and Computer Graphics, v.9 n.4, p.512-524, October
Xinguo Liu , Yizhou Yu , Heung-Yeung Shum, Synthesizing bidirectional texture functions for real-world surfaces, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.97-106, August 2001
Ce Liu , Heung-Yeung Shum , William T. Freeman, Face Hallucination: Theory and Practice, International Journal of Computer Vision, v.75 n.1, p.115-134, October   2007
Thomas Leung , Jitendra Malik, Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons, International Journal of Computer Vision, v.43 n.1, p.29-44, June 2001
Gozde Unal , Anthony Yezzi , Hamid Krim, Information-Theoretic Active Polygons for Unsupervised Texture Segmentation, International Journal of Computer Vision, v.62 n.3, p.199-220, May 2005
Daniel C. Alexander , Bernard F. Buxton, Statistical Modeling of Colour Data, International Journal of Computer Vision, v.44 n.2, p.87-109, September 2001
Matthias Heiler , Christoph Schnrr, Natural Image Statistics for Natural Image Segmentation, International Journal of Computer Vision, v.63 n.1, p.5-19, June      2005
Aaron Hertzmann , Charles E. Jacobs , Nuria Oliver , Brian Curless , David H. Salesin, Image analogies, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.327-340, August 2001
Stefan Roth , Michael J. Black, On the Spatial Statistics of Optical Flow, International Journal of Computer Vision, v.74 n.1, p.33-50, August    2007
Xiaofeng Ren , Charless C. Fowlkes , Jitendra Malik, Learning Probabilistic Models for Contour Completion in Natural Images, International Journal of Computer Vision, v.77 n.1-3, p.47-63, May       2008
D. A. Forsyth , J. Haddon , S. Ioffe, The Joy of Sampling, International Journal of Computer Vision, v.41 n.1-2, p.109-134, January-February 2001
Nikos Paragios , Rachid Deriche, Geodesic Active Regions and Level Set Methods for Supervised Texture Segmentation, International Journal of Computer Vision, v.46 n.3, p.223-247, February-March 2002
Daniel Cremers , Mikael Rousson , Rachid Deriche, A Review of Statistical Approaches to Level Set Segmentation: Integrating Color, Texture, Motion and Shape, International Journal of Computer Vision, v.72 n.2, p.195-215, April 2007
texture analysis and synthesis;minimax entropy;texture modeling;markov random field;feature pursuit;visual learning;maximum entropy