New multivariate product density estimators.
Let X be an Rd-valued random variable with unknown density f. Let X1 ...., Xn be i.i.d, random variables drawn from f. The objective is to estimate f(x), where We study the pointwise convergence of two new density estimates, the Hilbert product kernel estimate d!/n i=1nj=1d1/2log n|xjXij|, where and the Hilbert k-nearest neighbor estimate k(d-1)!/2dnlogd-1(n/(k(d-1)!)) j=1d|xj-X(k)j|, where and X(k) is the kth nearest neighbor of x when points are ordered by increasing values of the product j=1d |xj-X(k)j|, and . The auxiliary results needed permit us to formulate universal consistency results (pointwise and in L1) for product kernel estimates with different window widths for each coordinate, and for rectangular partitioning and tree estimates. In particular, we show that locally adapted smoothing factors for product kernel estimates may make the kernel estimate inconsistent even under standard conditions on the bandwidths.
Introduction
.
The objective of this note is to study two new multivariate density estimates that avoid the messy
problem of smoothing factor selection (in one case, at least), are invariant to a#ne transformations of
the coordinates, and provide easy means to jointly estimate all 2 d
marginal densities of a density f
on IR d . As a by-product, we will be able to study the universal consistency of product kernel estimates
and of rectangular partition estimates.
independent observations of an IR d -valued random vector X with unknown
density f . The classical kernel estimate of f is
where h > 0 is a smoothing factor depending upon n, K is an absolutely integrable function (the kernel),
and K h 1956). Observe that for the kernel
the smoothing factor h is canceled and we obtain
One may wonder what happens in this situation, now that the smoothing factor is absent. This problem
is dealt with by Devroye and Krzy -
zak (1998), who showed the following.
Theorem 1. The Hilbert estimate
fn (x) =V d n log n
where V d is the volume of the unit ball in IR d , is weakly consistent at almost all x, that is,
in probability at almost all x.
We use the name Hilbert estimate because of the related Hilbert integral with a similar ker-
nel. The Hilbert estimate is not invariant under a#ne transformations of the coordinates. That is, in
transparent notation, where fn (x; Y ) denotes the density estimate at x # IR d given data Y # (IR d
where b+aY denotes the sample of size n in which each i-th coordinate of each observation is transformed
by In kernel density estimation, the invariance may be obtained if one uses product kernels.
However, this causes additional problems as each component kernel requires its own smoothing factor. If
we apply that principle to the Hilbert estimate, we obtain the product Hilbert estimate
d!
d
Y
log n|x i -X ji |
.
With this estimate, which once again has no smoothing factor, the invariance mentioned above follows
readily. However, the weak pointwise consistency does not hold in IR d for d > 1 for all densities. Counterexamples
will be provided below. In fact, for the product kernel estimate
d
are positive smoothing parameters, it is known that if h
h, h # 0 and
nh d
#, then fn # f in probability at almost all x when K is a bounded compact support density
(Devroye and Gy-orfi, 1985), and R |f almost surely for any K with R
However, it is less known that if we allow the individual smoothing factors to tend to zero at di#erent
rates and are allowed to depend upon x, this universality is lost! The natural conditions on h i would
seem to be
lim
and
lim
d
Y
Within these conditions, and with K the uniform density on [-1, 1], there exist densities f for which
almost everywhere in probability. It turns out that a su#cient condition on f for almost
everywhere pointwise convergence is R f log d-1 (f The proof
and the counterexamples follow in section 3. The paper then explores su#cient conditions for various
types of convergence of the estimates, and discusses in this context the Jessen-Marcienkiewicz-Zygmund
condition. Consistency is proved for the multivariate Hilbert product estimate, the product nearest
neighbor estimate (which hasa nearest neighbor ranking that is invariant under scaling of the axes, a very
desirable feature in high-dimensional data collection!), the ordinary product kernel estimate and finally,
tree-based and rectangular partitioning estimates.
2. Sums of products of inverse uniforms.
be the product of d i.i.d. uniform [0, 1] random variables. Its density is given
by
log d-1 (1/z)
(d - 1)!
Let Z 1 , . , Zn be i.i.d. and distributed as Z. Then we have:
Lemma S1.n log d n
in probability.
Proof. The result follows from Theorem 2 of Rogozin, 1971 and Theorem 8.8.1 of Bingham, Goldie and
Teugels, 1987 about stability of sums of i.i.d. random variables
function F , then Sn/an # 1 in probability if R x
is a slowly varying function and
an is chosen such that l(a n )/a n # 1/n. Take
We have R x
log d x/d! and so we can take log d x/d! and an = n(log d n)/d!. Indeed,
an
(n(log d
n)/d!))/d!
(n log d
3. The Saks rarity theorem and its implications.
To understand the reasons for defining conditions on f , it helps to understand why we have to do
so. The reasons go back to the theory of di#erentiation (de Guzman, 1981, is a good reference). Consider
a function f on IR d , together with a collection B of bounded measurable sets with the property that for
every x # IR d , there exists a sequence , . from B with diameters (written diam(.)) decreasing to
zero and such that x # . Such a collection is called a basis. The collection of sets in B containing x
is denoted by B(x). We define upper and lower derivatives of f by
and
respectively (de Guzman, 1981, p. 105). we say that di#erentiates f if D+ (f, almost
everywhere (x). For example, it is known that if B is the collection of all balls or all hypercubes, then
di#erentiates all integrable functions f . This is a form of the celebrated Lebesgue density theorem,
and is at the basis of the pointwise convergence properties of kernel estimates and indeed most density
estimates. Let B 2 denote the interval basis, that is, the collection of all products of d finite intervals
containing at least two points. This is the collection of all bounded rectangles of positive measure aligned
with the axes. We will require the following result:
Lemma A1. (the jessen-marcinkiewicz-zygmund theorem, 1935). B 2 di#erentiates L(1+log
that is, all functions f on IR d
for which
While this includes most densities f , there are indeed exceptions. A good account of these is
chapter 7 of de Guzman (1981, p. 167). First of all, according to the Saks rarity theorem (Saks, 1935),
there exists a nonnegative function f on IR 2 such that D+ (f, almost everywhere (x) (with respect
to B 2 ). Later, Marstrand (1977) found an f with this property that works for all orientations of the axes
(each orientation of the axes has a di#erent collection B 2 ). El Helou (1978) found an f with the latter
property and in addition
Z |f | log a (1
for all a # (0, 1). Thus, with respect to B 2 , the logarithmic condition on f is nearly necessary.
Consider now a standard kernel estimate on IR 2 with product kernel
4 I |x1 |#1 I |x2 |#1 . If
we have di#erent smoothing factors for each coordinate, the form is
I |x1 -X j1 |#h1 I |x2 -X j2 |#h2
It is easy to see that
But by the result mentioned above, there exists a density f # 0<a<1 L(1 a such that at almost
all x, there exist h #, such that Efn (x) #. Note that the
results do not imply this when h 1 and h 2 are not allowed to depend upon x. As the variance of fn is
O(Efn (x)/(nh 1 h 2 )), it is easy to see that if nh 1 h 2 #, then fn (x)/Efn (x) # 1 in probability at almost
all x, and thus, fn (x) # in probability at almost all x. Therefore, if one adapts the smoothing factors
to x, it is not true any longer that kernel estimates are pointwise consistent for all densities!
We turn finally to the basis B 3 of all rectangles in IR 2
(rotated with respect to all possible
orientations). Here the situation is extremely volatile (de Guzman, 1981, p. 224), as B 3 does not even
di#erentiate the characteristic functions of bounded measurable sets. The counterexamples on which
di#erentiability fails include densities is a bounded measurable set of area 1/c that is a
specially selected subset of the Nikodym set N on [0, 1] 2 (N has measure one, but for each x # N , there
exists a line l(x) through x such that l(x) # {x}). Take such an f . The implication is that there
exist inconsistent kernel estimates of the rotation kind: for almost every x, there exist smoothing factors
#, the standard conditions on smoothing
and orthonormal rotation matrices A(n, x) such that the kernel estimate
K(A(n, x)(x -X j
at almost all x, where as before
4 I |x1 |#1 I |x2 |#1 . Thus, adaptation to both x and allowing adaptive
rotations makes kernel estimates potentially inconsistent even on bounded densities with compact support.
In the remainder of the paper, a rectangle is a set from B 2 , and not from B 3 .
4. Weak pointwise consistency of the multivariate Hilbert product kernel estimate.
In this section, we prove the main consistency theorem:
Theorem 2. Assume that f is a density with R f log d-1
(1+f) < #, and for which R g log s-1
for all its marginal densities g, where s denotes the dimension of the domain of g. Then the multivariate
Hilbert product kernel estimate is weakly pointwise consistent almost everywhere: at almost all x,
in probability.
Proof. Let M denote the space of all d - d diagonal matrices with diagonal elements 1 or -1. Clearly,
be vectors from IR d . For fixed x # IR d , define the
flipped density
where y # x means that y i # x i for all i. Observe in particular that f #
and that f # is a bona-
fide density with support on the positive quadrant with origin x. By the Jessen-Marcinkiewicz-Zygmund
theorem (Lemma A1), we have, if B 2 denotes the interval basis on IR d , for almost all x,
lim
This implies that for almost all x,
lim
x (z) dz
x (x) .
Here [x, y] denotes the rectangle Q d
To see this, note the following: let B be of the form [x, y].
Then as diam(B) # 0 (while varying y, not x),
M#M
Z MB
x
dz .
This little excursion allows us to study the behavior of f #
x instead of f . Interestingly, as
we see that the same flipping applied to our estimator fn leaves
fn unaltered. Thus, to show that fn # f at almost all x is equivalent to showing that fn (x) # f #
x (x)/2 d
at almost all x.
The remainder of the proof requires the introduction of the marginal densities fS and f #
x,S , where
thus, fS is the marginal density of f with respect to all components whose index is in
S, and similarly, f #
x,S is the marginal density of f #
x with respect to all components whose index is in S.
We call x a jmz point (after Jessen, Marcinkiewicz and Zygmund) if for all S #,
lim
Here xS is the |S|-dimensional vector composed of components of x whose index is in S, and B is the
collection of rectangles in this |S|-dimensional space. By Lemma A1, almost all points are jmz points.
Fix such an x for the remainder of the proof. For # > 0, we can thus find # > 0 such that simultaneously
for all S #, for all yS # xS
R [xS ,yS
x,S (z) dz
x,S
x,S
Also, recall that f #
x,S
Lemma B1. Let be a jmz point of f with f(x) > 0. Let # and # be as above. Let
be a random vector with density f # . Let S # {1, . , d} be a nonempty set of indices.
Let A be the event that x i < conditional on A, there exist i.i.d. uniform
[0, 1] random variables U i such that
where # denotes stochastic domination.
Proof. We assume without loss of generality that
of f #
x and f #
S instead of f #
x,S . Conditional on A, X has density f # /p, where
Note that
S (0)# d .
Furthermore, if F is the multivariate distribution function for the conditional X , then as x is a jmz point,
and therefore, as it is well-known that F (X 1 ,
which can be seen by applying the
probability integral transform to each conditional distribution function in the conditional decomposition
of F , we see that
d
Y
We now return to the proof of our theorem, where # and # remain as defined earlier. We enlarge
the data by considering an infinite i.i.d. sequence Y 1 , Y 2 , ., all distributed as X . Let B(Y
be the collection of indices i in Y ji with Y ji # [x i , point. Let
S #, and let T be the collection of the first n indices j with B(Y
by Lemma B1, where U 1 , . , U d are as in Lemma B1. By Lemma S1, the right-hand-side is in probability
asymptotic to
Thus, the contribution of all Y j , j # T , when |S| < d, is asymptotically negligible. Assume first f #
Then, taking d, we can no longer a#ord to artificially increase the data size as we did above. Thus, let
T now be those j # n for which |B(Y j d. Note that T is binomial (n, p), where
By the independence of T and the U i 's in Lemma B1, it is easy to see that
and the right-hand-side is in probability asymptotic to
(np)
x (x)# d log d
A similar weak lower bound is obtained, and by letting # 0, we obtain the sought result. The contribution
of those terms in the density estimate with |S| = d is o(n log d n) when (as seen from the last
chain of inequalities as well), just as with all S of size less than d. Thus, the proof of theorem 2 is
complete.
5. Lack of strong convergence.
For all f , it is true that at almost all x with f(x) > 0, the Hilbert product kernel estimates
cannot possibly converge to f in a strong sense. Rather than to prove the full-blown universal theorem,
we restrict ourselves to the uniform density on the real line and recall the following result from Devroye
and Krzy -
zak (1998), which is applicable as for the Hilbert product kernel estimate coincides with
the standard Hilbert kernel estimate.
Theorem 3. Let f be the uniform density on [0, 1]. Then, for any x # [0, 1], P{fn (x) # log log n
so that there is no strong convergence at any point in the support.
The poor rate of convergence of the estimate is best seen by considering points outside the support
of f . If x is at least distance s away from the support of f , then fn (x) # c/(s d (log n) d ) for some constant
c only depending upon k and d.
6. A product nearest neighbor estimate.
The k-nearest neighbor density estimate of Fix and Hodges (1951) and Loftsgaarden and Que-
senberry (1965) is
where X (k,x) is the k-th nearest neighbor of x among Its properties are well-understood
(Moore and Yackel, 1977; Devroye and Wagner, 1977; Mack, 1980; Bhattacharya and Mack, 1987; Mack
and Rosenblatt, 1979). For example, at almost all x, we have g k,n (x) # f(x) as n # if
k #. The k-nearest neighbor density is not scale-invariant because the relative order of the distances
when the coordinates are linearly scaled. To remedy this, we introduce the
product k-nearest neighbor density estimate
log d-1 (n/k(d - 1)!)2 d
is a permutation of X 1 , . , X d according to increasing values of
This permutation is invariant under linear transformations of the coordinate
axes (but not rotations). Interestingly, this product estimate has not been considered before except in
the trivial case we obtain the standard univariate k-th nearest neighbor estimate. As the
choice of scale is a perpetual cause of concern in estimation, the product k-th nearest neighbor estimate
should be particularly useful. We will prove its weak consistency:
Theorem 4. If k # such that k #, k/ log n # 0, and if R f log d-1 (f +1) < #, and if R g log d-1 (g+
lower-dimensional marginals of f , then at almost all x (that is, at all jmz points for f and
all lower-dimensional marginals f ), g k,n (x) # f(x) in probability.
Proof of Theorem 4. We only sketch a rough outline. Mimicking the proof of Theorem 2, we note first
that we may wish to consider the flipped density at x, which is 2 d f(y), with y # x, all coordinates of y are
at least equal to those for x. We will consider a small square of size # in each coordinate with bottom lower
vertex at x. We will show that the k-th nearest neighbor of x is with probability tending to one in this
square. Indeed, if Y +(Y 1 , . , Y d ) has density f , and x is a jmz point, then Q d
approximately
distributed as
where the U i 's are i.i.d. uniform [0, 1] random variables, and f #
As
U i has density log d-1 (1/z)/(d - 1)!, z > 0, and distribution function # z log d-1 (1/z)/(d - 1)! as
z # 0, we see that the order statistics of a sample of size n drawn from Q d
are approximately
distributed as F inv (1/n)/f # (x), F inv (2/n)/f # (x), and so forth, where F inv is the inverse distribution
function of
good approximation is F inv (u) # u(d - 1)!/(log d-1 (1/(u(d - 1)!))). Thus, the
k-th nearest neighbor has a value
concentrated in probability about
(d - 1)!k/n
. (D)
The concentration follows from k #. We only need to show that the probability that the k-th nearest
neighbor is in the square tends to one. To this end, we show that the nearest neighbor among all points
that have m coordinates outside the square and d - m coordinates in the square is at distance (always
measured by Q d
asymptotically much larger than (D). Without loss of generality, fix the
first m # 1 (m < d) coordinates. If x is also a jmz point for the marginal density for the last d - m
coordinates, then given that the first m coordinates are outside the square and the remaining ones inside
(and assuming that # is small enough), the nearest neighbor distance is asymptotically of the order of
1/(n log d-m-1 n) and thus it is improbable that the k-th nearest neighbor point can have any coordinate
outside the small square. This concludes the sketch of the proof.
7. Product kernel estimates.
In this section, we consider product kernel estimates defined by
d
are univariate kernels with R K 1. The smoothing factors
h n,i are for now deterministic sequences. Product kernel estimates are of interest because they o#er scale
invariance if the h n,i 's are proportional to scale (e.g., make h n,i proportional to a weighted sum of the
n). This may introduce big di#erences between the h n,i 's. In
fact, such di#erences may be desirable in situations like this one: let f be the product of
two univariate densities, a smooth one and a jagged one. For each density, one may want to pick di#erent
smoothing factors, and even di#erent orders for the kernel. In those cases, h n,1 and h n,2 may tend to
zero at di#erent rates. As each density is locally a product density (as it resembles a uniform density), it
is really important to consider product kernels with d individually picked smoothing parameters.
It is interesting that the literature o#ers little help with respect to the universal consistency
properties of these estimates with respect to pointwise or L 1 convergence. The "natural" conditions on
the h n,i 's would appear to be
lim
and
lim
d
Y
In this section, we prove two basic consistency results, which we have not been able to find in the vast
literature.
Theorem 5. Let fn be the product kernel estimate, and let each component kernel K i be absolutely
integrable. Then under the natural conditions (1) and (2),
lim
for all f .
Theorem 6. Let fn be the product kernel estimate, and let all kernels K i be bounded, of compact
support, and Riemann approximable, that is, in the L# sense, each K i is in the closure of the space of
functions that are finite weighted sums of indicators of finite intervals. Then, under the natural conditions
(1) and (2),
lim
at almost all x provided that R f log d-1 (1
Note. The conditions on the kernels are satisfied by kernels K i that are continuous, bounded and of
compact support. With a bit of e#ort, we can extend Theorem 6 to include kernels K i with K i
O(1/|x|) as |x| #
Perhaps the easiest proof of Theorem 5, and the most transparent one, uses the embedding
device from Devroye (1985) (where the embedding is used to handle the L 1 consistency of variable kernel
estimates), which may be summarized in the following Lemma:
Lemma B2. Let f and g be two densities on IR d , and let be a density estimate
based on an i.i.d. samples of size n drawn from f . Assume that
sup
for some constant C. Then, if g n is the density estimate based upon an i.i.d. sample of size n drawn from
g, we have
Proof. Consider a uniform Poisson point process on IR d
- [0, #) 2 . Let (U, V, T ) be a typical point in
this process. Keep only those points with V < f(U) or V < g(U ). For T < t, there is almost surely a
finite number of such points (with a Poisson (2t) distribution), so that we may order the points according
to increasing values of T , obtaining (U 1 , be the first n values of U j for
which Yn be the first n values of U j for which V j < g(U j ). Now, throw away
the Poisson point process, which was only needed to couple the two samples. Interestingly, X 1 , . ,
i.i.d. and drawn from f and Y 1 , . , Yn is i.i.d. and drawn from g. Also, for every i,
R max(f, g)
dx
R max(f, g)
Each one of the triples (U generates a common point in both samples,
and thus, the number of common points is stochastically greater than a binomial (n, p) random variable.
The number of points in one sample not seen in the other sample is not more than a binomial (n, 1 - p)
random variable, and its expected value does not exceed n(1 -
We may assume that fn and gn are based on the (coupled) X j and Y j samples respectively.
Then
as R |f n - gn | # C(n-N)
n by applying the triangle inequality n -N times.
Proof of Theorem 5. We first verify that Lemma B2 applies to the kernel estimate with kernel K.
Indeed, if
#n
Z
d
Y
Z
d
Y
=n
d
Y
so that Lemma B2 applies with
|. If all kernels K i are nonnegative, then simply, 2.
By Lemma 2 then, it su#ces to prove theorem 5 for all continuous densities g of compact support, as those
densities are dense in the L 1 space of all densities. Indeed, pick g continuous and of compact support
such that R |f - g| < #. Let fn and g n denote the product kernel estimate, but based on samples drawn
from densities f and g respectively. Applying Lemma B2, we have
As # is arbitrary, it su#ces therefore to prove Theorem 5 for all such g.
Since each K i is measurable and absolutely integrable, we may approximate it in the L 1 sense
by a sum of indicator functions of intervals. Thus, for # > 0, we find a finite number of intervals [a ij
and coe#cients k ij such that
where
The L i can even be picked such that R L also that R |L i | < R |K #, and that R |K i | # 1 (as
1). Define the constant
d
Y
But if fn and g n are two product kernel estimates with the same data but di#erent product kernels K i ,
d
Y
d
Y
d
Y
d
Y
d
Y
d
Y i=3
Y
d
Y
d
Y i=2
d
Y i=3
Y
#D .
Again, by the arbitrary nature of #, it su#ces to consider kernels that are products of finite sums of
weighted indicator functions of intervals. Let N be a bound on the number of indicators for any of the
component kernels. It is easy to see then, by forming the Cartesian grid of these intervals, that such an
estimate is equivalent to a kernel estimate with kernel
is the shorthand notation for a rectangle of IR d with vertices a j and b j , and # j is a real
number. Also, R denote the volume of [a j , b j ]. Introduce the notation
and note that M j is a bona fide kernel with integral one. Then we have, letting fn
denote the kernel estimate with kernel L, and f a continuous density of compact support, as
where f nj is the kernel estimate with kernel M j . The upper bound tends to zero in the mean if each
individual term tends to zero. Thus, it su#ces to prove the Theorem for kernels that are indicator
functions of rectangles.
But by Theorem 6, we have E|f nj - f | # 0 for all bounded f at almost all x. But then
by dominated convergence. This concludes the proof of Theorem 5.
Proof of Theorem 6. Let x be a jmz point for f . Let the kernel be proportional to an
indicator a bounded rectangle necessarily containing the origin, where
the vertices of R. By varying each coordinate in turn, we see
that
d
where z i are points of IR d and s Denote the volume of [0, z i ] by p i . Let Mn denote the d - d
diagonal matrix with elements h n,1 , . , h n,d on the diagonal. By Lemma A1 and condition (1),
and thus
d
because R K = 1. We conclude that Efn f(x), and that
#(R)n Q d
and therefore,
R x+MnR f
#(R)n
x is a jmz point, then
by Lemma A1 and (2).
We now turn to more general kernels. It will take some work to generalize the above results
to a reasonably big class of kernels. It is easy to verify from the last chain of inequalities that under
condition (2), for any bounded kernel, Var{fn (x)} # 0 at points x at which Efn (x) # f(x). Thus,
Theorem 6 is valid for all bounded kernels for which (1) implies that Efn (x) # f(x) at almost all x.
We find for each # > 0 a finite collection of rectangles R i and constants # i such
that
sup
x
where s > 0 is a large positive integer. From this, by integration, we note that
Then
|Efn
where in the last step, we used (1) and the first part of the proof for kernels that are indicators of
rectangles. Since # was arbitrary, the proof is complete.
8. Tree-based and rectangular partitioning density estimates.
Let A 1 , A 2 , . be a sequence of partitions of IR d into rectangles (which do not have to be open or
closed). For x, let An (x) denote the rectangle in An to which x belongs. Then the partitioning estimate
of f is given by
where #(.) is Lebesgue measure, and #(.) denotes the number of data points falling in a set. We assume
for now that the sequence of partitions is picked before the data are collected. The present estimates
contain all standard histogram estimates and indeed most tree-based density estimates. The purpose
of this section is to discuss its universal consistency, pointwise and in L 1 . For L 1 consistency, there is
a rather general theorem by Abou-Jaoude (1976a, 1976c) of which Theorem 7 below is a special case.
However, we include it here, as our proof is short and uses new tools. The L# convergence was dealt with
by Abou-Jaoude (1976b), but it is irrelevant here. There are just a few general consistency theorems for
partitioning estimates, such as estimates that partition the space via order statistics (Hanna and Abou-
Jaoude, 1981) or via multivariate rectangular partitions (Gessaman, 1970). The most di#cult problem,
from a universal convergence point of view, is the pointwise convergence. We give just such a theorem
below.
Theorem 7. Let fn be a partitioning density estimate. Assume that diam(An (x)) # 0 at almost all x,
and that n#(An (x)) # at almost all x. Then
lim
(There is no condition on f .)
Theorem 8. Let fn be a partitioning density estimate. Assume that diam(An (x)) # 0 at almost all x,
that n#(An (x)) # at almost all x, and that R f log d-1 (1 f) < #. Then
at almost all x.
Proof. Let x be a jmz point for f . Then
R An
if f(x) > 0. Thus, by Lemma A1, the theorem is proved at almost all points with f(x) > 0. When
x is a jmz point, then
by Lemma A1 again.
Proof of Theorem 7. We simply use the result of Theorem 8 and bounded convergence: indeed, if
# f at almost all x, and each fn is a bona fide (deterministic) density, then R |f
(1974) (see also Devroye and Gy-orfi, 1985) has shown that we may add the phrases "in probability" or
"almost surely" on both sides in case fn is a data-dependent sequence of estimates. Now note that the
condition of Lemma B2 is satisfied for any partition estimate in which the partition does not depend
upon the data with 2: indeed, consider two determistic samples di#ering in only the i-th point, and
let the corresponding partitioning estimates be called fn and gn . Let #An (x) and #An (x) denote the
cardinalities of An (x) under both samples. Then
A#An
Z A
A#An
|#A -#A|
=n |#An
#n
So, now let fn and gn denote the same partitioning estimate, but based on samples drawn from densities
f and g respectively, where g is a density that will be picked later. Applying Lemma B2, we have
At this point, we pick picked so large that that R |f -g| #. By
Theorem 8, at almost all x, E|gn (x)-g(x)| # 0, as g is bounded. but then E R |g
by bounded convergence. Thus,
As # was arbitrary, the proof is complete.
9.

Acknowledgment

The authors thank an attentive referee for pointing out a shortcut in a proof.
10.



--R

"Conditions n-ecessaires et su#santes de convergence L 1 en probabilit-e de l'histogramme pour une densit-e,"


"An approximation to the density function,"
"On the L 1 -Error in Histogram Density Estimation: The Multidimensional Case"
Cambridge University Press
Real variable Methods in Fourier Analysis
"The equivalence of weak, strong and complete convergence in L1 for kernel density esti- mates,"
"A note on the L1 consistency of variable kernel estimates,"

"The Hilbert kernel regression estimate,"
"On the Hilbert density estimate,"
"The strong uniform consistency of nearest neighbor density estimates,"
Recouvrement du tore T q par des ouverts al-eatoires
Discriminatory analysis
"A consistent nonparametric multivariate density estimator based on statistically equivalent blocks,"
"Consistency conditions for probability estimators and integrals of density estimators,"

"Note on the di#erentiability of multiple integrals,"
"A nonparametric estimate of a multivariate density function,"
"Asymptotic normality of multivariate k-NN density estimates,"
"Multivariate k-nearest neighbor density estimates,"
"A counter-example in the theory of strong di#erentiation,"
"Consistency properties of nearest neighbor density function esti- mators,"
"On the estimation of a probability density function and the mode,"
"The distribution of the first ladder moment and height and fluctuation of a random walk,"
"Remarks on some nonparametric estimates of a density function,"
"On the strong derivatives of functions of an interval,"
"A useful convergence theorem for probability distributions,"
Empirical Processes with Applications to Statistics
Measure and Integral
--TR
The Hilbert Kernal regression estimate
saks rarity theorem;nearest neighbor estimate;Jessen-Marcinkiewicz-Zygmund theorem;convergence;kernel estimate;bandwidth selection;nonparametric estimation;density estimation