Noname manuscript No.
(will be inserted by the editor)

Nonparametric estimation of multivariate
distribution function for truncated and censored
lifetime data

arXiv:1710.07468v1 [stat.ME] 20 Oct 2017

Valery Baskakov · Anna Bartunova

Received: 03-05-2017 / Accepted: date

Abstract In this article we consider a number of models for the statistical
data generation in different areas of insurance, including life, pension and
non-life insurance. Insurance statistics are usually truncated and censored,
and often are multidimensional. There are algorithms for estimating the distribution function for such data but they are applicable for one-dimensional
case. The most effective of them are implemented, for example, in SAS system. We propose a nonparametric estimation of the distribution function for
multidimensional truncated-censored data in the form of quasi-empirical distribution and a simple iterative algorithm for its calculating. The accuracy of
estimating the distribution function was verified by the Monte Carlo method.
A comparative analysis of the quasi-empirical distribution with alternative estimates showed that in the one-dimensional case the proposed estimate almost
coincides with the estimates calculated using the HPSEVERITY procedure,
which is a part of SAS/ETS. We didn’t make the comparative analysis in the
multidimensional case due to the lack of analogues of such algorithms. But our
algorithm has passed years of testing in the valuation of employees liabilities
in accordance with IAS 19 ”Employee benefits”. As an example, the article
provides an assessment of the joint function of distribution of workers age and
seniority of a large Russian energy enterprise. The proposed estimates can also
be used in other areas, such as medicine, biology, demography, reliability, etc.
Keywords Nonparametric estimation · Censored and truncated data ·
Multivariate distribution function · Survival analysis · Iterative algorithm
International Actuarial Advisory Company, Llc
Khoroshevskoe shosse 22-132, Moscow, 123007, Russia
V. Baskakov
Tel.: +7-903-100-2660
E-mail: chief@actuaries.ru
A. Bartunova
Tel.: +44-7376-183291
E-mail: BartunovaAnna@gmail.com

2

Valery Baskakov, Anna Bartunova

1 Introduction
Indicators of the loss amount and the level of insurance risk represent the
greatest interest for an insurance actuary. In general, these indicators are multidimensional random variables, for whose complete description a distribution
function is used. Therefore, the estimation of distribution functions or their
individual parameters (e.g., moments, quantiles, etc.) is always an important
and often the most difficult step in an actuarial work. The complexity of estimating the distribution function of insurance risk parameters is objectively due
to the complexity of insurance statistics structure and the underdevelopment
of its statistical analysis methods.
Among the indicators characterizing risk level there is usually a time factor,
that is rather natural, since insurance is always a time process. Usually this
time factor significantly complicates statistical data structure (e.g.,Baskakov
et al. [2] and many other authors), taking into account that actually it is often
time data that are censored or truncated.
In this article a set of examples shows that insurance companies data are
usually censored and truncated, a method of nonparametric estimation of multivariate distribution function with truncated and censored data is described,
and a simple iterative algorithm for distribution function estimation is developed.

2 Examples of truncated and censored data
Example 1. Classical life insurance. Let a t-aged person purchase a life insurance policy in the year of y. An insurance company includes this persons information in its database and since that moment the monitoring of this specified
individual life expectancy starts. Therefore, the insurance company observes
a conditional random variable ξ(x | x > t) life expectancy of the insured
at the age of t, i.e., provided that his age is x > t. If we consider the whole
company portfolio, it is obvious that this situation is replicated many times,
with each individual being observed under his individual condition. Thus, the
insurance company database is a collection of information about conditional
random variables with random condition (age of policy purchase is in general
a random value)
ξk (x | x > tk ), k = 1, . . . , N ∗
(1)
with N ∗ being the number of elements in the insurer database.
Such data type in relation to an unconditional random variable ξ(x) is
called truncated data. This name is justified by the fact that compared with
traditional statistical sampling
ξk (x), k = 1, . . . , N

(2)

with N the sample size. In a truncated sample (1) the N ∗ value is the number
of observations that cannot be interpreted as statistical sampling volume in

Estimation of multivariate distributions for truncated and censored lifetime data

3

Fig. 1 The scheme of development of left-truncated and right-censored data

the conventional sense, since the total sample N from which the truncated
sampling N > N ∗ was obtained. Indeed, in this data collection scheme an
observer is unable to access the information about events happened before the
truncation moment tk . It is illustrated in Fig. 1 showing the Lexis diagram
(e.g. Winsch et al. [11]). The truncated data of the type considered with a
variety of truncation Tk = (0, tk ) is called left-truncated.
Unfortunately, these are not all the challenges actuary faces during insurance statistics analysis. Additional problems are associated with various
reasons for termination of the observations of an individual at the age of xk .
Some of them are as follows:
1. The claim occurrence, i.e., death of an individual;
2. The death of an individual due to a cause not being an insurance event
(e.g., suicide or death while practicing extreme sports);
3. The cancellation of an insurance contract;
4. The need for actuarial calculations during the period when an individual
is alive, etc.
Note that only in the first case the realization of a random variable is fully
observed, which corresponds to the data collection in classical mathematical
statistics. In all other cases, the insurer knows only that the insured event will
happen/would have happened later, if the observation did not stop, and that
during the observation from the age tk to the age xk the claim has not occurred.
With x∗k as the moment of the event occurrence (in this case the death of the
individual) the observer knows only up to a set Ck ∋ x∗k , such data are called
censored data. The censored data case considered above, with Ck = (xk , ∞), is
called right-censored (the information right from the censoring moment xk is
as if being wiped/censored). Thus the considered statistical information type
refers to the samples randomly left-truncated and right-censored.
In fact, a case when the insurance information is censored and/or truncated
is typical. Insurance always covers certain time interval and is accompanied
by a loss (possibly zero loss). Regardless of the insurance type and statistics
collection process, the time indicator is always censored (just due to the fact
that actuarial calculations are carried out periodically, when not all the con-

4

Valery Baskakov, Anna Bartunova

tracts are completed and, therefore, not all the insured events occur) and as
shown above, the time indicator may be further truncated. Similar remarks
may be applied to the value of loss amount, with the only difference that the
loss amount is not always censored. Let us consider the following example.
Example 2. Motor insurance. Let a policyholder in a motor insurance contract
limits the insurance sum by the uc value equal, for example, to 50-100% of
the vehicle cost, and/or use the deductible ut , for example, from 0 to 40% of
the sum insured. Similarly with example 1, it is clear that uc insurance sum
acts as a censoring moment, while the ut deductible acts as the truncation
moment. As a result, we again have left-truncated and right-censored sample.
Example 3. State death registration. In Russia death registration is provided by
a registry office called a Body of Civil Status Fcts Registration (ZAGS) formed
by state power bodies of the Russian Federation subjects1 . Death certificate
record, in particular, contains the following information: family name, name,
patronymic, date and place of birth, gender, citizenship, nationality (if the
nationality is indicated in the certificate of death), date and place of death
and cause of death (based on the document confirming the fact of death).
Let us consider the data registered in ZAGS offices during the calendar year
y, i.e., in the interval [y,y+1). It is easy to notice that, according to the law,
ZAGS gathers information only about conditional variables: life expectancy
of a certain population, provided that their death occurred during y calendar
year. The information on life expectancy of those deceased prior to the start of
the y year, and on the age of those alive at the beginning of the following y + 1
year is not provided to ZAGS. Therefore, such population mortality data are
bilaterally truncated with the truncation set Tk = (0, tk ) ∪ (tk + 1, ∞), where
tk is the age of k individual at the beginning of observations y. The scheme
of bilaterally truncated2 data is represented in Fig. 2. It differs from the chart
shown in Fig. 1 by the amount of information available about k = 3 individual.
If you have previously known the age of this individual and the fact that in
y+1 he was alive, in this scheme the individual (3) data are missing completely
it is possible to talk only about his existence with a certain probability.
The bilaterally truncated data considered in the previous example appear in the literature quite rarely. For example, the International Database
of Longevity contains data with the same structure on supercentenarians,
which Gampe [6] used in the study of human mortality beyond age 110. The
most studied cases of the truncated-censored data are left-truncated and rightcensored samples, which are the subject of many publications, for example, the
fundamental work by Klein and Moeschberger [8]. However, the data available
to an actuary to address the most interesting and practically important tasks
1

Federal law no. 143-FZ of November 15, 1997, On acts of civil state.
Note that a more complex data structure occurs when it is required to estimate mortality
rates for only one reason in this case deaths for alternative reasons act as censoring value,
and mortality data are for the main reason additionally censored on the right (see Fig. 2, a
k=4 individual).
2

Estimation of multivariate distributions for truncated and censored lifetime data

5

Fig. 2 The scheme of bilaterally truncated data design

are, as a rule, multidimensional and have a complex structure. Let us consider
some relevant examples.
Example 4. Lifetime joint annuity. This annuity is issued to two individuals
(usually spouses), and paid fully or partially until the death of both policyholders. With spouses generally living together, having common budget and,
therefore, the same income, their way of life and living conditions being often
similar, there is a reason to believe that their life expectancy is positively correlated random variables. In this regard, to calculate their joint annuity, you
should use the joint distribution function of the spouses lifetime.
The statistical data structure concerning each spouses life expectancy that
is available to an insurance company, coincides with the one that the company
observes in classical life insurance (see Example 1). In the two-dimensional case
the situation changes fundamentally, and a censored set Ck and a truncated
set Tk are more complicated due to their higher dimension:

and


(m > mk ; w



(m = mk ; w
Ck =
(m = mk ; w



(m > mk ; w

= wk ),
> wk ),
= wk ),
> wk ),

with
with
with
with

mk = yk + tk ; yk > wk − τk ;
wk = yk + τk ; yk > mk − tk ;
mk − tk < yk ; wk − τk < yk ;
mk − tk = wk − τk = yk

Tk = (m < tk ; w < ∞) ∪ (m < ∞; w < τk ), k = 1, . . . , n

(3)

(4)

with m and w the current age of husband and wife respectively;
tk , mk and τk , wk the age of truncation (of the insurance contract conclusion), the age of observation (censoring) termination due to death of husband
and wife respectively or other reasons;
yk the time since the contract conclusion until the date as of which data is
collected;
n the number of policies sold.

6

Valery Baskakov, Anna Bartunova

Fig. 3 Censoring Ck and truncating Tk of a two-dimensional (M, W ) vector set

To visualize possible sets Ck and Tk are represented on Fig. 3. To simplify
we put that t1 =, . . . , = t4 and τ1 =, . . . , = τ4 , i.e., Tk sets are the same3 for all
four policies, and the relationship between the ages of censoring and spouses
death is selected in such a way as to show the diversity of Ck sets, with k
corresponding to the number of rows in (3).
Example 5. Selective mortality tables. In general, selective mortality table is
based on the conditional distribution function, e.g., distribution function of life
expectancy of individuals with pure endowment or with term life contracts,
or having a disability, etc. For these distributions, the selection condition is
a categorical variable, and the mortality tables construction is reduced to the
problem discussed in Example 1. The only difference is that the statistical
sample is based on the selection. However, a quantitative variable can often
be used as a selection condition. In the case of disability such a condition is
the age at which a person may become disabled. In this case it is convenient to
talk about the joint distribution of the lifetime X and the age of disability I,
where marginal distributions at any fixed age of disability i will form selective
mortality tables.
Fig. 4 shows the structure of statistical information on the random vector
(X, I) available for insurance companies. The scope of this vector has a specific
triangular representation type and is determined by the system of equations:
X > 0, I > 0 and X ≥ I, where equality is achieved in the case of death
of a non-disabled persons (k = 3). In case of death of a disabled (k = 1)
there is also a complete realization of the (X, I) vector, but with X > I. Note
that in general, the observations are truncated and censored. Data truncating
coincides with the start of observations of an individual at the age of tk ,
and the censoring occurs as a result of observations termination for reasons
not associated with his death (k = 4, 5 or 6) and/or when the observation
3 This suggestion does not affect information value of the chartsince T set is defined by
k
formula (4) and with accuracy to the coordinates of the (tk , τk ) point is the same for all
the policies sold.

Estimation of multivariate distributions for truncated and censored lifetime data

7

Fig. 4 Censoring Ck and truncating Tk of a two-dimensional (X, I) vector set

concerns a person being already disabled and the age of disability is unknown
(k = 2 or 6).
The figure shows possible options of censored sets Ck and a truncated
set Tk , which, for clarity, are the same for all six policies. Meanwhile the
expressions for sets Ck and Tk will not be written explicitly, assuming that a
keen reader can do it himself.
Example 6. Selective table of employees decrement. In the evaluation of social
liabilities in accordance with IAS 19 ”Employee benefits” some social benefits
are often connected with the age of an employee and others with his length of
service. Traditionally the age of an employee is the basis to construct mortality
and retirement tables and the length of service is used to construct turnover
table. This approach complicates the analysis, as different bases of decrement
tables do not allow to construct a multiple decrement table consistent with
them directly, without additional assumptions. An attempt to use age as the
basis to construct all these tables leads to the loss of accuracy of estimations
of financial flows linked to the employees length of service.
The way to cope with this situation is almost obvious. When assessing the
enterprises social liabilities it is necessary to use selective turnover decrement
tables where the basis is an employees age, and the selection condition is his
age at the time of recruitment. The complexity of the available statistical
information and the underdeveloped methods of statistical analysis determine
the obvious problems of such approach.
A standard scheme of data collection on turnover of an enterprise is as
follows. Human resources department provides the employees data according
to the payroll at the beginning and the end of reporting period, including date
of birth, date of recruitment, date and reason for employment termination.
Usually the following reasons for job termination are mentioned: resignation,
retirement, death and others. The structure of such statistical information is
shown in Fig. 5. You may notice that it is similar (but not identical) to data

8

Valery Baskakov, Anna Bartunova

Fig. 5 Censoring Ck and truncating Tk of a two-dimensional (X, E) vector set

structure considered in the previous example. In this case a truncation set is
more sophisticated
Tk = (x < tk , e < x) ∪ (x ≥ tk , e <?k )

(5)

and a possible type of censoring sets is less diverse

 (xk , ek ), with δk = 1;
(x; ek ) : x > xk , with δk = 2;
Ck =

(x, e) : e ≤ x − xk + ek , with δk = 3,

(6)

with δk an indicator of termination reason equal to 1, 2 or 3, if a k-employee
died, resigned, retired or quitted an employer for any other reason.
Example 7. Ratemaking and reserving. Baskakov et al. [2], and Baskakova et
al. [1] have discussed ratemaking and reserving using multidimensional censored data of an insurance company. It was assumed that the statistical sample
included only contracts without deductibles. The removal of this restriction
and the incorporation of all the data in the sample, including contracts with
deductibles, will obviously lead to the truncation of the observed losses (see
Example 2). However, we will not give expressions for censoring and truncating sets arising in such a scheme of data collection. We believe that the
examples previously given are enough for a reader familiar with the basics of
insurance to be able to deal with the insurance statistics structure and build
censoring and truncating sets for almost any insurance scheme, including the
above given.
It is obvious that the examples of the truncated and censored insurance
data are not limited to the ones discussed above, they can be countless. It is
more difficult to give an example of complete data. We, however, will not do
this and move on to the development of methods of multidimensional statistical
analysis of truncated and censored data.

Estimation of multivariate distributions for truncated and censored lifetime data

9

3 Formulating the problem
Let us remember that in case of complete data xk , k = 1, , n the empirical
distribution is defined as
n

P̄n (B) =

1X
I(xk ∈ B),
n

(7)

k=1

with B a measurable set, B ∈ Rm ;
xk = (x1k , , xm
k ) a random vector;
I(xk ∈ B) an indicator equal to 1, with xk ∈ B or 0, with xk ∈
/ B.
If data is Ck , k = 1, . . . , n censored, Baskakov [3] uses quasi-empirical
distribution as its complete analogue
n

1X
P̄n (B) =
P̄n (B | Ck ),
n

(8)

k=1

which is a fixed point of a functional
n

Pn (B) =

1X
Pn (B | Ck ), P (B) ∈ P,
n

(9)

k=1

with P (B | Ck ) conditional distribution;
P class of all possible distributions on Rm ;
B measurable set, B ⊂ Rm .
Estimation (8) generalizes the usual empirical distribution (indeed with
Ck = xk , P (B | xk ) = 1 or 0 is true, with xk ∈ B or xk ∈
/ B, respectively)
and, under certain circumstances (Baskakov [3]) inherits its basic features:
consistency, asymptotic normality, etc. At the same time P ?n (B) is generalized
maximum likelihood estimation of P (B) on P.
This article states the problem of quasi-empirical distribution generalization (8) on a general case of censored and truncated data
{Ck , Tk }, k = 1, . . . , n,

(10)

with Ck and Tk censoring and truncating sets respectively for k realization of
a random variable observed.
Hereinafter no distinction is made between complete and censored observations, as well as between truncated and non-truncated observations due to
the fact that the complete observation can be viewed as a censored one with
Ck = {xk }, and non-truncated as a censored one with Tk = ∅.

10

Valery Baskakov, Anna Bartunova

4 Quasi-empirical distribution
Before proceeding to the construction of estimation Pn (B) of the unconditional distribution P (B) of a random variable ξ on the basis of truncated and
censored data (10), let us remember that as a result of truncating the data
by a set Tk , range Ω of possible values of the k realization of the random
variable ξ is truncated to a set (Ω \ Tk ), k = 1, . . . , n. In other words, not all
realizations of ξ variable are seen in the data collection process, but only those
meeting the condition xk ∈ (Ω \ Tk ), k = 1, . . . , n. That allows to talk about
the conditional distribution Pn (B | (Ω \ Tk )) of the random variable ξ. The
data collection features mentioned above are the basis for the proposed estimation Pn (B). Its idea is very simple: each truncated (conditional) observation
is converted into the analogous non-truncated (unconditional) observation by
adding a certain number of complete data from the P (B) distribution, with
simultaneous and adequate increase in the sample volume. The result is a nontruncated sample from the unconditional distribution P (B) on which a Pn (B)
evaluation is built.
Lets consider the k realization of xk of an observed random variable ξ from
the conditional distribution P (B | (Ω \ Tk )). It is obvious that the equation
P (xk ∈ (Ω \ Tk ) | (Ω \ Tk )) = 1 is true. However, the same unconditional
probability is
P (xk ∈ (Ω \ Tk )) = 1 − P (xk ∈ Tk ) < 1,
(11)
with P (xk ∈ Tk ) > 0.
Experience demonstrates that the only one realization of a random value ξ
has been observed on a set (Ω \ Tk ). Suppose that while observing the random
variable ξ from the distribution P (B) on the set (Ω \ Tk ) the only one realization of the random variable ξ will be observed as well. Then simultaneously on
the set Tk P (Tk )/(1 − P (Tk )) realizations of this variable will be observed on
average. Thus it is necessary to add n∗k = P (Tk )/(1 − P (Tk )) of non-observed
realizations x∗k ∈ Tk to one observed realization xk ∈ (Ω \ Tk ), to convert one
truncated observation xk ∈ (Ω \ Tk ) from the distribution P (B | (Ω \ Tk ))
into a non-truncated observation from the distribution P (B). Each truncated
observation is replaced by an equivalent number
1+

1
P (Tk )
=
,
1 − P (Tk )
1 − P (Tk )

(12)

of non-truncated observations from the distribution P (B).
Now let us consider the truncated and censored sample (10) as a whole.
Similarly to the censored data let us call a fixed point of a functional

n 
1 X
P (B ∩ Tk )
, P (B) ∈ P,
(13)
Pn (B) =
P (B | Ck ) +
N
1 − P (Tk )
k=1

the quasi-empirical distribution, e.g.,

n 
1 X
P̄n (B ∩ Tk )
,
P̄n (B) =
P̄n (B | Ck ) +
N
1 − P̄n (Tk )
k=1

(14)

Estimation of multivariate distributions for truncated and censored lifetime data

11

with N sample volume adjusted according to (12), equal to
N=

n
X

1
,
1
−
P̄
n (Tk )
k=1

(15)

Lets explain the meaning of the second summand in the expression (14):
a numerator P̄n (B ∩ Tk ) allocates a part of probability measure concentrated
on the truncation sets Tk , k = 1, . . . , n to a set B , and a denominator 1 −
P̄n (B∩Tk )) normalizes it to enforce the condition (12) If we are going to repeat
previous arguments but normalize not P̄n (B ∩ Tk ), but P̄n (B | Ck ) (formally
its done by of both summands by 1 − P̄n (Tk ), we will have equivalent (to some
extend) estimation of quasi-empirical distribution
P̄n (B) =

n

1X
P̄n (B | Ck ) · (1 − P̄n (Tk )) + P̄n (B ∩ Tk ) .
n

(16)

k=1

In this case the sample volume is equal to n, as if B = Ω then

P̄n (Ω | Ck ) · (1 − P̄n (Tk )) + P̄n (Ω ∩ Tk ) ≡ 1, k = 1, . . . , n.

It is easy to check that with B = (x1 < x1 , . . . , xm < xm ), F̄n (x1 , . . . , xm ) =
P̄n (B) is a distribution function. It should also be mentioned that with Tk ≡ ∅,
k = 1, . . . , n in (10), the expression (14) transforms into (8). Due to natural
restrictions on the scope of the article we will not touch such a complex and
important issue as the dependence of the vector xk = (x1k , . . . , xm
k ) components. To simplify we assume that all components involved in the process of
observations censoring and truncation, but not clearly included n the list of
components of the estimated distribution function, are independent from the
latter.
To construct the estimation of distribution function in the form (14) or
(16) an iterative procedure like the EM algorithm (e.g., Dempster et al. [4])
can be used, that includes the following steps.
(0)
1. Set the initial approximation of estimation P̄n (B), B ∈ Rm , e.g., a
uniform one.
2. Calculate the adjusted sample volume according by formula
n
X
1
,
N (0) =
(0)
k=1 1 − P̄n (Tk )
(1)

3. Calculate a new value of P̄n (B) by formula
P̄n(1) (B)
or
P̄n(1) (B) =

n
(0)
1 X
P̄n (B ∩ Tk )
= (0)
P̄n(0) (B | Ck ) +
(0)
N
(1 − P̄n (Tk ))
k=1

!

n

1 X  (0)
P̄n (B | Ck ) · (1 − P̄n(0) (Tk )) + P̄n(0) (B ∩ Tk ) .
n
k=1

(0)

(1)

4. Return to step (2), replacing P̄n (·) for P̄n (·)) etc.
5. Stop with the required accuracy achieved.

12

Valery Baskakov, Anna Bartunova

Table 1 Various types of truncated and censored observations

Censoring
No (complete)
Right
Left
Interval

Truncation
No

Right

Left

Bilateral

+
+
+
+

+
–
+
+

+
+
–
+

+
–
–
+

5 Estimation of one-dimensional distribution function (general
case)
Let us consider the truncated and censored sample (10) from the distribution
F (x) of a one-dimensional random variable ξ. The censoring and truncating
sets are respectively equal to
Ck = (c1k , c2k );
Tk = (−∞, t1k ) ∪ (t2k , ∞),

(17)

and their boundaries are related by t1k ≤ c1k < c2k ≤ t2k , k = 1, . . . , n. Note that
special cases of this type of data: left-truncated and right-censored (examples
1 and 2) with Tk = (−∞, t1k ) and Ck = (c1k , ∞), as well as the bilaterally
truncated and complete (example 3) with Tk = (−∞, t1k ) ∪ (t2k , ∞) and Ck =
{xk } were discussed above.
Its obvious that the range of one-dimensional truncated and censored data
generated by the sets (17) is much wider than those previously considered.
It is possible to distinguish 12 different types of truncated and censored observations listed in table 1. Note that not all combinations of censoring and
truncation are possible. In the table invalid combinations are marked by (−),
they include right (left)-censored and right (left)-truncated or bilaterally truncated observations.
The estimation of quasi-empirical distribution function F̄ (x), with all sample elements are bilaterally truncated and censored by an interval, i.e., with
all Ck and Tk , k = 1, . . . , n looking as (17), is a fixed point of a functional4


n
1 X F min(c2k , x) − F min(c1k , x)
+
F (x) =
N
F (c2k ) − F (c1k )
k=1

!
F max(x, t2k ) − F (t2k ) + F min(x, t1k )
(18)
F (t2k ) − F (t1k )
with N an adjusted sample volume equal to
N=
4

n
X

F (t2k )
k=1

1
.
− F (t1k )

Further, for definiteness, we will consider quasi-empirical distribution (14).

(19)

Estimation of multivariate distributions for truncated and censored lifetime data

13

Fig. 6 Nonparametric estimation of the distribution function for left-truncated and rightcensored sample depending on its volume

Note that in the special cases of censoring and truncation (see table. 1) certain
boundaries of sets (17) are changed to their limits, that allows to simplify the
expressions for summands in (18). So, with k observation
— complete, we put c1k ≤ xk ≤ c2k , c1k → xk and c2k → xk , then Ck = {xk }
and

!
F min(c2k , x) − F min(c1k , x)
= I(xk ≤ x);
lim
F (c2k ) − F (c1k )
c1k →xk ; c2k →xk
— right-censored, then c2k = t2k = ∞ and F (c2k ) = F (max?(x, t2k )) =
F (t2k ) = 1;
– left-censored, then t1k = c1k = −∞ and F (min?(c1k , x)) = F (min?(x, t1k )) =
F (t1k ) = 0;
– right nontruncated, then t2k = ∞ and F (t2k ) = 1;
– left nontruncated, then t1k = −∞ and F (t1k ) = 0.
Expressions for the summands in (18) for various combinations of truncation and censoring of observations, can be found in table. 2.
Fig. 6 illustrates estimations of one-dimensional distribution function for
left-truncated and right-censored data from a uniform distribution on the interval [0,50]. During the simulation the non-truncated sample volume N got
the values 100, 1000 and 5000, and other model parameters did not vary.
Therefore, all the main characteristics of these samples were fairly close to
each other. The truncated samples volume N ∗ ?0.68N , the truncation degree
(the share of truncated observations in the sample) ranged from 0.71 to 0.72,
and the censoring degree (proportion of censored observations in the sample)
ranged from 0.72 to 0.74.
The calculation results show that with increasing sample volume the estimation accuracy increases and becomes satisfactory achieving the volume
comparable to an amount of insurance companies data. This allows to recommend the proposed estimation of distribution function in the form (18) for
practical work.

14

Valery Baskakov, Anna Bartunova

Table 2 Summands of quasi-empirical distribution depending on the observation type
Observation type

Formula

Complete,
nontruncated

I(xk ≤ x)
F (x)−F (min(c1
,x))
k

Right-censored,
nontruncated

1−F (c1 )
k

F (min(c2
,x))
k

Left-censored,
nontruncated

F (c2 )
k

Interval-censored,
nontruncated

F (min(c2
,x))−F (min(c1
,x))
k
k
F (c2 )−F (c1 )
k
k

Complete,
left-truncated

I(xk ≤ x) +

1−F (c1 )

k

F (min(c2
,x))−F (min(c1
,x))
k
k
F (c2 )−F (c1 )
k

+

F (min(x,t1
)
k )
1−F (t1 )

k

I(xk ≤ x) +

k

F (max(x,t2
) −F (t2
)
k )
k
F (t2 )
k

Right-censored,
right-truncated

—
F (min(c2
,x))
k

Left-censored,
right-truncated

F (c2 )

+

F (max(x,t2
) −F (t2
)
k )
k
F (t2 )

k

k

F (min(c2
,x))−F (min(c1
,x))
k
k
F (c2 )−F (c1 )
k

I(xk ≤ x) +

k

Right-censored,
bilaterally truncated

—
F (min(c2
,x))−F (min(c1
,x))
k
k
k

k

F (t2 )
k

—

F (c2 )−F (c1 )

+

)
) −F (t2
F (max(x,t2
k
k )

F (max(x,t2
) −F (t2
)+F (min(x,t1
)
k )
k
k )
F (t2 )−F (t1 )
k
k

Right-censored,
bilaterally truncated

Interval-censored,
bilaterally truncated

1−F (t1 )

—

Complete,
right-truncated

Complete,
bilaterally truncated

+

)
F (min(x,t1
k )

k

Left-censored,
left-truncated

Interval-censored,
right-truncated

1−F (t1 )
k

F (x)−F (min(c1
,x))
k

Right-censored,
left-truncated

Interval-censored,
left-truncated

F (min(x,t1
)
k )

+

F (max(x,t2
) −F (t2
)+F (min(x,t1
)
k )
k
k )
F (t2 )−F (t1 )
k

k

6 Estimation of multivariate distributions for truncated and
censored data
Let us consider the estimation of the joint distribution of age and length of
service of an employees as an example of performing nonparametric estimation
of a multivariate distribution function based on truncated and censored data.
HR department is usually a supplier of the relevant information, the collection

Estimation of multivariate distributions for truncated and censored lifetime data

15

Fig. 7 The estimation of joint distribution of employees age and length of service

scheme and the structure of data obtained were discussed in detail in Example
6.
Table 3 and Fig. 7 represent the estimation of the joint function of age
and length of service distribution of a large energy enterprise employees based
on the data of HR department from 01.01.2010 to 31.12.2013. The sample
volume was 26839, including 4328 resigned employees and 231 employees who
died during this period. The company recruited 5129 new employees instead
of those who quitted the employer. Therefore, the data truncation degree was
approximately 0.81, and the censoring degree was 0.83.
It should be noted that the estimation of joint distribution function of the
employees age and length of service allows to design the selective decrement
tables depending on the recruitment age and thus to solve completely the task
considered in Example 6.

16

Table 3 The estimation of the joint distribution function of the employees age and length of service
Length of service
Age

4

8

12

16

20

24

28

32

36

40

44

48

52

0.0001
0.0003
0.0006
0.0011
0.0017
0.0025
0.0034
0.0045
0.0060
0.0077
0.0097
0.0122
0.0151
0.0187
0.0227
0.0275
0.0334
0.0405
0.0493
0.0600
0.0729
0.0879
0.1044
0.1214
0.1391
0.1570
0.1761
0.1955
0.2164
0.2373

0.0001
0.0009
0.0027
0.0058
0.0072
0.0104
0.0120
0.0139
0.0177
0.0205
0.0256
0.0300
0.0353
0.0443
0.0516
0.0611
0.0714
0.0853
0.1035
0.1233
0.1498
0.1797
0.2169
0.2599
0.3078
0.3595
0.4154
0.4746
0.5377
0.6033

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0145
0.0169
0.0211
0.0253
0.0310
0.0370
0.0431
0.0531
0.0617
0.0723
0.0840
0.0998
0.1201
0.1421
0.1728
0.2072
0.2517
0.3039
0.3642
0.4310
0.5040
0.5826
0.6680
0.7582

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0216
0.0261
0.0321
0.0392
0.0457
0.0561
0.0651
0.0761
0.0893
0.1055
0.1269
0.1495
0.1809
0.2164
0.2629
0.3185
0.3845
0.4594
0.5432
0.6354
0.7376
0.8482

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0222
0.0268
0.0329
0.0412
0.0478
0.0583
0.0674
0.0785
0.0918
0.1084
0.1307
0.1536
0.1855
0.2213
0.2687
0.3252
0.3931
0.4714
0.5610
0.6617
0.7759
0.9021

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0222
0.0268
0.0329
0.0412
0.0481
0.0593
0.0689
0.0803
0.0937
0.1103
0.1329
0.1559
0.1881
0.2241
0.2719
0.3287
0.3975
0.4774
0.5705
0.6783
0.8055
0.9522

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0222
0.0268
0.0329
0.0412
0.0481
0.0597
0.0693
0.0808
0.0943
0.1109
0.1335
0.1565
0.1889
0.2250
0.2728
0.3297
0.3988
0.4793
0.5736
0.6841
0.8175
0.9770

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0222
0.0268
0.0329
0.0412
0.0481
0.0597
0.0693
0.0808
0.0943
0.1109
0.1335
0.1565
0.1889
0.2250
0.2728
0.3298
0.3989
0.4795
0.5741
0.6853
0.8209
0.9860

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0222
0.0268
0.0329
0.0412
0.0481
0.0597
0.0693
0.0809
0.0944
0.1110
0.1336
0.1567
0.1891
0.2253
0.2731
0.3301
0.3992
0.4798
0.5744
0.6858
0.8225
0.9911

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0222
0.0268
0.0329
0.0412
0.0481
0.0597
0.0693
0.0809
0.0944
0.1110
0.1336
0.1567
0.1891
0.2253
0.2731
0.3301
0.3992
0.4798
0.5744
0.6860
0.8234
0.9960

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0222
0.0268
0.0329
0.0412
0.0481
0.0597
0.0693
0.0809
0.0944
0.1110
0.1336
0.1567
0.1891
0.2253
0.2731
0.3301
0.3992
0.4798
0.5744
0.6860
0.8236
0.9972

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0222
0.0268
0.0329
0.0412
0.0481
0.0597
0.0693
0.0809
0.0944
0.1110
0.1336
0.1567
0.1891
0.2253
0.2731
0.3301
0.3992
0.4798
0.5744
0.6860
0.8237
0.9986

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0222
0.0268
0.0329
0.0412
0.0481
0.0597
0.0693
0.0809
0.0944
0.1110
0.1336
0.1567
0.1891
0.2253
0.2731
0.3301
0.3992
0.4798
0.5744
0.6860
0.8237
0.9989

0.0001
0.0009
0.0045
0.0077
0.0092
0.0126
0.0146
0.0172
0.0222
0.0268
0.0329
0.0412
0.0481
0.0597
0.0693
0.0809
0.0944
0.1110
0.1336
0.1567
0.1891
0.2253
0.2731
0.3301
0.3992
0.4798
0.5744
0.6860
0.8237
1.0000

Valery Baskakov, Anna Bartunova

20
22
24
26
28
30
32
34
36
38
40
42
44
46
48
50
52
54
56
58
60
62
64
66
68
70
72
74
76
78

0

Estimation of multivariate distributions for truncated and censored lifetime data

17

7 Conclusion
The authors propose nonparametric estimation of the distribution function for
multidimensional truncated and censored data. It coincides with the estimation of distribution function in multidimensional censored data in the form
of quasi-empirical distribution considered by Baskakov [3] for nontrancated
data, and, in fact, is its direct generalization. Very simple and effective iterative algorithms for constructing estimations of quasi-empirical distribution in
the form (14) and (16) is developed in SAS/IML environment for a number of
schemes of data truncation and censoring, including multidimensional ones.
The accuracy of estimation of distribution function for truncated and censored data was tested by Monte Carlo method. The comparative analysis of
quasi-empirical distribution (18) and alternative estimations showed that for
one-dimensional case the estimation coincides with the estimations obtained
using the HPSEVERITY procedure, which is a part of SAS/ETS. A HPSEVERITY procedure uses the modified Lai and Ying [9] and Klein and
Moeschberger [8] estimation of Kaplan-Meier (Kaplan and Meier [7]) for lefttruncated and right-censored data. In other cases, the algorithm for constructing nonparametric estimation of the maximum likelihood proposed by Turnbull [10] is used, and with truncated observations its modification proposed
by Frydman [5].
Similar problems for multidimensional case have not been considered in the
literature known to us, therefore similar analysis is not possible. However, the
fact that the proposed algorithm shows good results for one-dimensional data
and a number of schemes of multidimensional truncated and censored data
inspire to hope that it will work satisfactorily beyond the previously examined
schemes for multidimensional truncated and censored data. Moreover, the algorithm has passed years of testing in the IAAC group of companies in the
evaluation of social liabilities in accordance with IAS 19 ”Employee benefits”.

References
1. Baskakova, A. and Baskakov, V. IBRN Reserves Estimate on the Bases of Multivariate
Censored Data of an Insurance Company, The Actuary [Russian], Vol. 5, 21 – 25 (2014)
2. Baskakov, V. and Baskakov I. On Ratemaking and Other Tasks in Non-Life Insurance,
The Actuary [Russian], Vol. 4, 37 – 41 (2010)
3. Baskakov, V. On an Analog of Empirical Distribution for Multivariate Censored date, J.
of Math. Science, Vol. 81 (4), 2779 – 2785 (1996)
4. Dempster, A., Laird N. and Rubin D. Maximum likelihood estimation from incomplete
data, Journal of the Royal Statistical Society, Series B, vol. 39, 1 — 38, (1977).
5. Frydman, H. A Note on Nonparametric Estimation of the Distribution Function from
Interval-Censored and Truncated Observations. Journal of the Royal Statistical Society,
Series B 56, 71—74, (1994).
6. Gampe, J. Human mortality beyond age 110. In Supercentenarians, edited by H. Maier,
J. Gampe, B. Jeune, J.-M. Robine, and J.W. Vaupel, number 7 in Demographic Research
Monographs, chapter III, pp. 219230. Springer, Heidelberg [et al.] (2010).
7. Kaplan, E. L. and Meier, P. Nonparametric Estimation from Incomplate Observations.
Journal of the American Statistical Association, Vol. 53, 457 — 481, (1958).

18

Valery Baskakov, Anna Bartunova

8. Klein, J. P., and Moeschberger, M. L. Survival Analysis: Techniques for Censored and
Truncated Data. Springer-Verlag, New York (2003).
9. Lai, T. L., and Ying, Z. Estimating a Distribution Function with Truncated and Censored
Data. Annals of Statistics, Vol. 19, 417 — 442, (1991).
10. Turnbull, B. W. The Empirical Distribution Function with Arbitrarily Grouped, Censored, and Truncated Data. Journal of the Royal Statistical Society, Series B 38, 290 —
295, (1976).
11. Winsch, G., Mouchart, M. and Duchene, J. The Life Table. Modelling Survival and
Death. Kluwer Academic Publishers, Netherlands (2002).

