Measuring inequality: application of semi-parametric methods to real life data
Tchilabalo Abozou Kpanzou Tertius de Wet Gane Samb Lo
stat.ME 62F10, 62G05, 62P05





nkpanzout@gmail.com, tdewet@sun.ac.za, gane-samb.lo@ugb.edu.sn


Measuring inequality: application of semi-parametric methods to real life data
    G. S. Lo ^(3,4,5)
     
==============================================================================




 A number of methods have been introduced in order to measure the inequality in various situations such as income and expenditure. In order to curry out statistical inference, one often needs to estimate the available measures of inequality. Many estimators are available in the literature, the most used ones being the non parametric estimators. <cit.> has developed semi-parametric estimators for measures of inequality and showed that these are very appropriate especially for heavy tailed distributions. In this paper we apply such semi-parametric methods to a practical data set and show how they compare to the non parametric estimators. A guidance is also given on the choice of parametric distributions to fit in the tails of the data.^1 Université de Kara, Togo (Tchilabalo Atozou Kpanzou)

^2Faculty of Economics and Management Science, University of Stellenbosch, Stellenbosch, South Africa

^3 LERSTAD, Gaston Berger University, Saint-Louis, Sénégal (Gane Samb Lo, Modou Ngom). 

^4 Affiliated to LSTA, Université Pierre et Marie Curie, Paris, France (Gane Samb LO)

^5 Associated to African University of Sciences and Technology, AUST, Abuja, Nigeria (Gane Samb Lo)



Corresponding author. Tchilabalo Abozou Kpanzou. Email :
nkpanzout@gmail.com.


keywords and phrases : Income distribution; Inequality measures; Confidence intervals; Extreme Value Theory.

AMS 2010 Classification subjects. 62F10; 62G05; 62P05






§ INTRODUCTION

An area of application of the statistical methods is that of measures of inequality. These are very popular in economics and have applications in many other branches of Science, see e.g. <cit.>; <cit.>; <cit.>. In order to curry out statistical inference, one often needs to estimate the available measures of inequality, namely the Gini, the Generalised entropy, the Atkinson, the quintile share ratio, just to mention a few. Many estimators are available in the literature, the most used ones being the non parametric estimators. Details on the definitions of inequality measures as well as non parametric estimators can be found in, e.g., <cit.>, <cit.>, <cit.>, <cit.> and <cit.>. Further on inequality and poverty measures can be found in <cit.> and <cit.>.


<cit.> has developed semi-parametric estimators for measures of inequality and showed that these are very appropriate especially for heavy tailed distributions. In this paper we apply such semi-parametric methods to a practical data set and show how they compare to the non parametric estimators. A guidance is also given on the choice of parametric distributions to fit in the tails of the data. The data used are claims data from a South African short term insurer.


The remainder of the paper is organized as follows. In Section <ref>, we briefly describe the semi-parametric methods used. Results on application to the aforementioned data set are given in Section <ref>. We give some concluding remarks in Section <ref>.



§ METHODOLOGY

In this section we present the semi-parametric estimation method. The procedure relies on the estimation of the underlying distribution in a semi-parametric setting.


Define a semi-parametric distribution function by

    F̃(x)=    F(x),                              x≤ x_0,
        F(x_0)+(1-F(x_0))F_θ(x),  x> x_0,


for a given x_0, where F_θ is a parametric distribution satisfying the condition F_θ(x_0)=0, and F is an unknown distribution. Note that θ can be a vector parameter. 
Choose x_0=Q(F,1-α),  α∈ [0,1], where Q denotes the quantile function associated with F.


We then have

    F(x_0)+(1-F(x_0))F_θ(x)   =1-α+(1-(1-α))F_θ(x)
       =1-α+α F_θ(x)
       =1-α(1-F_θ(x)).


It follows that

    F̃(x)=    F(x),                  x≤ Q(F,1-α),
        1-α(1-F_θ(x)),  x> Q(F,1-α),


where F_θ satisfies the condition F_θ(Q(F,1-α))=0.


Estimating θ by θ, and estimating F by the empirical distribution function F_n, we estimate the underlying distribution semi-parametrically as

    F̃_n(x)=    F_n(x),                 x≤ Q(F_n,1-α),
        1-α(1-F_θ(x)),  x> Q(F_n,1-α).


Equation (<ref>) is very important as it estimates the underlying distribution. However, a choice of the parametric distribution F_θ is required in order to make the estimation process possible. We address that issue by making use of results from Extreme Value Theory (EVT). See e.g. Beirlant et al. <cit.> for these results.


Given a certain threshold u, we consider the conditional distribution of the exceedance of u given that u was exceeded. We consider two types of exceedances:

	
  * X-u given X > u (absolute exceedance).
	
  * X/u given X > u (relative exceedance).


From EVT, if F belongs to the domain of attraction of the generalized extreme value distribution, the following limiting results hold when u→∞.

	
  *  The distribution of X-u|X > u converges to the generalized Pareto distribution (GPD)


    G(x;σ,γ)=1-(1+γ x/σ)^-1/γ,  x>0,

	
	
where σ>0 is the scale parameter and γ>0 is the extreme value index (EVI).
	
  *  The distribution of X/u|X > u converges to the strict Pareto (Pa) distribution
	
    F_P(x)=1-x^-1/γ,  x>1,  γ>0,

	where γ is the EVI.
	
  * A second order approximation of the distribution of the relative exceedance is the perturbed Pareto distribution (PPD) defined by the survival function
	
    G(x;γ,c,τ) = 1-G(x;γ,c,τ)=(1-c)x^-1/γ+cx^-1/γ-τ,

	where x>1, γ>0, τ>0 and c∈(-1/τ,1). The idea here is to fit such a PPD to the relative exceedance, aiming for a more accurate estimation of the unknown tail. See <cit.> for more details.
	

Once the semi-parametric estimators for the distribution function are obtained, we plug them in the functional form of the inequality measures to obtain their SP estimators. Here we only give the estimator for Gini when fitting the GPD in order to illustrate the procedures we are aiming to apply.


Recall that given a distribution function F of a random variable X with mean μ, the ordinary Gini coefficient can be defined as


    I_G=1/μ∫_0^∞ F(x)(1-F(x))dx.


Consider a random sample X_1,X_2,…,X_n from F, with associated order statistics X_1,n≤ X_2,n≤…≤ X_n,n, and suppose the threshold above which a parametric distribution is fitted is x_0=Q(F,1-α). The Gini coefficient is then estimated semi-parametrically as


    I_SPG=1/μ∫_0^∞F̃_n(x)(1-F̃_n(x))dx,


where F̃_n is given in Equation (<ref>), and μ is the estimator of μ using F̃_n, that is


    μ=∫_0^∞ xdF̃_n(x).


Estimate the threshold u by X_n-k,n, and assume that for a large n, the GPD is a reasonable approximation to the distribution of the exceedances X_n-k+1,n-X_n-k,n,X_n-k+2,n-X_n-k,n,…,X_n,n-X_n-k,n for a given k. A semi-parametric estimator for I_G is then given by

    I_SPG=1/μ∑_i=1^n-k-1i/n(1-i/n)(X_i+1,n-X_i,n)+kσ[2n-k-γ(n-k)]/n^2μ(1-γ)(2-γ),
 

where σ and γ are estimators for the unknown scale and shape parameters σ and γ of the GPD using the exceedances, with γ<1, and 

    μ=1/n∑_i=1^n-kX_i,n+k/n(X_n-k,n+σ/1-γ)
 

is an estimator for μ.


Theoretical details and other estimators can be found in <cit.>. We remind that the aim of this work is to show how the semi-parametric methods can be applied to real life data, and so we direct the reader to the previous reference for all the theoretical derivations.




§ NUMERICAL APPLICATIONS AND INTERPRETATIONS

Here we consider claims data from a South African short term insurer. These consist of a portfolio of claims from 1 July 2004 to 21 July 2006. The dates used, were the dates the claims occurred, and not the dates the claims were registered. The claim amounts were the total claim amounts and any excesses paid by the client were ignored. The claim amounts were adjusted for inflation to July 2006 as base month. Finally, any negative or zero claim amounts were deleted from the data set. Negative amounts occur if, for instance, the value of the items salvaged from the wreck exceeds the claim amount in value. The final sample size is 16104. We will refer to the data set as Portfolio.


The histogram and the boxplot for the data are shown in Fig. <ref>., giving an idea of the tails of the distribution.



We see from Fig. <ref>. that the data have heavy tails and so we can use them to illustrate our methods. Table <ref> gives some descriptive statistics (Median, Median Absolute Deviation (MAD) and maximum value).


We next give the estimates of the inequality measures, starting with the non parametric ones summarised in Table <ref>.


The semi-parametric estimates for the inequality measures using the methods described in Section <ref> (fitting the GPD, the strict Pareto and the PPD in the tails) are now calculated. We use 10% of the data (upper order statistics) in each case both to estimate the parameters in the tail distribution and to fit that distribution. For comparison purposes we put together both the non parametric and semi-parametric estimates. The results are given in Table <ref>. We only show these for the Gini coefficient, the generalised entropy with parameter 0 (GE0), the Atkinson with parameter 1 (A1) and the quintile share ratio (QSR). The non parametric estimators are denoted by NP, the semi-parametric (SP) ones by SPGPD (when fitting the GPD in the tails), SPPa (when fitting the Pa) and SPPPD (when fitting the PPD).



We see from these tables that the SP estimates are not far away from the NP estimates. The interpretation is that for the portfolio data considered, we have a high level of inequality in the distribution. In fact, Gini is around 60%, GE0 around 70%, A1 around 50%, and the QSR shows that the 20% upper claims are worth 35 times the 20% lowers claims. For the interpretation of inequality measures, see e.g. <cit.>.


However, the advantage of using the semi-parametric estimators relies on the fact that they are more resistant to extreme values in the data, as shown by <cit.>. In addition, although we do not show confidence intervals in this paper, note that the coverage probabilities are very satisfactory for confidence intervals based on SP methods. This is also demonstrated in <cit.>.


The question that is now raised is how one can choose the right parametric distribution to fit in the tails. Such a choice is done among the GPD, the Pa and the PPD. In order to decide which one to use in a specific application, we suggest the use of a measure of representativeness of the sample to each distribution. In our applications, we use the one given by <cit.>. Below is the description of that measure.


Let X=(X_1,X_2,…,X_n) be a simple random sample of size n from a distribution F and denote by X_1,n≤ X_2,n≤…≤ X_n,n its associated order statistics. A representativeness index is given by


    R(X,F)=1-12n/4n^2-1∑_i=1^n(F(X_i.n)-2i-1/2n)^2.


This index is used to measure how well the sample X represents its parent distribution. Therefore, it can be used to select the model one should use when having to choose between a number of different models.


In our case, we use the measure to decide between the three parametric choices, GPD, strict Pareto and PPD. To do this we calculate R(X,GPD), R(X,Pa) and R(X,PPD), and the largest value determines the preferred distribution to use.


Applying the measure of representativeness to 10% of the upper values of the Portfolio data considered, we obtain the results in Table <ref>.


These results show that the strict Pareto and the PPD are well represented by the data in the tails and so either of them can be used. The corresponding estimators are the most reliable, and should be considered for assessing the inequality in the given situation.


Given a practical data set, in order to use a semi-parametric method, a two-step approach would be to first determine which of the three parametric distributions to use in the tail estimation. The corresponding semi-parametric procedure can then be used as a preferred choice to estimate the desired measures.




§ CONCLUSIONS

In this paper we have illustrated the application of semi-parametric estimators to real life data. Such estimators indeed perform very well, especially in the case of heavy tailed distributions. They are based on semi-parametric estimators of the underlying distribution using results from Extreme Value Theory. As the name indicates, a part of a semi-parametric estimator is made up with non parametric estimation and the other part (the tails) with parametric estimation. Three options have been considered for fitting such parametric distributions, namely the generalized Pareto, the strict Pareto and the perturbed Pareto. Given a data set, one therefore needs to decide which one of the three he should use for the estimation. We have addressed this question by suggesting the use of the measure of representativeness proposed by <cit.>. In order to illustrate the way one should use it, this measure has also been applied to the portfolio data set considered.


From the analysis done in this paper, we make the following recommendations. Suppose one has a data set from a heavy tailed distribution. Then:


	

  (i) Choose a threshold above which a particular distribution fits well (we have the three possibilities mentioned above);

  (ii) Use the measure of representativeness to choose the best fitting distribution;

  (iii) Calculate the estimates of the inequality measures using the estimators corresponding to the distribution obtained in (ii).

Applying this procedure guarantees, to some extent, that the conclusions drawn from the analysis will be more reliable.


99

[Allison (1978)]allison1978 Allison P.D. (1978). Measures of inequality. American Sociological Review, 43, 865–880.

[Beirlant et al. (2004)]beirlantetal2004 Beirlant J., Goegebeur Y., Segers J. and Teugels J. (2004). Statistics of Extremes: Theory and  Applications. Wiley, New York.

[Bertino (2006)]bertino2006 Bertino S. (2006). A Measure of Representativeness of a Sample for Inferential Purposes. International Statistical Review, 74, 149–159.

[Cowell and Flachaire (2007)]cowellflach2007 Cowell F.A. and Flachaire E. (2007). Income Distribution and Inequality Measurement: The Problem of Extreme Values. Journal of Econometrics, 141, 1044-1072.

[Creedy (2014)]creedy2014 Creedy J. (2014). Interpreting Inequality Measures and Changes in Inequality. Working Papers in Public Finance, Victoria Business School, 11, 1-21. 

[EU-SILC (2004)]eusilc2004 EU-SILC 131/04 (2004). Common Cross-sectional EU Indicators Based on EU-SILC; The Gender Pay Gap. Eurostat, Luxembourg.

[Hullinger and Schoch (2009)]hullingerschoch2009 Hulliger B. and Schoch T. (2009). The Robustification of the Quintile Share Ratio, in the Proceedings of New Technologies and Techniques in Statistics (NTTS). Eurostat, Brussel.

[Kpanzou (2011)]kpanzou2011 Kpanzou T.A. (2011). Statistical Inference for Inequality Measures based on Semi-Parametric Estimators. Ph.D. dissertation, Stellenbosch University.

[Kpanzou (2014)]kpanzou2014 Kpanzou, T.A. (2014). Asymptotic distribution of the quintile share ratio estimator. Afrika Statistika, 9 (1), 659-670.

[Kpanzou (2015)]kpanzou2015 Kpanzou, T.A. (2015). On the Influence Function of the Quintile Share Ratio. Communications in Statistics – Simulation and Computation, 44, 2492–2499.

[Langel and Tillé (2011)]langeltille2011 Langel M. and Tillé Y. (2011). Statistical inference for the quintile share ratio. Journal of Statistical Planning and Inference, 141, 2976-2985.

[Lo (2013)]lo2013 Lo, G. S. (2013). The Generalized Poverty Index. Far East Journal of Theoretical Statistics, 42(1), 1-22.

[Lo and Mergane (2013)]loandmergane2013 Lo, G. S. and Mergane P. D. (2013). On the influence of the Theil-like inequality measure
on the growth. Applied Mathematics, 4, 986-1000. doi:10.4236/am.2013.47136, arXiv:1210.3190.




