Combining Empirical Likelihood and Robust Estimation Methods for Linear Regression Models
Şenay Özdemir Olcay Arslan
stat.OT



Combining Empirical Likelihood and Robust Estimation Methods for Linear Regression Models




aŞenay Özdemir and bOlcay Arslan




aDepartment of Statistics, Afyon Kocatepe University, 03200, Afyonkarahisar, Turkey, senayozdemir@aku.edu.tr

bDepartment of Statistics, Ankara University, 06100, Ankara, Turkey




Abstract

 Ordinary least square (OLS),  maximum likelihood (ML) and  robust methods are the widely used methods to estimate the parameters of a linear regression model. It is well known that these methods perform well under some  distributional assumptions on  error terms. However,  these distributional  assumptions on the errors may not be appropriate for some data sets. In these case, nonparametric  methods  may be considered to carry on the regression analysis. Empirical likelihood (EL) method is one of these nonparametric methods.  The EL method maximizes a function, which is multiplication of the unknown probabilities corresponding to each observation, under some constraints  inherited  from  the normal equations in OLS estimation method.  However, it is well known that the OLS method has poor performance when there are some outliers in the data.  In this paper, we consider the EL method with robustifyed constraints. The robustification of the constraints  is done by using the  robust M estimation  methods  for regression.  We provide a small simulation study and a real data example to demonstrate the capability of the robust EL method to handle unusual observations in the data. The simulation and real data results reveal that robust constraints are needed when heavy tailedness and/or outliers are possible in the data.




Keywords:Empirical likelihood; Linear regression; M estimation



Class codes:







§ INTRODUCTION


Consider the linear regression model


    Y_i=X_i^Tβ+ε_i  for   i=1,2,…,n


where Y_i  ∈ R is the response variable, 𝐗_i∈ R^k is the k-dimensional vector of the explanatory variables, β∈ R^k is an unknown k-dimensional parameter vector and ε_i's are the independent and identically distributed (iid) errors with E(ε_i)=0 and Var(ε_i)=σ^2. The regression equation given in (<ref>) can also be written in matrix notation as


    Y=Xβ+ε,


where X_n× k is the design matrix, Y is the response vector, and ε is the vector of ε _i.


The  simplest way to estimate the parameters of a linear regression model is to use the OLS method. The OLS estimators for β can be obtain by minimizing the following objective function:



    1/n∑_i=1^n(Y_i- X_i^Tβ)^2.


Taking the derivative of this function with respect to β  and setting to zero gives the following estimating equation, which is known as the normal equations in the OLS:


    1/n∑_i=1^n X_i(Y_i-X_i^Tβ)=0.


From this equation  the OLS  estimator β̂ is obtained. An unbiased estimator for  σ^2  can be obtained  using   residuals sum of squares  after the OLS estimator for the regression parameter  is obtained.  The following  objective function, which is the negative of the log-likelihood function  under normally distributed error terms,can also be minimized to obtain estimators for   β  and σ^2.


    1/2σ^2∑_i=1^n(Y_i-X_i^Tβ)^2 + nlogσ.


The ML  estimators  will be the solutions of the equation given in (<ref>) and  the following equation


    1/n∑_i=1^n((Y_i- X_i^Tβ)^2-σ^2)=0.


Note that the OLS  and the ML estimators for the regression parameters are the same under the normality assumption on error.


It is known that the OLS estimators (also the ML estimators) are the minimum variance unbiased estimates for model parameters under the assumption that ε_i are normally distributed. However, these estimators are dramatically  effected  when fundamental assumptions are unfulfilled by the nature of the data. It is well known that OLS estimators (or ML estimators under the normality assumption)  are very sensitive to outliers or departure from normality ( such as heavy-tailed error).  Even  a single  outlier may drastically  affect  the OLS estimators.  Therefore,  robust regression estimation methods  alternatives to the OLS estimation method  have been developed  to deal with outliers and/or heavy-tailed error distributions.  In this paper,  we will use the M regression estimation method  (Huber (1964)) that will be briefly described  in the following paragraph. One can see the books by Huber(1981) for the details of the robust regression estimation  methods,  including M estimation methods.

The widely  used method of robust regression is the M-estimation   introduced by Huber (1964,1973). Since, this class of estimators can be regarded as a generalization of maximum-likelihood estimation the  term M estimation is used.    The   M estimation method is designed to minimize an   objective function that is less  rapidly increasing  then the OLS objective  function.  This is done using a ρ  function  that is less rapidly incising than the squared function.   The M estimator for the regression parameter β is obtained by minimizing the following objective function with respect to β.



    1/n∑_i=1^nρ(Y_i-X_i^Tβ).



The function ρ   should be nonnegative,  nondecreasing and ρ(0)=0. The M estimation method   includes  OLS,  ML and  Least absolute deviation (LAD) methods  with the choice of   ρ(t)=t^2,   -log(f(t)), and |t|, respectively.  In this minimization problem,  contribution of each observation to  the sum  is determined by the function ρ in the sense that  the  observations with large residuals have small contributions to the sum.   Consequently, since  the outlying observation will   have small  contribution to the sum due to the ρ function  they will  not drastically effect the resulting   estimator.  If ρ is differentiable, differentiating (<ref>)   with respect to β and setting to zero yields the following  M estimation equation


    1/n∑_i=1^nψ(Y_i-X_i^Tβ)X_i=0,


where ψ=ρ'.

Note that in  robust statistical analysis, the influence function  is one way   to measure the robustness of an estimator and it is desired to have a bounded influence function.  In M estimation method the function ψ determines the shape of the influence function. Therefore, if ψ  is nonincreasing  (may be equal to zero beyond some threshold  or tend to zero in larger values of its argument) the corresponding M estimator will  have bounded influence  function.  In regression M estimation bounded ψ  function only  controls the large residuals; that is,  the estimator will be resistent to the outliers in  y direction.  To   control the outliers in x-direction we have to use other robust   estimation   methods such as   generalized M  estimation or MM regression estimation methods.   In this paper,  we will  only combine  the regression M estimation with the EL method and save the   others for our next project.  In robust statistics literature   several  different ρ  functions are proposed,  but  Huber  and Tukey (bisquare) ρ  functions are the widely  used ones.  The Huber ρ  and   ψ  functions are



    ρ=(Y_i-X_i^Tβ)^2    ,|Y_i-X_i^Tβ|≤ k 
    
               2k|Y_i-X_i^Tβ|-k^2    ,|Y_i-X_i^Tβ|>k



    ψ=(Y_i-X_i^Tβ)     ,|Y_i-X_i^Tβ|≤ k 
    
               sgn(Y_i-X_i^Tβ)k     ,|Y_i-X_i^Tβ|>k


Similarly,  The bisquare (Tukey) ρ  and   ψ  functions are



    ρ=k^2/6(1-(1-(Y_i-X_i^Tβ/k)^2)^3)     ,|Y_i-X_i^Tβ|≤ k 
    k^2/6    ,|Y_i-X_i^Tβ|>k



    ψ=(1-(Y_i-X_i^Tβ/k))^2(Y_i-X_i^Tβ)     ,|Y_i-X_i^Tβ|≤ k
    
          0     ,|Y_i-X_i^Tβ|>k



Concerning the estimation of   σ^2    along  with  β  the  following objective   function can be minimized to obtain  M estimators for  β  and σ^2  (see Huber and Ronchetti, 2009, Chapter 7.7).




    1/n∑_i=1^nρ(Y_i-X_i^Tβ/σ)σ+aσ,



Taking the derivative of this objective function  and setting to  zero  yield  the following M estimating   equations   for  β  and σ^2



    1/n∑_i=1^nψ(Y_i-X_i^Tβ/σ) X_i=0,



    1/n∑_i=1^nρ_0(Y_i-X_i^Tβ/σ)=a



where ρ_0(r) =rψ(r)-ρ(r).   a>0 is added in order to get consistency  of the scale estimator under normality assumption of the error and to get the classical  OLS estimator for the scale when ρ(x)=x^2/2. Note that, since throughout this study  we are only interested  for estimating  the regression parameters  we can assume that σ is fixed.  This can be done either by estimating σ beforehand or assuming that it is known, as it is suggested in the book by Maronna et al. (2006) (Chepters 4.4.1  and4.4.2.).

One can see the books Huber (2009) and Maronna (2006) for further  details on  M estimators for regression and its properties.



If  we do not want to use OLS or robust methods  or not want to assume any distribution for the error terms we can also use nonparametric methods as alternatives to  these mentioned methods  for estimation and inference in a linear regression model. One of these nonparametric methods is the EL method  which was introduced by Owen(1988). This method is an alternative to likelihood method when there is no distribution assumption for error terms. In EL method, it is assumed that each observation has unknown p_i probability weight for i=1,2,…,n. The aim of the EL method is to estimate the regression parameters by maximizing an empirical likelihood function defined as the multiplication of these p_i s' , which is developed modifying  nonparametric likelihood ratio function, under some constraints  related to the parameters of interest. However, since the constraints used in EL  method (see Section 2)  are   very similar  to   the normal equations in   OLS or  likelihood equations under normally distributed errors  the corresponding estimators will be very sensitive to the non-normality or unusual  observations (outliers)  in data.  In this paper,  we will carry on EL estimation in regression using robust constraint  borrowed  from the robust estimation equations   described in previous paragraph. Since,  we will   combine the M estimation methods with the EL method we expect that the   resulting   estimator for the regression parameters will be resistent  to the outliers   in y-direction.  Simulation study  and real  data example results  show  that this expectation comes true.

Note that the reason we keep 1/n multiplication in   all the objective  functions  and the estimating equations is to emphasize that each observation equally contributes (1/n) to the minimization procedure.   In the EL method, which will be  described  in Section 2, this term will be replaced by an unknown probability p_i corresponding to each observation. Therefore, in empirical likelihood procedure   contribution of each observation will  be different  and this contributions (probabilities) need to be estimated.

The rest of the paper is organized as follows. In next section, after we briefly describe the EL method we will move on the EL estimation  with robust constraint for linear regression models. In Section 3 we will provide a small  simulation study and  a real data example to  illustrate the necessity  of robust constraints in EL  estimation. The paper is finalized  with a conclusion section.




§ EMPIRICAL LIKELIHOOD  ESTIMATION FOR THE PARAMETERS OF A LINEAR REGRESSION MODEL WITH ROBUST CONSTRAINTS


In this section,   we will first outline   the  EL method with classical constraints.    Then, we will move on our proposal that combines   the EL estimation method  with some robust constraints.



 §.§ Empirical Likelihood  Estimation

Consider the linear regression model given in equation (<ref>)  with the same assumptions  given there. The EL estimators will be the solution of the   the following constrained optimization problem.  Specifically, if we are only interested for estimating the regression parameter β the EL method maximizes the  following empirical likelihood function


    L(β)=∏_i^np_i


with  respect to p_i≥ 0 and β  under the  following constraints



    ∑_i^np_i=1



    ∑_i^np_i(Y_i-X_i^Tβ)X_i=0


where p_1,p_2,…,p_n are the probability weights of the observations.  Note that the constraint given in equation (<ref>) is very similar to the normal equation given in (<ref>).  The only difference is  the unknown probability weights assigned for each observation. In former equation each observation has equal probability 1/n, but in later each observation has a different p_i probability  and these probabilities are unknown and  need to be estimated. Further,  if  we are interested  for estimating σ^2 along with the regression parameters the following constraint, which is  motivated  form the equation (<ref>), is  added to   constrained optimization problem defined above.


    ∑_i^np_i[(Y_i-X_i^Tβ)^2-σ^2]=0.



The empirical likelihood method can be used estimating parameters, constructing confidence regions, testing statistical hypothesis, etc. Briefly, it is a useful tool for making statistical inference when it is not too easy to assign a distribution to data. There are several remarkable studies on the EL method after it was introduced by Owen (1988, 1990, 1991). In these papers, Owen   used the empirical likelihood for constructing confidence regions and estimating parameters of linear regression models. Hall and Scala (1990) studied on main features of the empirical likelihood, Kolaczyk (1994) adapted it in generalized linear regression model, Qin and Lawless (1994) combined general estimating equations and the empirical likelihood, Chen et. all (1993,1994 1996, 2003,2009) handled this method for constructing confidence regions, parameter estimation with additional constraints, Newey and Smith  (2004) studied about properties of generalized methods of moments and generalized empirical likelihood estimators, Bondell and Stefanski (2013) suggested a robust estimator modified the generalized empirical likelihood. Recently, Ozdemir and Arslan (2017) have suggested an  alternative   algorithm to obtain EL  estimates  using the primal optimization problem. As we have already  pointed out we are proposing to assign robust constraints   to gain  robustness.

The estimation of p_is and the model parameters β and σ^2 can be done by maximizing the function given in equation (<ref>) under the constraints <ref>  and <ref>. For simplicity we assume that variance is known  and consider only estimation of β.   Therefore, the EL  function should be jointly maximized with respect to p_is  and β. One way to handle this constraint maximization problem is  as follows.  First, fix the regression parameter  vector β and consider maximizing  the log-EL  function  with respect to [p_1,p_2,…,p_n]^T  under the constraints given above. This procedure is called as  profiled out. Once this is done, the profile likelihood  will be a function of β.  Then,  we can maximize the profile likelihood to obtain  the EL  estimator of β. This problem will be easily   handled  using Lagrange multiplier  method.   Therefore, setting the problem in Lagrangian form, the Lagrange function associated  with this constrained maximization  problem is



    L(p,β,λ_0,λ_1^T)=∑_i=1^nlog(p_i)-λ_0(∑_i=1^np_i - 1)-nλ_1^T(∑_i=1^n p_iX_i(Y_i-X_i^Tβ))


where 𝐩=[p_1,p_2,…,p_n]^T   are the vector of probabilities, and  λ_0∈ R^1 and λ_1^T∈ R^p are the Lagrange multipliers.  Taking the derivatives of equation (<ref>) with respect to each p_i, and setting them to zero yields


    p_i=1/n(1+λ^TX_i(Y_i-X_i^Tβ)),


and λ=-nλ_1. By using equation (<ref>) in the log-EL function we find


    L(β,λ)= -∑_i=1^nlog(1+λ^TX_i(Y_i-X_i^Tβ))-nlog n.


For  a given β  the Lagrange multiplier λ_1(β)  will  be obtained  as the solution of the following  minimization problem


    λ_1(β)= argmin_λ_1(-L(β,λ_1)).


Note that since this minimization problem will not have an explicit solution  numerical  methods  should be used to find the minimizer.  Now, using this in equation    yields  the  following   function


    L(β)= -∑_i=1^nlog(1+λ_1(β)^TX_i(Y_i-X_i^Tβ))-nlog n.


Then, this function  will  be   maximized  to   obtain  the EL  estimator   for the regression coefficient  vector β.    That is the EL estimator of β  will  be


    β̂_EL= argmax_βL(β).


Again, numerical algorithms will be required to handle this maximization problem.    Note that  one can see   Owen(1989,2001)or Kitamura (2006)  for further   details about the computational issues of the EL  method.   In the following subsection we will  use  similar steps  to   maximize the EL  function under robust  constraints.





 §.§ Empirical Likelihood Estimation  with Robust Constraints


In this subsection   we will turn our attention  to the EL estimation using   robust constraints instead of the classical constraints.  In the EL method  the   unknown probabilistic weights p_i  are assigned for each observation  so that each observation will  have different  contribution to the estimation procedure. In some sense,  this  can be considered  as a weighting procedure of the normal equations in terms of probabilities for each observation. However, from our limited experience we observe that the probabilistic  weights are not satisfactorily   reduce the  effect of outliers on the estimators. Therefore,  extra care should be taken to reduce  the outliers  affect on estimation procedure. This   can be done by adapting the M estimating equation given in previous section.

Suppose that we are  interested   for  estimating  the regression parameter β. Then,   we will  maximize  the following EL function


    L_EL(β)=∏_i=1^np_i


under the  constraints


    ∑_i=1^np_i = 1


and


    ∑_i=1^np_iψ(Y_i-X_i^Tβ)X_i=0,


where  p_i≥ 0. The second   constraint  can be regarded  as robust version  of the classical   constraint  related    to  the regression parameters. Here,   ψ function is a  nonincreasing function of the residuals for Huber case  and a decreasing   function of   residuals   for Tukey   function.  Thus,   unusual observations with large   residuals  will  receive   small ψ values   so that the corresponding observations will  not completely ruin   the  estimation procedure.  Further, if we are also interested for  estimating  σ^2 along  with the  regression  parameters the simultaneous M estimating equations given in  equations (<ref>)-(<ref>)  can  be   adapted and used in the maximization   problem  instead of  the classical   constraints for  β and σ^2  given in  equations (<ref>)and (<ref>). The adaptation of those  equations    will be done  using  p_i's  instead  of  using  1/n. However,   since in this paper  we are only interested  for the regression parameters we will  solve the above maximization   problem and   not  add  a robust  constraint for σ^2.

The robust EL  estimator for β will be defined  as the value of β  that maximizes  the log-EL  function  under the  constraints   given in equations (<ref>)  and (<ref>).  We will  again  use the  maximization procedure   described  in previous subsection. In this case, the  Lagrange function     will  be




    L(p,β,λ_0,λ_1^T)=∑_i=1^nlog(p_i)-λ_0(∑_i=1^np_i - 1)-nλ_1^T(∑_i=1^n  p_iψ(Y_i-X_i^Tβ)X_i).


Taking the derivatives of the Lagrange function  with  respect to  p_i, λ_0 and λ_1, setting them to zero and solving the  corresponding first order  equations for this  maximization problem yield


    p_i=1/n(1+λ^Tψ(Y_i-X_i^Tβ)X_i),  for    i=1,2,...,n,


and λ=-nλ_1.   Then,  using these   p_is    in the log-EL function  we get the  following objective function


    L(β,λ)= -∑_i=1^nlog(1+λ^Tψ(Y_i-X_i^Tβ)X_i)-nlog n,


which is the function of λ and β.    Further,   for a given regression parameter vector β the Lagrange multiplier λ   can be obtained from  the following minimization problem



    λ(β)=argmin_λ-∑_i=1^nlog(1+λ^Tψ(Y_i-X_i^Tβ)X_i)-nlog n.


Since, the  solution of this minimization problem  cannot be obtained explicitly,  numerical   algorithm  should be used to get the solution. Using λ(β),  the profile log-EL  function will  be

 
    L(β)= -∑_i=1^nlog(1+λ(β)^Tψ(Y_i-X_i^Tβ)X_i)-nlog n.



Then, the robust  EL  estimator   for the  regression parameter vector β  will  be obtained as

    β̂_REL= argmax_β(L(β)).

Again,  numerical algorithms  are necessary   to  preform  this  maximization  problem to  obtain  the robust  EL  estimates for the regression parameter   vector β.



§ SIMULATION STUDY


To evaluate the performance of the proposed robust EL estimation procedure, we conduct a small simulation study.  We compare the classical  EL  method with the  robust   EL method (EL  with robust constraints) to  estimate  the parameters  of a  linear regression model.  We assume that  the error   variance is  known and  only deal with     estimating the regression parameters.  We take the  sample sizes as n=30,50,100  and  the dimensions  of the unknown   regression parameter  vector as 2,5,15.  The  standard normal  distribution (N(0,1)) and the contaminated   normal  distribution ((0.90)N(0,1)+(0.10)N(20,1))  are  used as the   error   distribution for the  regression model. The second error distribution is chosen  as to add  some  outliers in the data.  The  regressors   are also generated from the standard normal  distributions.   The unknown parameter vectors are also determined randomly from a normal distribution with mean μ and variance τ.  The dependent variable is generated using the regression model given in equation (<ref>).  The MSE values and the relative efficiency of estimators with respect to the OLS are calculated to compare the performance of the   considered estimators.  All  the simulation  scenarios are repeated  100  times. The relative efficiencies are calculated using  the formula


    RE=∑_j=1^100β_j-β/∑_j=1^100β_j^OLS-β


where β indicates the EL, EL-Hub, EL-Tukey, while β^OLS symbolizes the OLS.

The simulation results are summarized in Tables <ref>-<ref>. From these tables we observe that, without contamination all the estimators  have similar  performance with small MSE values  for the dimensions 2 and 5. When  k=15 the MSE values  of the estimators  are getting worse for  all the cases compare to  the  smaller dimension cases,  but  even for this case   the EL and  robust  EL  estimators   seem   superior   to the OLS estimators.  Smaller   behavior  is observed from the table of relative efficiencies.

In Tables <ref> and <ref>  we report  the simulation results for the contaminated   error distribution.   That is,    these   are the simulation   results for the  outlier  case.   From  these  result we observe that when we introduce contamination  the  robust EL  estimators  have better  performance  compare  to the  classical EL and the OLS estimators  in terms of the MSE  values.    Among   the two  robust   estimators   the Tukey    case  seems superior   to the Huber case in terms   of the MSE values.  Again,   when  the dimension  is large and  the  sample size is  small   all  the estimators  have large  MSE  values. However,  compare   to the classical  ones   the MSE values for the robust  estimators  are relatively  small. Overall,  from our limited simulation study  we observe that for all the simulation settings considered in this study, the robust  EL  estimators
have  comparable  performance in terms of MSE  values. Therefore, robust   constraints   should be considered   in  case of  potential     outliers   in the   data set.





§ A REAL DATA EXAMPLE:INTERNATIONAL PHONE CALLS FROM BELGIUM 


To illustrate proposed robust EL   estimators  we will  use a data set that is a widely used in  robust regression estimation  literature to  evaluate the performance  of the robust  estimation methods.  The data set, which is  taken from  the   book  by Rousseeuw and Leory (1987), contains of  the total number (in tens of millions) of international phone calls made over the years. Table <ref>   displays the data set.  The scatter plot (Figure <ref>) of the  data set shows an increasing  trend  over the years.  However,  it can be noticed  that  the data set  is heavily   contaminated  and these points  are the outliers in y-direction. Therefore,  using  classical  methods such as  OLS,   will  not provide adequate estimation. For instance,   the fitted line  obtained from the OLS method    is  highly  affected  from the contaminated   points (Figure <ref>).   Therefore,  robust  regression estimation methods have been  used   to  get   better  fit  to this data set.  One can see  Rousseeuw and Leory (1987) for further details   and the results  of the robust  regression estimation methods.

Now  we will turn our attention   to the EL  estimation for the regression parameters.  If we   apply   classical   and robust   EL  estimation methods considered  in this paper    we   obtain the   regression estimates   provided   in Table  <ref>.   This table also  contains the  OLS  estimates.   From   this table we can see  that  unlike  the OLS  methods,   the  EL  methods    give  closers   fits that are obtained from the   robust regression estimation  methods.  We   further   observe  from   Figure <ref> that  among the EL    estimates the  robust EL estimates obtained from the Tukey   constraint   gives  the  best fit  to data avoiding  contaminated   points.   The  line obtained from   these  estimates   fits the  majority  of  the data points.





§ CONCLUSION


  The EL  method can be used instead of classical estimation like the OLS when there is no distribution assumption on error terms. The EL method maximizes a function  of  unknown probabilities corresponding to each observation under some constraints  related  to the unknown probabilities and the moments equations of the unknown parameters of interest. In the classical  EL  estimation for regression the constraints  related  to  the  parameters   of interest  are very similar to the normal equations in the OLS method. The only difference is: in OLS case we take average, however in EL case the constraints are a weighted form of normal equations obtained in the OLS method. Here, the weights are the unknown probabilities used in the EL function. However, it is well known that the OLS method  and hence the  corresponding  equations  have  poor performance when  there are some outliers in the data. Although, the observations in the constraints are weighted using the unknown probabilities,  these weights are not satisfactory to deal with the outliers. Therefore,   some  extra care should be  taken  to reduce the affect of outliers on the  estimation procedure.   In this paper, we  have  considered the  EL method with robustifyed constraints. The robustification has been  done by using some weight functions borrowed from robust M estimation. We  have  provided  a small simulation study and a real data example to demonstrate the capability of the robust EL method to handle unusual observations in the data. The simulation and the real data example results have showed  that   the  robust constraints have plausible affect on the  estimators  when heavy tailedness and/or outliers are possible in the data.




99

[1]Bondel2013 Bondell, H. D. and Stefanski, L.A.  2013. Efficient Robust Regression via Two-Stage Generalized Empirical Likelihood, Journal of the American Statistical Association, 108:502, 644-655.

[2]Chen1993Chen, S.X. 1993. On the accuracy of empirical likelihood confidence regions for linear regression model. Ann. Inst. Statist. Math., 45, 621-637.

[3]Chen1994Chen, S.X. 1994. Empirical likelihood confidence intervals for linear regression coefficients.J. Multiv. Anal., 49, 24-40.

[4]Chen1996Chen, S.X. 1996. Empirical likelihood confidence intervals for nonparametric density estimation. Biometrika, 83, 329-341.

[5]Chen2003Chen, S.X. and Cui, H. 2003. An extended empirical likelihood for generalized linear models. Statist. Sinica, 13, 69-81.

[6]Chen2009Chen, S.X. and  Keilegom, I. V. 2009.  A review on empirical likelihood methods for regression. TEST, 19,3,415-447.

[7]Hall1990Hall, P. and La Scala, B. 1990. Methodology and algorithms of empirical likelihood. Internat. Statist. Review, 58, 109-127.

[8]Hampel1986Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J. and Stahel, W. A. 1986. Robust Statistics:The Approach Based on Influence Functions. New York: Wiley.
[9]Huber1964Huber P.J. 1964. Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35.1 , 73–101.

[10]Huber1973 Huber P.J. 1973. Robust regression: asymptotics, conjectures and Monte Carlo. The Annals of Statistics , 799–821.


[11]Huber2009 Huber P. and Ronchetti E. M. 2009.Robust Statistics. New York: Wiley. 

[12]Kitamura2006 Kitamura Y. 2006. Empirical likelihood methods in econometrics: Theory and practice. Available at: http://citeseerx.ist.psu.edu/viewdoc/download? doi=10.1.1.126.9883&rep=rep1&type=pdf

[13]Kolaczyk1994Kolaczyk, E.D. 1994. Empirical likelihood for generalized linear model. Statist. Sinica, 4, 199-218.

[N14]Newey2004 Newey W. and Smith  R.J. 2004. Higher-order properties of GMM and generalized empirical likelihood estimators. Econometrica, 72, 219–255.

[15]Maronna2006Maronna, R.A., Martin, R.D. and Yohai, V.J. 2006.Robust Statistics: Theory and Methods. Wiley, New York.

[16]Owen1988Owen, A.B. 1988. Empirical likelihood ratio confidence intervals for a single functional. Biometrika, 75, 237-249.

[17]Owen1990Owen, A.B. 1990. Empirical likelihood confidence regions. Ann. Statist., 18, 90-120.

[18]Owen1991Owen, A.B. 1991. Empirical likelihood for linear models. Ann. Statist., 19, 1725-1747.

[19]Owen2001Owen, A. 2001. Empirical Likelihood. Chapman and Hall, New York.

[20]Owen2013Owen, A. 2013. Self-concordance for empirical likelihood. Canadian Journal of Statistics, 41(3), 387-397.

[21]Ozdemir2017Ozdemir,S. and Arslan, O. 2017. An alternative algorithm of the empirical likelihood estimation for the parameter of a linear regression model. (submitted paper: May,2017 )

[22]Rousseeuw1987Rousseeuw, J. and Leroy, A.M. 1987. Robust regression and outlier Detection. New York: Wiley. p:26.

[23]Qin1994Qin, J. and Lawless, J. 1994. Empirical likelihood and general estimating equations. Ann. Statist., 22, 300-325.
































