# Asymptotic distribution for MLE of shifted exponential distribution

Suppose we have $$X_1,...,X_n$$ iid the shifted exponential distribution:

$$f(x)=\lambda e^{-\lambda(x-\theta)}, x\ge \theta$$

I have figured out both the MLE for $$\lambda$$ and $$\theta$$, which are $$\hat \lambda = \frac{1}{\bar X - X_{min}}$$ and $$\hat \theta =X_{min}$$.

I also found the asymptotic distribution of $$\hat \theta$$:

$$\sqrt{n}(\hat \theta-\theta) \rightarrow 0$$

Now I'm stuck at deriving the asymptotic distribution of $$\hat \lambda$$ and showing that it is a consistent estimator. How do you do this?

Thanks!

• The asymptotic distribution of $\hat\theta$ is using the wrong scale: it should be $n$ not $\sqrt n$. See this answer. Jun 4 '20 at 4:35
• To arrive at a non-degenerate limiting distribution of $\hat\theta$, you ought to use $n(\hat\theta-\theta)\sim \mathsf{Exp}(1)$ as mentioned above. This is an exact distribution which is naturally also the asymptotic distribution. Jun 4 '20 at 7:27

For consistency, by the weak law of large numbers $$\bar X_n \stackrel{\text p}\to \frac 1\lambda + \theta$$ and $$X_\min \stackrel{\text p}\to \theta$$ so by Slutsky $$\bar X_n - X_\min \stackrel{\text p}\to \frac 1\lambda.$$ By assumption $$\lambda > 0$$ so the map $$x \mapsto x^{-1}$$ is continuous, and the continuous mapping theorem finishes the job.

For the asymptotic distribution, by the standard CLT we know $$\sqrt n (\bar X_n - \theta -\lambda^{-1}) \stackrel{\text d}\to \mathcal N(0, \lambda^{-2})$$. Let $$Y_n = \sqrt n (\bar X_n - \theta - \lambda^{-1})$$ and consider $$\sqrt n (\bar X_n - X_{\min,n} - \lambda^{-1}) = \sqrt n ([\bar X_n - \theta - \lambda^{-1}] - [X_{\min,n} - \theta])\\ = Y_n - Z_n$$ where $$Z_n := \sqrt n (X_{\min,n} - \theta)$$. You already worked out the asymptotic distribution of $$Z_n$$ so we can use that along with Slutsky again to conclude $$Y_n - Z_n \stackrel{\text d}\to \mathcal N(0, \lambda^{-2}).$$

You can now finish this off with the delta method.

Although you are also asking about the estimator $$\hat{\lambda}$$, I am going to note some things about $$\hat{\theta}$$. In this particular case it is quite easy to obtain the exact distribution of this estimator. Since you have a series of shifted exponential random variables, you can define the values $$Y_i = X_i - \theta$$ and you then have the associated series $$Y_1,Y_3,Y_3 ... \sim \text{IID Exp}(\lambda)$$. This gives the exact distribution:

$$\hat{\theta} = X_{(1)} = \theta+ Y_{(1)} \sim \theta + \text{Exp}(n \lambda).$$

Note that this gives the pivotal quantity $$n(\hat{\theta} - \theta) \sim \text{Exp}(\lambda)$$. You can prove that $$\hat{\theta}$$ is a consistent estimator by computing the probability of a deviation larger than a specified level. For all $$\varepsilon >0$$ we have:

\begin{aligned} \mathbb{P}(|\hat{\theta} - \theta| < \varepsilon) = \mathbb{P}(\hat{\theta} - \theta< \varepsilon) = \exp(-n \lambda \varepsilon). \\[6pt] \end{aligned}

We therefore get the limiting result:

\begin{aligned} \lim_{n \rightarrow \infty} \mathbb{P}(|\hat{\theta} - \theta| < \varepsilon) = \lim_{n \rightarrow \infty} \exp(-n \lambda \varepsilon) = 0, \\[6pt] \end{aligned}

which is the required condition for weak consistency (i.e., convergence in probability of the estimator to the parameter it is estimating).

• any idea why exactly does the asymptotic normality of MLE not hold in this case? Any regularity condition broke? Nov 21 '20 at 19:07
• @MaverickMeerkat: The MLE occurs at a boundary point of the likelihood function, which breaks the ordinary regularity conditions.
– Ben
Nov 21 '20 at 20:51
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 1, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 30, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.9935221076011658, "perplexity": 237.10212992076586}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2021-43/segments/1634323583087.95/warc/CC-MAIN-20211015222918-20211016012918-00276.warc.gz"}