 to be $0$.
$$
0 = \frac{\sum _{i = 1}^ n X_ i}{\theta } - \frac{nK - \sum _{i = 1}^ n X_ i}{1 - \theta }.
$$
The above holds when
$$
p = \frac{1}{nK} \sum _{i = 1}^ n X_ i.
$$
Therefore, the right-hand side is the MLE for this statistical model.

#### $\chi^2$-Test for a Family of Distribution

Now, we return to the following more general statistical set-up.

Let $X_1, \ldots, X_ n \stackrel{iid}{\sim } \mathbf{P}$ denote iid discrete random variables supported on ${0,..., K}$. We will decide between the following null and alternative hypotheses.
$$
H_0:  \mathbf{P}\in \{  \text {Bin}(K, \theta ) \} _{\theta \in (0,1)}\\
H_1:  \mathbf{P}\notin \{  \text {Bin}(K, \theta ) \} _{\theta \in (0,1)},
$$
Let $f_\theta$​ denote the PMF of the distribution $\text {Bin}(K, \theta )$​, and let $\hat{\theta}$​ denote the MLE of the parameter $\theta$ from the previous problem.

Further, let $N_j$​ denote the number of times that $j (j \in \{0,1,..., K\})$​ appears in the data set $\, X_1, \ldots, X_ n\,$​ (So that $\sum _{j=0}^{K} N_ j =n$​) The $\chi^2$ test statistic for this hypothesis test is defined to be 
$$
T_ n := n\sum _{j =0}^ K \frac{\left( \frac{N_ j}{n} - {{f_{\widehat{\theta }}(j)}}  \right)^2}{{{f_{\widehat{\theta }}(j)}}  }.
$$
This statistic is different from before. Previously, under the null hypothesis, $\mathbf{P}(X=j)=p_ j\,$ for some fixed $p_j$. Here, instead, we use $f_{\widehat{\theta }}(j)\,$ to estimate $\mathbf{P}(X=j)\,$. This statistic still converges in distribution to a $\chi^2$​ distribution, but the number of degrees of freedom is smaller.

#### Degrees of Freedom for Test for a Family of Distribution

More generally, to test if a distribution $\mathbf{P}$​ is described by some member of a family of discrete distributions $\{  \mathbf{P}_{\theta } \} _{\theta \in \Theta \subset \mathbb {R}^ d}$​ where $\Theta \subset \mathbb {R}^ d$​ is $d$​-dimensional, with support $\{0,1,2,...,K\}$​ and PMF $f_\theta$​, i.e. to test the hypotheses:
$$
H_0: \mathbf{P}\in \{  \mathbf{P}_\theta \} _{\theta \in \Theta }\\
H_1: \mathbf{P}\notin \{  \mathbf{P}_\theta \} _{\theta \in \Theta }\\
$$
then if indeed $\mathbf{P}\in \{  \mathbf{P}_{\theta } \} _{\theta \in \Theta \subset \mathbb {R}^ d}$​ (i.e. the null hypothesis $H_0$ holds), and if in addition some technical assumption hold, then we have that
$$
T_ n:= n\sum _{j =0}^ K \frac{\left( \frac{N_ j}{n} - f_{\widehat{\theta }}(j) \right)^2}{ f_{\widehat{\theta }}(j) } \xrightarrow [n \to \infty ]{(d)} \chi ^2_{(K+1) - d - 1}.
$$
Note that $K+1$​​​ is the support size of $\mathbf{P}_\theta$​​​ (for all $\theta$​​).

In our example testing for a binomial distribution, the parameter $\theta$ is one-dimensional, i.e. $d=1$. Therefore, under the null hypothesis $H_0$, it holds that 
$$
T_ n \xrightarrow [n \to \infty ]{(d)} \chi ^2_{(K+1) - 1 - 1} = \chi ^2_{K-1}.
$$

> #### Exercise 88
>
> Consider the same statistical set-up as above. In particular, we have the test statistic
> $$
> T_ n := n \sum _{j =0}^ K \frac{\left( \frac{N_ j}{n} - f_{\widehat{\theta }}(j) \right)^2}{ f_{\widehat{\theta }}(j) }.
> $$
> where $\hat{\theta}$ is the MLE for the binomial statistical model $(\{ 0,1, \ldots, K\}, \{  \text {Bin}(K, \theta ) \} _{\theta \in (0,1)})$.
>
> We define our test to be
> $$
> \psi _ n = \mathbf{1}( T_ n > \tau ),
> $$
> where $\tau$​ is a threshold that you will specify. For the remainder of this page, we will assume that $K=3$​ (The sample space is $\{0,1,2,3\}$​).
>
> 1. What value of $\tau$​​ should be chosen so that $\psi_n$​​ is a test of asymptotic level $5\%$​​​?
>
> 2. Suppose we observe a data set consisting of $1000$​​ Observations as described in the following format: $N_i$​​, number of observations of $i$​):
>    $$
>    \begin{aligned}
>    i \quad & \quad N_i\\
>    0 \quad & \quad 339\\
>    1 \quad & \quad 455\\
>    2 \quad & \quad 180\\
>    3 \quad & \quad 26\\
>    \end{aligned}
>    $$
>    What is the value of the test statistic $T_n$​ for this data set? What is the p-value of this data set with respect to the test $\psi_{1000}$​? If $\psi_n$ is designed to have level $5\%$, would you reject or fail to reject on the given data set?
>
> **Solution:**
>
> 1. Since $K=3$​ and $d=1$​, we know that the limiting distribution of $T_n$​ is $\chi^2_2$​. Thus, the asymptotic level is the value $\tau$​ such that 
>    $$
>    \lim _{n \to \infty } P( T_ n > \tau ) = P( Z > \tau ) = 0.05
>    $$
>    Where $Z \sim \chi_2^2$. Hence, $\tau$ should be chosen to be $5.991$.
>
> 2. Observe that the MLE is given by
>    $$
>    \widehat{\theta } = \frac{1}{3 \cdot 1000} (455 + 2 \cdot 180 + 3 \cdot 26 ) \approx 0.29767.
>    $$
>    Thus for this data set,
>    $$
>    \begin{aligned}
>    T_n &= 1000 \cdot \bigg( \frac{\left(\frac{339}{1000} - \binom {3}{0} (0.2977^0) (0.7023)^{3 - 0} \right)^2}{\binom {3}{0} (0.2977^0) (0.7023)^{3 - 0} } + \frac{\left(\frac{455}{1000} - \binom {3}{1} (0.2977^1) (0.7023)^{3 - 1} \right)^2}{\binom {3}{1} (0.2977^1) (0.7023)^{3 - 1} } + \\
>    &\frac{\left(\frac{180}{1000} - \binom {3}{2} (0.2977^2) (0.7023)^{3 - 2} \right)^2}{\binom {3}{2} (0.2977^2) (0.7023)^{3 - 2} } + \frac{\left(\frac{26}{1000} - \binom {3}{3} (0.2977^3) (0.7023)^{3 - 3} \right)^2}{\binom {3}{3} (0.2977^3) (0.7023)^{3 - 3} } \bigg)\\
>    & \approx 0.8829
>    \end{aligned}
>    $$
>    The asymptotic p-value for this data set is given by
>    $$
>    \lim _{n \to \infty } P(T_ n > 0.8829) = P(Z > 0.8829).
>    $$
