Masaki Yoshioka,Fuyuhiko Tanaka
Information-Geometric Approach for a One-Sided Truncated Exponential Family
truncated exponential family,non-regular model,common scale parameter,alpha-parallel prior,information geometry,statistical manifold,62B11,62F12
In information geometry, there has been extensive research on the deep connections between differential geometric structures, such as the Fisher metric and the α-connection, and the statistical theory for statistical models satisfying regularity conditions. However, the study of information geometry for non-regular statistical models is insufficient, and a one-sided truncated exponential family (oTEF) is one example of these models. In this paper, based on the asymptotic properties of maximum likelihood estimators, we provide a Riemannian metric for the oTEF. Furthermore, we demonstrate that the oTEF has an α = 1 parallel prior distribution and that the scalar curvature of a certain submodel, including the Pareto family, is a negative constant.
Information-Geometric Approach for a One-Sided Truncated Exponential Family

In information geometry, there has been extensive research on the deep connections between differential geometric structures, such as the Fisher metric and the α-connection, and the statistical theory for statistical models satisfying regularity conditions. However, the study of information geometry for non-regular statistical models is insufficient, and a one-sided truncated exponential family (oTEF) is one example of these models. In this paper, based on the asymptotic properties of maximum likelihood estimators, we provide a Riemannian metric for the oTEF. Furthermore, we demonstrate that the oTEF has an α = 1 parallel prior distribution and that the scalar curvature of a certain submodel, including the Pareto family, is a negative constant.

Information geometry is the study of the structure of statistical models using differential geometry. From the standpoint of geometry, a statistical model consisting of a collection of parameterized probability distributions can be regarded as a manifold. Then, when the statistical model satisfies a certain regularity condition, Chentsov’s theorem leads to a natural differential geometric structure [1]. This natural differential geometric structure consists of the Riemann metric defined from the Fisher information matrix and a one-parameter family of affine connections, called the Fisher metric and the α-connection, respectively. Information geometry of regular models has been studied for a long time, and deep relationships between the geometric structures and statistical models have been revealed (Amari [2] and Amari and Nagaoka [3]). However, the geometric properties of statistical models which do not satisfy regularity conditions are not sufficiently investigated. One reason is that Chentsov’s theorem cannot be applied to non-regular models; thus, a natural geometric structure is not established. For example, in regular models, it is possible to define the Fisher information matrix in two forms, but in statistical models where the support of the probability density function depends on the parameters, they do not coincide. In previous work, Amari [4] discussed the relationship between the Finsler geometry and non-regular models especially for the translation family. In the present study, we discuss information geometry for a one-sided truncated exponential family (oTEF) [5], a typical non-regular model. An oTEF is a statistical model with two parameters: the natural parameter θ and the truncation parameter γ. The support of the probability density function depends on the truncation parameter γ, which is what makes such a model non-regular. However, similar to the exponential family, the derivatives of the log-likelihood function, moments, and KL divergence can be explicitly described. Additionally, an oTEF model is a model containing many practical examples, such as the Pareto distribution family, truncated normal distribution family, and truncated exponential distribution family. The statistical properties of the oTEF have been studied for a long time, particularly in the area of parameter estimation theory (Bar-Lev [5], Akahira [6], and Akahira [7]). Recently, Nielsen [8] studied the relationship between the Kullback–Leibler divergence and the Bhattacharyya distance in the oTEF. Shemyakin [9] explored the Hellinger information matrix and its application to noninformative priors for multiparameter non-regular cases, which includes the oTEF, within the framework of Bayesian statistics. The family of Pareto distributions, a type of oTEF, is a topic of study in information geometry as well (Rylov [10], Peng et al. [11], Li et al. [12], and Sun et al. [13]). As a result, it is known that the Pareto model has constant curvature with respect to the Levi-Civita connection. However, it is impossible to introduce a natural geometric structure, so the geometric structure of the Pareto distribution family has been formally defined based on regular models. Consequently, the consistency with statistical theory remains to be determined. Moreover, multiple geometric structures exist depending on the regular model expressions used as references. For example, Peng et al. [11], Rylov [10], and Sun et al. [13] all define the geometric structure of the Pareto distribution family based on the Fisher metric and α-connection, but these structures are different. Therefore, the question of which geometric structure should be used remains unresolved. The results of the present study consist of two theorems: one regarding the α-parallel prior distribution and another concerning curvature. First of all, by comparing the relationship between the Fisher metric and the asymptotic behavior of the maximum likelihood estimators in regular models with those in the oTEF, we define the Riemannian metric for the oTEF. Additionally, we use a formal definition for the α-connection, similar to Sun et al. [13]. Under this geometric structure, we demonstrate two geometric properties. The first property is that the oTEF always has an α-parallel prior distribution with α = 1. The α-parallel prior distribution is a type of noninformative prior distribution used in Bayesian statistics and is an information-geometric extension of Jeffreys prior distribution. Although the α-parallel prior distribution with α = 1 always exists, this is not true for other α values. Such a phenomenon does not occur in regular models, highlighting the non-regularity. The second research result is related to the curvature of a submodel of the oTEF. We consider a submodel of the oTEF that includes common scale Pareto distribution families [14] and common location exponential distribution families [15] and demonstrate that its scalar curvature is constant. This result, stating that the scalar curvature of this submodel is constant, corresponds to a higher-dimensional version of the fact that the Pareto model has constant curvature [10]. The remainder of this paper is organized as follows. In Section 2, we review the geometric structure of regular statistical models and one-sided truncated exponential families, providing an overview of the relevant background. In Section 3, we provide a Riemannian metric and affine connection in the oTEF, giving the family a differential geometric structure. Under the geometric structure, we study the α-parallel prior to the oTEF in Section 4. Finally, Section 5 gives our derivation of the curvature for a submodel of the oTEF.

In this section, we review basic definitions and notation of information geometry for regular statistical models. See work by Amari [2] for more details. Let be a family of distributions on a sample space parametrized by , where is an open subset of . We treat the statistical model as an n-dimensional manifold. In this case, parameter takes the role of coordinates of the manifold . Furthermore, we assume that parameter and distribution have a one-to-one correspondence. Furthermore, suppose that all distributions in are absolutely continuous with respect to a dominating measure . We denote the probability density function of distribution with respect to as . In information geometry, we assume certain conditions on . Definition 1 (Regular statistical models). A statistical model is said to be regular if it satisfies the following conditions. 1. The support of the density functionis independent of . 2. For all , the function is on Θ. 3. Let . The elements of } are linearly independent as functions on χ. 4. For any functions in the following, partial differentiation and integration with respect to the measure μ are interchangeable as The above four conditions are referred to as regularity conditions. We will treat non-regular statistical models in later sections. In preparation, this subsection discusses the geometric structure of regular statistical models. Next, we introduce the Fisher metric and α-connection. Let be a regular statistical model and X be a random variable distributed according to . We usually use the following definition of the Riemannian metric and affine connections on . Definition 2 (Fisher metric). The Fisher metric is the Riemannian metric on defined by for . Here, the symbol denotes the expectation with respect to the observation X. Definition 3 (α-connection). The α-connection is an affine connection defined by the coefficients for and on the coordinate system θ. If satisfies the regularity conditions, the Fisher metric and the α-connection have several properties. For instance, there is a formula [16] convenient for calculating the Fisher metric: Furthermore, the α-connection and the −α-connection are dual. In other words, holds for all tangent vectors A, B, and C, where denotes the inner product given by the Fisher metric. In particular, the 0-connection is self-dual. Furthermore, the 0-connection corresponds to the Levi-Civita connection, defined by the coefficients

Next, we introduce volume elements on a regular statistical model and its relation to Bayesian statistics [17]. In Bayesian statistics, for a given statistical model , we need a probability distribution over the model parameter space, which is called a prior distribution or simply a prior. We often denote a prior density as . ( and ) A volume element on an n-dimensional oriented model manifold corresponds to a prior density function over the parameter space () in a one-to-one manner. For a prior , its corresponding volume element is an n-form (differential form of degree n) and is written as in the local coordinates. For example, in the two-dimensional Euclidian space (), the volume element is given by in the Cartesian coordinates . In the polar coordinates , it is written as . Now, let us explain noninformative priors (see, e.g., work by Robert [18] for details). If we have specific information on the parameter in advance, then the prior should reflect it, in which case it is often called a subjective prior. If not, we adopt a certain criterion and use a prior obtained through the criterion. Such priors are called noninformative priors or objective priors. In particular, the Jeffreys prior, which is given by , is the standard noninformative prior [19]. In information geometry, the Jeffreys prior is regarded as a 0-parallel volume element and was extended to an α-parallel volume element by Takeuchi and Amari [17]. The extensions are called α-parallel priors. They showed that the geometrical properties of regular models are deeply related to the existence of α-parallel priors. When a suitable geometric structure is defined on a non-regular statistical model, it is interesting to see the relationship between the geometrical properties and the existence of α-parallel priors. Later sections will discuss this topic. In preparation, in this subsection, we briefly summarize some definitions and facts of α-parallel priors for regular models. To define α-parallel priors, we introduce a geometric property of affine connections. Definition 4 (Equiaffine). Let be an n-dimensional manifold with an affine connection induced by a covariant derivative ∇. An affine connection ∇ is equiaffine if there exists a volume element ω such that holds everywhere in . Furthermore, such a volume element ω is said to be a parallel volume element with respect to ∇. The necessary and sufficient condition for an affine connection ∇ to be equiaffine is described by its curvature. The following proposition holds for a manifold with an affine connection ∇. Let be the components of the Riemannian curvature tensor [3] of ∇, defined as where denotes the connection coefficients of ∇. Proposition 1 (Nomizu and Sasaki [20]). The following conditions are equivalent: ∇ is equiaffine, , where . The Levi-Civita connection is always equiaffine (Nomizu and Sasaki [20]). Returning to the statistical model, we define the -parallel prior distribution. Let be an n-dimensional regular statistical manifold. It is shown that is equiaffine for some if and only if it is equiaffine for all . Such a statistical manifold is said to be statistically equiaffine. In a statistically equiaffine manifold, we may represent the -parallel volume element as for the coordinates , where . We take as a prior distribution on the parameter space . Definition 5 (-parallel prior). In a statistically equiaffine manifold, for fixed , we call the above form of an α-parallel prior. Since the 0-connection (Levi-Civita connection) is equiaffine, there always exists a 0-parallel prior, known as the Jeffreys prior. Takeuchi and Amari [17] give a necessary and sufficient condition for α-parallel priors to exist. Proposition 2 (Takeuchi and Amari [17]). For a model manifold , if then the α-parallel prior exists for any . Otherwise, only the 0-parallel prior exists.

In this section, we introduce a one-sided truncated exponential family, which is a typical non-regular model. Definition 6 (One-sided Truncated Exponential Family (Bar-Lev [5])). A one-sided truncated exponential family of distributions (oTEF) is a family of distributions with the density functions with respect to the Lebesgue measure. Here, are known parameters, C is a continuous function, and are absolutely continuous with over the interval . We say is the natural parameter and is the truncation parameter. We also denote the interval as I. In oTEF , the support of the density function depends on the truncation parameter . Then, the oTEF does not satisfy the regularity conditions. An oTEF differs from an exponential family at this point. The density of an exponential family has a support that is independent of the parameters. Furthermore, this truncation parameter does not allow for the interchange of partial differentiation and integration with respect to the Lebesgue measure. For instance, this means instead of . Here, we introduce two properties of the function . First, the partial derivative coincides with and is always positive. This fact can be verified as follows. Since p is a probability density function, Therefore, by differentiating both sides with respect to , we obtain and Second, the following lemma holds. Lemma 1. For the function in (13), there always exists θ and γ such that . Proof. Suppose that is always 0. Under this assumption, we will show that , leading to a contradiction that is not a parameter. If , the function can be expressed as where . Then, Differentiating both sides with respect to , we obtain Further differentiating both sides with respect to , we have and all are constants. Therefore, □ The family of Pareto distributions is an example of the oTEF. Pareto distributions have the following density functions, with the natural parameter and the truncation parameter . This family is used to describe various natural and social phenomena [21]. To discuss geometric structures in subsequent sections, we focus on the asymptotic behavior of maximum likelihood estimators in the oTEF. Our discussion of these estimators follows previous works. Consider random variables independent and identically distributed according to , and let be the order statistics of the sample. Let and denote the maximum likelihood estimators for and , respectively. Bar-Lev [5] showed the existence and uniqueness of and . Here, and is a root of the maximum likelihood equation for . The first-order asymptotic variances of and are given by These are essential to our argument in the next section. Additionally, Akahira [6] and Akahira and Ohyauchi [22] obtained the second-order asymptotic loss.

This section gives definitions of a geometric structure of the oTEF. We take as an -dimensional manifold with coordinates . Since does not satisfy regularity conditions, its geometric structure is not determined in a natural way.

In this subsection, we define a Riemannian metric on oTEF as follows. Definition 7 (Riemannian metric in the oTEF). Let belong to the oTEF. The Riemannian metric of the oTEF is defined by for . This metric is also represented as Note that can be expressed in terms of the function as follows: This is similar to the case of exponential family distributions. We will now explain how we came to the above definition. Consider an exponential family for and a statistical model for . is an n-dimensional submanifold of obtained by fixing the truncation parameter and satisfies the regularity conditions. Additionally, is a one-dimensional submanifold of obtained by fixing the natural parameter . Since is regular, the Riemannian metric on should be the Fisher metric defined in Definition 2. This idea induces the components to be the components of the Fisher metric. The remaining task is to define , the inner products of and , and , the metric on . In this context, we review the statistical interpretation of the Fisher metric in regular models. Let be a regular model with parameters . As mentioned in Section 2.1, the Fisher metric is a Riemannian metric defined by the Fisher information matrix. Expanding the variance of the maximum likelihood estimator , we have where the first-order coefficient corresponds to . On the other hand, in the oTEF, the Riemannian metric is determined from the coefficient of the variance of the maximum likelihood estimator. As shown in Section 2.3, the variances of and are expressed as where is the matrix . Then, similarly to the Fisher information matrix, appears as the reciprocal of the first-term coefficient. From this, the Riemannian metric of is defined as Furthermore, it should be noted that is negligible up to . Moreover, whether is known does not affect the first-order term of [22]. This is also true for the estimation of [23]. Based on these facts, we assume that and are orthogonal, and define As a result, the Riemannian metric in Definition 7 is obtained. This Riemannian metric is equal to the formally defined Fisher metric on the oTEF. In other words, the equality holds for . The right-hand side is the same as the definition of the Fisher metric on regular statistical models. However, the Riemann metric does not satisfy the equation For , we have but for some and , by Lemma 1. This is influenced by the fact that it does not satisfy the regularity conditions.

Next, we define an affine connection in the oTEF. In this study, we adopt two types of connections: the Levi-Civita connection and the -connection. The first affine connection is the Levi-Civita connection: introduced from the Riemannian metric. Of course, this Levi-Civita connection is also a metric and self-dual, the same as in the regular case. Rylov [10] and Li et al. [12] previously adopted this same affine connection. Second, we define an affine connection in the oTEF as an analogy for the -connection in the regular statistical model defined in Definition 3. Definition 8. For a given , the α-connection in the oTEF is defined by the connection coefficients where is a log-likelihood function. The above definition is obtained by substituting the oTEF probability density function into Equation (4) for the α-connection in the regular model. In particular, for the Pareto distribution family, it coincides with the α-connection given by Sun et al. [13] (see the equation in their paper). Note that, the log-likelihood function is not differentiable with respect to γ at . We calculate the expectation over the interval instead of . This -connection is torsion-free, The connection coefficients for the -connection are given by where . The -connection in the oTEF differs from the one in the regular model in several aspects. First, the 0-connection does not correspond to the Levi-Civita connection. This is due to the inability to express the -connection coefficients in the following form: This transformation involves an interchange of the order of differentiation and integration, which is one of the regularity conditions. Therefore, in the non-regular model oTEF, the two sides do not match. Additionally, the dual connection of the -connection in the oTEF does not become the -connection. This can be verified as follows. The partial derivative and connection coefficients and are given by Thus, Therefore, since and , the duality does not hold.

In this section, as part of investigating the properties of the α-connection in the oTEF, we deal with the α-parallel priors. We show that there exists an α-parallel prior distribution for α = 1. The existence of α-parallel priors depends on the geometric properties of statistical models, and they are not guaranteed to exist in general. Therefore, it is necessary to investigate the existence of α-parallel priors in the case of the oTEF. Note that the Levi-Civita connection is always equiaffine, and it is known to have the Jeffreys prior as a parallel prior distribution. Therefore, we do not deal with it in this section. In the oTEF, attention is needed to be paid to the conditions for the existence of the α-parallel prior distributions. In the case of regular models, Proposition 2 provides a necessary and sufficient condition for the existence of α-parallel priors. Sun et al. [13] revealed that the Pareto distribution family does not satisfy this condition and claims that it does not have α-parallel priors. However, the above deduction is incorrect, since Proposition 2 does not hold for an oTEF distribution. In proof [17] of Proposition 2, the connection coefficients of the α-connection are written in the same form as (55) by the Levi-Civita connection and the cubic tensor . However, this form cannot represent the α-connection in Definition 8. Then, in the oTEF, Proposition 2 does not confirm that the equation is a necessary and sufficient condition for the existence of α-parallel priors. Therefore, in this study, we use the necessary and sufficient conditions for general affine connections to investigate the existence of the α-parallel prior. The following theorem reveals the existence of α-parallel priors in the oTEF. Theorem 1. Consider as belonging to the oTEF with densities of the form as those in Definition 13 with the natural parameter and the truncation parameter γ. If , then the connection is equiaffine and there exists a one-parallel prior. Moreover, this one-parallel prior can be represented as for . Proof. First, we prove that is equiaffine. By Proposition 1, the necessary and sufficient condition for to be equiaffine is for everywhere in , where denotes α-Riemannian curvature tensors. This condition can be represented as since By Definition 7, we have Thus, Therefore, the formula holds when and is equiaffine. Second, we find the one-parallel prior. Let be the density of the one-parallel volume element. According to Proposition 1 in the paper by Takeuchi and Amari [17], we have In the case of the oTEF, its representation is Therefore, the one-parallel prior for is given as □ Moreover, the above one-parallel prior coincides with a certain reference prior. A reference prior, proposed by Bernardo [24], is a noninformative prior distribution derived from an information-theoretic perspective. Specifically, it is defined as a prior distribution that maximizes the expectation of the KL divergence between the posterior and prior distributions, and in the case of no nuisance parameters, it coincides with the Jeffreys prior [24]. However, this is not necessarily the case when nuisance parameters are present. Ghosh and Mukerjee [25] provided a new formulation for reference priors with nuisance parameters by considering the maximization of a functional with an appropriate penalty term. Furthermore, Ghosal [26] extended this reference prior to non-regular models where the support of the density depends on the parameters. When applied to an oTEF model with as the parameter of interest and as the nuisance parameter, the reference prior of Ghosal [26] is given by Thus, it coincides with the one-parallel prior in Theorem 1.

This section finds a submodel of the oTEF with constant scalar curvature for the Levi-Civita connection. We do not use the α-connection in Definition 8. Previous works about the information geometry of non-regular cases have mainly studied the geometric structure of Pareto distributions for the Levi-Civita connection. They adopted the formally defined Fisher metric as in (42), which is consistent with our Riemannian metric. Rylov [10] found that the family of Pareto distributions has a constant curvature with respect to the Levi-Civita connection. Li et al. [12] showed that its geometrical structure is isometric to the Poincaré upper half-plane and applied this geometrical structure to Bayesian inference by considering the Jeffreys prior. We extend these previous works on Pareto distributions to n dimensions. For , let be a smooth function on the interval I and let be a random variable following a two-parameter truncated exponential distribution, which has density function with respect to the Lebesgue measure, where . An oTEF includes this distribution. Consider , a joint distribution of independent random variables with the common redefined truncation parameter and the natural parameter . has the density function with respect to the Lebesgue measure on . Here, is a rectangle and is . is a family of distributions with parameters and . includes practical examples such as a family of several Pareto distributions with a common scale parameter (Rohatgi and Saleh [14]) and a family of truncated exponential distributions with a common location parameter (Ghosh and Razmpour [15]). Truncated exponential distributions with a common are sometimes applied to reliability and life testing. Please assume that the case where the first failure of n products can occur only after a common minimum time has elapsed, and these products have unknown and possibly unequal failure rates . This truncation parameter takes the role of a “guarantee time”, so estimation of the parameter is vital to determining the warranty period. Note that the geometric structure of the above common truncation parameter model is equivalent to that of the family of Pareto distributions when . Theorem 2. Riemannian manifold has a constant scalar curvature of . Proof. Let denote , where . The Riemannian metric for the common truncation parameter model is given by for . The tangent vectors are mutually orthogonal. By (46), we have and for , where Hence, we obtain for . Therefore, the scalar curvature is given by □

This paper considered the geometric structure of a one-sided truncated exponential family (oTEF) with parameters and . We constructed a Riemannian metric based on the asymptotic properties of the maximum likelihood estimators. Under this, we showed that the formally defined -connection admits -parallel priors when . Our result gives geometric meaning to a specific reference prior. Furthermore, we proved that the scalar curvature of some submodels of the oTEF obtained by making common across multiple distributions is constant. It is essential to discuss suitable affine conditions for the oTEF. First, we need to reveal the statistical meaning of the -connection in the oTEF. -connection coefficients are expected to appear in higher-order terms of the variance of maximum likelihood estimators. Additionally, instead of the -connection, we can construct a family of affine connections to be equiaffine by connecting the -connection and the Levi-Civita connection. It is also interesting to consider affine connections induced by the third derivatives of divergences.