Abstract
Abstract. Entropy functionals (i.e. convex integral functionals) and extensions of these functionals are minimized on convex sets. This paper is aimed at reducing as much as possible the assumptions on the constraint set. Dual equalities and characterizations of the minimizers are obtained with weak constraint qualifications.
Presentation of the results. Our aim is to reduce as much as possible the restrictions on the convex set C.
Denoting the minimizer Q of (1.3), the geometric picture is that some level set of I is tangent at Q to the constraint set T −1 o C. Since these sets are convex, they are separated by some affine hyperplane and the analytic description of this separation yields the characterization of Q. Of course Hahn-Banach theorem is the key. Standard approaches require C to be open with respect to some given topology in order to be allowed to apply it. In the present paper, one chooses to use a topological structure which is designed for the level sets of I to "look like" open sets, so that Hahn-Banach theorem can be applied without assuming to much on C.
Presentation of the results. Our aim is to reduce as much as possible the restrictions on the convex set C.
This strategy is implemented in [17] in an abstract setting suitable for several applications. It is a refinement of the standard saddle-point method [22] where convex conjugates play an important role. The proofs of the present article are applications of the general results of [17] .
Presentation of the results. Our aim is to reduce as much as possible the restrictions on the convex set C.
Clearly, for the problem (1.3) to be attained, T
−1
o C must share a supporting hyperplane with some level set of I. This is the reason why it is assumed to be closed with respect to the above mentioned topological structure. This will be the only restriction to be kept together with the interior specification (1.4) below.
−1
Dual equalities and primal attainment are obtained under the weakest possible assumption:
−1
C ∩ T o dom I = ∅ where dom I := {Q ∈ M Z ; I(Q) < ∞} is the effective domain of I and T o dom I is its image by T o . The main result of this article is the characterization of the minimizers of (1.3) in the interior case which is specified by
−1
where icor (T o dom I) is the intrinsic core of T o dom I. The notion of intrinsic core does not rely on any topology; it gives the largest possible interior set. For comparison, a usual form of constraint qualification required for the representation of the minimizers of (1.3) is int (C) ∩ T o dom I = ∅ (1.5) where int (C) is the interior of C with respect to a topology which is not directly connected to the "geometry" of I. In particular, int (C) must be nonempty; this is an important restriction. The constraint qualification (1.4) is weaker.
−1
An extension of Problem (1.3) is also investigated. One considers an extensionĪ of the entropy I to a vector space L Z which contains M Z and may also contain singular linear forms which are not σ-additive. The extended problem is minimizeĪ(ℓ) subject to T o ℓ ∈ C, ℓ ∈ L Z (1.6)
−1
Even if I is strictly convex,Ī isn't strictly convex in general so that (1.6) may admit several minimizers. There are situations where (1.3) is not attained in M Z while (1.6) is attained in L Z . Other relations between these minimization problems are investigated by the author in [18] with probabilistic questions in mind.
−1
1.3. Literature about entropy minimization. Entropy minimization problems appear in many areas of applied mathematics and sciences. The literature about the minimization of entropy functionals under convex constraints is considerable: many papers are concerned with an engineering approach, working on the implementation of numerical procedures in specific situations. In fact, entropy minimization is a popular method to solve ill-posed inverse problems.
−1
Rigorous general results on this topic are quite recent. Let us cite, among others, the main contribution of Borwein and Lewis: [1] , [2] , [3] , [4] , [5] , [6] together with the paper [23] by Teboulle and Vajda. In these papers, topological constraint qualifications of the type of (1.5) are required. Such restrictions are removed here.
−1
With a geometric point of view, Csiszár [8, 9] provides a complete treatment of (1.3) with the relative entropy (see Section 6.1) under the weak assumption (1.4). The behavior of minimizing sequences of general entropy functionals is studied in [10] . By means of a method different from the saddle-point approach, the author has already studied in [15, 16] entropy minimization problems under affine constraints (corresponding to C reduced to a single point) and more restrictive assumptions on γ * . The present article extends these results.
−1
Outline of the paper. The minimization problems (1.3) and (1.6) are described in details at Section 2. In Section 3, the main results of [17] about the extended saddle-point method are recalled. Section 4 is devoted to the extended problem (1.6) and Section 5 to (1.3). One presents important examples of entropies and constraints at Section 6.1.
−1
Notation. Let X and Y be topological vector spaces. The algebraic dual space of X is X * , the topological dual space of X is X ′ . The topology of X weakened by Y is σ(X, Y ) and one writes X, Y to specify that X and Y are in separating duality. Let f : X → [−∞, +∞] be an extended numerical function. Its convex conjugate with respect to X, Y is f
−1
Its subdifferential at x with respect to X, Y is ∂ Y f (x) = {y ∈ Y ; f (x + ξ) ≥ f (x) + y, ξ , ∀ξ ∈ X}. If no confusion occurs, one writes ∂f (x). The intrinsic core of a subset A of a vector space is icor A = {x ∈ A; ∀x ′ ∈ aff A, ∃t > 0, [x, x + t(x ′ − x)[⊂ A} where aff A is the affine space spanned by A. icordom f is the intrisic core of the effective domain of f : dom f = {x ∈ X; f (x) < ∞}. The indicator of a subset A of X is defined by
−1
The support function of A ⊂ X is ι * A (y) = sup x∈A x, y , y ∈ Y. One writes I ϕ (u) := Z ϕ(z, u(z)) R(dz) = Z ϕ(u) dR and I = I γ * for short, instead of (1.1).
Presentation of the minimization problems (P C ) and (P C )
The problem (1.3) and its extension (1.6) are introduced. Their correct mathematical statements necessitate the notion of Orlicz spaces. The definitions of good and bad constraints are given and the main assumptions are collected at the end of this section.
Presentation of the minimization problems (P C ) and (P C )
2.1. Orlicz spaces. To state the minimization problem (1.3) and its extension correctly, one will need to talk in terms of Orlicz spaces related to the function γ * . Let us recall some basic definitions and results. A set Z is furnished with a σ-finite nonnegative measure R on a σ-field which is assumed to be R-complete. A function ρ : Z × R is said to be a Young function if for R-almost every z, ρ(z, ·) is a convex even [0, ∞]-valued function on R such that ρ(z, 0) = 0 and there exists a measurable function z → s z > 0 such that 0 < ρ(z, s z ) < ∞. In the sequel, every numerical function on Z is supposed to be measurable.
A subspace of interest is
Now, let us identify the R-a.e. equal functions. The corresponding spaces of equivalence classes are denoted L ρ (Z, R) and E ρ (Z, R).
A subspace of interest is
Of course E ρ ⊂ L ρ . Note that if ρ doesn't depend on z and ρ(s o ) = ∞ for some s o > 0, E ρ reduces to the null space and if in addition R is bounded, L ρ is L ∞ . On the other hand, if ρ is a finite function which doesn't depend on z and R is bounded, E ρ contains all the bounded functions.
A subspace of interest is
Duality in Orlicz spaces is intimately linked with the convex conjugacy. The convex conjugate ρ * of ρ is defined by ρ * (z, t) = sup s∈R {st − ρ(z, s)}. It is also a Young function so that one may consider the Orlicz space L ρ * .
Theorem 2.2 (Representation of E
Proof. For a proof of this result, see ([12] , Thm 4.8).
Theorem 2.2 (Representation of E
A continuous linear form ℓ ∈ L ′ ρ is said to be singular if for all u ∈ L ρ , there exists a decreasing sequence of measurable sets (A n ) such that R(∩ n A n ) = 0 and for all n ≥ 1,
Theorem 2.2 (Representation of E
When ρ is a finite function, this result is ( [13] , Theorem 2.2). The general result is proved in [19] , with ρ not depending on z but the extension to a z-dependent ρ is obvious.
Theorem 2.2 (Representation of E
In the decomposition (2.4), ℓ a is called the absolutely continuous part of ℓ while ℓ s is its singular part. Proof. This result is ( [13] , Proposition 2.1).
Theorem 2.2 (Representation of E
The function ρ is said to satisfy the ∆ 2 -condition if
Theorem 2.2 (Representation of E
If s o = 0, the ∆ 2 -condition is said to be global. When R is bounded, in order that E ρ = L ρ , it is enough that ρ satisfies the ∆ 2 -condition. When R is unbounded, this equality still holds if the ∆ 2 -condition is global. Consequently, if ρ satisfies the
Theorem 2.2 (Representation of E
reduces to the null vector space.
2.2.
The minimization problem (P C ). Before introducing an extended minimization problem, let us state properly the basic problem (1.3).
2.2.
Relevant Orlicz spaces. Since γ * z is closed convex for each z, it is the convex conjugate of some closed convex function γ z . Defining
2.2.
where m satisfies (1.2), one sees that for R-a.e. z, λ z is a nonnegative convex function and it vanishes at 0. Hence,
2.2.
is a Young function. We shall use Orlicz spaces associated with λ ⋄ and λ * ⋄ . We denote the space of R-absolutely continuous signed measures having a density in the
2.2.
Constraint. In order to define the constraint, take X o a vector space and a function θ : Z → X o . One wants to give a meaning to the formal constraint Z θ dQ = x with Q ∈ L λ * ⋄ R and x ∈ X o . Suppose that X o is the algebraic dual space of some vector space
2.2.
Assuming that
2.2.
Hölder's inequality in Orlicz spaces allows to define the constraint operator
2.2.
Minimization problem. Consider the minimization problem
2.2.
where C o is a convex subset of X o . One sees with γ * z (t) = λ * z (t − m(z)) that I γ * (Q) = I λ * (Q − mR). Therefore, the problem (P Co ) is equivalent to
2.2.
with ℓ = Q − mR. If the function m satisfies m ∈ L λ * ⋄ , one sees with (2.8) and Hölder's inequality in Orlicz spaces that the vector x o = Z θm dR ∈ X o is well-defined in the weak sense. Therefore, (P Co ) is
2.2.
The extended minimization problem (P C ). If the Young function λ ⋄ doesn't satisfy the ∆ 2 -condition (2.6), for instance if it has an exponential growth at infinity as in (6.1) or even worse as in (6.3), the small Orlicz space E λ⋄ may be a proper subset of L λ⋄ . Consequently, for some functions θ, the integrability property
2.2.
or equivalently
2.2.
holds. In this situation, analytical complications occur (see Section 4). This is the reason why constraints satisfying (A ∀ θ ) are called good constraints, while constraints satisfying
2.2.
If the constraint is bad, it may happen that (P C ) is not attained in L λ * ⋄ R. This is the reason why it is worth introducing its extension (P C ) which may admit minimizers and is defined by
2.2.
λ⋄ are respectively the absolutely continuous part and the singular part of ℓ, see Theorem 2.3. The extensionĪ has the following form
2.2.
In a similar way to (2.9), the assumption (A
2.2.
Important examples of entropies with λ ⋄ not satisfying the ∆ 2 -condition are the usual (Boltzmann) entropy and its variants, see Section 6.1 and (6.1) in particular. When λ ⋄ satisfies the ∆ 2 -condition (2.6), (P C ) is (P C ).
2.4.
Assumptions. Let us collect the assumptions on R, γ * and θ.
Assumptions (A).
(A R ) It is assumed that the reference measure R is a σ-finite nonnegative measure on a space Z endowed with some R-complete σ-field.
Assumptions (A).
is z-measurable for all t and for R-almost every z ∈ Z, γ * (z, ·) is a lower semicontinuous strictly convex [0, +∞]-valued function on R which attains its (unique) minimum at m(z) with γ
Assumptions (A).
(1) for any y ∈ Y o , the function z ∈ Z → y, θ(z) ∈ R is measurable; (2) for any y ∈ Y o , y, θ(·) = 0, R-a.e. implies that y = 0;
Assumptions (A).
Remarks 2.13. Some technical remarks about the assumptions.
Assumptions (A).
(a) Since γ * z is a convex function on R, it is continuous on the interior of its domain. Under our assumptions, γ * is (jointly) measurable, and so are γ and m. Hence, λ is also measurable.
Assumptions (A).
It allows to consider Problem (P C ) rather than (P Co ). If this assumption is not satisfied, our results still hold for (P Co ), but their statement is a little heavier, see Remark 4.10-d below.
Assumptions (A).
(d) Since X o and Y o are in separating duality, (A 2 θ ) states that the vector space spanned by the range of θ "is essentially" X o . This is not an effective restriction.
Preliminary results
The aim of this section is to recall for the convenience of the reader some results of [14, 16, 17] .
Convex minimization problems under weak constraint qualifications.
The main results of [17] are presented.
Convex minimization problems under weak constraint qualifications.
where C is a convex subset of X o . This will be used later with Φ = I λ on the Orlicz space
Convex minimization problems under weak constraint qualifications.
It is useful to define the constraint operator T o by means of its adjoint
Convex minimization problems under weak constraint qualifications.
Hypotheses. Let us give the list of the main hypotheses.
Convex minimization problems under weak constraint qualifications.
The definitions of the vector spaces X and Y which appear in the last assumption are stated below. For the moment, let us only say that if C is convex and
Convex minimization problems under weak constraint qualifications.
Several primal and dual problems. These variants are expressed below in terms of new spaces and functions. Let us first introduce them.
Convex minimization problems under weak constraint qualifications.
-The norms | · | Φ and | · | Λ . Let Φ ± (u) = max(Φ(u), Φ(−u)). By (H Φ1 ) and (H Φ2 ), {u ∈ U o ; Φ ± (u) ≤ 1} is a convex absorbing balanced set. Hence its gauge functional which is defined for all u ∈ U o by |u| Φ := inf{α > 0; Φ ± (u/α)) ≤ 1} is a seminorm. Thanks to hypothesis (H Φ3 ), it is a norm. Taking (H T 1 ) into account, one can define
Convex minimization problems under weak constraint qualifications.
Thanks to (H Φ ) and (H T ), it is a norm and
Convex minimization problems under weak constraint qualifications.
where any x in Y ′ is identified with its restriction to Y o . We also have to consider the algebraic dual spaces L * and X * of L and X . -The operators T and
Convex minimization problems under weak constraint qualifications.
-The optimization problems. They are:
Convex minimization problems under weak constraint qualifications.
Statement of the results. It is assumed that (H Φ ), (H T ) and (H C ) hold.
Theorem 3.2 (Primal attainment and dual equality).
(a) The problems (P o ) and (P) are equivalent: they have the same solutions and
Theorem 3.2 (Primal attainment and dual equality).
(b) We have the dual equalities
Theorem 3.2 (Primal attainment and dual equality).
Moreover,l ∈ L andω ∈ X * satisfy (3.
4) if and only ifl solves (P o ) andω solves (D).
The assumption C ∩ icor (T o dom Φ * ) = ∅ is equivalent to C ∩ icordom Λ * o = ∅ and the representation formula (3.4-c) is equivalent to Young's identity
4) if and only ifl solves (P o ) andω solves (D).
Formula (3.4-c) can be made a little more precise by means of the following regularity result.
4) if and only ifl solves (P o ) andω solves (D).
Theorem 3.6. Any solutionω of (D) shares the following properties
If in addition the level sets of
where Y ′′ and U ′′ are the topological bidual spaces of Y and U. This occurs if Φ, or equivalently Φ * , is an even function.
3.2.
Convex conjugates in a Riesz space. The following results are taken from [14, 16] . For the basic definitions and properties of Riesz spaces, see [7, Chapter 2] . Let U be a Riesz vector space for the order relation ≤ . Since U is a Riesz space, any u ∈ U admits a nonnegative part: u + := u ∨ 0, and a nonpositive part:
3.2.
Of course, u = u + − u − and as usual, we state:
3.2.
Remark 3.7. Recall that there is a natural order on the algebraic dual space E * of a Riesz vector space E which is defined by: e * ≤ f * ⇔ e * , e ≤ f * , e for any e ∈ E with e ≥ 0. A linear form e * ∈ E * is said to be relatively bounded if for any f ∈ E, f ≥ 0, we have sup e:|e|≤f | e * , e | < +∞. Although E * may not be a Riesz space in general, the vector space E b of all the relatively bounded linear forms on E is always a Riesz space. In particular, the elements of E b admit a decomposition in positive and negative parts e * = e * + − e * − .
3.2.
Let Φ be a [0, ∞]-valued function on U which satisfies the following conditions: Proposition 3.10. Assume (3.8) and (3.9) and suppose that L is a Riesz space. 
3.2.
Proof. 
Solving (P C )
The general assumptions (A) are imposed and we study (P C ).
Several function spaces and cones.
To state the extended dual problem (D C ) below, notation is needed. If λ is not an even function, one has to consider
Several function spaces and cones.
which are Young functions and the corresponding Orlicz spaces.
Definitions 4.2. For any relatively bounded linear form
λ⋄ , one writes:
Definitions 4.2. For any relatively bounded linear form
where λ ± are defined at (4.1) and 
continuous with respect to relative topology generated by the strong topology of L
Let ρ be a Young function. By Theorem 2.3, we have
continuous with respect to relative topology generated by the strong topology of L
′ , let us denote the restrictions ζ 1 = ζ |L ρ * R and 
4.2.
The ingredients of the saddle-point method. One applies the abstract results of Section 3.1 with
4.2.
This gives U = L λ⋄ with the Orlicz norm 
4.2.
This last identity is a dual equality as in Theorem 3.2-b with Φ = ι B where B is the unit ball of L λ⋄ and C = {x}.
4.2.
for some subset Y ⊂ Y L and some functions y ∈ Y → a y ∈ R. For comparison, note that if C is only supposed to be convex, (y,a)∈A ℓ ∈ L ′ λ⋄ ; y, θ , ℓ > a with A ⊂ Y × R is the general shape of T −1 C.
4.3.
The main result. Let us define
4.3.
which is the convex conjugate of Γ(y) = I γ ( y, θ ), y ∈ Y o . The dual problem (D) associated with (P C ) and (P C ) is
4.3.
The extended dual problem is maximize inf
4.3.
λ⋄ is the extension of T * o which is defined at Section 3.1,
4.3.
As R is assumed to be σ-finite, there exists a measurable partition (
4.3.
with respect to the topologies associated with · λ + and · λ − ; (3) C satisfies (4.5) with y, θ ∈ L λ⋄ for all y ∈ Y. Then: (a) The dual equality for (P C ) is
-cluster points and every such point is a solution to (P C ).
Suppose that in addition we have
-cluster points and every such point is a solution to (P C ).
where
-cluster points and every such point is a solution to (P C ).
is a measurable function in the strong closure of
.8) if and only ifl solves
Proposition 4.9. For the assumption (2) of Theorem 4.6 to be satisfied, it is enough that one of these conditions holds (i) λ is even or more generally 0 < lim inf t→∞
.8) if and only ifl solves
(ii) lim t→∞ λ + λ − (t) = +∞ and λ − satisfies the ∆ 2 -condition (2.6).
.8) if and only ifl solves
Proof. It is enough to work with a bounded measure R.
.8) if and only ifl solves
As γ * is assumed to be strictly convex, zero is in the interior of dom λ and L λ⋄ contains the space B of all bounded measurable functions. But B is dense in E λ − and the result follows. 
.8) if and only ifl solves
instead of (P C ). The transcription of Theorem 4.6 is as follows. Denote
.8) if and only ifl solves
and replace respectively (P C ), C, Γ * ,x and γ by (P Co ), C o , Λ * ,x and λ wherẽ x = θ,l − mR is well-defined. The statement (b) must be replaced by the following one:
.8) if and only ifl solves
andl is a solution of (P Co ). Proof of Theorem 4.6. It is an application of Theorems 3.2 and 3.3. We use the notation and framework of Section 3.1. With (4.4) and Theorem 3.
.8) if and only ifl solves
Equality (a) comes from Proposition 3.10-a and equality (b) is a dual equality of the type of Theorem 3.2-b applied with
.8) if and only ifl solves
which holds for any Young function ρ. This identity is proved by Fougères, Giner, Kozek and Rockafellar [11, 13, 21] under the assumptions (A R ) and (A (2), we obtain thatĪ
.8) if and only ifl solves
taking advantage of the direct sum ℓ = ⊕ k ℓ |Z k acting on u = (u |Z k ) k≥1 which lead to the nonnegative series Φ(u) = ⊕ k Φ(u |Z k ) and Φ
.8) if and only ifl solves
• Reduction to m = 0. We have seen at (2.10) that the transformation Q ℓ = Q − mR corresponds to the transformations γ λ and (P C ) (2.10). This still works with (P C ) and one can assume from now on without loss of generality that m = 0 and γ = λ. The assumption (A 2 γ * ) will not be used during the rest of the proof. This allows Remark 4.10-d.
.8) if and only ifl solves
• Verification of (H Φ ) and (H T ). Suppose that W = {z ∈ Z; λ(z, s) = 0, ∀s ∈ R} is such that R(W ) > 0. Then, any ℓ such that u1 W , ℓ > 0 for some u ∈ L λ⋄ satisfies Φ * (ℓ) = +∞. Therefore, one can remove W from Z without loss of generality. Once, this is done, the hypothesis (H Φ ) is satisfied under the assumption (A
.8) if and only ifl solves
• The computation ofΦ in the case where λ is even. Since Φ is even, Theorem 3.6 tells us that domΦ is included in the σ(L 
.8) if and only ifl solves
-closure of dom I λ and we dropped the restrictions ζ |L for simplicity.
.8) if and only ifl solves
• Extension to the case where λ is not even. By Proposition 3.10-b, we haveΦ(ζ) =
.8) if and only ifl solves
λ⋄ and +∞ otherwise. It follows that
.8) if and only ifl solves
if ζ ∈ K ′′ λ and +∞ otherwise. In particular, we have
.8) if and only ifl solves
This provides us with the dual problems (D C ) and (D C ).
.8) if and only ifl solves
• Proof of (a) and (b). Apply Theorem 3.2. 2 Let us go on with the proof of (c). By Theorem 3.3, 
.8) if and only ifl solves
where u 1 = u and u 2 = u act respectively on L λ * R and L s λ . This direct sum structure leads us to
.8) if and only ifl solves
which again is the direct sum of the absolutely continuous and singular components of
.8) if and only ifl solves
• 
.8) if and only ifl solves
is the pointwise limit of a sequence (T * y n ) n≥1 with y n ∈ Y. As T * y n (z) = y n , θ(z) , we see that [T * ω ] a 1 (z) = θ(z),ω for some linear formω on X o . If R is unbounded, it is still assumed to be σ-finite: there exists a sequence (Z k ) of measurable subsets of Z such that ∪ k Z k = Z and R(Z k ) < ∞ for each k. Hence, for each k and all z ∈ Z k , (T * ω ) a (z) = θ(z),ω k for some linear formω k on X o , from which (4.16) follows.
.8) if and only ifl solves
•Proof of (c). It follows from the previous considerations and Theorem 3.3.
.8) if and only ifl solves
•Proof of (d). Statement (d)-1 follows from Theorem 3.2. Statement (d)-2 is immediately deduced from (c). Finally, (d)-3 is (3.5).
Solving (P C )
The general assumptions (A) are imposed and we study (P C ) under the additional good constraint assumption (A ∀ θ ) which imposes that the convex set C is such that
Solving (P C )
for some subset Y ∈ X * o such that y, θ ∈ E λ⋄ for all y ∈ Y and some function y ∈ Y → a y ∈ R. The dual problem (D) associated with (P C ) is (D C ) and the extended dual problem is
Solving (P C )
where Y is the convex cone of all linear forms ω on X o which are such that -the function ω, θ(·) X * o ,Xo is measurable; 
Solving (P C )
Proof. It is a corollary of the proof of Theorem 4.6. One applies the abstract results of Section 3.1 with
Solving (P C )
This gives U = E λ⋄ with the Orlicz norm |u| Φ = u λ⋄ and L = L λ * ⋄ R. The space Y := Y E is the completion of Y o endowed with the norm |y| Λ = y, θ λ⋄ . It is isomorphic to the closure of the subspace { y, θ ;
Solving (P C )
E is identified with L λ * ⋄ R/ker T and its norm is given by |x| *
Solving (P C )
As in the proof of Theorem 4.6, one reduces to the case where m = 0 without loss of generality.
Solving (P C )
The assumption (2) implies that λ is a finite function. It follows that E ′ λ⋄ = L λ * ⋄ , the convex conjugate Φ * of Φ with respect to the duality E λ⋄ , L λ * ⋄ is Φ * = I λ * (see [20] ) and the corresponding extended functionΦ is
Solving (P C )
and +∞ otherwise. With these correspondences, the proof of the theorem is an immediate translation of the proof of Theorem 4.6. λ⋄ ; θ, ℓ ≥ a y = y∈Y f R ∈ L λ * ⋄ R; Z y, θ f dR ≥ a y whenever (A ∀ θ ) holds. But, the drawback is that the unnecessary assumption (2) of Theorem 4.6 has to be kept.
Examples
Standard examples of entropy minimization problems are presented. 6.1. Some examples of entropies. The entropies defined below occur naturally in statistical physics, probability theory, mathematical statistics and information theory. s ∈ R for all z ∈ Z. Note that H(Q|R) < ∞ implies that Q is nonnegative.
Examples
Relative entropy. The reference measure R is assumed to be a probability measure and one denotes P Z the set of all probability measures on Z. The relative entropy of Q ∈ M Z with respect to R ∈ P Z is the following variant of the Boltzmann entropy:
Examples
It is (6.2) with the additional constraint that Q(Z) = 1 :
I(Q|R) = H(Q|R) + ι {Q(Z)=1}
When minimizing the Boltzmann entropy Q → H B (Q|R) on a constraint set which is included in P Z , we have for all P, Q ∈ P Z , H B (Q|R) = I(Q|P ) + Z log dP dR dQ which is meaningful for each Q ∈ P Z which integrates dP dR .
I(Q|R) = H(Q|R) + ι {Q(Z)=1}
Reverse relative entropy. The reference measure R is assumed to be a probability measure. The reverse relative entropy is
I(Q|R) = H(Q|R) + ι {Q(Z)=1}
It corresponds to γ * z (t) = − log t + t − 1 if t > 0 +∞ if t ≤ 0 , m(z) = 1 and
I(Q|R) = H(Q|R) + ι {Q(Z)=1}
for all z ∈ Z, with the additional constraint that Q(Z) = 1.
I(Q|R) = H(Q|R) + ι {Q(Z)=1}
6.2. Some examples of constraints. Let us consider two standard constraints which are the moment constraints and the marginal constraints.

{'paper_id': '9983728', 'title': 'Minimization of entropy functionals', 's2_url': 'https://api.semanticscholar.org/CorpusID:9983728', 'source': 'S2ORC'}