0 & 1\\ 1 & 0 
    \end{matrix}\right], 
        \left[\begin{matrix}
    0 & 1\\0 & 1\\ 0 & 1 
    \end{matrix}\right], 
        \left[\begin{matrix}
    1 & 0\\1 & 0\\ 0 & 1 
    \end{matrix}\right]$$ and can be written as a union of Cartesian products of convex sets, illustrated in Figure \[figure:convexrepresentable\], $$G = \bigcup_{\theta\in\Delta_A} G_\theta,\qquad
    G_\theta = \{ \theta \} \times \{ \theta \} \times \Delta_A. 
    $$

As mentioned in Section \[sec:optimalcontrol\], the set $F\subseteq\Delta{W\times A}$ of joint distributions that are compatible with the representable conditionals $G=f_\beta(\Delta_{S,A})\subseteq\Delta_{W,A}$, may not be convex. In the following we describe large convex subsets of $F$, depending on the properties of $G$. We use the following definitions.

-   Given a set of distributions ${\mathcal{P}}\subseteq\Delta_W$ and a set of kernels ${\mathcal{G}}\subseteq\Delta_{W,A}$, let $${\mathcal{P}}\ast {\mathcal{G}}: =\Big\{ q(w,a) = p(w)g(a|w) \in \Delta_{W\times A} \colon p\in{\mathcal{P}}, g\in {\mathcal{G}}\Big\}$$ denote the set of joint distributions over world states and actions, with world state marginals in ${\mathcal{P}}$ and conditional distributions in ${\mathcal{G}}$.

-   For any $V \subseteq W$ let $$\Delta_W(V) : = \Big\{ p\in \Delta_W \colon {\operatorname{supp}}(p):=\{w\in W\colon p(w)>0 \}\subseteq V\Big\}$$ denote the set of world state distributions with support in $V$.

-   Given a subset $V\subseteq W$ and a set of kernels ${\mathcal{G}}\subseteq\Delta_{W,A}$, let $${\mathcal{G}}|_V := \Big\{ h\in \Delta_{V,A}\colon h(\cdot|w) = g(\
0 & 1\\ 1 & 0 
     \end{matrix}\right ], 
         \left[\begin{matrix } 
     0 & 1\\0 & 1\\ 0 & 1 
     \end{matrix}\right ], 
         \left[\begin{matrix } 
     1 & 0\\1 & 0\\ 0 & 1 
     \end{matrix}\right]$$ and can be written as a union of Cartesian products of convex sets, illustrate in name   \[figure: convexrepresentable\ ], $ $ G = \bigcup_{\theta\in\Delta_A } G_\theta,\qquad 
     G_\theta = \ { \theta \ } \times \ { \theta \ } \times \Delta_A. 
     $ $ 

 As mentioned in Section   \[sec: optimalcontrol\ ], the set $ F\subseteq\Delta{W\times A}$ of joint distribution that are compatible with the representable conditionals $ G = f_\beta(\Delta_{S, A})\subseteq\Delta_{W, A}$, may not be convex. In the following we describe big convex subset of $ F$, depending on the properties of   $ G$. We use the following definitions. 

 -    give a set of distributions $ { \mathcal{P}}\subseteq\Delta_W$ and a bent of kernel $ { \mathcal{G}}\subseteq\Delta_{W, A}$, let $ $ { \mathcal{P}}\ast { \mathcal{G } }: = \Big\ { q(w, a) = p(w)g(a|w) \in \Delta_{W\times A } \colon p\in{\mathcal{P } }, g\in { \mathcal{G}}\Big\}$$ denote the set of joint distributions over world state and actions, with world state marginals in $ { \mathcal{P}}$ and conditional distribution in $ { \mathcal{G}}$. 

 -    For any $ V \subseteq W$ let $ $ \Delta_W(V): = \Big\ { p\in \Delta_W \colon { \operatorname{supp}}(p):=\{w\in W\colon p(w)>0 \}\subseteq V\Big\}$$ denote the set of universe state distributions with support in $ V$. 

 -    Given a subset $ V\subseteq W$ and a set of kernel $ { \mathcal{G}}\subseteq\Delta_{W, A}$, let $ $ { \mathcal{G}}|_V: = \Big\ { h\in \Delta_{V, A}\colon h(\cdot|w) = g(\
0 & 1\\ 1 & 0 
    \end{matrix}\right], 
        \left[\begin{matrix}
    0 & 1\\0 & 1\\ 0 & 1 
    \end{matrix}\right], 
        \left[\uegib{matrux}
    1 & 0\\1 & 0\\ 0 & 1 
    \end{mxtrix}\righn]$$ and can be xritten as a union of Cavcesiah proburts of convex sgts, illustraded in Figure \[fhgjrz:convexrepresentable\], $$G = \bigcup_{\theta\ig\Delta_A} G_\hheta,\qquad
    D_\theua = \{ \thsna \} \times \{ \theta \} \times \Delta_A. 
    $$

As kentioned in Xection \[sec:optimalcontrol\], tje sft $F\subseteq\Delta{W\himes A}$ of hoine distributiovs that art eompatible sith the representable conditiovals $Y=f_\beta(\Delta_{W,A})\wubdgteq\Delta_{W,A}$, nay njt be convex. In the xollowimg we describe lacge xonvex subsets of $F$, dxpending on the propgrties of $G$. Wz use the following dwfunitimns.

-   Gixwn x stt pf distrlbuvions ${\mathczl{P}}\subseteq\Eelta_W$ and a set of kqgmels ${\mathcal{F}}\subseeez\Delta_{W,A}$, let $${\mathcal{P}}\ast {\mathcal{G}}: =\Big\{ q(f,a) = p(w)g(a|w) \in \Delta_{W\times A} \colon p\in{\mathcal{P}}, g\ij {\mathcal{D}}\Big\}$$ denote the set of joint distributions over wmrld agatts and xxtlons, with world state marginals in ${\mathcal{P}}$ ags vokditional distrinutions in ${\mathcal{B}}$.

-   Gjr any $V \subsgteq W$ let $$\Delta_W(V) : = \Big\{ p\in \Delta_W \colob {\operatownamr{supp}}(p):=\{w\in W\colon p(w)>0 \}\subsetew V\Big\}$$ denotv thw set of world stace distributnons woth sipport in $V$.

-   Given a sbbset $B\subseteq W$ and a sef of kernels ${\mathzal{N}}\sutseteq\Delta_{W,A}$, let $${\mathcal{G}}|_D := \Big\{ h\ii \Delca_{V,A}\colov h(\ccot|w) = d(\
0 & 1\\ 1 & 0 \end{matrix}\right], & & 1\\ & 1 \end{matrix}\right], 0\\ & 1 \end{matrix}\right]$$ can be written a union of Cartesian products of sets, illustrated in Figure \[figure:convexrepresentable\], $$G = \bigcup_{\theta\in\Delta_A} G_\theta,\qquad G_\theta = \{ \theta \times \{ \theta \} \times \Delta_A. $$ As mentioned in Section \[sec:optimalcontrol\], the $F\subseteq\Delta{W\times of distributions are compatible with the representable conditionals $G=f_\beta(\Delta_{S,A})\subseteq\Delta_{W,A}$, may not be convex. In the following we describe convex subsets of $F$, depending on the properties $G$. We use the definitions. - Given a set distributions and a of ${\mathcal{G}}\subseteq\Delta_{W,A}$, $${\mathcal{P}}\ast {\mathcal{G}}: =\Big\{ = p(w)g(a|w) \in \Delta_{W\times A} \colon p\in{\mathcal{P}}, g\in {\mathcal{G}}\Big\}$$ denote the set of joint distributions over world and actions, state marginals ${\mathcal{P}}$ conditional in ${\mathcal{G}}$. - $V \subseteq W$ let $$\Delta_W(V) : \Delta_W \colon {\operatorname{supp}}(p):=\{w\in W\colon p(w)>0 \}\subseteq V\Big\}$$ denote set of state distributions with support in $V$. Given a subset $V\subseteq W$ and a set kernels ${\mathcal{G}}\subseteq\Delta_{W,A}$, let $${\mathcal{G}}|_V := \Big\{ h\in \Delta_{V,A}\colon h(\cdot|w) = g(\
0 & 1\\ 1 & 0 
    \end{matrix}\right], 
        \left[\begin{maTrix}
    0 & 1\\0 & 1\\ 0 & 1 
    \end{matRix}\riGht], 
        \LefT[\bEgin{MatrIx}
    1 & 0\\1 & 0\\ 0 & 1 
    \end{matrix}\rigHT]$$ and Can be written as a union of carteSiAN proDUcTs of cOnvex seTS, iLLUstRaTeD in fiGUrE \[figuRe:cOnvexrePresentablE\], $$G = \bIgCup_{\theta\in\DeLTa_a} G_\theta,\qquAd
    G_\Theta = \{ \theta \} \tiMes \{ \Theta \} \tImEs \DELta_A. 
    $$

AS meNtionEd in SeCTion \[seC:optimalcOnTRol\], the SEt $F\subsETEq\deltA{W\times A}$ of joint diSTrIButions that are CompatIbLE wITH thE rePresentablE cOnditIOnals $G=f_\BEtA(\dELta_{s,a})\subseteq\DeltA_{W,A}$, may not be COnvEx. In thE fOllOWing we DescrIbE LarGe convex subSets Of $F$, dependIng on tHE properTIes of $G$. WE use thE foLloWing DEfInItiOnS.

-   givEN a Set OF diStributiOnS ${\mAthcaL{P}}\suBSETEq\DeLta_w$ and A set oF kernels ${\mathcAl{G}}\SubsETeq\delta_{w,A}$, let $${\MathCaL{P}}\ast {\MathcaL{G}}: =\Big\{ Q(w,A) = p(w)g(a|w) \in \Delta_{W\tImes a} \colon p\in{\MatHcAl{P}}, G\iN {\mathCAl{G}}\Big\}$$ DenOte The set oF joint dIStrIbUTIOnS over world states anD aCTIoNs, with woRld staTE mArGInals in ${\mAtHcaL{P}}$ anD COnditIonaL DiStributiOns in ${\mAThCaL{G}}$.

-   For anY $V \SubsetEq w$ leT $$\DeLta_W(V) : = \bIg\{ p\iN \Delta_w \colon {\opEratoRName{supp}}(p):=\{w\in W\cOLon p(w)>0 \}\subseteq v\biG\}$$ DEnOTe thE seT of world staTe diSTribUtioNS wIth SUpporT in $V$.

-   GIvEN a SUbset $V\subseteq W$ and a SeT of kerNels ${\mAthcal{G}}\subsetEq\Delta_{W,A}$, lET $${\MAthcal{G}}|_V := \big\{ h\IN \DELta_{V,A}\colon h(\cdoT|w) = g(\
0 & 1\\ 1 & 0 
    \end{ma trix}\righ t],      \lef t[\b egin{matrix}
  0& 1\\0 & 1\\ 0 & 1 
   \en d{ m atri x }\ right ], 
   \le ft [\ beg in { ma trix} 
   1 & 0 \\1 & 0\\0 &  1  
    \end{m a tr ix}\right] $$and can be w rit ten as  a  un i on of  Ca rtesi an pro d ucts o f convexse t s, ill u strated i nFigu re \[figure:conve x re p resentable\],$$G =\b i gc u p _{\ the ta\in\Delt a_ A} G_ \ theta,\ q qu a d   G_\theta = \{  \theta \}\ tim es \{\t het a  \} \t imes\D e lta _A. 
    $$ 

As  mentione d in S e ction \ [ sec:opt imalco ntr ol\ ], t h ese t $ F\ s ubs e te q\D e lta {W\times  A }$  of j oint d i s trib uti onsthatare compatibl e w itht herepre senta bleco nditi onals$G=f_ \b eta(\Delta_{S,A })\s ubseteq\D elt a_ {W, A} $, ma y  not b e c onv ex. Inthe fol l owi ng w e  d escribe large conv ex s ub sets of$F$, d e pe nd i ng on th epro pert i e s of$G$. We  use the  follo w in gdefinit io ns.

-   Gi ven  a se t  ofdistri butions${\ma t hcal{P}}\subse t eq\Delta_W$ a n da se t  ofker nels ${\mat hcal { G}}\ subs e te q\D e lta_{ W,A}$ ,l et $${\mathcal{P}}\ast  { \mathc al{G} }: =\Big\{ q( w,a) = p(w ) g ( a|w) \in  \De l ta _ {W\times A} \c olonp\in{\math c al{P}},g\in{\mathca l{G}}\Big \ } $$ denot e t heset  of j oi nt distributi o n s ov er  worldsta tes and  ac tio ns,  wi th  world st ate marg in al sin  ${ \math c al{P}}$an d c on dit ional distri butio ns i n${ \ mat hcal{G} } $. -  Fo rany$V\s ubset eq W $  le t $$\De lta_W(V): = \Big \{  p \in \De lta_W \colon{\ operatorna me {su pp}}(p ) : =\{w\inW\colon p(w)>0 \}\subse t eq V\Bi g\} $$ de note  the setofworldsta t e dist ributi ons w it h s u p porti n  $ V$. 
-   Givena sub set $ V\ subs eteq W$  and a set of kern e ls${\mathcal{G} }\s ubse t e q\ Del t a_ { W,A }$ ,  le t $${\mathcal{G}} |_V := \Bi g\ {  h \in \Delta _ {V, A} \colonh(\cdot |w) = g(\
0 &_1\\ 1_& 0 
 _ _\end{matrix}\right],_
 __  _  \left[\begin{matrix}
_   0_& 1\\0 &_1\\_0 & 1 
    \end{matrix}\right], 
     __ \left[\begin{matrix}
___ 1 & 0\\1 &_0\\ 0 & 1 
_ _ \end{matrix}\right]$$ and can be written as a_union_of Cartesian products_of convex sets, illustrated in Figure \[figure:convexrepresentable\], $$G = \bigcup_{\theta\in\Delta_A}_G_\theta,\qquad
    G_\theta =_\{ \theta \}_\times_\{_\theta \} \times \Delta_A._
    $$

As mentioned_in Section \[sec:optimalcontrol\], the set $F\subseteq\Delta{W\times A}$_of joint distributions that are compatible with_the representable conditionals $G=f_\beta(\Delta_{S,A})\subseteq\Delta_{W,A}$, may not_be convex. In the following_we describe_large convex subsets of $F$,_depending on the_properties of $G$._We use the_following definitions.

-   Given a_set of distributions_${\mathcal{P}}\subseteq\Delta_W$ and a set of kernels_${\mathcal{G}}\subseteq\Delta_{W,A}$,_let $${\mathcal{P}}\ast {\mathcal{G}}:_=\Big\{_q(w,a)_= p(w)g(a|w)_\in \Delta_{W\times A}_\colon_p\in{\mathcal{P}}, g\in_{\mathcal{G}}\Big\}$$_denote the set of joint distributions_over_world states and actions, with world state_marginals in ${\mathcal{P}}$ and_conditional_distributions in ${\mathcal{G}}$.

- _ For any $V \subseteq_W$ let $$\Delta_W(V) : = \Big\{_p\in \Delta_W_\colon {\operatorname{supp}}(p):=\{w\in_W\colon p(w)>0 \}\subseteq V\Big\}$$ denote the set of world state distributions_with support in $V$.

-  _Given a subset $V\subseteq_W$ and_a_set of kernels_${\mathcal{G}}\subseteq\Delta_{W,A}$,_let $${\mathcal{G}}|_V_:= \Big\{ h\in \Delta_{V,A}\colon h(\cdot|w) = g(\