 in Figure \[fig:meta\_ex\] (left), for instance, the edge weight connecting $s_1$ and $s_2$ is the probability that a trajectory reaches $s_2$ before it can return to $s_1$, when executing $\pi_1$. These probabilities may be estimated either by sampling (Monte Carlo simulations), or computed analytically. The first approach is trivially implemented, with the usual advantages (e.g. parallelism, access to a black-box simulator is all is needed) and disadvantages (slow convergence); here we develop the latter, which leads to a concise set of linear problems to be solved, and sheds light on both the mathematical and computational structure. Since the bottlenecks partition the statespace into disjoint sets, the probabilities $\widetilde{P}(s,a,s')$ can be quickly computed in each [cluster]{}separately.

Let $a$ be the action corresponding to executing a policy $\pi_{\ensuremath{\mathsf{c}}\xspace}$ in [cluster]{} ${\ensuremath{\mathsf{c}}\xspace}$. Then $$\widetilde{P}(s,a,s') = H_{s,s'},\quad  \text{ for all } s,s'\in {\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace},$$ where $H$ is the [*minimal non-negative solution*]{}, for each $s'\in{\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace}$, to the linear system $$\label{eqn:trans_probs}
H_{s,s'} =  {P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(s,s') + \sum_{s''\in{\accentset{\circ}{{\ensuremath{\mathsf{c}}\xspace}}}}{P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(s,s'')H_{s'',s'},\quad s\in{\ensuremath{\mathsf{c}}\xspace}, s'\in{\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace}\,,$$ or in matrix-vector form, $$\label{eqn:trans_probs_matvec}
\bigl(I - {P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(I-J)\bigr)H = {P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\
in Figure   \[fig: meta\_ex\ ] (left), for instance, the edge weight connect $ s_1 $ and $ s_2 $ is the probability that a trajectory pass $ s_2 $ before it can return to $ s_1 $, when carry through $ \pi_1$. These probability may be estimated either by sampling (Monte Carlo pretense), or calculate analytically. The first approach is trivially implemented, with the usual advantages (for example parallelism, access to a black - box simulator is all is needed) and disadvantage (slow convergence); here we originate the latter, which leads to a concise set of analogue problems to be solved, and sheds sparkle on both the mathematical and computational structure. Since the constriction partition the statespace into disjoint sets, the probability $ \widetilde{P}(s, a, s')$ can be quickly computed in each [ cluster]{}separately. 

 Let $ a$ be the action corresponding to execute a policy $ \pi_{\ensuremath{\mathsf{c}}\xspace}$ in [ cluster ] { }   $ { \ensuremath{\mathsf{c}}\xspace}$. Then $ $ \widetilde{P}(s, a, s') = H_{s, s'},\quad   \text { for all } s, s'\in { \ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace},$$ where $ H$ is the [ * minimal non - negative solution * ] { }, for each $ s'\in{\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace}$, to the linear system $ $ \label{eqn: trans_probs } 
 H_{s, s' } =   { P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(s, s') + \sum_{s''\in{\accentset{\circ}{{\ensuremath{\mathsf{c}}\xspace}}}}{P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(s, s'')H_{s'',s'},\quad s\in{\ensuremath{\mathsf{c}}\xspace }, s'\in{\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace}\,,$$ or in matrix - vector form, $ $ \label{eqn: trans_probs_matvec } 
 \bigl(I - { P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(I - J)\bigr)H = { P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\
 in Figure \[fig:meta\_ex\] (left), fov instance, the ebte weijht conhecting $r_1$ and $s_2$ is the probability tiat q trahectory reaches $s_2$ befofe it can return ro $s_1$, qhen execuvjng $\pi_1$. Bkese lvobabnlmties may be esjimated eithar by sampling (Munce Carlo simulations), or computed anajyticalky. The first apptoach ys tdpvlally implemented, with the usual advanteges (e.g. parallekism, access to a black-box dimupator is all is nefded) and diwadvwbtages (slow zonvergenct); kere we devglop the latter, which leads to a concnse set of oibeag problems ti be folved, and smvds ligvt on bpth the mathemstiral qnd computational strncture. Since the botjlenecks pdrcition the statespace ibto dhsjohnt rwts, tht pcobzbilitles $\widetilde{L}(s,a,s')$ can be quickly computed im qqch [cluster]{}selaratejy.

Jet $a$ be the action corresponding to extcutihg a policy $\pi_{\ensurematy{\mathsf{c}}\xspace}$ in [cludter]{} ${\ensurqmath{\mathsf{c}}\xspace}$. Then $$\widetilde{P}(s,a,s') = H_{s,s'},\quad  \taxt{ fkf aol } w,s'\ln {\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xs[zct},$$ wmere $H$ is the [*mikimal non-negative xopuyyon*]{}, for each $r'\in{\ensbdejath{\partial{\ensuremwth{\mathff{c}}\xspace}}\xspact}$, to yhe linear system $$\label{eqn:teans_probs}
H_{s,s'} =  {P_{\ensuremath{\mathsy{c}}\xspace}^{\pi_{\enrurekath{\msthsf{c}}\xspace}}}(s,s') + \sum_{s''\in{\aecentsst{\circ}{{\ensurfmath{\mathaw{c}}\xspace}}}}{P_{\ensuremxth{\kadhsf{c}}\xspace}^{\pi_{\ensuremath{\matrsf{c}}\xspacx}}}(s,s'')H_{s'',x'},\quad s\kn{\enxuremaeh{\mathsf{c}}\xdpace}, s'\in{\ensuremath{\partiwl{\ensorematv{\mathsf{c}}\xsoace}}\xspace}\,,$$ or in matrix-vector form, $$\label{eqn:trsnv_prmbs_matvee}
\bigl(I - {P_{\ensuremath{\iathsf{c}}\xspace}^{\pn_{\ensuremcth{\matfsf{c}}\xspace}}}(P-J)\bigr)H = {'_{\ensuremath{\mwthsf{c}}\xspace}^{\ph_{\
in Figure \[fig:meta\_ex\] (left), for instance, the connecting and $s_2$ the probability that it return to $s_1$, executing $\pi_1$. These may be estimated either by sampling Carlo simulations), or computed analytically. The first approach is trivially implemented, with the advantages (e.g. parallelism, access to a black-box simulator is all is needed) and (slow here develop latter, which leads to a concise set of linear problems to be solved, and sheds light both the mathematical and computational structure. Since the partition the statespace into sets, the probabilities $\widetilde{P}(s,a,s')$ can quickly in each Let be action corresponding to a policy $\pi_{\ensuremath{\mathsf{c}}\xspace}$ in [cluster]{} ${\ensuremath{\mathsf{c}}\xspace}$. Then $$\widetilde{P}(s,a,s') = H_{s,s'},\quad \text{ for all } s,s'\in {\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace},$$ where is the solution*]{}, for $s'\in{\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace}$, the system $$\label{eqn:trans_probs} H_{s,s'} + \sum_{s''\in{\accentset{\circ}{{\ensuremath{\mathsf{c}}\xspace}}}}{P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(s,s'')H_{s'',s'},\quad s\in{\ensuremath{\mathsf{c}}\xspace}, s'\in{\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace}\,,$$ or in \bigl(I - {P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(I-J)\bigr)H = {P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\
 in Figure \[fig:meta\_ex\] (left), for iNstance, the Edge wEigHt cOnNectIng $s_1$ And $s_2$ is the probaBIlitY that a trajectory reacheS $s_2$ befOrE It caN ReTurn tO $s_1$, when eXEcUTIng $\Pi_1$. thEse PrOBaBilitIes May be esTimated eitHer By Sampling (MontE caRlo simulatIonS), or computed aNalYticalLy. the FIrst aPprOach iS triviALly impLemented, wItH The usuAL advantAGEs (E.g. paRallelism, access to A BlACk-box simulator Is all iS nEEdED) And DisAdvantages (SlOw conVErgence); HErE WE DevELop the latter, wHich leads to A ConCise seT oF liNEar proBlems To BE soLved, and shedS ligHt on both tHe mathEMatical ANd compuTationAl sTruCturE. siNcE thE bOTtlENeCks PArtItion the StAtEspacE intO DISJoinT seTs, thE probAbilities $\wideTilDe{P}(s,A,S')$ caN be quIckly CompUtEd in eAch [cluSter]{}sEpArately.

Let $a$ be thE actIon corresPonDiNg tO eXecutINg a polIcy $\Pi_{\eNsuremaTh{\mathsF{C}}\xsPaCE}$ IN [cLuster]{} ${\ensuremath{\maThSF{C}}\xSpace}$. TheN $$\widetILdE{P}(S,A,s') = H_{s,s'},\quaD  \tExt{ For aLL } S,s'\in {\eNsurEMaTh{\partiaL{\ensurEMaTh{\Mathsf{c}}\XsPace}}\xsPaCe},$$ wHerE $H$ is tHE [*minImal noN-negativE soluTIon*]{}, for each $s'\in{\eNSuremath{\partiAL{\eNSUrEMath{\MatHsf{c}}\xspace}}\xSpacE}$, To thE linEAr SysTEm $$\labEl{eqn:TrANs_PRobs}
H_{s,s'} =  {P_{\ensuremath{\mAtHsf{c}}\xsPace}^{\pI_{\ensuremath{\maThsf{c}}\xspacE}}}(S,S') + \Sum_{s''\in{\acCentSEt{\CIrc}{{\ensuremath{\mAthsf{C}}\xspace}}}}{P_{\enSUremath{\mAthsf{C}}\xspace}^{\pI_{\ensuremaTH{\Mathsf{c}}\xSpaCe}}}(s,S'')H_{s'',S'},\quAD S\iN{\ensuremath{\maTHSf{c}}\xSpAce}, s'\in{\eNsuRemath{\pArtIal{\EnsUreMaTh{\mathsf{c}}\Xspace}}\xsPaCe}\,,$$ Or In MatRix-veCTor form, $$\lAbEl{eQn:TraNs_proBS_matveC}
\bigl(i - {P_{\enSuReMAth{\Mathsf{c}}\XSpACE}^{\pi_{\eNsUrEmatH{\maThSf{c}}\xsPace}}}(i-j)\biGr)H = {P_{\ensUremath{\maThsF{C}}\xspAcE}^{\pI_{\
 in Figure \[fig:meta\_ex\ ] (left),for i nst anc e,  the  edg e weight conne c ting  $s_1$ and $s_2$ is th e pro ba b ilit y  t hat a  trajec t or y rea ch es  $s _2 $  b efore  it  can re turn to $s _1$ ,when executi n g$\pi_1$. T hes e probabilit ies  may b eest i mated  ei therby sam p ling ( Monte Car lo simula t ions),o r  c ompu ted analytically. Th e  first approac h is t ri v ia l l y i mpl emented, w it h the usual a d va n t a ges (e.g. paralle lism, acces s  to  a bla ck -bo x  simul atoris all  is needed)  and  disadvan tages( slow co n vergenc e); he rewedeve l op  t hela t ter ,  w hic h  le ads to a  c on ciseseto f l inea r p robl ems t o be solved,and  she d s l ighton bo th t he  math ematic al an dcomputational s truc ture. Sin ceth e b ot tlene c ks par tit ion  the st atespac e  in to d i sj oint sets, the pro ba b i li ties $\w idetil d e{ P} ( s,a,s')$  c anbe q u i cklycomp u te d in eac h [clu s te r] {}separ at ely.
Le t $ a$be th e  act ion co rrespond ing t o  executing a p o licy $\pi_{\e n su r e ma t h{\m ath sf{c}}\xspa ce}$ in [ clus t er ]{} ${\en surem at h {\ m athsf{c}}\xspace}$.  T hen $$ \wide tilde{P}(s,a, s') = H_{s , s ' },\quad \te x t{ for all } s,s' \in { \ensuremat h {\partia l{\en suremath {\mathsf{ c } }\xspace }}\ xsp ace },$ $ wh ere $H$ is th e [*mi ni mal non -ne gativesol uti on* ]{} ,for each$s'\in{\ en su re ma th{ \part i al{\ensu re mat h{ \ma thsf{ c }}\xsp ace}} \xsp ac e} $ , t o the l i ne a r  sys te m$$\l abe l{ eqn:t rans _ pro bs}
H_{ s,s'} = {P_ { \ens ur em ath{\ma thsf{c}}\xspa ce }^{\pi_{\e ns ure math{\ m a thsf{c}} \xspace}}}(s,s') + \sum _ {s''\in {\a ccent set{ \circ}{{\ ens uremat h{\ m athsf{ c}}\xs pace} }} }{P _ { \ensu r e ma th{ \m athsf{c}}\ x s pac e}^{\ pi _{\e nsurema th{\mathsf{c}}\xsp a ce} }}(s,s'')H_{s '', s'}, \ q ua d s \ in { \en su r ema t h {\mathsf{c}}\xs pace}, s'\ in { \e nsuremath{ \ par ti al{\ens uremath {\mat h sf{c}}\ xspace}}\ xspace}\, ,$ $ or i n m atrix-vect or form,  $$\label { eqn:t r an s_pro bs_ matvec }\bi gl(I- {P_{ \ ens urema th{\ma th sf{c}} \xspa ce }^{\pi_{ \ensuremath{\mathsf{c}} \xspac e}}}( I-J )\bigr)H= { P _{\ ensuremat h{\m athsf{c}}\ xsp ace }^{\p i_{ \ 
 in_Figure \[fig:meta\_ex\] (left),_for instance, the edge_weight connecting_$s_1$_and $s_2$_is_the probability that_a trajectory reaches_$s_2$ before it can_return to $s_1$,_when_executing $\pi_1$. These probabilities may be estimated either by sampling (Monte Carlo simulations), or_computed_analytically. The_first_approach_is trivially implemented, with the_usual advantages (e.g. parallelism, access_to a_black-box simulator is all is needed) and disadvantages_(slow_convergence); here we_develop the latter, which leads to a concise set_of linear problems to be solved,_and sheds light_on_both_the mathematical and computational_structure. Since the bottlenecks partition the_statespace into disjoint sets, the probabilities_$\widetilde{P}(s,a,s')$ can be quickly computed in each_[cluster]{}separately.

Let $a$ be the action corresponding_to executing a policy $\pi_{\ensuremath{\mathsf{c}}\xspace}$_in [cluster]{} ${\ensuremath{\mathsf{c}}\xspace}$._Then $$\widetilde{P}(s,a,s') = H_{s,s'},\quad _\text{ for all_} s,s'\in_{\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace},$$ where $H$_is the [*minimal non-negative solution*]{}, for_each $s'\in{\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace}$, to_the linear system $$\label{eqn:trans_probs}
H_{s,s'} = _{P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(s,s')_+ \sum_{s''\in{\accentset{\circ}{{\ensuremath{\mathsf{c}}\xspace}}}}{P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(s,s'')H_{s'',s'},\quad s\in{\ensuremath{\mathsf{c}}\xspace},_s'\in{\ensuremath{\partial{\ensuremath{\mathsf{c}}\xspace}}\xspace}\,,$$_or_in matrix-vector_form, $$\label{eqn:trans_probs_matvec}
\bigl(I -_{P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\ensuremath{\mathsf{c}}\xspace}}}(I-J)\bigr)H_= {P_{\ensuremath{\mathsf{c}}\xspace}^{\pi_{\