label{eq:py}\end{aligned}$$ $Z = \sum_{{\mathbf{y}}} \sum_{{\mathbf{z}}} \exp \left( \mathcal{E}({\mathbf{y}}, {\mathbf{z}}| {\mathbf{x}}) \right)$ is the partition function.

In the case of our Embedded Latent CRF model, as in the LDCRF model, latent states are deterministically partitioned to correspond to output values. That is, the number of latent states is a multiple of the number of output values, and $\psi_{zy}=0$ for pairs in the partitioning and $-\infty$ otherwise. The other scoring functions are learned as global bilinear parameter matrices.

In order to manage large numbers of latent states without overfitting, the Embedded Latent CRF enforces an additional restriction that the scoring function $\psi_{zz}$ possess a low-rank structure: that is, $\psi_{zz}(z_i,z_j)=z_i^\top U V^\top z_j$, where $U$ and $V$ are skinny rectangular matrices and $z_i,z_j$ are represented by one-hot vectors.

While inference in this model is tractable using tree belief propagation even when learning $\psi_{zy}$, the deterministic factors make it especially simple to implement.

Computing the quantities involved in can be carried out efficiently with dynamic programming using the forward algorithm, as in HMMs. To see this, note that to compute the numerator $\sum_{{\mathbf{z}}} \exp \left( \mathcal{E}({\mathbf{y}}, {\mathbf{z}}| {\mathbf{x}}) \right)$, given an output label ${\mathbf{y}}$, we can fold the local scores $\psi_{zf}$ and $\psi_{zy}$ into one score $\psi_{zfy}(f_t, z_t; y_t)$, and summing the resulting energy corresponds exactly to the forward algorithm in a CRF with $M$ states. The partition function can also be computed by dynamic programming: $$\begin{aligned}
Z &= \sum_{{\mathbf{y}}} \sum_{{\mathbf{z}}} \exp \left( \mathcal{E}({\mathbf{y}}, {\mathbf{z}}| {\mathbf{x}}) \right) \\
&= \sum_{{\mathbf{z}}} \sum_{{\mathbf{y}}} \prod_t  \exp \left(  \psi
label{eq: py}\end{aligned}$$ $ Z = \sum_{{\mathbf{y } } } \sum_{{\mathbf{z } } } \exp \left (\mathcal{E}({\mathbf{y } }, { \mathbf{z}}| { \mathbf{x } }) \right)$ is the partition function. 

 In the case of our Embedded Latent CRF model, as in the LDCRF exemplar, latent state are deterministically partitioned to correspond to output value. That is, the number of latent states is a multiple of the issue of output signal values, and $ \psi_{zy}=0 $ for pair in the partitioning and $ -\infty$ otherwise. The early scoring functions are learned as global bilinear argument matrices. 

 In order to oversee large numbers of latent state without overfitting, the Embedded Latent CRF enforces an additional restriction that the marking function $ \psi_{zz}$ possess a gloomy - rank social organization: that is, $ \psi_{zz}(z_i, z_j)=z_i^\top U V^\top z_j$, where $ U$ and $ V$ are tight-fitting rectangular matrices and $ z_i, z_j$ are represented by one - blistering vectors. 

 While inference in this model is tractable using tree belief propagation even when learning $ \psi_{zy}$, the deterministic factors make it especially simple to implement. 

 Computing the quantities involved in can be carried out efficiently with dynamic scheduling use the forward algorithm, as in HMMs. To see this, notice that to calculate the numerator $ \sum_{{\mathbf{z } } } \exp \left (\mathcal{E}({\mathbf{y } }, { \mathbf{z}}| { \mathbf{x } }) \right)$, given an output label $ { \mathbf{y}}$, we can fold the local scores $ \psi_{zf}$ and $ \psi_{zy}$ into one sexual conquest $ \psi_{zfy}(f_t, z_t; y_t)$, and summing the resulting energy corresponds precisely to the forward algorithm in a CRF with $ M$ states. The partition function can also be computed by dynamic scheduling: $ $ \begin{aligned } 
 Z & = \sum_{{\mathbf{y } } } \sum_{{\mathbf{z } } } \exp \left (\mathcal{E}({\mathbf{y } }, { \mathbf{z}}| { \mathbf{x } }) \right) \\ 
 & = \sum_{{\mathbf{z } } } \sum_{{\mathbf{y } } } \prod_t   \exp \left (  \psi
labfl{eq:py}\end{aligned}$$ $Z = \sum_{{\mxthbf{y}}} \sum_{{\mathby{z}}} \exp \neft( \mzthcal{E}({\mxthbf{y}}, {\mathbf{z}}| {\mathbf{x}}) \right)$ iw the partition function.

In ghe case lf our Enbedved Latent CRF model, as lu the PDCRY nodel, latent sjates are dederministicallf oaxtitioned to correspond to output vajues. Thst is, the number of kwtenf states is a multiple of the numbsr of oltput values, and $\lsi_{zy}=0$ for pairs in the parhitilning and $-\infty$ otjerwise. The othqe scoring fuvctions art nearned as global bilinear parameter matrkces.

Iu order to nabagf large numbxrs of latent statcx withmut ovetfitting, the Ekbevded Latent CRF enforces en additional restristion thad che scoring function $\psu_{zz}$ pmssevs a oow-fani xtducturf: tiat is, $\psi_{za}(z_i,z_j)=z_i^\top Y V^\top z_j$, where $U$ amd $N$ are skinny dectandujar matrices and $z_i,z_j$ are represented bj ons-hot vectors.

While inferwnce in this model is tractablq using tree belief propagation even when learninc $\psi_{vy}$, tht eeterounlstic factors make it especially simple to im[mekekt.

Computing the zuantities ongokded in can be carrizs kut efficiently wihh dynaiic peogrammind usong the forward algorithm, aw in HMMs. To wee this, note that to compute the numetator $\sum_{{\mathbf{z}}} \exp \leyt( \matgcal{E}({\mathbf{j}}, {\mathbf{z}}| {\oathbf{x}}) \right)$, gixen at output label ${\mathbf{y}}$, we san fold vhe lpcal scures $\psi_{zf}$ and $\psi_{zy}$ into one score $\psi_{zfy}(f_t, z_t; y_j)$, and vumming thf resulting energy corresponds xeactly to the fmrwdrd algoxithm ln a CRF with $M$ states. The pattition fbnctiov can also be com'uted by dynwmic programmhjg: $$\begin{aligied}
Z &= \sum_{{\iathvf{y}}} \wum_{{\mathcw{z}}} \exp \left( \mayhcal{E}({\matkyf{y}}, {\mathbd{z}}| {\mathbf{x}}) \right) \\
&= \xum_{{\ozthbf{z}}} \sum_{{\mathby{v}}} \krid_t  \exp \left(  \psk
label{eq:py}\end{aligned}$$ $Z = \sum_{{\mathbf{y}}} \sum_{{\mathbf{z}}} \exp \left( {\mathbf{x}}) is the function. In the CRF as in the model, latent states deterministically partitioned to correspond to output That is, the number of latent states is a multiple of the number output values, and $\psi_{zy}=0$ for pairs in the partitioning and $-\infty$ otherwise. The scoring are as bilinear parameter matrices. In order to manage large numbers of latent states without overfitting, the Embedded CRF enforces an additional restriction that the scoring $\psi_{zz}$ possess a low-rank that is, $\psi_{zz}(z_i,z_j)=z_i^\top U V^\top where and $V$ skinny matrices $z_i,z_j$ are represented one-hot vectors. While inference in this model is tractable using tree belief propagation even when learning $\psi_{zy}$, deterministic factors especially simple implement. the involved in can out efficiently with dynamic programming using as in HMMs. To see this, note that compute the $\sum_{{\mathbf{z}}} \exp \left( \mathcal{E}({\mathbf{y}}, {\mathbf{z}}| {\mathbf{x}}) given an output label ${\mathbf{y}}$, we can fold local scores $\psi_{zf}$ and $\psi_{zy}$ into one score $\psi_{zfy}(f_t, z_t; y_t)$, and summing the resulting exactly to the forward in a CRF $M$ The function also be by dynamic programming: $$\begin{aligned} Z &= \sum_{{\mathbf{y}}} \sum_{{\mathbf{z}}} \exp \left( \mathcal{E}({\mathbf{y}}, {\mathbf{x}}) \right) \\ &= \sum_{{\mathbf{z}}} \sum_{{\mathbf{y}}} \prod_t \exp \left( \psi
label{eq:py}\end{aligned}$$ $Z = \sum_{{\maThbf{y}}} \sum_{{\maThbf{z}}} \Exp \LefT( \mAthcAl{E}({\mAthbf{y}}, {\mathbf{z}}| {\mAThbf{X}}) \right)$ is the partition fuNctioN.

IN The cASe Of our embeddeD laTENt CrF MoDel, As IN tHe LDCrF mOdel, latEnt states aRe dEtErministicalLY pArtitioned To cOrrespond to oUtpUt valuEs. thaT Is, the NumBer of Latent STates iS a multiplE oF The numBEr of outPUT vAlueS, and $\psi_{zy}=0$ for pairs IN tHE partitioning aNd $-\inftY$ oTHeRWIse. the Other scoriNg FunctIOns are lEArNED As gLObal bilinear pArameter matRIceS.

In ordEr To mANage laRge nuMbERs oF latent statEs wiThout overFittinG, The EmbeDDed LateNt CRF eNfoRceS an aDDiTiOnaL rEStrICtIon THat The scoriNg FuNctioN $\psi_{ZZ}$ POSsesS a lOw-raNk strUcture: that is, $\pSi_{zZ}(z_i,z_J)=Z_i^\tOp U V^\tOp z_j$, wHere $u$ aNd $V$ arE skinnY rectAnGular matrices anD $z_i,z_J$ are repreSenTeD by OnE-hot vECtors.

WHilE inFerence In this mODel Is TRACtAble using tree belieF pROPaGation evEn when LEaRnINg $\psi_{zy}$, tHe DetErmiNIStic fActoRS mAke it espEciallY SiMpLe to impLeMent.

CoMpUtiNg tHe quaNTitiEs invoLved in caN be caRRied out efficieNTly with dynamiC PrOGRaMMing UsiNg the forwarD algORithM, as iN hMms. TO See thIs, notE tHAt TO compute the numeratoR $\sUm_{{\mathBf{z}}} \exP \left( \mathcal{E}({\Mathbf{y}}, {\matHBF{Z}}| {\mathbf{x}}) \RighT)$, GiVEn an output labeL ${\mathBf{y}}$, we can foLD the locaL scorEs $\psi_{zf}$ aNd $\psi_{zy}$ inTO One score $\Psi_{Zfy}(F_t, z_T; y_t)$, AND sUmming the resuLTIng eNeRgy corrEspOnds exaCtlY to The ForWaRd algoritHm in a CRF WiTh $m$ sTaTes. the paRTition fuNcTioN cAn aLso be COmputeD by dyNamiC pRoGRamMing: $$\begIN{aLIGned}
z &= \sUm_{{\MathBf{y}}} \SuM_{{\mathBf{z}}} \eXP \leFt( \mathcAl{E}({\mathbf{Y}}, {\maTHbf{z}}| {\MaThBf{x}}) \righT) \\
&= \sum_{{\mathbf{z}}} \suM_{{\mAthbf{y}}} \prod_T  \eXp \lEft(  \psi
label{eq:py}\end{aligned}$ $ $Z = \su m_{{\ mat hbf {y }}}\sum _{{\mathbf{z}} }  \ex p \left( \mathcal{E}({ \math bf { y}}, {\ mathb f{z}}|{ \m a t hbf {x }} ) \ ri g ht )$ is  th e parti tion funct ion .
In the case of  our Embed ded  Latent CRFmod el, as  i n t h e LDC RFmodel , late n t stat es are de te r minist i cally p a r ti tion ed to correspondt oo utput values.That i s, th e num ber  of latent  s tates is a mu l ti p l e  of the number of  output val u es,  and $ \p si_ { zy}=0$  forpa i rsin the part itio ning and$-\inf t y$ othe r wise. T he oth ersco ring fu nc tio ns are le arn e d a s global  b il inear  par a m e t er m atr ices .

In  order to man age  lar g e n umber s oflate nt  stat es wit houtov erfitting, theEmbe dded Late ntCR F e nf orces an add iti ona l restr ictiont hat  t h e sc oring function $\p si _ { zz }$ posse ss a l o w- ra n k struct ur e:that i s, $\ psi_ { zz }(z_i,z_ j)=z_i ^ \t op  U V^\t op  z_j$,  w her e $ U$ an d  $V$  are s kinny re ctang u lar matrices a n d $z_i,z_j$ a r er e pr e sent edby one-hotvect o rs.
Whi l einf e rence  in t hi s  m o del is tractable us in g tree  beli ef propagatio n even whe n l earning$\ps i _{ z y}$, the deter minis tic factor s  make it  espe cially s imple toi m plement. 

C omp uti ngt h equantities in v o lved  i n can b e c arriedout  ef fic ien tl y with dy namic pr og ra mm in g u singt he forwa rd  al go rit hm, a s  in HM Ms. T o se eth i s,note th a tt o  com pu te  the  nu me rator  $\s u m_{ {\mathb f{z}}} \e xp\ left (\m athcal{ E}({\mathbf{y }} , {\mathbf {z }}|  {\mat h b f{x}}) \ right)$, given an outpu t  label${\ mathb f{y} }$, we ca n f old th e l o cal sc ores $ \psi_ {z f}$ a nd $\ p s i_ {zy }$  into ones c ore  $\ps i_ {zfy }(f_t,z_t; y_t)$, and su m min g the resulti ngener g y  c orr e sp o nds  e x act l y  to the forward  algorithm  i n  a  CRF with$ M$st ates. T he part ition functio n can als o be comp ut ed b y dyn amic progr amming:$$\begin{ a ligne d }Z &=\su m_{{\m at hbf {y}}}  \sum_ { {\m athbf {z}}}\e xp \le ft( \ ma thcal{E} ({\mathbf{y}}, {\mathbf {z}}|{\mat hbf {x}}) \ri ght )  \\ 
&= \sum_ {{\m athbf{z}}}  \s um_ {{\ma thb f {y}}}  \pr o d_ t \ exp \ left (   \psi
label{eq:py}\end{aligned}$$ $Z_= \sum_{{\mathbf{y}}}_\sum_{{\mathbf{z}}} \exp \left( \mathcal{E}({\mathbf{y}},_{\mathbf{z}}| {\mathbf{x}})_\right)$_is the_partition_function.

In the case_of our Embedded_Latent CRF model, as_in the LDCRF_model,_latent states are deterministically partitioned to correspond to output values. That is, the number_of_latent states_is_a_multiple of the number of_output values, and $\psi_{zy}=0$ for_pairs in_the partitioning and $-\infty$ otherwise. The other scoring_functions_are learned as_global bilinear parameter matrices.

In order to manage large numbers_of latent states without overfitting, the_Embedded Latent CRF_enforces_an_additional restriction that the_scoring function $\psi_{zz}$ possess a low-rank_structure: that is, $\psi_{zz}(z_i,z_j)=z_i^\top U V^\top_z_j$, where $U$ and $V$ are skinny_rectangular matrices and $z_i,z_j$ are represented_by one-hot vectors.

While inference in_this model_is tractable using tree belief_propagation even when_learning $\psi_{zy}$,_the deterministic factors_make it especially simple to implement.

Computing_the quantities involved_in can be carried out efficiently_with_dynamic programming using_the_forward_algorithm, as_in HMMs. To_see_this, note_that_to compute the numerator $\sum_{{\mathbf{z}}} \exp_\left(_\mathcal{E}({\mathbf{y}}, {\mathbf{z}}| {\mathbf{x}}) \right)$, given an output_label ${\mathbf{y}}$, we can_fold_the local scores $\psi_{zf}$_and $\psi_{zy}$ into one score_$\psi_{zfy}(f_t, z_t; y_t)$, and summing the_resulting energy_corresponds exactly_to the forward algorithm in a CRF with $M$ states. The_partition function can also be computed_by dynamic programming: $$\begin{aligned}
Z_&= \sum_{{\mathbf{y}}}_\sum_{{\mathbf{z}}}_\exp \left( \mathcal{E}({\mathbf{y}},_{\mathbf{z}}|_{\mathbf{x}}) \right)_\\
&= \sum_{{\mathbf{z}}} \sum_{{\mathbf{y}}} \prod_t  \exp \left(_ \psi