 <span style="font-variant:small-caps;">Bleu</span> for German$\rightarrow$Czech. Pivot adapter provides an additional boost of maximum +0.7% <span style="font-variant:small-caps;">Bleu</span> or -0.7% <span style="font-variant:small-caps;">Ter</span>.

Cross-lingual encoder pre-training is proved to be not effective in the plain transfer setup. It shows no improvements over plain transfer in French$\rightarrow$German, and 0.4% <span style="font-variant:small-caps;">Bleu</span> worse performance in German$\rightarrow$Czech. We conjecture that the cross-lingual encoder needs a lot more data to be fine-tuned for another decoder, where the encoder capacity is basically divided into two languages at the beginning of the fine-tuning. On the other hand, the pivot adapter directly improves the connection to an individually pre-trained decoder, which works nicely with small fine-tuning data.

Pivot adapter gives an additional improvement on top of the cross-lingual encoder; up to +0.4% <span style="font-variant:small-caps;">Bleu</span> in French$\rightarrow$German and +0.6% <span style="font-variant:small-caps;">Bleu</span> in German$\rightarrow$Czech. In this case, we extract source and pivot sentence representations from the same shared encoder for training the adapter.

Step-wise pre-training gives a big improvement up to +1.2% <span style="font-variant:small-caps;">Bleu</span> or -1.6% <span style="font-variant:small-caps;">Ter</span> against plain transfer in French$\rightarrow$German. It shows the best performance in both tasks when combined with the cross-lingual encoder: up to +1.2% <span style="font-variant:small-caps;">Bleu</span> in French$\rightarrow$German and +2.6% <span style="font-variant:small-caps;">Bleu</span> in German$\rightarrow$Czech, compared to the multilingual baseline. Step-wise pre-training prevents the cross-lingual encoder from degeneration, since the pivot$\rightarrow$target pre-training (Step
< span style="font - variant: small - caps;">Bleu</span > for German$\rightarrow$Czech. Pivot adapter provides an extra rise of maximal +0.7% < span style="font - variant: small - caps;">Bleu</span > or -0.7% < span style="font - variant: little - caps;">Ter</span >. 

 Cross - lingual encoder pre - education is proved to be not effective in the plain transfer apparatus. It shows no improvements over homely transfer in French$\rightarrow$German, and 0.4% < span style="font - variant: modest - caps;">Bleu</span > bad performance in German$\rightarrow$Czech. We conjecture that the cross - lingual encoder needs a batch more data to be fine - tuned for another decoder, where the encoder capacitance is basically divided into two languages at the beginning of the fine - tuning. On the other hand, the pivot arranger directly better the connection to an individually pre - trained decoder, which works nicely with belittled fine - tuning data. 

 Pivot adapter gives an additional improvement on top of the cross - lingual encoder; up to +0.4% < span style="font - variant: small - caps;">Bleu</span > in French$\rightarrow$German and +0.6% < span style="font - variant: small - caps;">Bleu</span > in German$\rightarrow$Czech. In this case, we extract source and pivot sentence representations from the same share encoder for training the adapter. 

 dance step - wise pre - training gives a big improvement up to +1.2% < straddle style="font - variant: small - caps;">Bleu</span > or -1.6% < span style="font - variant: small - caps;">Ter</span > against plain transportation in French$\rightarrow$German. It shows the best performance in both tasks when combined with the cross - lingual encoder: up to +1.2% < span style="font - variant: small - caps;">Bleu</span > in French$\rightarrow$German and +2.6% < span style="font - variant: small - caps;">Bleu</span > in German$\rightarrow$Czech, compared to the multilingual service line. Step - judicious pre - training prevents the cross - lingual encoder from degeneration, since the pivot$\rightarrow$target pre - training (Step
 <spwn style="font-variant:small-gaps;">Bleu</span> for German$\cightardow$Czech. Pivot adapter provides an avdituonal boost of maximum +0.7% <spav style="fojt-varianr:smaoo-caps;">Bleu</s'zn> or -0.7% <span sfnle="fout-tariant:small-capx;">Ter</span>.

Crmss-lingual encmddr pre-training is proved to be not efsective ij the plain trwnsftr fetul. It shows no improvements over plzin traisfer in French$\tightarrow$German, and 0.4% <span stype="font-variant:small-faps;">Bleu</spab> wowwe performanze in German$\rightarrow$Dzech. We conjecture that the cruss-liugual encodgx nefgs a lot moce datw to be fine-blned fos anothrr decoder, wheve thx enxoder capacity is basmcally divided into jwo languacea at the beginnint if thg fina-tunkbg. Un uhe ofher hwnd, the pivot adapter dieectly improves the cjbnection to ah indidiqually pre-trained decoder, which works npcelg with small fine-tuning data.

Pivot adapter giges an adqitional improvement on top of the cross-lingual etcodec; jp uo +0.4% <spav shyle="font-variant:small-caps;">Bleu</span> in French$\rigrfatrpw$German and +0.6% <fpan style="fpnh-vstiant:small-caps;">Cleu</spch> jn German$\rightarroa$Czech. Yn thus case, wt exttact source and pivot sentebce represennatiins from the same dhared encober fot traiming the adapter.

Step-wisz pre-tdaining givfs a big joprovement up to +1.2% <sian vtyle="fonu-xariant:small-caps;">Bjeu</span> oc -1.6% <spcn style="wont-fariane:small-caps;">Her</span> against plain trwnsfet in Fsench$\rightwrrow$German. It shows the best pxcformance in nodh nasks wheu comblned with the cwoss-lingual eneoder: up to +1.2% <rpan style="ront-varmant:small-capf;">Bleu</span> in Xtench$\rightarrmw$German and +2.6% <spqn styld="wont-variant:smakl-caps;">Blel</skan> in Gernan$\rightarrow$Czech, coolared to the muotioingual baselinr. Sgep-ride pwa-training pravengs gne cruss-lingual cncuder from degeneration, shnce the pivot$\rightarrpw$barget prg-training (Step
<span style="font-variant:small-caps;">Bleu</span> for German$\rightarrow$Czech. Pivot adapter provides boost maximum +0.7% style="font-variant:small-caps;">Bleu</span> or -0.7% is to be not in the plain setup. It shows no improvements over transfer in French$\rightarrow$German, and 0.4% <span style="font-variant:small-caps;">Bleu</span> worse performance in German$\rightarrow$Czech. We conjecture the cross-lingual encoder needs a lot more data to be fine-tuned for another where encoder is divided into two languages at the beginning of the fine-tuning. On the other hand, the pivot directly improves the connection to an individually pre-trained which works nicely with fine-tuning data. Pivot adapter gives additional on top the encoder; to +0.4% <span in French$\rightarrow$German and +0.6% <span style="font-variant:small-caps;">Bleu</span> in German$\rightarrow$Czech. In this case, we extract source and pivot sentence from the encoder for the Step-wise gives a big to +1.2% <span style="font-variant:small-caps;">Bleu</span> or -1.6% plain transfer in French$\rightarrow$German. It shows the best in both when combined with the cross-lingual encoder: to +1.2% <span style="font-variant:small-caps;">Bleu</span> in French$\rightarrow$German and +2.6% style="font-variant:small-caps;">Bleu</span> in German$\rightarrow$Czech, compared to the multilingual baseline. Step-wise pre-training prevents the cross-lingual encoder from the pivot$\rightarrow$target pre-training (Step
 <span style="font-variant:small-Caps;">Bleu</spAn> for gerMan$\RiGhtaRrow$czech. Pivot adapTEr prOvides an additional boosT of maXiMUm +0.7% <spAN sTyle="fOnt-variANt:SMAll-CaPs;">bleU</sPAn> Or -0.7% <spaN stYle="font-Variant:smaLl-cApS;">Ter</span>.

CrosS-LiNgual encodEr pRe-training is ProVed to bE nOt eFFectiVe iN the pLain trANsfer sEtup. It shoWs NO improVEments oVER pLain Transfer in French$\rIGhTArrow$German, and 0.4% <Span stYlE="FoNT-VarIanT:small-caps;">blEu</spaN> Worse peRFoRMANce IN German$\rightaRrow$Czech. We COnjEcture ThAt tHE cross-LinguAl ENcoDer needs a loT morE data to be Fine-tuNEd for anOTher decOder, whEre The EncoDEr CaPacItY Is bASiCalLY diVided intO tWo LanguAges AT THE begInnIng oF the fIne-tuning. On thE otHer hANd, tHe pivOt adaPter DiRectlY improVes thE cOnnection to an inDiviDually pre-TraInEd dEcOder, wHIch worKs nIceLy with sMall finE-TunInG DATa.

pivot adapter gives aN aDDItIonal impRovemeNT oN tOP of the crOsS-liNguaL ENcodeR; up tO +0.4% <SpAn style="fOnt-varIAnT:sMall-capS;">BLeu</spaN> iN FrEncH$\righTArroW$GermaN and +0.6% <span Style="FOnt-variant:smalL-Caps;">Bleu</span> iN geRMAn$\RIghtArrOw$Czech. In thIs caSE, we eXtraCT sOurCE and pIvot sEnTEnCE representations froM tHe same ShareD encoder for trAining the aDAPTer.

Step-wIse pRE-tRAining gives a biG imprOvement up tO +1.2% <Span stylE="font-Variant:sMall-caps;">BLEU</span> or -1.6% <sPan StyLe="fOnt-VARiAnt:small-caps;">TER</Span> AgAinst plAin TransfeR in freNch$\RigHtArrow$GermAn. It showS tHe BeSt PerFormaNCe in both TaSks WhEn cOmbinED with tHe croSs-liNgUaL EncOder: up tO +1.2% <SpAN StylE="fOnT-varIanT:sMall-cAps;">BLEu</sPan> in FrEnch$\rightArrOW$GerMaN aNd +2.6% <span sTyle="font-variaNt:Small-caps;">BLeU</spAn> in GeRMAn$\rightaRrow$Czech, compared to the mULtilingUal BaselIne. STep-wise prE-trAining PreVEnts thE cross-LinguAl EncODEr froM DEgEneRaTion, since tHE PivOt$\rigHtArroW$target Pre-training (Step
 <span style="font-variant :small-cap s;">B leu </s pa n> f or G erman$\rightar r ow$C zech. Pivot adapter pr ovide sa n ad d it ional  boosto fm a xim um  + 0.7 %< sp an st yle ="font- variant:sm all -c aps;">Bleu</ s pa n> or -0.7 % < span style=" fon t-vari an t:s m all-c aps ;">Te r</spa n >.

Cr oss-lingu al encode r  pre-tr a i ni ng i s proved to be no t  e f fective in the  plain  t r an s f erset up. It sho ws  no i m proveme n ts o v erp lain transfer  in French$ \ rig htarro w$ Ger m an, an d 0.4 %< spa n style="fo nt-v ariant:sm all-ca p s;">Ble u </span>  worse  pe rfo rman c ein  Ge rm a n$\ r ig hta r row $Czech.We  c onjec ture t h a t th e c ross -ling ual encoder n eed s al otmoredatato b efine- tunedfor a no ther decoder, w here  the enco der  c apa ci ty is basica lly  di vided i nto two lan gu a g e sat the beginning o ft h efine-tun ing. O n  t he other ha nd , t he p i v ot ad apte r  d irectlyimprov e sth e conne ct ion to  a n i ndi vidua l ly p re-tra ined dec oder, which works ni c ely with smal l  f i n e- t unin g d ata.

Pivot  ada p tergive s  a n a d ditio nal i mp r ov e ment on top of thecr oss-li ngual  encoder; upto +0.4% < s p a n style= "fon t -v a riant:small-ca ps;"> Bleu</span >  in Fren ch$\r ightarro w$Germana n d +0.6%<sp ansty le= " f on t-variant:sma l l -cap s; ">Bleu< /sp an> inGer man $\r igh ta rrow$Czec h. In th is  c as e,  we  extr a ct sourc eand  p ivo t sen t ence r epres enta ti on s  fr om thes am e shar ed  e ncod erfo r tra inin g  th e adapt er.

Step -wi s e pr e- tr aininggives a big i mp rovement u pto+1.2%< s pan styl e="font-variant:small-c a ps;">Bl eu< /span > or  -1.6% <s pan  style ="f o nt-var iant:s mall- ca ps; " > Ter</ s p an > a ga inst plain t ran sferin  Fre nch$\ri ghtarrow$German. I t  sh ows the bestper form a n ce  in bo t h t as k s w h e n combined with  the cross -l i ng ual encode r : u pto +1.2 % <span  styl e ="font- variant:s mall-caps ;" >Ble u < /sp an> in Fre nch$\rig htarrow$G e rmana nd  +2.6 % < span s ty le= "font -varia n t:s mall- caps;" >B leu</s pan>in  German$ \rightarrow$Czech, comp ared t o the  mu ltilingua l b a sel ine. Step -wis e pre-trai nin g p reven tst he cr oss- l in gua l  enco derf rom degen e ra tio n ,  s ince the pi v o t $\r ighta rro w $targe t pr e-training (Step
 <span_style="font-variant:small-caps;">Bleu</span> for_German$\rightarrow$Czech. Pivot adapter provides_an additional_boost_of maximum_+0.7%_<span style="font-variant:small-caps;">Bleu</span> or_-0.7% <span style="font-variant:small-caps;">Ter</span>.

Cross-lingual_encoder pre-training is proved_to be not_effective_in the plain transfer setup. It shows no improvements over plain transfer in French$\rightarrow$German,_and_0.4% <span_style="font-variant:small-caps;">Bleu</span>_worse_performance in German$\rightarrow$Czech. We conjecture_that the cross-lingual encoder needs_a lot_more data to be fine-tuned for another decoder,_where_the encoder capacity_is basically divided into two languages at the beginning_of the fine-tuning. On the other_hand, the pivot_adapter_directly_improves the connection to_an individually pre-trained decoder, which works_nicely with small fine-tuning data.

Pivot adapter_gives an additional improvement on top of_the cross-lingual encoder; up to +0.4%_<span style="font-variant:small-caps;">Bleu</span> in French$\rightarrow$German and_+0.6% <span_style="font-variant:small-caps;">Bleu</span> in German$\rightarrow$Czech. In this_case, we extract_source and_pivot sentence representations_from the same shared encoder for_training the adapter.

Step-wise_pre-training gives a big improvement up_to_+1.2% <span style="font-variant:small-caps;">Bleu</span>_or_-1.6%_<span style="font-variant:small-caps;">Ter</span>_against plain transfer_in_French$\rightarrow$German. It_shows_the best performance in both tasks_when_combined with the cross-lingual encoder: up to_+1.2% <span style="font-variant:small-caps;">Bleu</span> in_French$\rightarrow$German_and +2.6% <span style="font-variant:small-caps;">Bleu</span>_in German$\rightarrow$Czech, compared to the_multilingual baseline. Step-wise pre-training prevents the_cross-lingual encoder_from degeneration,_since the pivot$\rightarrow$target pre-training (Step