 train time approaches.

What’s new? To the best of our knowledge, the splitting criterion, the boosting statement, the LOMtree algorithm, the swapping guarantee, and the experimental results are all new here.

Prior Work
----------

Only a few authors address logarithmic time training. The Filter tree [@BeygelzimerLR09] addresses consistent (and robust) multiclass classification, showing that it is possible in the statistical limit. The Filter tree does not address the partition problem as we do here which as shown in our experimental section is often helpful. The partition finding problem is addressed in the conditional probability tree [@BeygelzimerLLSS09], but that paper addresses conditional probability estimation. Conditional probability estimation can be converted into multiclass prediction [@Bishop:2006:PRM:1162264], but doing so is not a logarithmic time operation.

Quite a few authors have addressed logarithmic testing time while allowing training time to be $O(k)$ or worse. While these approaches are intractable on our larger scale problems, we describe them here for context. The partition problem can be addressed by recursively applying spectral clustering on a confusion graph [@BengioWG10] (other clustering approaches include [@journals/informaticaSI/MadzarovGC09]). Empirically, this approach has been found to sometimes lead to badly imbalanced splits [@DengSBL11]. In the context of ranking, another approach uses $k$-means hierarchical clustering to recover the label sets for a given partition [@weston13].

The more recent work [@conf/cvpr/ZhaoX13] on the multiclass classification problem addresses it via sparse output coding by tuning high-cardinality multiclass categorization into a bit-by-bit decoding problem. The authors decouple the learning processes of coding matrix and bit predictors and use probabilistic decoding to decode the optimal class label. The authors however specify a class similarity which is $\mathcal{O}(k^2)$ to compute (see Section $2.1.1$ in [@conf/cvpr/ZhaoX13]), and hence this approach is in a different complexity class than ours (this is also born out experimentally). The variant of the popular error correcting output code scheme for solving multi-label prediction problems with large output spaces under the assumption of output sparsity was also considered in [@DBLP:journals/corr/abs-0902-
train time approaches. 

 What ’s new? To the good of our cognition, the splitting criterion, the boosting instruction, the LOMtree algorithm, the swapping guarantee, and the experimental solution are all new here. 

 Prior oeuvre 
 ---------- 

 entirely a few authors cover logarithmic meter training. The Filter tree   [ @BeygelzimerLR09 ] address consistent (and robust) multiclass categorization, showing that it is possible in the statistical limit. The Filter tree does not address the partition problem as we do here which as read in our experimental section is often helpful. The partition determination problem is addressed in the conditional probability tree   [ @BeygelzimerLLSS09 ], but that composition addresses conditional probability appraisal. Conditional probability estimation can be converted into multiclass prediction   [ @Bishop:2006: PRM:1162264 ], but doing so is not a logarithmic time process. 

 Quite a few authors have addressed logarithmic testing time while allowing training time to be $ O(k)$ or worse. While these approaches are intractable on our larger scale problems, we describe them here for context. The partition problem can be addressed by recursively applying apparitional bunch on a confusion graph   [ @BengioWG10 ] (other clustering approaches include   [ @journals / informaticaSI / MadzarovGC09 ]). Empirically, this overture has been found to sometimes lead to badly imbalanced split   [ @DengSBL11 ]. In the context of ranking, another approach uses $ k$-means hierarchical clustering to recover the label set for a given partition   [ @weston13 ]. 

 The more recent work   [ @conf / cvpr / ZhaoX13 ] on the multiclass classification problem address it via sparse output coding by tuning high - cardinality multiclass categorization into a bit - by - snatch decoding problem. The authors decouple the learning process of coding matrix and bit predictors and use probabilistic decoding to decode the optimal class label. The authors however assign a class similarity which is $ \mathcal{O}(k^2)$ to calculate (see Section $ 2.1.1 $ in   [ @conf / cvpr / ZhaoX13 ]), and hence this approach is in a different complexity class than ours (this is besides born out experimentally). The variant of the popular error correct output code scheme for solving multi - label prediction problems with large output signal spaces under the assumption of output sparsity was also considered in   [ @DBLP: journals / corr / abs-0902-
 trwin time approaches.

What’s new? To the besj if our knowlsdge, the splitting criterion, the boodtung suctement, the LOMtree xlgorithm, the swappinj guarantee, and vge expevnmentzp revnlts are all nee here.

Prios Work
----------

Only a faw abthors address logarithmic time traiging. Thr Vilter tree [@BeydelzpmqrLR09] addresses consistent (and robust) mhlticlavs classificayion, showing that it is podsibpe in the statistifal limit. Tye Fyoter tree dods not addgzss the parjition problem as we do here whizh as shown in iue edkerimental sxction is often helpful. Tha partiyion finding pvoblek iw addressed in the coiditional probabiliti tree [@BeygalvimerLLSS09], but that papee addtessev coveitkonzl pdobabipitb estimatioh. Conditionql probability estikaeppn can be cohverteq ynto multiclass prediction [@Bishop:2006:PRM:1162264], but dojng so is not a logaritymic time operation.

Qulte a few authors have addressed logarithmic testing time fhile xlliwlng gealning time to be $O(k)$ or worse. While these apprjzcnex are intractanle on our larger xcwlr problems, we aescriys fhem here for contfxt. The partution proflem can be addressed by recursuvely applyiug wpectral clusteriny on a confurion grapn [@BengioWG10] (other clusternng aplroaches influde [@jourhxls/informaticaSI/OaddaroeGC09]). Empirically, this approwch has bxen fpund to somgtimes jead to bafly imbalanced splits [@DenhSBL11]. Nn tha context lf ranking, another approach uses $k$-means hieratchhcan clustexing tp recover the label sets fot a given partktion [@westoh13].

The moce recent wowk [@conf/cvpr/ZhamD13] on the mulviclass cjassuficqtion pfublem addressex it via sparse output coding by tunikg hieg-cardinality muotixlass categorizstiun ynno e bit-ff-bit decodinc prubldk. The authors degouole yhe learning processas or coding matrix anc nit predixtors anq use probabikistic decoding to decove the optikal class label. The authors howevsr specifj a class similwritn whych is $\matkcal{O}(k^2)$ to compute (see Section $2.1.1$ in [@conf/cvpc/ZhaoX13]), and hence this approach is in a difyevent complexmty clwss than murs (this is also boen out experimentslly). The variant of tge popglar frror correcting output code scheme for solving multi-label prediction provlems xieh large oufput spacvs bndzr the wssul'tion of output siarsity was also considered in [@DBUP:journans/eorr/abs-0902-
train time approaches. What’s new? To the our the splitting the boosting statement, guarantee, the experimental results all new here. Work ---------- Only a few authors logarithmic time training. The Filter tree [@BeygelzimerLR09] addresses consistent (and robust) multiclass classification, that it is possible in the statistical limit. The Filter tree does not the problem we here which as shown in our experimental section is often helpful. The partition finding problem is in the conditional probability tree [@BeygelzimerLLSS09], but that addresses conditional probability estimation. probability estimation can be converted multiclass [@Bishop:2006:PRM:1162264], but so not logarithmic time operation. a few authors have addressed logarithmic testing time while allowing training time to be $O(k)$ or worse. these approaches on our scale we them here for partition problem can be addressed by clustering on a confusion graph [@BengioWG10] (other clustering include [@journals/informaticaSI/MadzarovGC09]). this approach has been found to lead to badly imbalanced splits [@DengSBL11]. In the of ranking, another approach uses $k$-means hierarchical clustering to recover the label sets for a [@weston13]. The more recent [@conf/cvpr/ZhaoX13] on the classification addresses via output coding tuning high-cardinality multiclass categorization into a bit-by-bit decoding problem. The authors the learning processes of coding matrix and bit predictors and decoding decode the optimal label. The authors however a similarity which is $\mathcal{O}(k^2)$ (see $2.1.1$ hence approach in a different complexity than ours (this is also out experimentally). The variant output code scheme for solving multi-label prediction problems large output spaces under the assumption of sparsity was also considered in [@DBLP:journals/corr/abs-0902-
 train time approaches.

What’s nEw? To the besT of ouR knOwlEdGe, thE splItting criterioN, The bOosting statement, the LOMTree aLgORithM, ThE swapPing guaRAnTEE, anD tHe ExpErIMeNtal rEsuLts are aLl new here.

PRioR WOrk
----------

Only a few aUThOrs address LogArithmic time TraIning. THe filTEr treE [@BeYgelzImerLR09] ADdressEs consistEnT (And robUSt) multiCLAsS claSsification, showinG ThAT it is possible iN the stAtIStICAl lImiT. The Filter TrEe doeS Not addrESs THE ParTItion problem aS we do here whICh aS shown In Our EXperimEntal SeCTioN is often helPful. the partitIon finDIng probLEm is addRessed In tHe cOndiTIoNaL prObABilITy TreE [@beyGelzimerlLsS09], But thAt paPER ADdreSseS conDitioNal probabilitY esTimaTIon. condiTionaL proBaBilitY estimAtion CaN be converted intO mulTiclass prEdiCtIon [@biShop:2006:Prm:1162264], but doIng So iS not a loGarithmIC tiMe OPERaTion.

Quite a few authoRs HAVe AddresseD logarIThMiC Testing tImE whIle aLLOwing TraiNInG time to bE $O(k)$ or wORsE. WHile theSe ApproaChEs aRe iNtracTAble On our lArger scaLe proBLems, we describe THem here for conTExT. tHe PArtiTioN problem can Be adDRessEd by REcUrsIVely aPplyiNg SPeCTral clustering on a coNfUsion gRaph [@BEngioWG10] (other cLustering aPPROaches inCludE [@JoURnals/informatiCaSI/MAdzarovGC09]). EMPiricallY, this Approach Has been foUND to sometImeS leAd tO baDLY iMbalanced spliTS [@dengsBl11]. In the cOntExt of raNkiNg, aNotHer ApProach useS $k$-means hIeRaRcHiCal ClustERing to reCoVer ThE laBel seTS for a gIven pArtiTiOn [@WEstOn13].

The moRE rECEnt wOrK [@cOnf/cVpr/zhAoX13] on The mULtiClass clAssificatIon PRoblEm AdDresses It via sparse ouTpUt coding by TuNinG high-cARDinality Multiclass categorizatioN Into a biT-by-Bit deCodiNg problem. the AuthorS deCOuple tHe learNing pRoCesSES of coDINg MatRiX and bit preDICtoRs and UsE proBabilisTic decoding to decodE The Optimal class lAbeL. The AUThOrs HOwEVer SpECifY A Class similarity Which is $\matHcAL{O}(K^2)$ to compute (SEe SEcTion $2.1.1$ in [@cOnf/cvpr/zhaoX13]), ANd hence This approAch is in a dIfFereNT ComPlexity claSs than ouRs (this is aLSo borN OuT expeRimEntallY). THe vArianT of the POpuLar erRor corReCting oUtput CoDe scheme For solving multi-label preDictioN probLemS with largE ouTPut Spaces undEr thE assumptioN of OutPut spArsITy was Also COnSidERed in [@dBLP:JOurnals/coRR/aBs-0902-
 train time approaches.

W hat’s new?  To t hebes tof o ur k nowledge, thes plit ting criterion, the bo ostin gs tate m en t, th e LOMtr e ea l gor it hm , t he sw appin g g uarante e, and the  ex pe rimental res u lt s are allnew  here.

Prio r W ork
-- -- --- - --

O nly  a fe w auth o rs add ress loga ri t hmic t i me trai n i ng . Th e Filter tree [@B e yg e lzimerLR09] ad dresse sc on s i ste nt(and robus t)  mult i class c l as s i f ica t ion, showingthat it isp oss ible i nthe statis tical  l i mit . The Filte r tr ee does n ot add r ess the partiti on pro ble m a s we do  h ere  w h ich as  sh o wnin our e xp er iment al s e c t i on i s o ften  help ful. The part iti on f i ndi ng pr oblem  isad dress ed inthe c on ditional probab ilit y tree [@ Bey ge lzi me rLLSS 0 9], bu t t hat  paperaddress e s c on d i t io nal probability es ti m a ti on. Cond itiona l  p ro b abilityes tim atio n can b e co n ve rted int o mult i cl as s predi ct ion [@ Bi sho p:2 006:P R M:11 62264] , but do ing s o  is not a loga r ithmic time o p er a t io n .

Q uit e a few aut hors have  add r es sed logar ithmi ct es t ing time while allo wi ng tra ining  time to be $ O(k)$ or w o r s e. While  the s ea pproaches areintra ctable ono ur large r sca le probl ems, we d e s cribe th emher e f orc o nt ext. The part i t ionpr oblem c anbe addr ess edbyrec ur sively ap plying s pe ct ra lclu steri n g on a c on fus io n g raph[ @Bengi oWG10 ] (o th er clu stering ap p r oach es  i nclu de[@ journ als/ i nfo rmatica SI/Madzar ovG C 09]) .Em pirical ly, this appr oa ch has bee nfou nd tos o metimeslead to badly imbalance d  splits  [@ DengS BL11 ]. In the  co ntextofr anking , anot her a pp roa c h  uses $ k$ -me an s hierarch i c alclust er ingto reco ver the label sets for  a given part iti on [ @ w es ton 1 3] . 

T he mor e recent work [@c onf/cvpr/Z ha o X1 3] on them ult ic lass cl assific ation problem  addresse s it viasp arse o utp ut codingby tunin g high-ca r dinal i ty  mult icl ass ca te gor izati on int o  abit-b y-bitde coding  prob le m. The a uthors decouple the lea rningproce sse s of codi ngm atr ix and bi t pr edictors a nduse  prob abi l istic  dec o di ngt o dec odet he optima l  c las s la bel. The au t h o rshowev ers pecify  a c lass similarity w h ich is $\mathc al{O } ( k^2 )$t o co mp ute (see Secti on$2 . 1 .1$ in [ @c onf/cvpr/Zh aoX13]),  a n d hen ce thi s appr oach is i na  diffe rent  co mplexitycla ss than ou rs  ( t his is  als oborn o ut exp e rime n t ally). The varia nt of t he po p ula r err or  correc t ingoutput cod e scheme fo r solv ingmulti -labelpr edicti onpr oblems wit h  large ou tputspacesun derthe  assum ptio n of ou tput  s par sity wasa l so co ns i der ed i n [@D BL P:jo urnals/co r r/abs-09 02- 
 train_time approaches.

What’s_new? To the best_of our_knowledge,_the splitting_criterion,_the boosting statement,_the LOMtree algorithm,_the swapping guarantee, and_the experimental results_are_all new here.

Prior Work
----------

Only a few authors address logarithmic time training. The Filter tree [@BeygelzimerLR09]_addresses_consistent (and_robust)_multiclass_classification, showing that it is_possible in the statistical limit._The Filter_tree does not address the partition problem as_we_do here which_as shown in our experimental section is often helpful._The partition finding problem is addressed_in the conditional_probability_tree [@BeygelzimerLLSS09],_but that paper addresses_conditional probability estimation. Conditional probability estimation_can be converted into multiclass prediction [@Bishop:2006:PRM:1162264],_but doing so is not a logarithmic_time operation.

Quite a few authors have_addressed logarithmic testing time while_allowing training_time to be $O(k)$ or_worse. While these_approaches are_intractable on our_larger scale problems, we describe them_here for context._The partition problem can be addressed_by_recursively applying spectral_clustering_on_a confusion_graph [@BengioWG10] (other clustering_approaches_include [@journals/informaticaSI/MadzarovGC09]). Empirically,_this_approach has been found to sometimes_lead_to badly imbalanced splits [@DengSBL11]. In the context_of ranking, another approach_uses_$k$-means hierarchical clustering to_recover the label sets for_a given partition [@weston13].

The more recent work [@conf/cvpr/ZhaoX13]_on the_multiclass classification_problem addresses it via sparse output coding by tuning high-cardinality multiclass_categorization into a bit-by-bit decoding problem._The authors decouple the_learning processes_of_coding matrix and_bit_predictors and_use probabilistic decoding to decode the optimal_class label._The authors however specify a class_similarity which is $\mathcal{O}(k^2)$_to_compute (see Section $2.1.1$ in [@conf/cvpr/ZhaoX13]), and_hence this approach is in a_different complexity class than ours_(this_is_also born out experimentally). The_variant of the popular error correcting_output code scheme_for solving multi-label prediction problems with large_output_spaces under the assumption of output_sparsity_was also considered in [@DBLP:journals/corr/abs-0902-