

### Marginal likelihood approximation influence

In this section the influence of the type of the marginal likelihood approximation on the anomaly detection results is studied.

The average results for both datasets (Figure \[fig:mean\_results\]) demonstrate that the type of the marginal likelihood approximation does not influence remarkably on anomaly detection performance. As the plug-in approximation requires less computational resources both in terms of time and memory (as there is no need to sample and store posterior samples and average among them) this type of approximation is recommended to be used for anomaly detection in the proposed framework.

### Learning algorithms comparison

This section compares the anomaly detection performance obtained by three learning algorithms.

The best results in terms of a mean area under a precision-recall curve are obtained by the EM learning algorithm, the worst results are obtained by the GS learning algorithm (Figure \[fig:mean\_results\] and Table \[tab:mean\_area\]). In Table \[tab:mean\_area\] for each learning algorithm the group of hyperparameters settings and the type of marginal likelihood approximation is chosen to have the maximum of the mean area under curves, where a mean is taken over independent runs of the same method and maximum is taken among different settings for the same learning algorithm.

  Dataset   EM       VB       GS
  --------- -------- -------- --------
  QMUL      0.3166   0.3155   0.2970
  Idiap     0.3759   0.3729   0.3673

  : Mean area under precision-recall curves[]{data-label="tab:mean_area"}

Figure \[fig:best\_worst\_curves\] presents the best and the worst precision-recall curves (in terms of the area under them) for the individual runs of the learning algorithms. The figure shows that among the individual runs the EM learning algorithm also demonstrates the most accurate results. Although, the minimum area under the precision-recall curve for the EM learning algorithm is less than the area under the corresponding curve for the VB algorithm. It means that the variance among the individual curves for the EM learning algorithm is larger in comparison with the VB learning algorithm.

The variance of the precision-recall curves for both VB and GS learning algorithms is relatively small. However, the VB learning algorithm has the curves higher than the curves obtained by the GS learning algorithm. It can be confirmed by examination of the best and worst precision-recall curves (Figure \[
# # # Marginal likelihood approximation influence 

 In this section the influence of the type of the bare likelihood estimate on the anomaly detection results is study. 

 The median results for both datasets (Figure \[fig: mean\_results\ ]) show that the type of the marginal likelihood estimate does not determine remarkably on anomaly signal detection performance. As the plug - in approximation requires less computational resource both in terms of time and memory (as there is no need to sample and store posterior sample and average among them) this type of approximation is recommend to be used for anomaly detection in the proposed model. 

 # # # Learning algorithms comparison 

 This department compares the anomaly detection performance receive by three learning algorithms. 

 The best results in terms of a beggarly sphere under a precision - recall curve are obtained by the EM learning algorithm, the worst results are obtained by the GS teach algorithm (Figure \[fig: mean\_results\ ] and Table \[tab: mean\_area\ ]). In Table \[tab: mean\_area\ ] for each learn algorithm the group of hyperparameters settings and the character of bare likelihood estimate is chosen to have the maximum of the mean area under curves, where a mean is taken over independent run of the same method and maximum is taken among different settings for the like learning algorithm. 

   Dataset    EM        VB        GS 
   --------- -------- -------- -------- 
   QMUL       0.3166    0.3155    0.2970 
   Idiap      0.3759    0.3729    0.3673 

  : Mean area under precision - recall curves[]{data - label="tab: mean_area " } 

 Figure \[fig: best\_worst\_curves\ ] presents the good and the worst precision - recall curves (in terms of the area under them) for the individual run of the learning algorithms. The figure usher that among the individual runs the EM learning algorithm also demonstrate the most accurate solution. Although, the minimum area under the preciseness - recall curve for the EM learning algorithm is less than the area under the corresponding curvature for the VB algorithm. It means that the discrepancy among the individual curves for the EM learning algorithm is bombastic in comparison with the VB learning algorithm. 

 The discrepancy of the precision - recall curve for both VB and GS learning algorithms is relatively small. However, the VB learning algorithm has the curve higher than the curves obtain by the GS learning algorithm. It can be confirmed by examination of the best and worst precision - recall curve (Figure \ [


### Magginal likelihood approxlmation influencg

Ib this sectikn the ivfluence of the type of the laeginao likelihood approximagion on tje anomaoy dttection results ma studicb.

The zyeragz cesults for botm datasets (Xigure \[fig:mean\_serupts\]) demonstrate that the type of thq margimap likelihood akproxpmwtioh does not influence remarkably on anomalj detection perfotmance. As the plug-in approdimahion requires less computatiobal wwsources botf in terms of time and memory (as there is no need to ramplz and store pistftior samples and wverage amonn them) dhis tyle of approximstimn us recommended to be nsed for anomaly detgction in dhz proposed framework.

### Oeqrninc alcorigyms cojperiaon

Thid sxction compzres the animaly detection pergownance obtaines by trrqe learning algorithms.

The best results pn tsrms of a mean area undwr a precision-recall furve are obtained by the EM learning algorithm, the worst sesulvs art ibtaivwd by the GS learning algorithm (Figure \[fig:mean\_rqauktx\] and Table \[tan:mean\_area\]). In Table \[twb:kgan\_area\] for eazh leaxhihg algorithm the ggoup of hypeeparametews srttings and the type of martinal likelikooe approximation is chosen to kave tne macimum of the mean area bnder durves, wherf a mean jr taken over inddpekdett runs of the same method and maxinum ns taken amomg difserent sethings for the same learnlng apgmrithm.

  Dahaset   EM       VB       GS
  --------- -------- -------- --------
  QMUL      0.3166   0.3155   0.2970
  Idiap     0.3759   0.3729   0.3673

  : Mean area under prgcision-reeall cjrves[]{data-lzbel="tab:kean_area"}

Figtre \[fig:best\_wosdt\_curves\] prevents thq bewt abd the durst precision-tecall cugvts (in termw of the area undev theo) for the indivibbao runs of the lrarvind wljoritrks. The figura shuws yhat xmong the ikdixidusl runs the EM learnhng zlgorithm also dempnftrates jhe most wccurate resukts. Although, the mlnimuk acea uncer the precision-recall curve for the EM lfarking algorithi is less than tke area under the corresponding curve foc the VB algorithm. It mwans that the variauct among the iidividtal curvev for the EM learnint algorithm is lavger in comparison witg the EB lewrning algorithm.

The variance of the precision-recall curves for both VB ane GS lxawning algorjthmx is gelctirely smwll. Ioxever, the VB learming algorithm has the curves hmgher than tke curves obtained by the GS kexrning algorijhm. It can be confirmed by examinayion of the best and worst precosion-recall curves (Figurc \[
### Marginal likelihood approximation influence In this influence the type the marginal likelihood results studied. The average for both datasets \[fig:mean\_results\]) demonstrate that the type of marginal likelihood approximation does not influence remarkably on anomaly detection performance. As the approximation requires less computational resources both in terms of time and memory (as is need sample store posterior samples and average among them) this type of approximation is recommended to be used anomaly detection in the proposed framework. ### Learning comparison This section compares anomaly detection performance obtained by learning The best in of mean area under precision-recall curve are obtained by the EM learning algorithm, the worst results are obtained by the GS algorithm (Figure Table \[tab:mean\_area\]). Table for learning algorithm the hyperparameters settings and the type of is chosen to have the maximum of the area under where a mean is taken over runs of the same method and maximum is among different settings for the same learning algorithm. Dataset EM VB GS --------- -------- -------- 0.3166 0.3155 0.2970 Idiap 0.3729 0.3673 : area precision-recall Figure presents the and the worst precision-recall curves (in terms of the area under for the individual runs of the learning algorithms. The figure among individual runs the learning algorithm also demonstrates most results. Although, the minimum the curve learning is than the area under corresponding curve for the VB It means that the for the EM learning algorithm is larger in with the VB learning algorithm. The variance the precision-recall curves for both VB and GS learning algorithms is relatively However, the algorithm has the curves higher than the curves by the GS learning It can be confirmed by examination of the best worst curves (Figure


### Marginal likelihood approxiMation inflUence

in tHis SeCtioN the Influence of the TYpe oF the marginal likelihood ApproXiMAtioN On The anOmaly deTEcTIOn rEsUlTs iS sTUdIed.

ThE avErage reSults for boTh dAtAsets (Figure \[fIG:mEan\_results\]) DemOnstrate that The Type of ThE maRGinal LikElihoOd apprOXimatiOn does not InFLuence REmarkabLY On AnomAly detection perfoRMaNCe. As the plug-in aPproxiMaTIoN REquIreS less compuTaTionaL ResourcES bOTH In tERms of time and mEmory (as therE Is nO need tO sAmpLE and stOre poStERioR samples and AverAge among tHem) thiS Type of aPProximaTion is RecOmmEndeD To Be UseD fOR anOMaLy dETecTion in thE pRoPosed FramEWORK.

### LeaRniNg alGoritHms comparison

thiS secTIon CompaRes thE anoMaLy detEction PerfoRmAnce obtained by tHree Learning aLgoRiThmS.

THe besT ResultS in TerMs of a meAn area uNDer A pRECIsIon-recall curve are oBtAINeD by the EM LearniNG aLgORithm, the WoRst ResuLTS are oBtaiNEd By the GS lEarninG AlGoRithm (FiGuRe \[fig:mEaN\_reSulTs\] and tAble \[Tab:meaN\_area\]). In TAble \[tAB:mean\_area\] for eaCH learning algoRItHM ThE GrouP of HyperparameTers SEttiNgs aND tHe tYPe of mArginAl LIkELihood approximation Is Chosen To havE the maximum of The mean areA UNDer curveS, wheRE a MEan is taken over IndepEndent runs OF the same MethoD and maxiMum is takeN AMong diffEreNt sEttIngS FOr The same learniNG AlgoRiThm.

  DataSet   eM       VB       GS
  --------- -------- -------- --------
  QmUL      0.3166   0.3155   0.2970
  idiAp     0.3759   0.3729   0.3673

  : MEan ArEa under prEcision-rEcAlL cUrVes[]{Data-lABel="tab:meAn_AreA"}

FIguRe \[fig:BEst\_worSt\_curVes\] pReSeNTs tHe best aND tHE WorsT pReCisiOn-rEcAll cuRves (IN teRms of thE area undeR thEM) for ThE iNdividuAl runs of the leArNing algoriThMs. THe figuRE Shows thaT among the individual runs THe EM leaRniNg algOritHm also demOnsTrates The MOst accUrate rEsultS. ALthOUGh, the MINiMum ArEa under the PREciSion-rEcAll cUrve for The EM learning algorIThm Is less than the AreA undER ThE coRReSPonDiNG cuRVE for the VB algoriThm. It means ThAT tHe variance AMonG tHe indivIdual cuRves fOR the EM lEarning alGorithm is LaRger IN ComParison witH the VB leArning algORithm.

tHe VariaNce Of the pReCisIon-reCall cuRVes For boTh VB anD Gs learnIng alGoRithms is Relatively small. However, tHe VB leArninG alGorithm haS thE CurVes higher Than The curves oBtaIneD by thE GS LEarniNg alGOrIthM. it can Be coNFirmed by eXAmInaTIOn Of the best anD WORst PreciSioN-Recall CurvEs (Figure \[


### Marginal likelihoodapproximat ion i nfl uen ce 

In  thi s section thei nflu ence of the type of th e mar gi n al l i ke lihoo d appro x im a t ion  o nthe  a n om aly d ete ction r esults isstu di ed.

The ave r ag e resultsfor  both datase ts(Figur e\[f i g:mea n\_ resul ts\])d emonst rate that  t h e type of them a rg inal  likelihood appro x im a tion does notinflue nc e  r e m ark abl y on anoma ly  dete c tion pe r fo r m a nce .  As the plug- in approxim a tio n requ ir esl ess co mputa ti o nal  resourcesboth  in terms  of ti m e and m e mory (a s ther e i s n o ne e dto  sa mp l e a n dsto r e p osterior  s am plesanda v e r ageamo ng t hem)this type ofapp roxi m ati on is  reco mmen de d tobe use d for  a nomaly detectio n in  the prop ose dfra me work. 
### L ear nin g algor ithms c o mpa ri s o n 
This section compa re s th e anomal y dete c ti on performa nc e o btai n e d bythre e  l earningalgori t hm s. 

The b es t resu lt s i n t ermso f amean a rea unde r a p r ecision-recall curve are obt a in e d  b y  the  EM  learning a lgor i thm,  the wo rst resul ts ar eo bt a ined by the GS lear ni ng alg orith m (Figure \[f ig:mean\_r e s u lts\] an d Ta b le \[tab:mean\_ar ea\]) . In Table \[tab:me an\_a rea\] fo r each le a r ning alg ori thm  th e g r o up  of hyperpara m e ters  s ettings  an d the t ype  of  ma rgi na l likelih ood appr ox im at io n i s cho s en to ha ve  th emax imumo f themeanarea  u nd e r c urves,w he r e  a m ea nis t ake noverinde p end ent run s of thesam e  met ho dand max imum is taken  a mong diffe re ntsettin g s  for the  same learning algorith m .

  Da tas et  EM      VB     GS
 -- - ------  ----- --- - -- --- - -  ---- - - -- 
 QM UL      0. 3 1 66  0.3 15 5  0.2970  Idiap     0.3759   0 .3729   0.367 3
  :M e an  ar e au nde rp rec i s ion-recall curv es[]{data- la b el ="tab:mean _ are a" }

Figu re \[fi g:bes t \_worst \_curves\ ] present stheb e stand the wo rst prec ision-rec a ll cu r ve s (in  te rms of  t heareaundert hem ) for  the i nd ividua l run sof the l earning algorithms. The  figur e sho wsthat amon g t h e i ndividual  run s the EM l ear nin g alg ori t hm al so d e mo nst r atesthem ost accur a te  re s u lt s. Although , t heminim uma rea un derthe precision-rec a ll curve for t he E M lea rni n g al go rithm is lesstha nt h e area u nd er the corr espondin gc urvefor th e VB a lgorith m .  I t  means  tha t t he varian ceam o ng thein di v idualcurv es  for t he EMl earn i n g algorithm is l arger i n com p ari son w it h the V B  lea rning algo rithm.

The  varia nceof th e preci si on-rec all  c urves forb oth VB an d GSlearnin galgo rit hms is  rel a t ively  sma ll . H owever, t h e  V B  l ea r nin g al gorit hm  has  the curv e s higher  th a n the c ur ves o btaine d  b y the GS lea rni ng al g o rithm. Itc an b e  c o nfirm ed byexamin ation o f  th ebest an d w o r st precis ion-recal l  cu rv es ( Figure \ [


### Marginal_likelihood approximation_influence

In this section the_influence of_the_type of_the_marginal likelihood approximation_on the anomaly_detection results is studied.

The_average results for_both_datasets (Figure \[fig:mean\_results\]) demonstrate that the type of the marginal likelihood approximation does not_influence_remarkably on_anomaly_detection_performance. As the plug-in approximation_requires less computational resources both_in terms_of time and memory (as there is no_need_to sample and_store posterior samples and average among them) this type_of approximation is recommended to be_used for anomaly_detection_in_the proposed framework.

### Learning_algorithms comparison

This section compares the anomaly_detection performance obtained by three learning_algorithms.

The best results in terms of a_mean area under a precision-recall curve_are obtained by the EM_learning algorithm,_the worst results are obtained_by the GS_learning algorithm_(Figure \[fig:mean\_results\] and_Table \[tab:mean\_area\]). In Table \[tab:mean\_area\] for_each learning algorithm_the group of hyperparameters settings and_the_type of marginal_likelihood_approximation_is chosen_to have the_maximum_of the_mean_area under curves, where a mean_is_taken over independent runs of the same_method and maximum is_taken_among different settings for_the same learning algorithm.

 _Dataset   EM  _ _ _VB       GS
  --------- --------_-------- --------
  QMUL  _   0.3166_ _0.3155_  0.2970
__Idiap _   0.3759   0.3729_ _0.3673

  : Mean area under_precision-recall curves[]{data-label="tab:mean_area"}

Figure \[fig:best\_worst\_curves\] presents_the_best and the worst precision-recall curves_(in terms of the area under_them) for the individual runs_of_the_learning algorithms. The figure shows_that among the individual runs the_EM learning algorithm_also demonstrates the most accurate results. Although,_the_minimum area under the precision-recall curve_for_the EM learning algorithm is less_than_the_area under the corresponding curve_for the VB algorithm. It means_that the variance among the individual curves for the_EM learning algorithm_is larger in comparison with_the_VB_learning algorithm.

The variance of the precision-recall curves for both VB_and GS_learning algorithms is_relatively small. However, the VB learning algorithm has the curves_higher than the curves obtained by the_GS learning algorithm. It can be confirmed by examination of the_best and worst precision-recall curves (Figure \[