-engine’ bias as in VGG-Face [@parkhi2015deep] and MS-Celeb-1M [@guo2016ms]. However, we found this cluster-based approach introduces new bias. MegaFace prefers small groups with highly duplicated images,, face captured from the same video. Limited by the base model for clustering, considerable groups in MegaFace contain noises, or sometimes mess up multiple people in the same group.

![(a) A visualization of size and estimated noise percentage of datasets. (b) Noise distribution of MS-Celeb-1M(v1) [@guo2016ms]. (c) Noise distribution of MegaFace [@nech2017level]. The two horizontal lines in each bar represent the lower- and upper-bounds of noise, respectively. See Sec. \[subsec:noise\_ratio\] for details.[]{data-label="fig:noiseDistribution"}](noiseDistribution_eccv.pdf){width="0.7\linewidth"}

An Approximation of Signal-to-Noise Ratio {#subsec:noise_ratio}
-----------------------------------------

Owing to the source of data and cleaning strategies, existing large-scale datasets invariably contain label noises. In this study, we aim to profile the noise distribution in existing datasets. Our analysis may provide a hint to future research on how one should exploit the distribution of these data.

It is infeasible to obtain the exact number of these noises due to the scale of the datasets. We bypass this difficulty by randomly selecting a subset of a dataset and manually categorize them into three groups – ‘correct identity assigned’, ‘doubtful’, and ‘wrong identity assigned’. We select a subset of 2.7M images from MegaFace [@nech2017level] and 3.7M images from MS-Celeb-1M [@guo2016ms]. For CASIA-WebFace [@yi2014learning] and CelebFaces [@sun2014deep; @celebface], we sampled 30 identities to estimate their signal-to-noise ratio. The final statistics are visualized in Figure \[fig:noiseDistribution\](a). Due to the difficulty in estimating the exact ratio, we approximate an upper and a lower bound of noisy data during the estimation. The lower-bound is more optimistic considering doubtful labels as clean data. The upper-bound is more pessimistic considering all doubtful cases as badly labeled. We provide more details on the estim
-engine ’ bias as in VGG - Face   [ @parkhi2015deep ] and MS - Celeb-1 M   [ @guo2016ms ]. However, we found this cluster - based overture introduce new diagonal. MegaFace prefers little groups with highly double double, , expression captured from the same television. Limited by the base mannequin for clustering, considerable group in MegaFace contain noises, or sometimes mess up multiple people in the same group. 

! [ (a) A visualization of size and estimate noise percentage of datasets. (b) Noise distribution of MS - Celeb-1M(v1)   [ @guo2016ms ]. (c) Noise distribution of MegaFace   [ @nech2017level ]. The two horizontal lines in each legal profession represent the lower- and upper - bounds of noise, respectively. See Sec.   \[subsec: noise\_ratio\ ] for details.[]{data - label="fig: noiseDistribution"}](noiseDistribution_eccv.pdf){width="0.7\linewidth " } 

 An Approximation of Signal - to - Noise proportion { # subsec: noise_ratio } 
 ----------------------------------------- 

 Owing to the reservoir of data and cleaning strategies, existing big - scale datasets invariably contain label noises. In this study, we aim to profile the noise distribution in existing datasets. Our analysis may provide a hint to future research on how one should exploit the distribution of these data. 

 It is infeasible to obtain the exact act of these noises due to the scale of the datasets. We bypass this trouble by randomly choose a subset of a dataset and manually categorize them into three groups – ‘ correct identity assigned ’, ‘ doubtful ’, and ‘ wrong identity assign ’. We select a subset of 2.7 M images from MegaFace   [ @nech2017level ] and 3.7 M image from MS - Celeb-1 M   [ @guo2016ms ]. For CASIA - WebFace   [ @yi2014learning ] and CelebFaces   [ @sun2014deep; @celebface ], we sampled 30 identities to estimate their signal - to - noise ratio. The final statistic are visualized in Figure   \[fig: noiseDistribution\](a). Due to the difficulty in estimating the exact ratio, we approximate an upper and a lower bound of noisy datum during the estimate. The lower - bound is more optimistic considering doubtful labels as uninfected data. The upper - bound is more pessimistic considering all doubtful case as ill labeled. We provide more detail on the estim
-englne’ bias as in VGG-Face [@pavkhi2015deep] and MS-Cglwb-1M [@guo2016ks]. Howsver, we wound this cluster-based apprlaxh inugoduces new bias. MegaWace prefvrs small gronps with highly vhplicatcb imafcs,, faee captured from the same eideo. Limited ty tke base model for clustering, considewable gtokps in MegaFacg conuaig nojses, or sometimes mess up multiple people in the same broup.

![(a) A visualization of dize and estimated noide percentate os datasets. (b) Voise distgnbution of JS-Celeb-1M(v1) [@guo2016ms]. (c) Noise distribugion pf MegaFacg [@uwch2017pgvel]. The two horidontal lines lm each bar relresent the loeer- ane upper-bounds of noisx, respectively. See Sgc. \[subsec:nohsz\_ratio\] for details.[]{datq-lqbel="fhg:nohseDkwtrkbuuioi"}](nojseDishriuution_eccv.psf){width="0.7\lineqidth"}

An Approximatipn if Signal-to-Nojse Raeij {#subsec:noise_ratio}
-----------------------------------------

Owing to the source ox dzta and cleaning strateties, existing large-scwle datasqts invariably contain label noises. In this study, we amm to pvofiuw hhe noise distribution in existing datasets. Otd snslysis may proyide a hint to futirf tgsearch on how one skkumd exploit the dishributijn of these daua.

It os infeasible to obtain the exact numbeg of these noises due co the scale of jhe dayasets. We bypass this dnfficumty by randlmly seledging a subset of a cadaset and manually categoryze them mnto chree gruups – ‘corrqct identihy assigned’, ‘doubtful’, and ‘wrony idettity assihned’. We select a subset of 2.7M imejes from MegaGawe [@nvch2017level] cnd 3.7M lmages from MS-Cqleb-1M [@guo2016ms]. For CASIA-WzbFace [@hi2014learning] and CenebFaces [@sun2014qeep; @celebfaca], we sampled 30 identityes ro ewtimate gheir signal-to-moise ratpo. The final statistics ere xjsualized in Fiybrt \[fug:noiseDistribitiun\](a). Dle vo thq difficulty hn ertiosting the exact vatko, wr approximate an uppar ahd a lower bound og koisy datq during the estimatipn. The lower-bound ls moce optmmistiv cjnsidering doubtful labels as dlean datw. Tme upper-bound is nore pessimixtic considering all doubtful cases as uadly labeled. We providw more details on tke estim
-engine’ bias as in VGG-Face [@parkhi2015deep] and However, found this approach introduces new with duplicated images,, face from the same Limited by the base model for considerable groups in MegaFace contain noises, or sometimes mess up multiple people in same group. ![(a) A visualization of size and estimated noise percentage of datasets. Noise of [@guo2016ms]. Noise distribution of MegaFace [@nech2017level]. The two horizontal lines in each bar represent the lower- and of noise, respectively. See Sec. \[subsec:noise\_ratio\] for details.[]{data-label="fig:noiseDistribution"}](noiseDistribution_eccv.pdf){width="0.7\linewidth"} Approximation of Signal-to-Noise Ratio ----------------------------------------- Owing to the source data cleaning strategies, large-scale invariably label noises. In study, we aim to profile the noise distribution in existing datasets. Our analysis may provide a hint future research one should the of data. It is obtain the exact number of these the scale of the datasets. We bypass this by randomly a subset of a dataset and categorize them into three groups – ‘correct identity ‘doubtful’, and ‘wrong identity assigned’. We select a subset of 2.7M images from MegaFace [@nech2017level] images from MS-Celeb-1M [@guo2016ms]. CASIA-WebFace [@yi2014learning] and [@sun2014deep; we 30 to estimate signal-to-noise ratio. The final statistics are visualized in Figure \[fig:noiseDistribution\](a). Due the difficulty in estimating the exact ratio, we approximate an a bound of noisy during the estimation. The is optimistic considering doubtful labels data. upper-bound considering doubtful as badly labeled. We more details on the estim
-engine’ bias as in VGG-Face [@parkHi2015deep] and Ms-CeleB-1M [@gUo2016mS]. HOwevEr, we Found this clustER-basEd approach introduces neW bias. meGAFacE PrEfers Small grOUpS WIth HiGhLy dUpLIcAted iMagEs,, face cAptured froM thE sAme video. LimiTEd By the base mOdeL for clusteriNg, cOnsideRaBle GRoups In MEgaFaCe contAIn noisEs, or sometImES mess uP MultiplE PEoPle iN the same group.

![(a) A viSUaLIzation of size aNd estiMaTEd NOIse PerCentage of dAtAsets. (B) noise diSTrIBUTioN Of MS-Celeb-1M(v1) [@guO2016ms]. (c) Noise diSTriBution Of megAface [@neCh2017levEl]. tHe tWo horizontaL linEs in each bAr reprESent the LOwer- and Upper-bOunDs oF noiSE, rEsPecTiVEly. sEe sec. \[SUbsEc:noise\_rAtIo\] For deTailS.[]{DATA-labEl="fIg:noIseDiStribution"}](noiSeDIstrIButIon_ecCv.pdf){WidtH="0.7\lInewiDth"}

An APproxImAtion of Signal-to-noisE Ratio {#subSec:NoIse_RaTio}
-----------------------------------------

OwINg to thE soUrcE of data And cleaNIng StRATEgIes, existing large-scAlE DAtAsets invAriablY CoNtAIn label nOiSes. in thIS Study, We aiM To Profile tHe noisE DiStRibutioN iN existInG daTasEts. OuR AnalYsis maY provide A hint TO future researcH On how one shoulD ExPLOiT The dIstRibution of tHese DAta.

IT is iNFeAsiBLe to oBtain ThE ExACt number of these noisEs Due to tHe scaLe of the dataseTs. We bypass THIS difficuLty bY RaNDomly selecting A subsEt of a datasET and manuAlly cAtegorizE them into THRee groupS – ‘coRreCt iDenTITy Assigned’, ‘doubtFUL’, and ‘WrOng idenTitY assignEd’. WE seLecT a sUbSet of 2.7M imaGes from MEgAFAcE [@nEch2017Level] ANd 3.7M imageS fRom mS-celEb-1M [@guO2016Ms]. For CaSIA-WEbFaCe [@Yi2014LEarNing] and cElEBfaceS [@sUn2014Deep; @CelEbFace], wE samPLed 30 IdentitIes to estiMatE TheiR sIgNal-to-noIse ratio. The fiNaL statisticS aRe vIsualiZED in FigurE \[fig:noiseDistribution\](a). DUE to the dIffIcultY in eStimating The Exact rAtiO, We apprOximatE an upPeR anD A Lower BOUnD of NoIsy data durING thE estiMaTion. the loweR-bound is more optimiSTic Considering doUbtFul lABElS as CLeAN daTa. tHe uPPEr-bound is more peSsimistic cOnSIdEring all doUBtfUl Cases as Badly laBeled. wE providE more detaIls on the eStIm
-engine’ bias as in VGG-Fa ce [@parkh i2015 dee p]an d MS -Cel eb-1M [@guo201 6 ms].  However, we found thi s clu st e r-ba s ed  appr oach in t ro d u ces  n ew  bi as .  M egaFa ceprefers  small gro ups  w ith highly d u pl icated ima ges ,, face capt ure d from  t hes ame v ide o. Li mitedb y thebase mode lf or clu s tering, c on side rable groups in M e ga F ace contain no ises,or so m e tim esmess up mu lt iplep eople i n  t h e sam e  group.

![(a ) A visuali z ati on ofsi zea nd est imate dn ois e percentag e of  datasets . (b)N oise di s tributi on ofMS- Cel eb-1 M (v 1)  [@ gu o 201 6 ms ].( c)Noise di st ri butio n of M e g aFac e [ @nec h2017 level]. The t wohori z ont al li nes i n ea ch  barrepres ent t he  lower- and upp er-b ounds ofnoi se , r es pecti v ely. S eeSec . \[sub sec:noi s e\_ ra t i o \]  for details.[]{da ta - l ab el="fig: noiseD i st ri b ution"}] (n ois eDis t r ibuti on_e c cv .pdf){wi dth="0 . 7\ li newidth "} 

An A pp rox ima tiono f Si gnal-t o-NoiseRatio {#subsec:noise _ ratio}
------ - -- - - -- - ---- --- ----------- ---- - ---- 

Ow i ng  to the s ource  o f  d a ta and cleaning str at egies,  exis ting large-sc ale datase t s invariab ly c o nt a in label noise s. In  this stud y , we aim  to p rofile t he noised i stributi oninexi sti n g  d atasets. Oura n alys is  may pr ovi de a hi nttofut ure  r esearch o n how on esh ou ld  ex ploit the dist ri but io n o f the s e data .

It  isin fe a sib le to o b ta i n  the  e xa ct n umb er  of t hese noi ses due  to the s cal e  ofth edataset s. We bypassth is difficu lt y b y rand o m ly selec ting a subset of a data s et andman ually  cat egorize t hem  intothr e e grou ps – ‘ corre ct  id e n titya s si gne d’ , ‘doubtfu l ’ , a nd ‘w ro ng i dentity  assigned’. We sel e cta subset of 2 .7M  ima g e sfro m  M e gaF ac e  [@ n e ch2017level] an d 3.7M ima ge s  f rom MS-Cel e b-1 M[@guo20 16ms].For C A SIA-Web Face [@yi 2014learn in g] a n d  Ce lebFaces [ @sun2014 deep; @ce l ebfac e ],  we s amp led 30  i den titie s to e s tim ate t heir s ig nal-to -nois eratio. T he final statistics are  visua lized  in  Figure \ [fi g :no iseDistri buti on\](a). D uetothe d iff i culty  ine st ima t ing t he e x act ratio ,  w e a p p ro ximate an u p p e r a nd alow e r boun d of  noisy data durin g  the estimatio n. T h e  lo wer - boun dis more optimi sti cc o nsiderin gdoubtful la bels ascl e an da ta. Th e uppe r-bound i sm ore pe ssim ist ic consid eri ng all dou bt fu l  cases  asba dly la beled. We p r o vide more detail s ont h e est i m
-engine’ bias_as in_VGG-Face [@parkhi2015deep] and MS-Celeb-1M [@guo2016ms]. However,_we found_this_cluster-based approach_introduces_new bias. MegaFace_prefers small groups_with highly duplicated images,,_face captured from_the_same video. Limited by the base model for clustering, considerable groups in MegaFace contain_noises,_or sometimes_mess_up_multiple people in the same_group.

![(a) A visualization of size_and estimated_noise percentage of datasets. (b) Noise distribution of_MS-Celeb-1M(v1) [@guo2016ms]._(c) Noise distribution_of MegaFace [@nech2017level]. The two horizontal lines in each bar_represent the lower- and upper-bounds of_noise, respectively. See_Sec. \[subsec:noise\_ratio\]_for_details.[]{data-label="fig:noiseDistribution"}](noiseDistribution_eccv.pdf){width="0.7\linewidth"}

An Approximation of Signal-to-Noise_Ratio {#subsec:noise_ratio}
-----------------------------------------

Owing to the source of_data and cleaning strategies, existing large-scale_datasets invariably contain label noises. In this_study, we aim to profile the_noise distribution in existing datasets._Our analysis_may provide a hint to_future research on_how one_should exploit the_distribution of these data.

It is infeasible_to obtain the_exact number of these noises due_to_the scale of_the_datasets._We bypass_this difficulty by_randomly_selecting a_subset_of a dataset and manually categorize_them_into three groups – ‘correct identity assigned’,_‘doubtful’, and ‘wrong identity_assigned’._We select a subset_of 2.7M images from MegaFace [@nech2017level]_and 3.7M images from MS-Celeb-1M [@guo2016ms]. For_CASIA-WebFace [@yi2014learning] and_CelebFaces [@sun2014deep; @celebface],_we sampled 30 identities to estimate their signal-to-noise ratio. The final_statistics are visualized in Figure \[fig:noiseDistribution\](a). Due_to the difficulty in_estimating the_exact_ratio, we approximate_an_upper and_a lower bound of noisy data during_the estimation._The lower-bound is more optimistic considering_doubtful labels as clean_data._The upper-bound is more pessimistic considering_all doubtful cases as badly labeled._We provide more details on_the_estim