 values of linear convolutional layers are computed by applying SVD to ${{{\uppercase{{\bm{t}}}}}}$ directly.

-   Circular Approximation: A block doubly circulant matrix ${{{\uppercase{{\bm{c}}}}}}$ is constructed according to -. The singular values are computed by applying SVD on ${{{\uppercase{{\bm{c}}}}}}$ directly.

-   Uniform Sampling: The block diagonal matrices ${{{\uppercase{{\bm{b}}}}}}_{j_1,j_2}$ is produced by uniformly sampling the spectral density matrix $F(\omega_1,\omega_2)$ with sampling grids $(\omega_1,\omega_2)=(-\pi+\frac{2\pi j_1}{n},-\pi+\frac{2\pi j_1}{n})$ for all $j_1, j_2 \in [n]$. The singular values are obtained by collecting all singular values of $\{{{{\uppercase{{\bm{b}}}}}}_{j_1,j_2}\}_{j_1,j_2=1}^n$. This corresponds to lines 1-10 in Algorithm \[alg:quantile\].

-   Quantile Interpolation: The singular values obtained from uniform sampling are arranged for each $1\le j \le \min\{c_{in},c_{out}\}$ in descending order. By quantile estimation using linear interpolation methods, the singular values are recomputed by selecting properly shifted sampling grids as outlined in Algorithm \[alg:quantile\].

The experiments are conducted on MATLAB 2020a, which is more friendly to matrix computation. For simplicity, we set $h_1=h_2$ and $w_1=w_2$, and the input size per channel is set to $10 \times 10$. Fig. \[fig:Fig-1\] presents the $(i-1)n+1$-th largest singular values ($i \in [n]$) of four methods with four different filter sizes. The first two filters are from the pre-trained GoogLeNet, and the last two are from the training process of ResNet-20. It can be observed that (1) both circular approximation and uniform sampling have identical singular values for different filter sizes, (2) quantile interpolation improves accuracy of the singular values over the circular approximation with negligible extra running time (see Section \[sec:experiments-sva\]), and (3) during the training process the improvement of the largest singular
values of linear convolutional layers are computed by apply SVD to $ { { { \uppercase{{\bm{t}}}}}}$ immediately. 

 -    Circular Approximation: A block doubly circulant matrix $ { { { \uppercase{{\bm{c}}}}}}$ is construct accord to -. The singular values are calculate by applying SVD on $ { { { \uppercase{{\bm{c}}}}}}$ directly. 

 -    Uniform Sampling: The block aslant matrix $ { { { \uppercase{{\bm{b}}}}}}_{j_1,j_2}$ is produced by uniformly sample the spectral density matrix $ F(\omega_1,\omega_2)$ with sampling grids $ (\omega_1,\omega_2)=(-\pi+\frac{2\pi j_1}{n},-\pi+\frac{2\pi j_1}{n})$ for all $ j_1, j_2 \in [ n]$. The remarkable values are obtained by collecting all singular values of $ \{{{{\uppercase{{\bm{b}}}}}}_{j_1,j_2}\}_{j_1,j_2=1}^n$. This match to lines 1 - 10 in Algorithm \[alg: quantile\ ]. 

 -    Quantile Interpolation: The singular values prevail from uniform sampling are arranged for each $ 1\le j \le \min\{c_{in},c_{out}\}$ in descending ordering. By quantile estimation using linear interpolation methods, the remarkable values are recomputed by selecting properly shifted sampling grids as outlined in Algorithm \[alg: quantile\ ]. 

 The experiments are conducted on MATLAB 2020a, which is more friendly to matrix computation. For simplicity, we set $ h_1 = h_2 $ and $ w_1 = w_2 $, and the input size per channel is set to $ 10 \times 10$. Fig.   \[fig: Fig-1\ ] presents the $ (i-1)n+1$-th largest remarkable values ($ i \in [ n]$) of four methods with four unlike filter size. The beginning two filters are from the pre - trained GoogLeNet, and the final two are from the training process of ResNet-20. It can be observed that (1) both circular estimate and uniform sampling have identical singular values for different filter sizes, (2) quantile interpolation improves accuracy of the singular values over the circular approximation with negligible excess running prison term (determine Section \[sec: experiments - sva\ ]), and (3) during the training process the improvement of the big singular
 vapues of linear convolutiunal layers are computxd by alplying RVD to ${{{\uppercase{{\bm{t}}}}}}$ directly.

-   Circylar Approximation: A buock doubpy circuoant natrix ${{{\uppxdcase{{\bm{g}}}}}}$ is dlnstxurted according jo -. The singglar values ara zolputed by applying SVD on ${{{\uppercase{{\fm{c}}}}}}$ dirrchly.

-   Uniform Fampkyng: Fhe block diagonal matrices ${{{\upperczse{{\bm{b}}}}}}_{j_1,o_2}$ is produced bu uniformly sampling the soectgal density matrix $F(\omega_1,\omegq_2)$ wiey sampling gfids $(\omega_1,\omega_2)=(-\pi+\frac{2\pj j_1}{n},-\pi+\frac{2\pi j_1}{n})$ for all $j_1, j_2 \in [v]$. The singular caoued are obtainxd by bollecting all singuldr valurs of $\{{{{\uppercasc{{\bm{b}}}}}}_{j_1,o_2}\}_{j_1,j_2=1}^n$. This corresponds to nines 1-10 in Algorithi \[alg:quandime\].

-   Quantile Intwrpolathon: Dhe rungjlad talhes obhaiied from unjform samplung are arranged fot qqch $1\le j \le \mjn\{c_{in},c_{jue}\}$ in descending order. By quantile estimdtikn using linear interpooation methods, the sijgular vajues are recomputed by selecting properly shifted sampmkng gvids qs outlined in Algorithm \[alg:quantile\].

The experimqhtx sre conducted jn MATLAB 2020a, wjivr is more frigndly to mztrix computation. Vor simklicitt, we set $r_1=h_2$ amd $w_1=w_2$, and the input size pee channel is wet to $10 \times 10$. Fig. \[yig:Fig-1\] preseuts thg $(i-1)n+1$-th largest singular valuer ($i \jn [n]$) of foug methods dith four differdnt fhlter sieds. The first two silters ace frpm the ore-ttained DoogLeNet, wnd tma last two are frol the tsaining prlcess of ResNet-20. It can be obsertxd that (1) both chrcllar apprjximabion and unifori sampling havg identiccl sineular valuvs for dihferent filtqr sizes, (2) quatjile interpoletion impwovew acxuracy uw the singular values onex the cirxular approximatiok witf negligible extxc tynning time (ser Sdctyoj \[xes:axperiments-sea\]), avd (3) curine the trainlng provess the improvement of fhe largest singulsr
values of linear convolutional layers are computed SVD ${{{\uppercase{{\bm{t}}}}}}$ directly. Circular Approximation: A is according to -. singular values are by applying SVD on ${{{\uppercase{{\bm{c}}}}}}$ directly. Uniform Sampling: The block diagonal matrices ${{{\uppercase{{\bm{b}}}}}}_{j_1,j_2}$ is produced by uniformly sampling the density matrix $F(\omega_1,\omega_2)$ with sampling grids $(\omega_1,\omega_2)=(-\pi+\frac{2\pi j_1}{n},-\pi+\frac{2\pi j_1}{n})$ for all $j_1, j_2 [n]$. singular are by collecting all singular values of $\{{{{\uppercase{{\bm{b}}}}}}_{j_1,j_2}\}_{j_1,j_2=1}^n$. This corresponds to lines 1-10 in Algorithm \[alg:quantile\]. - Interpolation: The singular values obtained from uniform sampling arranged for each $1\le \le \min\{c_{in},c_{out}\}$ in descending order. quantile using linear methods, singular are recomputed by properly shifted sampling grids as outlined in Algorithm \[alg:quantile\]. The experiments are conducted on MATLAB 2020a, which more friendly computation. For we $h_1=h_2$ $w_1=w_2$, and the per channel is set to $10 \[fig:Fig-1\] presents the $(i-1)n+1$-th largest singular values ($i [n]$) of methods with four different filter sizes. first two filters are from the pre-trained GoogLeNet, the last two are from the training process of ResNet-20. It can be observed that circular approximation and uniform have identical singular for filter (2) interpolation improves of the singular values over the circular approximation with negligible extra time (see Section \[sec:experiments-sva\]), and (3) during the training process of largest singular
 values of linear convolutionAl layers arE compUteD by ApPlyiNg SVd to ${{{\uppercase{{\bm{T}}}}}}$ DireCtly.

-   Circular ApproximatIon: A bLoCK douBLy CircuLant matRIx ${{{\UPPerCaSe{{\Bm{c}}}}}}$ Is COnStrucTed AccordiNg to -. The sinGulAr Values are comPUtEd by applyiNg SvD on ${{{\uppercasE{{\bm{C}}}}}}$ direcTlY.

-   UnIForm SAmpLing: THe blocK DiagonAl matriceS ${{{\uPPercasE{{\Bm{b}}}}}}_{j_1,j_2}$ is PROdUced By uniformly sampliNG tHE spectral densiTy matrIx $f(\OmEGA_1,\omEga_2)$ With sampliNg Grids $(\OMega_1,\omeGA_2)=(-\pI+\FRAc{2\pI J_1}{n},-\pi+\frac{2\pi j_1}{n})$ fOr all $j_1, j_2 \in [n]$. THE siNgular VaLueS Are obtAined By COllEcting all siNgulAr values oF $\{{{{\upperCAse{{\bm{b}}}}}}_{j_1,J_2}\}_{J_1,j_2=1}^n$. This CorresPonDs tO linES 1-10 iN ALgoRiTHm \[aLG:qUanTIle\].

-   quantile inTeRpolaTion: tHE SInguLar ValuEs obtAined from unifOrm SampLIng Are arRangeD for EaCh $1\le j \Le \min\{c_{In},c_{ouT}\}$ iN descending ordeR. By qUantile esTimAtIon UsIng liNEar intErpOlaTion metHods, the SIngUlAR VAlUes are recomputed by SeLECtIng propeRly shiFTeD sAMpling grIdS as OutlINEd in ALgorIThM \[alg:quanTile\].

ThE ExPeRiments ArE conduCtEd oN MAtLAB 2020a, WHich Is more Friendly To matRIx computation. FOR simplicity, we SEt $H_1=H_2$ AnD $W_1=w_2$, anD thE input size pEr chANnel Is seT To $10 \TimES 10$. Fig. \[fIg:Fig-1\] PrESeNTs the $(i-1)n+1$-th largest sinGuLar valUes ($i \iN [n]$) of four methoDs with four DIFFerent fiLter SIzES. The first two fiLters Are from the PRe-traineD GoogleNet, and The last twO ARe from thE trAinIng ProCESs Of ResNet-20. It can BE ObseRvEd that (1) bOth CirculaR apProXimAtiOn And uniforM samplinG hAvE iDeNtiCal siNGular valUeS foR dIffErent FIlter sIzes, (2) qUantIlE iNTerPolatioN ImPROves AcCuRacy Of tHe SinguLar vALueS over thE circular AppROximAtIoN with neGligible extra RuNning time (sEe secTion \[seC:EXperimenTs-sva\]), and (3) during the trainiNG procesS thE imprOvemEnt of the lArgEst sinGulAR
 values of linear convolut ional laye rs ar e c omp ut ed b y ap plying SVD to$ {{{\ uppercase{{\bm{t}}}}}} $ dir ec t ly.-  Cir cular A p pr o x ima ti on : A  b l oc k dou bly  circul ant matrix  ${ {{ \uppercase{{ \ bm {c}}}}}}$isconstructedacc ording  t o - .  Thesin gular  value s  are c omputed b ya pplyin g  SVD on $ {{ {\up percase{{\bm{c}}} } }} $  directly.

-  Unif or m  S a m pli ng:  The block  d iagon a l matri c es $ { {{\ u ppercase{{\bm {b}}}}}}_{j _ 1,j _2}$ i spro d uced b y uni fo r mly  sampling t he s pectral d ensity matrix$ F(\omeg a_1,\o meg a_2 )$ w i th  s amp li n g g r id s $ ( \om ega_1,\o me ga _2)=( -\pi + \ f r ac{2 \pi  j_1 }{n}, -\pi+\frac{2\ pij_1} { n}) $ for  all$j_1 ,j_2 \ in [n] $. Th esingular values  are  obtained  by  c oll ec tinga ll sin gul arvaluesof $\{{ { {\u pp e r c as e{{\bm{b}}}}}}_{j_ 1, j _ 2} \}_{j_1, j_2=1} ^ n$ .T his corr es pon ds t o lines  1-1 0  i n Algori thm \[ a lg :q uantile \] .

-  Q uan til e Int e rpol ation:  The sin gular values obtaine d  from uniform sa m p li n g ar e a rranged for  eac h  $1\ le j \l e \ m in\{c _{in} ,c _ {o u t}\}$ in descending  o rder.By qu antile estima tion using l i near int erpo l at i on methods, th e sin gular valu e s are re compu ted by s electingp r operly s hif ted  sa mpl i n ggrids as outl i n ed i nAlgorit hm\[alg:q uan til e\] .
Th e experim ents are  c on du ct edon MA T LAB 2020 a,  wh ic h i s mor e  frien dly t o ma tr ix com putatio n .F o r si mp li city , w eset $ h_1= h _2$  and $w _1=w_2$,and thein pu t sizeper channel i sset to $10  \ tim es 10$ . Fig. \[f ig:Fig-1\] presents the $(i-1)n +1$ -th l arge st singul arvalues  ($ i  \in [ n]$) o f fou rmet h o ds wi t h  f our  d ifferent f i l ter  size s.  The  firsttwo filters are fr o m t he pre-traine d G oogL e N et , a n dt hela s t t w o  are from the t raining pr oc e ss  of ResNet - 20.  I t can b e obser ved t h at (1)both circ ular appr ox imat i o n a nd uniform  samplin g have id e ntica l  s ingul arvalues  f ordiffe rent f i lte r siz es, (2 )quanti le in te rpolatio n improves accuracy ofthe si ngula r v alues ove r t h e c ircular a ppro ximation w ith  ne gligi ble extra  run n in g t i me (s ee S e ction \[s e c: exp e r im ents-sva\]) , a nd(3) d uri n g thetrai ning process thei mprovement ofthel a rge sts ingu la r
 values_of linear_convolutional layers are computed_by applying_SVD_to ${{{\uppercase{{\bm{t}}}}}}$_directly.

-_  Circular_Approximation: A block_doubly circulant matrix ${{{\uppercase{{\bm{c}}}}}}$_is constructed according_to_-. The singular values are computed by applying SVD on ${{{\uppercase{{\bm{c}}}}}}$ directly.

-  _Uniform_Sampling: The_block_diagonal_matrices ${{{\uppercase{{\bm{b}}}}}}_{j_1,j_2}$ is produced by_uniformly sampling the spectral density_matrix $F(\omega_1,\omega_2)$_with sampling grids $(\omega_1,\omega_2)=(-\pi+\frac{2\pi j_1}{n},-\pi+\frac{2\pi j_1}{n})$ for all_$j_1,_j_2 \in [n]$._The singular values are obtained by collecting all singular_values of $\{{{{\uppercase{{\bm{b}}}}}}_{j_1,j_2}\}_{j_1,j_2=1}^n$. This corresponds to_lines 1-10 in_Algorithm_\[alg:quantile\].

-_  Quantile Interpolation:_The singular values obtained from uniform_sampling are arranged for each $1\le_j \le \min\{c_{in},c_{out}\}$ in descending order. By_quantile estimation using linear interpolation methods,_the singular values are recomputed_by selecting_properly shifted sampling grids as_outlined in Algorithm_\[alg:quantile\].

The experiments_are conducted on_MATLAB 2020a, which is more friendly_to matrix computation._For simplicity, we set $h_1=h_2$ and_$w_1=w_2$,_and the input_size_per_channel is_set to $10_\times_10$. Fig. \[fig:Fig-1\]_presents_the $(i-1)n+1$-th largest singular values ($i_\in_[n]$) of four methods with four different_filter sizes. The first_two_filters are from the_pre-trained GoogLeNet, and the last_two are from the training process_of ResNet-20._It can_be observed that (1) both circular approximation and uniform sampling have_identical singular values for different filter_sizes, (2) quantile interpolation_improves accuracy_of_the singular values_over_the circular_approximation with negligible extra running time (see_Section \[sec:experiments-sva\]),_and (3) during the training process_the improvement of the_largest_singular