2$ and $R=r^2$. The $L$ encoded subtasks are partitioned into $N$ arrays of size $R$, and each worker is assigned a distinct array of $R$ subtasks. Each worker sequentially works through its $R$ subtasks. At the completion of each subtask, it transmits the result to the master node.

Order of processing
-------------------

Through an example, we illustrate that optimizing the order in which the processors complete sub-tasks can provide a further reduction in computation time. We consider a simple distributed setup in which the master node needs to multiply two matrices $\mathbf A=[\mathbf A_1,\mathbf A_2]$ and $\mathbf B =[\mathbf B_1, \mathbf B_2]$, where $\mathbf A_i$ and $\mathbf B_j$ are $m \times q$ matrices and $k=2$, and there are $n^2 = 9$ worker nodes. We first divide each $\mathbf A_i$ and $\mathbf B_j$ into $r=4$ sub-matrices $\mathbf A_i = [\mathbf A_{i1}, \ldots,\mathbf A_{i4}]$ and $\mathbf B_j = [\mathbf B_{j1}, \dots, \mathbf B_{j4}]$ respectively. We then form an $8 \times 8$ array as described in \[product code\]. Then by encoding rows and columns of this array using $(12,8)$ MDS codes, $(n \times r)^2 = 144$ subtasks are generated. These subtasks are arranged in a $12 \times 12$ array, as shown in Fig. \[fig:MATMUL2\]. We assign $r^2=16$ subtasks to each processor, as depicted by the $4 \times 4$ blocks in Fig. \[fig:MATMUL2\]. For illustrative purpose we assume that the four white processors are stragglers and complete their tasks in 4 sec. We assume that the other five gray processors are non-stragglers and finish their tasks after 1 sec. In one approach all processors perform their tasks according to a diagonal schedule illustrated in Fig. \[fig:MATMUL2\]a. In the second, they follow the column-wise scheduling depicted in Fig. \[fig:MATMUL2\]b. If the diagonal schedule is used, the master node can complete the matrix-matrix multiplication $\
2 $ and $ R = r^2$. The $ L$ encoded subtasks are partitioned into $ N$ arrays of size $ R$, and each proletarian is impute a distinct array of $ R$ subtasks. Each worker sequentially work through its $ R$ subtasks. At the completion of each subtask, it transmits the result to the headmaster node. 

 club of processing 
 ------------------- 

 Through an example, we exemplify that optimize the order in which the central processing unit complete sub - undertaking can provide a further reduction in computation time. We consider a elementary distributed setup in which the master node need to multiply two matrices $ \mathbf A=[\mathbf A_1,\mathbf A_2]$ and $ \mathbf B =[ \mathbf B_1, \mathbf B_2]$, where $ \mathbf A_i$ and $ \mathbf B_j$ are $ m \times q$ matrices and $ k=2 $, and there be $ n^2 = 9 $ worker nodes. We first divide each $ \mathbf A_i$ and $ \mathbf B_j$ into $ r=4 $ sub - matrix $ \mathbf A_i = [ \mathbf A_{i1 }, \ldots,\mathbf A_{i4}]$ and $ \mathbf B_j = [ \mathbf B_{j1 }, \dots, \mathbf B_{j4}]$ respectively. We then form an $ 8 \times 8 $ array as described in   \[product code\ ]. Then by encoding rows and columns of this array use $ (12,8)$ MDS code, $ (n \times r)^2 = 144 $ subtasks are generated. These subtasks are arranged in a $ 12 \times 12 $ array, as shown in Fig.   \[fig: MATMUL2\ ]. We assign $ r^2=16 $ subtasks to each processor, as depicted by the $ 4 \times 4 $ block in Fig.   \[fig: MATMUL2\ ]. For exemplifying purpose we wear that the four white processors are stragglers and complete their undertaking in 4 sec. We assume that the other five gray processors are non - straggler and finish their tasks after 1 sec. In one approach all processors perform their tasks according to a diagonal schedule illustrated in Fig.   \[fig: MATMUL2\]a. In the second, they follow the column - fresh scheduling depicted in Fig.   \[fig: MATMUL2\]b. If the aslant agenda is use, the master node can complete the matrix - matrix multiplication $ \
2$ anf $R=r^2$. The $L$ encoded subtarks are partitioned invo $N$ ardays of rize $R$, and each worker is asditned q distinct array of $R$ rubtasks. Vach workwr stquentially works through its $R$ dubtcsjs. At the compketion of aach subtask, id grcnsmits the result to the master nodq.

Order pf processing
-------------------

Thrjugh wn esample, we illustrate that optimizihg the mrder in whicn the processors complete dub-twsks can provide a further reeuctyin in computxtion time. We consider a simple distributed setup in dhich the mastet hofg needs to mnltiplj two matrices $\mathbf A=[\mathbg A_1,\mathbf A_2]$ anc $\methbd B =[\mathbf B_1, \mathbf B_2]$, where $\mathbf A_i$ agd $\mathbf B_l$ are $m \times q$ matrixew and $k=2$, atd tfwre art $n^2 = 9$ workeg nkdes. We fidst divide wach $\mathbf A_i$ and $\kaeybf B_j$ into $r=4$ sub-maeryces $\mathbf A_i = [\mathbf A_{i1}, \ldots,\mathbf A_{p4}]$ ans $\mathbf B_j = [\mathbf B_{j1}, \eots, \mathbf B_{j4}]$ respecjively. We ehen form an $8 \times 8$ array as described in \[product code\]. Gheu by evxofing rows and columns of this array using $(12,8)$ MDF cpdvs, $(n \times r)^2 = 144$ suntasks are generatrd. Tngse subtasks ate arraufes in a $12 \times 12$ arrwy, as srown un Fig. \[fig:IATMIL2\]. We assign $r^2=16$ subtasks to wach processje, as depicted by tke $4 \times 4$ buockx in Gig. \[fig:MATMUL2\]. For illustxative purpose we assume tgxt the four whitd pgocevsors art stragglers and cjmplete tieir casks in 4 sev. We afsume that the other five gray professots are non-straggpers and finish their tasks aftxc 1 sec. In one a[prmach all proccssors perform eheir tasks aceording co a dkagonal scgedule mllustrated yn Fig. \[fig:MATMGP2\]a. In the serond, they foloow rhe coljon-wise schedulong depicneb in Fig. \[fug:MATMUL2\]b. If the dlagonxm schedule is uwed, the master nodr cxn solpkeea the matrix-katrkx oiltipuication $\
2$ and $R=r^2$. The $L$ encoded subtasks into arrays of $R$, and each array $R$ subtasks. Each sequentially works through $R$ subtasks. At the completion of subtask, it transmits the result to the master node. Order of processing ------------------- an example, we illustrate that optimizing the order in which the processors complete can a reduction computation time. We consider a simple distributed setup in which the master node needs to multiply matrices $\mathbf A=[\mathbf A_1,\mathbf A_2]$ and $\mathbf B B_1, \mathbf B_2]$, where A_i$ and $\mathbf B_j$ are \times matrices and and are = 9$ worker We first divide each $\mathbf A_i$ and $\mathbf B_j$ into $r=4$ sub-matrices $\mathbf A_i = [\mathbf A_{i1}, A_{i4}]$ and = [\mathbf \dots, B_{j4}]$ We then form \times 8$ array as described in by encoding rows and columns of this array $(12,8)$ MDS $(n \times r)^2 = 144$ subtasks generated. These subtasks are arranged in a $12 12$ array, as shown in Fig. \[fig:MATMUL2\]. We assign $r^2=16$ subtasks to each processor, as the $4 \times 4$ in Fig. \[fig:MATMUL2\]. illustrative we that four white are stragglers and complete their tasks in 4 sec. We assume the other five gray processors are non-stragglers and finish their 1 In one approach processors perform their tasks to diagonal schedule illustrated in In second, column-wise depicted Fig. \[fig:MATMUL2\]b. If the schedule is used, the master can complete the matrix-matrix
2$ and $R=r^2$. The $L$ encoded subtasks aRe partitioNed inTo $N$ ArrAyS of sIze $R$, And each worker iS AssiGned a distinct array of $R$ sUbtasKs. eAch wORkEr seqUentialLY wORKs tHrOuGh iTs $r$ SuBtaskS. At The compLetion of eaCh sUbTask, it transmITs The result tO thE master node.

ORdeR of proCeSsiNG
-------------------

ThroUgh An exaMple, we ILlustrAte that opTiMIzing tHE order iN WHiCh thE processors compleTE sUB-tasks can proviDe a furThER rEDUctIon In computatIoN time. wE considER a SIMPle DIstributed setUp in which thE MasTer nodE nEedS To multIply tWo MAtrIces $\mathbf A=[\MathBf A_1,\mathbf a_2]$ and $\maTHbf B =[\matHBf B_1, \mathBf B_2]$, wheRe $\mAthBf A_i$ ANd $\MaThbF B_J$ Are $M \TiMes Q$ MatRices and $K=2$, aNd There Are $n^2 = 9$ WORKEr noDes. we fiRst diVide each $\mathbF A_i$ And $\mAThbF B_j$ inTo $r=4$ suB-matRiCes $\maThbf A_i = [\MathbF A_{I1}, \ldots,\mathbf A_{i4}]$ aNd $\maThbf B_j = [\matHbf b_{j1}, \DotS, \mAthbf b_{J4}]$ respeCtiVelY. We then Form an $8 \tIMes 8$ ArRAY As Described in \[product CoDE\]. thEn by encoDing roWS aNd COlumns of ThIs aRray USIng $(12,8)$ MDs codES, $(n \Times r)^2 = 144$ suBtasks ARe GeNerated. thEse subTaSks Are ArranGEd in A $12 \times 12$ Array, as sHown iN fig. \[fig:MATMUL2\]. We ASsign $r^2=16$ subtaskS To EACh PRoceSsoR, as depicted By thE $4 \TimeS 4$ bloCKs In FIG. \[fig:MaTMUL2\]. foR IlLUstrative purpose we aSsUme thaT the fOur white proceSsors are stRAGGlers and CompLEtE Their tasks in 4 seC. We asSume that thE Other fivE gray ProcessoRs are non-sTRAgglers aNd fIniSh tHeiR TAsKs after 1 sec. In oNE ApprOaCh all prOceSsors peRfoRm tHeiR taSkS accordinG to a diagOnAl ScHeDulE illuSTrated in fiG. \[fiG:MaTMuL2\]a. In THe secoNd, theY folLoW tHE coLumn-wisE ScHEDuliNg DePictEd iN FIg. \[fig:mATMul2\]b. IF the diaGonal scheDulE Is usEd, ThE master Node can compleTe The matrix-mAtRix MultipLICation $\
2$ and $R=r^2$. The $L$ en coded subt asksare  pa rt itio nedinto $N$ array s  ofsize $R$, and each wor ker i sa ssig n ed  a di stincta rr a y  of  $ R$  su bt a sk s. Ea chworkersequential lywo rks throughi ts  $R$ subta sks . At the com ple tion o feac h  subt ask , ittransm i ts the  result t ot he mas t er node . 
O rder  of processing
-- - -- - -------------
Throu gh an e xam ple , we illus tr ate t h at opti m iz i n g  th e  order in whi ch the proc e sso rs com pl ete sub-ta sks c an pro vide a furt herreduction  in co m putatio n  time.We con sid era si m pl edis tr i but e dset u p i n whichth emaste r no d e n eeds  to  mul tiply  two matrices  $\ math b f A =[\ma thbfA_1, \m athbf  A_2]$  and$\ mathbf B =[\mat hbfB_1, \mat hbf  B _2] $,  wher e  $\mat hbf  A_ i$ and$\mathb f  B_ j$ a r e$m \times q$ matri ce s an d $k=2$,  and t h er ea re $n^2=9$work e r  node s. W e  f irst div ide ea c h$\ mathbfA_ i$ and  $ \ma thb f B_j $  int o $r=4 $ sub-ma trice s  $\mathbf A_i=  [\mathbf A_{ i 1} , \l d ots, \ma thbf A_{i4} ]$ a n d $\ math b fB_j = [\m athbf  B _ {j 1 }, \dots, \mathbf B _{ j4}]$respe ctively. We t hen form a n $ 8 \times  8$a rr a y as described  in \ [product c o de\]. Th en by  encodin g rows an d columnsofthi s a rra y us ing $(12,8)$M D S co de s, $(n\ti mes r)^ 2 =  14 4$sub ta sks are g enerated .Th es esub tasks are arra ng edin  a$12 \ t imes 1 2$ ar ray,  a ss how n in Fi g .\ [ fig: MA TM UL2\ ].We  assi gn $ r ^2= 16$ sub tasks toeac h  pro ce ss or, asdepicted by t he  $4 \times  4 $ b locksi n  Fig. \[ fig:MATMUL2\]. For illu s trative  pu rpose  weassume th atthe fo urw hite p rocess ors a re  st r a ggler s an d c om plete thei r tas ks in  4  sec . We as sume that the othe r  fi ve gray proce sso rs a r e  n on- s tr a ggl er s  an d finish their ta sks after1s ec . In one a p pro ac h all p rocesso rs pe r form th eir tasks  accordin gto a d iag onal sched ule illu strated i n  Fig. \[ fig:M ATM UL2\]a .Inthe s econd, the y fol low th ecolumn -wise  s chedulin g depicted in Fig. \[fi g:MATM UL2\] b.If the di ago n alscheduleis u sed, the m ast ernodecan compl etet he  ma t rix-m atri x  multipli c at ion $ \
2$ and_$R=r^2$. The_$L$ encoded subtasks are_partitioned into_$N$_arrays of_size_$R$, and each_worker is assigned_a distinct array of_$R$ subtasks. Each_worker_sequentially works through its $R$ subtasks. At the completion of each subtask, it transmits_the_result to_the_master_node.

Order of processing
-------------------

Through an example,_we illustrate that optimizing the_order in_which the processors complete sub-tasks can provide a_further_reduction in computation_time. We consider a simple distributed setup in which_the master node needs to multiply_two matrices $\mathbf_A=[\mathbf_A_1,\mathbf_A_2]$ and $\mathbf B_=[\mathbf B_1, \mathbf B_2]$, where $\mathbf_A_i$ and $\mathbf B_j$ are $m_\times q$ matrices and $k=2$, and there_are $n^2 = 9$ worker nodes._We first divide each $\mathbf_A_i$ and_$\mathbf B_j$ into $r=4$ sub-matrices_$\mathbf A_i =_[\mathbf A_{i1},_\ldots,\mathbf A_{i4}]$ and_$\mathbf B_j = [\mathbf B_{j1}, \dots,_\mathbf B_{j4}]$ respectively._We then form an $8 \times_8$_array as described_in \[product_code\]._Then by_encoding rows and_columns_of this_array_using $(12,8)$ MDS codes, $(n \times_r)^2_= 144$ subtasks are generated. These subtasks_are arranged in a_$12_\times 12$ array, as_shown in Fig. \[fig:MATMUL2\]. We assign_$r^2=16$ subtasks to each processor, as_depicted by_the $4_\times 4$ blocks in Fig. \[fig:MATMUL2\]. For illustrative purpose we assume that_the four white processors are stragglers_and complete their tasks_in 4_sec._We assume that_the_other five_gray processors are non-stragglers and finish their_tasks after_1 sec. In one approach all_processors perform their tasks_according_to a diagonal schedule illustrated in_Fig. \[fig:MATMUL2\]a. In the second, they follow_the column-wise scheduling depicted in_Fig. \[fig:MATMUL2\]b._If_the diagonal schedule is used,_the master node can complete the_matrix-matrix multiplication $\