 table $D$. With table $D$ a prefix sum computation is started in row major layout such that $D$ now contains the relative output destination of each row within each meta-run. In a CRCW model, with the same assignment of elements as before, tiles can now be written to their destination to form the output using table $D$.

For CREW, when first creating $D$, one can ceil the tile sizes to full blocks. The resulting layout will obviously contain blocks that are not entirely filled, but contain empty memory cells. However, using the contraction described above, one can extract these empty cells.

Hence, the number of I/Os to copy tiles independently in parallel is $\O{\frac{H}{PB}}$. In order to write the transposed matrix directly, the destination for each tile has to be determined. This can be done using a prefix sum computation, requiring $\O{\log P}$ I/Os. The whole step to finalise the shuffle step has I/O-complexity $\O{\frac{H}{PB} + \log P}$.

#### Parallel (associative) reduce function

Assuming a parallelisable reduce, each processor shall perform multiple reduce functions simultaneously on a subset of elements with intermediate key in a certain range. In a final step, the results of these partial reduce executions are then collected and reduced to the final result.

To this end, the range of intermediate keys is partitioned into $\ceil{N_RPw/H}$ ranges of up to $\ceil{H/(Pw)}$ keys. @fullpaper Using the range-bounded load-balancing algorithm, elements (still ordered in meta-runs) are assigned to processors such that each processor gets elements from at most two pieces of row indices. This can be achieved by using the tuple $(\text{\it meta-run index}, \text{\it row index})$ as key. Then, elements are assigned to processors such that each processor gets elements from at most two ranges of row indices. If a processor got assigned elements that belong to the same reduce function, elements can be reduced immediately by the processor. Afterwards, for each key range, elements can be gathered to form the final result of the reduce function. This is possible with $\O{\frac{H}{PB} + \log P}$ I/Os.

### Complete sorting / merging

For some choices of parameters, especially for small instances, it can be optimal to simply apply
table $ D$. With table $ D$ a prefix sum computation is depart in rowing major layout such that $ D$ now contains the relative output signal destination of each row within each meta - rivulet. In a CRCW mannequin, with the same grant of elements as before, tile can now be written to their finish to form the output using table $ D$. 

 For CREW, when first make $ D$, one can ceil the tile sizes to full blocks. The result layout will obviously contain blocks that are not entirely fill, but contain empty memory cells. However, using the contraction described above, one can press out these empty cells. 

 Hence, the number of I / Os to copy tiles independently in parallel is $ \O{\frac{H}{PB}}$. In order to write the transposed matrix directly, the destination for each tile has to be determined. This can be done using a prefix sum computation, necessitate $ \O{\log P}$ I / Os. The whole dance step to finalise the shuffle step has I / O - complexity $ \O{\frac{H}{PB } + \log P}$. 

 # # # # Parallel (associative) deoxidize affair 

 Assuming a parallelisable reduce, each processor shall do multiple reduce functions simultaneously on a subset of elements with intermediate key in a certain range. In a final step, the results of these partial reduce executions are then collected and reduce to the final consequence. 

 To this end, the range of intermediate keys is partitioned into $ \ceil{N_RPw / H}$ range of up to $ \ceil{H/(Pw)}$ keys. @fullpaper Using the range - restrict load - balancing algorithm, elements (still regulate in meta - runs) are assigned to processors such that each processor gets component from at most two pieces of quarrel indices. This can be achieve by using the tuple $ (\text{\it meta - run index }, \text{\it rowing index})$ as samara. Then, elements are assign to processors such that each processor gets elements from at most two scope of row indices. If a central processing unit got assigned elements that belong to the same reduce function, elements can be reduced immediately by the central processing unit. Afterwards, for each key range, elements can be gathered to form the final result of the reduce affair. This is potential with $ \O{\frac{H}{PB } + \log P}$ I / Os. 

 # # # arrant sorting / merging 

 For some choices of parameters, especially for minor instances, it can be optimal to simply give
 tahle $D$. With table $D$ a prenix sum computatnin is vtartes in row major layout such that $D$ nox cobtainw the relative output aestinatiln of eaxh riq within eedh meta-vbn. In w CREW model, with thg same assigtment of elemettr cs before, tiles can now be written tj their dfstination to sorm ehe kltkut using table $D$.

For CREW, when fidst creeting $D$, one can ceil the tile sizes to fupl bpocks. The resultinh layout wiol ofciously contxin blocks that are noj entirely filled, but contain emoty mzmory cells. Hiweggr, using the contgaction descrlned abmve, one can extract tmese xmptt cells.

Hence, the numbxr of I/Os to copy tijes indepanbently in parallel is $\O{\drac{H}{KB}}$. In ordde tu wdive fhe trwns'osed matris directly, rhe destination for ewbn tile has tk be dqtqrmined. This can be done using a prefix suj computation, requiring $\O{\log P}$ I/Os. The whole step to sinalise the shuffle step has I/O-complexity $\O{\frac{H}{[B} + \lke P}$.

#### Pwfqlpel (associative) reduce function

Assuming a parwmltlixable reduce, ecch processor shslp lgrform multiplg reducz fhnctions simultanelusly og a sybset of tlememts with intermediate key ib a certain gangw. In a final step, che results uf tnese lartial reduce executious are then collefted and ddduced to the fival rasult.

To ufis end, the range of internedicte keys is kartitijned into $\feil{N_V[w/H}$ ranges of up tl $\ceip{H/([w)}$ keys. @fuplpaper Using the range-bounded load-balancing slcorpthm, elemznts (sbill ordered in meta-runs) are cssigned to pfocessors auch thet each procqssor gets elalents from av most twj piwces of row kndices. This csn be achpered by usung the tuple $(\text{\lt meja-dun index}, \text{\ic tiw index})$ as keu. Tfen, epekegds are assigted go otocesrors such tmat eacn processor gets elekenta from at most two rwnges of row indyces. If a provessor got assignef elekenvs thay bglong to the same reduce functikn, elemenhs gan be reduceq imnediately by the processor. Afterwards, for each key cange, elements can be gqthered to form the ninal result of tre reduce function. This is powsible with $\O{\frac{M}{PB} + \log P}$ I/Os.

### Complets sorthng / lerging

For some choices of parameters, especially for small instances, it cqn be optimal to sjmplu appny
table $D$. With table $D$ a prefix is in row layout such that output of each row each meta-run. In CRCW model, with the same assignment elements as before, tiles can now be written to their destination to form output using table $D$. For CREW, when first creating $D$, one can ceil tile to blocks. resulting layout will obviously contain blocks that are not entirely filled, but contain empty memory cells. using the contraction described above, one can extract empty cells. Hence, the of I/Os to copy tiles in is $\O{\frac{H}{PB}}$. order write transposed matrix directly, destination for each tile has to be determined. This can be done using a prefix sum computation, $\O{\log P}$ whole step finalise shuffle has I/O-complexity $\O{\frac{H}{PB} P}$. #### Parallel (associative) reduce function reduce, each processor shall perform multiple reduce functions on a of elements with intermediate key in certain range. In a final step, the results these partial reduce executions are then collected and reduced to the final result. To this range of intermediate keys partitioned into $\ceil{N_RPw/H}$ of to keys. Using the load-balancing algorithm, elements (still ordered in meta-runs) are assigned to processors that each processor gets elements from at most two pieces indices. can be achieved using the tuple $(\text{\it index}, row index})$ as key. are to each gets from at most two of row indices. If a got assigned elements that function, elements can be reduced immediately by the Afterwards, for each key range, elements can gathered to form the final result of the reduce function. This is with $\O{\frac{H}{PB} P}$ I/Os. ### Complete sorting / merging For choices of parameters, especially small instances, it can be optimal to simply apply
 table $D$. With table $D$ a prefix suM computatiOn is sTarTed In Row mAjor Layout such that $d$ Now cOntains the relative outpUt desTiNAtioN Of Each rOw withiN EaCH MetA-rUn. in a cRcw mOdel, wIth The same Assignment Of eLeMents as beforE, TiLes can now bE wrItten to their DesTinatiOn To fORm the OutPut usIng tabLE $D$.

For CrEW, when fiRsT CreatiNG $D$, one caN CEiL the Tile sizes to full blOCkS. the resulting laYout wiLl OBvIOUslY coNtain blockS tHat arE Not entiRElY FILleD, But contain empTy memory celLS. HoWever, uSiNg tHE contrActioN dEScrIbed above, onE can Extract thEse empTY cells.

HENce, the nUmber oF I/OS to Copy TIlEs IndEpENdeNTlY in PAraLlel is $\O{\fRaC{H}{pB}}$. In oRder TO WRIte tHe tRansPosed Matrix directlY, thE desTInaTion fOr eacH tilE hAs to bE deterMined. thIs can be done usinG a prEfix sum coMpuTaTioN, rEquirINg $\O{\log p}$ I/OS. ThE whole sTep to fiNAliSe THE ShUffle step has I/O-compLeXITy $\o{\frac{H}{PB} + \Log P}$.

#### PaRAlLeL (AssociatIvE) reDuce FUNctioN

AssUMiNg a paralLelisaBLe ReDuce, eacH pRocessOr ShaLl pErforM MultIple reDuce funcTions SImultaneously oN A subset of elemENtS WItH InteRmeDiate key in a CertAIn raNge. IN A fInaL Step, tHe resUlTS oF These partial reduce eXeCutionS are tHen collected aNd reduced tO THE final reSult.

tO tHIs end, the range oF inteRmediate keYS is partiTioneD into $\ceiL{N_RPw/H}$ ranGES of up to $\cEil{h/(Pw)}$ KeyS. @fuLLPaPer Using the raNGE-bouNdEd load-bAlaNcing alGorIthM, elEmeNtS (still ordEred in meTa-RuNs) ArE asSigneD To procesSoRs sUcH thAt eacH ProcesSor geTs elEmEnTS frOm at mosT TwO PIeceS oF rOw inDicEs. this cAn be AChiEved by uSing the tuPle $(\TExt{\iT mEtA-run indEx}, \text{\it row inDeX})$ as key. Then, ElEmeNts are ASSigned to Processors such that each pROcessor GetS elemEnts From at mosT twO rangeS of ROw indiCes. If a ProceSsOr gOT AssigNED eLemEnTs that beloNG To tHe samE rEducE functiOn, elements can be redUCed Immediately by The ProcESSoR. AfTErWArdS, fOR eaCH Key range, elementS can be gathErED tO form the fiNAl rEsUlt of thE reduce FunctIOn. This iS possible With $\O{\frac{h}{Pb} + \log p}$ i/os.

### COmplete sorTing / mergIng

For somE ChoicES oF paraMetErs, espEcIalLy for Small iNStaNces, iT can be OpTimal tO simpLy Apply
 table $D$. With table $D$  a prefixsum c omp uta ti on i s st arted in row m a jorlayout such that $D$ n ow co nt a inst he  rela tive ou t pu t des ti na tio no feachrow  within  each meta -ru n.  In a CRCW m o de l, with th e s ame assignme ntof ele me nts as be for e, ti les ca n  now b e written  t o  their destina t i on  toform the output u s in g  table $D$.

F or CRE W, wh e n  fi rst  creating$D $, on e  can ce i lt h e  ti l e sizes to fu ll blocks.T heresult in g l a yout w ill o bv i ous ly containbloc ks that a re not entirel y  filled , butcon tai n em p ty  m emo ry cel l s.  Ho w eve r, using  t he  cont ract i o n desc rib ed a bove,  one can extr act  the s e e mptycells .

H en ce, t he num ber o fI/Os to copy ti lesindepende ntl yinpa ralle l  is $\ O{\ fra c{H}{PB }}$. In ord er t o  w rite the transpose dm a tr ix direc tly, t h ede s tination  f oreach t ile h as t o  b e determ ined.T hi scan bedo ne usi ng  apre fix s u m co mputat ion, req uirin g  $\O{\log P}$I /Os. The whol e  s t e pt o fi nal ise the shu ffle step  has I/ O-c o mplex ity $ \O { \f r ac{H}{PB} + \log P} $. 

####  Para llel (associa tive) redu c e function 

As s um i ng a paralleli sable  reduce, e a ch proce ssorshall pe rform mul t i ple redu cefun cti ons s im ultaneously o n a su bs et of e lem ents wi thint erm edi at e key ina certai nra ng e.  In  a fi n al step,  t here sul ts of theseparti al r ed uc e  ex ecution s  a r e  the nco llec ted  a nd re duce d  to  the fi nal resul t.To t hi send, th e range of in te rmediate k ey s i s part i t ioned in to $\ceil{N_RPw/H}$ ran g es of u p t o $\c eil{ H/(Pw)}$key s. @fu llp a per Us ing th e ran ge -bo u n ded l o a d- bal an cing algor i t hm,  elem en ts ( still o rdered in meta-run s ) a re assigned t o p roce s s or s s u ch tha te ach p rocessor gets e lements fr om at  most twop iec es  of row  indice s. Th i s can b e achieve d by usin gthet u ple  $(\text{\ it meta- run index } , \te x t{ \it r owindex} )$  as  key.  Then, ele ments  are a ss ignedto pr oc essors s uch that each processor  getseleme nts  from atmos t  tw o rangesof r ow indices . I f a  proc ess o r got  ass i gn ede lemen ts t h at belong to  th e sa me reduce f u n c tio n, el eme n ts can  bereduced immediate l y by the proce ssor . Aft erw a rds,  f or each key ra nge ,e l ements c an  be gathere d to for mt he fi nal re sult o f the r e d uc e  funct ion.  Th is is pos sib le with $\ O{ \f r ac{H}{ PB}+\log P }$ I/O s .

# # #  Complete sortin g / m e r gingFor  some  c hoiceso f pa rameters,especiallyfor sm allinsta nces, i tcan be  op ti mal to sim p ly apply
 table_$D$. With_table $D$ a prefix_sum computation_is_started in_row_major layout such_that $D$ now_contains the relative output_destination of each_row_within each meta-run. In a CRCW model, with the same assignment of elements as_before,_tiles can_now_be_written to their destination to_form the output using table_$D$.

For CREW,_when first creating $D$, one can ceil the_tile_sizes to full_blocks. The resulting layout will obviously contain blocks that_are not entirely filled, but contain_empty memory cells._However,_using_the contraction described above,_one can extract these empty cells.

Hence,_the number of I/Os to copy_tiles independently in parallel is $\O{\frac{H}{PB}}$. In_order to write the transposed matrix_directly, the destination for each_tile has_to be determined. This can_be done using_a prefix_sum computation, requiring_$\O{\log P}$ I/Os. The whole step_to finalise the_shuffle step has I/O-complexity $\O{\frac{H}{PB} +_\log_P}$.

#### Parallel (associative)_reduce_function

Assuming_a parallelisable_reduce, each processor_shall_perform multiple_reduce_functions simultaneously on a subset of_elements_with intermediate key in a certain range._In a final step,_the_results of these partial_reduce executions are then collected_and reduced to the final result.

To_this end,_the range_of intermediate keys is partitioned into $\ceil{N_RPw/H}$ ranges of up to_$\ceil{H/(Pw)}$ keys. @fullpaper Using the range-bounded_load-balancing algorithm, elements (still_ordered in_meta-runs)_are assigned to_processors_such that_each processor gets elements from at most_two pieces_of row indices. This can be_achieved by using the_tuple_$(\text{\it meta-run index}, \text{\it row index})$_as key. Then, elements are assigned_to processors such that each_processor_gets_elements from at most two_ranges of row indices. If a_processor got assigned_elements that belong to the same reduce_function,_elements can be reduced immediately by_the_processor. Afterwards, for each key range,_elements_can_be gathered to form the_final result of the reduce function._This is possible with $\O{\frac{H}{PB} + \log P}$ I/Os.

###_Complete sorting /_merging

For some choices of parameters,_especially_for_small instances, it can be optimal to simply apply