 in $K$ will be done using iterative methods. In this case, the computational effort to computing (\[score\]) or (\[ascore\]) is roughly linear in the number of solves required (although see Section \[sec4\] for methods that make $N$ solves for a common matrix $K$ somewhat less than $N$ times the effort of one solve), so that (\[ascore\]) is much easier to compute than (\[score\]) when $N/n$ is small. An attractive feature of the approximation (\[ascore\]) is that if at any point one wants to obtain a better approximation to the score function, it suffices to consider additional $U_j$’s in (\[ascore\]). However, how exactly to do this if using the dependent sampling scheme for the $U_j$’s in Section \[sec4\] is not so obvious.

Since this stochastic approach provides only an approximation to the MLE, one must compare it with other possible approximations to the MLE. Many such approaches exist, including spectral methods, low-rank approximations, covariance tapering and those based on some form of composite likelihood. All these methods involve computing the likelihood itself and not just its gradient, and thus all share this advantage over solving (\[ascore\]). Note that one can use randomized algorithms to approximate $\log\det K$ and thus approximate the loglikelihood directly \[@zhangY\]. However, this approximation requires first taking a power series expansion of $K$ and then applying the randomization trick to each term in the truncated power series; the examples presented by @zhangY show that the approach does not generally provide a good approximation to the loglikelihood. Since the accuracy of the power series approximation to $\log\det K$ depends on the condition number of $K$, some of the filtering ideas described by @steinchenanitescufiltering and used to good effect in Section \[sec4\] here could perhaps be of value for approximating $\log\det K$, but we do not explore that possibility. See @aune for some recent developments on stochastic approximation of log determinants of positive definite matrices.

Let us consider the four approaches of spectral methods, low-rank approximations, covariance tapering and composite likelihood in turn. Spectral approximations to the likelihood can be fast and accurate for gridded data \[@whittle [@guyon; @dahlhaus]\], although even for gridded data they may require some prefiltering to work well \[@stein1995\]. In addition,
in $ K$ will be done using iterative methods. In this case, the computational campaign to calculate (\[score\ ]) or (\[ascore\ ]) is roughly linear in the numeral of solves ask (although see Section \[sec4\ ] for method acting that have $ N$ solves for a common matrix $ K$ slightly less than $ N$ times the effort of one solve), so that (\[ascore\ ]) is a lot easier to compute than (\[score\ ]) when $ N / n$ is small. An attractive feature of the estimate (\[ascore\ ]) is that if at any point one wants to receive a better approximation to the mark function, it suffices to consider extra $ U_j$ ’s in (\[ascore\ ]). However, how exactly to do this if using the pendent sampling dodge for the $ U_j$ ’s in Section \[sec4\ ] is not so obvious. 

 Since this stochastic approach provides only an approximation to the MLE, one must compare it with early possible approximations to the MLE. Many such approaches exist, including spectral methods, broken - rank and file approximations, covariance tapering and those based on some form of composite likelihood. All these methods involve computing the likelihood itself and not barely its gradient, and therefore all partake this advantage over solving (\[ascore\ ]). Note that one can use randomize algorithm to approximate $ \log\det K$ and thus approximate the loglikelihood directly \[@zhangY\ ]. However, this approximation requires first taking a baron series expansion of $ K$ and then applying the randomization trick to each term in the truncated baron series; the examples presented by @zhangY show that the overture does not generally supply a good approximation to the loglikelihood. Since the accuracy of the power series approximation to $ \log\det K$ depend on the condition number of $ K$, some of the filtering idea described by @steinchenanitescufiltering and used to good effect in Section \[sec4\ ] here could possibly be of value for approximating $ \log\det K$, but we do not explore that possibility. See @aune for some recent development on stochastic approximation of logarithm determinants of positive definite matrices. 

 Let us think the four approaches of spectral method acting, low - rank and file approximations, covariance sharpen and composite likelihood in turn. Spectral approximations to the likelihood can be fast and accurate for gridded data \[@whittle [ @guyon; @dahlhaus]\ ], although even for gridded datum they may require some prefiltering to work well \[@stein1995\ ]. In accession,
 in $K$ will be done using ittrative methods. Iu this rase, ths computxtional effort to computing (\[dcire\]) oe (\[ascore\]) is roughly livear in tje numbee of wolves reqnjred (albkough dee Vxction \[sec4\] for kethods thdt make $N$ solvas flr a common matrix $K$ somewhat less ehan $N$ yiles the effort of pge sklve), so that (\[ascore\]) is much easier to comkute than (\[score\]) whrn $N/n$ is small. An attractige ffature of the apprlximation (\[awcorq\]) is that if xt any point one wants to obtain a better approximatiun to the score fynchhon, it suffmces tj consider additionan $U_j$’s im (\[ascore\]). Howevcr, hox exqctly to do this if uving the dependent sampling seheme for the $U_j$’s in Wextion \[sec4\] is vit ro kbtiohs.

Sincf tiis stochasfic approacy provides only an sp[gpximation to the MJE, one must compare it with other possiblt appdoximations to the MLE. Nany such approaches gxist, incltding spectral methods, low-rank approximations, covdrianre takevlng xbd those based on some form of composite likelirkoc. Sll these methjds involve clmloting the likeuihood itaelf and not just lts graqient, and thus all share this advantage over wolving (\[ascoge\]). Nite that one can ude randomizzd algprithks to approximate $\log\dec K$ ans thus apprlximate tgd loglikelihood airvctlf \[@zhangY\]. However, this apprjximation reqbires fifst jaking w power segies cfpansion of $K$ and hhen cpplyhng the rajdomization trick to each term mi the truncatgd [owvr series; the cxamples preseneed by @zhangY xhow thct the approach soes nov generally [rovide a goog approximatimn to thq lotlikwlihood. Rince the accutacy of tkt power seeies approximation to $\mog\det K$ dependw ob the condition nuobew lf $K$, vome of the xiltdrivb idexs describeb cy @syeinchenanitescufiltarinf and used to good enfect in Wection \[fec4\] here coulc perhaps be of vapue fmr epproxomajing $\log\det K$, but we do not expmore that pofsibility. Seq @auke fjr some reeent developments on stochastic approximetion of log determinanrs of positive defiuiue matrices.

Lev us cjnsider tve four approaches od spectral methodx, low-rank approximatikns, coeariajce tapering and composite likelihood in turn. Spectral approximations to rhe likelihood can be gast dnb aecurate for jridded data \[@whitnle [@guyon; @dahlhaus]\], although even for grigdzd data they may require some pfefiltering tu work well \[@stein1995\]. In addjtion,
in $K$ will be done using iterative this the computational to computing (\[score\]) in number of solves (although see Section for methods that make $N$ solves a common matrix $K$ somewhat less than $N$ times the effort of one so that (\[ascore\]) is much easier to compute than (\[score\]) when $N/n$ is An feature the (\[ascore\]) is that if at any point one wants to obtain a better approximation to the function, it suffices to consider additional $U_j$’s in However, how exactly to this if using the dependent scheme the $U_j$’s Section is so obvious. Since stochastic approach provides only an approximation to the MLE, one must compare it with other possible approximations the MLE. approaches exist, spectral low-rank covariance tapering and on some form of composite likelihood. involve computing the likelihood itself and not just gradient, and all share this advantage over solving Note that one can use randomized algorithms to $\log\det K$ and thus approximate the loglikelihood directly \[@zhangY\]. However, this approximation requires first taking series expansion of $K$ then applying the trick each in truncated power the examples presented by @zhangY show that the approach does not provide a good approximation to the loglikelihood. Since the accuracy power approximation to $\log\det depends on the condition of some of the filtering by and effect Section here could perhaps be value for approximating $\log\det K$, we do not explore some recent developments on stochastic approximation of log of positive definite matrices. Let us consider four approaches of spectral methods, low-rank approximations, covariance tapering and composite likelihood turn. Spectral the likelihood can be fast and accurate for data \[@whittle [@guyon; @dahlhaus]\], even for gridded data they may require some prefiltering work \[@stein1995\]. In
 in $K$ will be done using iteratiVe methods. IN this CasE, thE cOmpuTatiOnal effort to coMPutiNg (\[score\]) or (\[ascore\]) is roughLy linEaR In thE NuMber oF solves REqUIRed (AlThOugH sEE SEctioN \[seC4\] for metHods that maKe $N$ SoLves for a commON mAtrix $K$ someWhaT less than $N$ tiMes The effOrT of ONe solVe), sO that (\[Ascore\]) IS much eAsier to coMpUTe than (\[SCore\]) wheN $n/N$ iS smaLl. An attractive feaTUrE Of the approximaTion (\[asCoRE\]) iS THat If aT any point oNe Wants TO obtain A BeTTER apPRoximation to tHe score funcTIon, It suffIcEs tO ConsidEr addItIOnaL $U_j$’s in (\[ascorE\]). HowEver, how exActly tO Do this iF Using thE depenDenT saMpliNG sChEme FoR The $u_J$’s In SECtiOn \[sec4\] is nOt So ObvioUs.

SiNCE THis sTocHastIc appRoach provides OnlY an aPProXimatIon to The MlE, One muSt compAre it WiTh other possible ApprOximationS to ThE MLe. MAny suCH approAchEs eXist, incLuding sPEctRaL METhOds, low-rank approximAtIONs, CovarianCe tapeRInG aND those baSeD on Some FORm of cOmpoSItE likelihOod. All THeSe Methods InVolve cOmPutIng The liKElihOod itsElf and noT just ITs gradient, and tHUs all share thiS AdVANtAGe ovEr sOlving (\[ascorE\]). NotE That One cAN uSe rANdomiZed alGoRItHMs to approximate $\log\dEt k$ and thUs appRoximate the loGlikelihooD DIRectly \[@zhAngY\]. hOwEVer, this approxiMatioN requires fIRst takinG a powEr series Expansion OF $k$ and then AppLyiNg tHe rANDoMization trick TO Each TeRm in the TruNcated pOweR seRieS; thE eXamples prEsented bY @zHaNgy sHow That tHE approacH dOes NoT geNeralLY proviDe a goOd apPrOxIMatIon to thE LoGLIkelIhOoD. SinCe tHe AccurAcy oF The Power seRies approXimATion To $\LoG\det K$ dePends on the conDiTion number Of $k$, soMe of thE FIltering Ideas described by @steinchENanitesCufIlterIng aNd used to gOod Effect In SECtion \[sEc4\] here Could PeRhaPS Be of vALUe For ApProximatinG $\LOg\dEt K$, buT wE do nOt exploRe that possibility. SEE @auNe for some receNt dEvelOPMeNts ON sTOchAsTIc aPPRoximation of log DeterminanTs OF pOsitive defINitE mAtrices.

let us coNsideR The four ApproacheS of spectrAl MethODS, loW-rank approXimationS, covarianCE tapeRInG and cOmpOsite lIkEliHood iN turn. SPEctRal apProximAtIons to The liKeLihood caN be fast and accurate for grIdded dAta \[@whIttLe [@guyon; @daHlhAUs]\], aLthough evEn foR gridded daTa tHey May reQuiRE some PrefILtEriNG to woRk weLL \[@stein1995\]. In aDDiTioN,
 in $K$ will be done using  iterative  meth ods . I nthis  cas e, the computa t iona l effort to computing(\[sc or e \])o r(\[as core\]) is r oug hl ylin ea r  i n the  nu mber of  solves re qui re d (althoughs ee  Section \ [se c4\] for met hod s that  m ake $N$ s olv es fo r a co m mon ma trix $K$so m ewhatl ess tha n $N $ ti mes the effort of on e  solve), so th at (\[ as c or e \ ])ismuch easie rto co m pute th a n( \ [ sco r e\]) when $N/ n$ is small .  An  attra ct ive featur e ofth e  ap proximation  (\[ ascore\])  is th a t if at any poi nt one  wa nts  too bt ai n a  b e tte r  a ppr o xim ation to  t he  scor e fu n c t i on,itsuff icesto consider a ddi tion a l $ U_j$’ s in(\[a sc ore\] ). How ever,  h ow exactly to d o th is if usi ngth e d ep enden t  sampl ing  sc heme fo r the $ U _j$ ’s i n  S ection \[sec4\] is  n o t  s o obviou s.

Si n ce  t h is stoch as tic  app r o ach p rovi d es  only an  appro x im at ion toth e MLE,  o nemus t com p areit wit h otherpossi b le approximati o ns to the MLE .  M a n ys uchapp roaches exi st,i nclu ding sp ect r al me thods ,l ow - rank approximations ,covari ancetapering andthose base d o n some f ormo fc omposite likel ihood . All thes e  methods  invo lve comp uting the l ikelihoo d i tse lfand n ot  just its gra d i ent,  a nd thus  al l share  th isadv ant ag e over so lving (\ [a sc or e\ ]).  Note that one  c anus e r andom i zed al gorit hmsto  a p pro ximate$ \l o g \det  K $andthu sappro xima t e t he logl ikelihood  di r ectl y\[ @zhangY \]. However,th is approxi ma tio n requ i r es first  taking a power seriese xpansio n o f $K$  and  then app lyi ng the  ra n domiza tion t rickto  ea c h  term i nthe  t runcated p o w erserie s;  the  exampl es presented by @z h ang Y show that t heappr o a ch  do e sn otge n era l l y provide a goo d approxim at i on  to the lo g lik el ihood.Since t he ac c uracy o f the pow er series  a ppro x i mat ion to $\l og\det K $ depends on th e  c ondit ion  numbe rof$K$,some o f  th e fil tering  i deas d escri be d by @st einchenanitescufilterin g andusedtogood effe cti n S ection \[ sec4 \] here co uld  pe rhaps  be of va luef or  ap p roxim atin g  $\log\de t  K $,b u twe do not e x p l ore  that  po s sibili ty.See @aune for som e  recent develo pmen t s  on  st o chas ti c approximatio n o fl o g determ in ants of pos itive de fi n ite m atrice s.

Le t us co n s id e r thefour  ap proachesofsp e ctral m et ho d s, low -ran kapprox imatio n s, c o v ariance tapering  andc o mposi t e l ikeli ho od in t u rn.Spectral a pproximatio ns tothelikel ihood c an  be fa stan d accurate for gridd ed da ta \[@w hi ttle  [@ guyon;  @da h l haus] \],al tho ugh evenf o rg ri dd e d d atatheyma y re quire som e  prefilt eri n g to wo rk  we l l  \[@st e in 1 9 95\]. In a ddi tion, 
 in_$K$ will_be done using iterative_methods. In_this_case, the_computational_effort to computing_(\[score\]) or (\[ascore\])_is roughly linear in_the number of_solves_required (although see Section \[sec4\] for methods that make $N$ solves for a common_matrix_$K$ somewhat_less_than_$N$ times the effort of_one solve), so that (\[ascore\])_is much_easier to compute than (\[score\]) when $N/n$ is_small._An attractive feature_of the approximation (\[ascore\]) is that if at any_point one wants to obtain a_better approximation to_the_score_function, it suffices to_consider additional $U_j$’s in (\[ascore\]). However,_how exactly to do this if_using the dependent sampling scheme for the_$U_j$’s in Section \[sec4\] is not_so obvious.

Since this stochastic approach_provides only_an approximation to the MLE,_one must compare_it with_other possible approximations_to the MLE. Many such approaches_exist, including spectral_methods, low-rank approximations, covariance tapering and_those_based on some_form_of_composite likelihood._All these methods_involve_computing the_likelihood_itself and not just its gradient,_and_thus all share this advantage over solving_(\[ascore\]). Note that one_can_use randomized algorithms to_approximate $\log\det K$ and thus_approximate the loglikelihood directly \[@zhangY\]. However,_this approximation_requires first_taking a power series expansion of $K$ and then applying the_randomization trick to each term in_the truncated power series;_the examples_presented_by @zhangY show_that_the approach_does not generally provide a good approximation_to the_loglikelihood. Since the accuracy of the_power series approximation to_$\log\det_K$ depends on the condition number_of $K$, some of the filtering_ideas described by @steinchenanitescufiltering and_used_to_good effect in Section \[sec4\]_here could perhaps be of value_for approximating $\log\det_K$, but we do not explore that_possibility._See @aune for some recent developments_on_stochastic approximation of log determinants of_positive_definite_matrices.

Let us consider the four_approaches of spectral methods, low-rank approximations,_covariance tapering and composite likelihood in turn. Spectral approximations_to the likelihood_can be fast and accurate_for_gridded_data \[@whittle [@guyon; @dahlhaus]\], although even for gridded data they_may require_some prefiltering to_work well \[@stein1995\]. In addition,