 are calculated from the experimental design $\cx$ and associated model response $\bfy$. Due to the high input dimensionality, a surrogate $\widehat{\cm}|\cx,\bfy$ may lead to poor generalisation performance or it may not even be computationally tractable. To reduce the dimensionality, the class of DR methods was introduced in [Section \[sec:Meth:DR\]]{}. A DR transformation, expressed by $\cz = g(\cx; \bfw)$, can provide a compressed experimental design, $\bfz^{(i)} \in \Rr^m\,,\, i=1\enu N$ with $m\ll M$. The surrogate $\widehat{\cm}|\cz,\bfy$ becomes tractable if $m$ is sufficiently small. The potential of $\widehat{\cm}|\cz,\bfy$ to achieve satisfactory generalisation performance depends on (i) the learning capacity of the surrogate itself and (ii) the assumption that the input-output map $\bfx \mapsto y$ can be sufficiently well approximated by a smaller set of features via the transformation $g(\cdot)$. This discussion focuses on the latter and assumes that the learning capacity of the surrogate is adequate. In case of unstructured inputs, the importance of each input variable may vary depending on the output of interest. In case of structured inputs, there is typically high correlation between the input components. Hence, in both families of problems a low-dimensional representation may often approximate well the input-output map.

Traditional DR approaches are focused on the discovery of the input manifold and not the input-output manifold. Performing an input compression without taking into account the associated output values may lead to a highly complex input-output map that is difficult to surrogate. In the DRSM (dimensionality reduction for surrogate modelling) approach proposed in this paper, we capitalise on this claim to try and find an optimal input compression scheme w.r.t. the generalisation performance of $\widehat{\cm}|\cz,\bfy$.

A nested optimisation problem {#sec:Meth:DRSM:1_nested_optim}
-----------------------------

The goal of DRSM is to optimise the parameters $\bfw$ of the compression scheme so that the auxiliary variables $\bfz = g(\bfx;\bfw)$ are suitable to achieve an overall accurate surrogate. The general formulation of this problem reads:

$$\label{eq:DRSM_general}
    \acc{\bfwh,\bf
are calculated from the experimental design $ \cx$ and associated model reception $ \bfy$. Due to the eminent input dimensionality, a surrogate $ \widehat{\cm}|\cx,\bfy$ may lead to inadequate generalisation performance or it may not even be computationally amenable. To reduce the dimensionality, the class of DR methods was precede in [ Section   \[sec: Meth: DR\ ] ] { }. A DR transformation, expressed by $ \cz = g(\cx; \bfw)$, can leave a compressed experimental design, $ \bfz^{(i) } \in \Rr^m\,,\, i=1\enu N$ with $ m\ll M$. The foster $ \widehat{\cm}|\cz,\bfy$ becomes tractable if $ m$ is sufficiently humble. The potential of $ \widehat{\cm}|\cz,\bfy$ to achieve satisfactory generalisation operation depends on (i) the learning capacitance of the surrogate itself and (ii) the assumption that the input signal - output function $ \bfx \mapsto y$ can be sufficiently well approximated by a small set of features via the transformation $ g(\cdot)$. This discussion focuses on the latter and assume that the learning capacity of the surrogate is adequate. In case of unstructured stimulation, the importance of each input variable may vary depending on the output of interest. In case of structured inputs, there is typically eminent correlation between the input component. Hence, in both families of problems a low - dimensional representation may frequently approximate well the input - output map. 

 Traditional DR approaches are focus on the discovery of the input manifold and not the input - output manifold. perform an input compression without taking into account the consociate output value may lead to a highly complex input - output map that is difficult to surrogate. In the DRSM (dimensionality decrease for surrogate modelling) approach proposed in this paper, we capitalise on this claim to try and find an optimum input compression scheme w.r.t. the generalisation performance of $ \widehat{\cm}|\cz,\bfy$. 

 A nested optimization problem { # sec: Meth: DRSM:1_nested_optim } 
 ----------------------------- 

 The goal of DRSM is to optimise the parameters $ \bfw$ of the compaction scheme so that the auxiliary variable star $ \bfz = g(\bfx;\bfw)$ are suitable to achieve an overall accurate surrogate. The cosmopolitan formulation of this problem reads: 

 $ $ \label{eq: DRSM_general } 
     \acc{\bfwh,\bf
 arf calculated from the exkerimental design $\cx$ and assocjated moael response $\bfy$. Due to the iigh inpuu dimensionality, a rurrogate $\widehat{\xm}|\cx,\ufy$ may lead to 'kor gencxaliszbion 'ecformance or it may not eeen be computadiunclly tractable. To reduce the dimensijnality, tje class of DR metnjds sas introduced in [Section \[sec:Meth:DR\]]{}. A DR tgansformation, exptessed by $\cz = g(\cx; \bfw)$, can orovlde a compressed edperimental desytn, $\bfz^{(i)} \in \Rf^m\,,\, i=1\enu N$ with $m\ll M$. Tge surrogate $\widehat{\cm}|\cz,\bfy$ becumes cractable id $n$ id sufficientoy smwll. The poteknial of $\widehay{\cm}|\cz,\bfy$ to acmieve sarisfactory generalisavion performance depgnds on (i) dhz learning capacity od rhe sorrogdte krseuf znv (ij) the wssnmption thaf the input-iutput map $\bfx \mapsuo r$ can be suffidientlr rell approximated by a smaller set of ftaturss via the transformatiin $g(\cdot)$. This discusslon focusqs on the latter and assumes that the learning ca[acitb uf uhc ryrgogate is adequate. In case of unstructured in[htx, nhe importance of each input vwrowble may vary depenbjnf on the output of intereft. In case of ftruvtured inputs, there is typixally high cjerelation between che input coopongnts. Hrnce, in both families oy probmems a low-dlmensionam representation oay oxten approximate well the ynput-outpnt ma'.

Traditiunal DR ap[roaches age foggsed on the discovfry oy the input manlfold and not the input-output meiifold. Perforkitg dn input compvession without taking into aecount tke assuciated ounput valuxs may lead eo a highly cmlplex input-ontput map thar is difficjut to surrogatr. In the BXSM (dimenwionality reductiok for aurrogate modeloint) approach proppsea ig nhix [dper, we capidalire um thir claim to bry and find an optimal inpgt ckmpression scheme e.r.b. the gengralisatijn performancr of $\widehat{\cm}|\cz,\bfj$.

A nevtev optikiswtion problem {#sec:Meth:DRSM:1_nestes_optim}
-----------------------------

The gowl of DRSM if to optimise thz parameters $\bfw$ of the compression schene so that the auxuliary variables $\bfe = n(\bfx;\bfw)$ are xuitafle to acvieve an overall accyrate surrogate. Tme general formulation of thhs prlblem reads:

$$\label{eq:DRSM_general}
    \acc{\bfwh,\bf
are calculated from the experimental design $\cx$ model $\bfy$. Due the high input lead poor generalisation performance it may not be computationally tractable. To reduce the the class of DR methods was introduced in [Section \[sec:Meth:DR\]]{}. A DR transformation, by $\cz = g(\cx; \bfw)$, can provide a compressed experimental design, $\bfz^{(i)} \in i=1\enu with M$. surrogate $\widehat{\cm}|\cz,\bfy$ becomes tractable if $m$ is sufficiently small. The potential of $\widehat{\cm}|\cz,\bfy$ to achieve satisfactory performance depends on (i) the learning capacity of surrogate itself and (ii) assumption that the input-output map \mapsto can be well by smaller set of via the transformation $g(\cdot)$. This discussion focuses on the latter and assumes that the learning capacity of surrogate is case of inputs, importance each input variable depending on the output of interest. structured inputs, there is typically high correlation between input components. in both families of problems a representation may often approximate well the input-output map. DR approaches are focused on the discovery of the input manifold and not the input-output an input compression without into account the output may to highly complex map that is difficult to surrogate. In the DRSM (dimensionality reduction surrogate modelling) approach proposed in this paper, we capitalise on to and find an input compression scheme w.r.t. generalisation of $\widehat{\cm}|\cz,\bfy$. A nested {#sec:Meth:DRSM:1_nested_optim} The is optimise parameters $\bfw$ of the scheme so that the auxiliary $\bfz = g(\bfx;\bfw)$ are accurate surrogate. The general formulation of this problem $$\label{eq:DRSM_general} \acc{\bfwh,\bf
 are calculated from the experImental desIgn $\cx$ And AssOcIateD modEl response $\bfy$. DUE to tHe high input dimensionalIty, a sUrROgatE $\WiDehat{\Cm}|\cx,\bfy$ MAy LEAd tO pOoR geNeRAlIsatiOn pErformaNce or it may Not EvEn be computatIOnAlly tractaBle. to reduce the dImeNsionaLiTy, tHE clasS of dR metHods waS IntrodUced in [SecTiON \[sec:MeTH:DR\]]{}. A DR tRANsFormAtion, expressed by $\cZ = G(\cX; \Bfw)$, can provide a CompreSsED eXPEriMenTal design, $\bFz^{(I)} \in \Rr^M\,,\, I=1\enu N$ wiTH $m\LL m$. the SUrrogate $\widehAt{\cm}|\cz,\bfy$ beCOmeS tractAbLe iF $M$ is sufFicieNtLY smAll. The potenTial Of $\widehat{\Cm}|\cz,\bfY$ To achieVE satisfActory GenEraLisaTIoN pErfOrMAncE DePenDS on (I) the learNiNg CapacIty oF THE SurrOgaTe itSelf aNd (ii) the assumpTioN thaT The Input-OutpuT map $\BfX \mapsTo y$ can Be sufFiCiently well apprOximAted by a smAllEr Set Of FeatuREs via tHe tRanSformatIon $g(\cdoT)$. thiS dISCUsSion focuses on the laTtER AnD assumes That thE LeArNIng capacItY of The sURRogatE is aDEqUate. In caSe of unSTrUcTured inPuTs, the iMpOrtAncE of eaCH inpUt variAble may vAry dePEnding on the outPUt of interest. IN CaSE Of STrucTurEd inputs, theRe is TYpicAlly HIgH coRRelatIon beTwEEn THe input components. HeNcE, in botH famiLies of problemS a low-dimenSIONal repreSentATiON may often approXimatE well the inPUt-output Map.

TrAditionaL DR approaCHEs are focUseD on The DisCOVeRy of the input mANIfolD aNd not thE inPut-outpUt mAniFolD. PeRfOrming an iNput compReSsIoN wIthOut taKIng into aCcOunT tHe aSsociATed outPut vaLues MaY lEAd tO a highlY CoMPLex iNpUt-OutpUt mAp That iS difFIcuLt to surRogate. In tHe DrsM (diMeNsIonalitY reduction for SuRrogate modElLinG) approACH proposeD in this paper, we capitalisE On this cLaiM to trY and Find an optImaL input ComPRessioN schemE w.r.t. tHe GenERAlisaTIOn PerFoRmance of $\wiDEHat{\Cm}|\cz,\bFy$.

a nesTed optiMisation problem {#sec:mEth:dRSM:1_nested_optIm}
-----------------------------

THe goAL Of dRSm Is TO opTiMIse THE parameters $\bfw$ oF the compreSsIOn Scheme so thAT thE aUxiliarY variabLes $\bfZ = G(\bfx;\bfw)$ Are suitabLe to achieVe An ovERAll Accurate suRrogate. THe general FOrmulATiOn of tHis ProbleM rEadS:

$$\labeL{eq:DRSm_GenEral}
    \aCc{\bfwh,\Bf
 are calculated from the e xperimenta l des ign  $\ cx $ an d as sociated model resp onse $\bfy$. Due to th e hig hi nput di mensi onality ,  a s urr og at e $ \w i de hat{\ cm} |\cx,\b fy$ may le adto  poor genera l is ation perf orm ance or it m aynot ev en  be compu tat ional ly tra c table.  To reduc et he dim e nsional i t y,  the  class of DR meth o ds was introduced  in [S ec t io n \[s ec: Meth:DR\]] {} . A D R  transf o rm a t i on, expressed by$\cz = g(\c x ; \ bfw)$,  c anp rovide  a co mp r ess ed experime ntal  design,$\bfz^ { (i)} \i n  \Rr^m\ ,,\, i =1\ enu  N$w it h$m\ ll M$. Th e s u rro gate $\w id eh at{\c m}|\ c z , \ bfy$  be come s tra ctable if $m$  is  suf f ici ently  smal l. T he  pote ntialof $\ wi dehat{\cm}|\cz, \bfy $ to achi eve  s ati sf actor y  gener ali sat ion per formanc e  de pe n d s  o n (i) the learning  c a p ac ity of t he sur r og at e  itselfan d ( ii)t h e ass umpt i on  that th e inpu t -o ut put map  $ \bfx \ ma pst o y $ can be s uffici ently we ll ap p roximated by a smaller set o f  f e a tu r es v iathe transfo rmat i on $ g(\c d ot )$. Thisdiscu ss i on focuses on the latt er  and a ssume s that the le arning cap a c i ty of th e su r ro g ate is adequat e. In  case of u n structur ed in puts, th e importa n c e of eac h i npu t v ari a b le  may vary dep e n ding  o n the o utp ut of i nte res t.Inca se of str ucturedin pu ts ,the re is typicall yhig hcor relat i on bet weenthein pu t  co mponent s .H e nce,  i nboth  fa mi liesof p r obl ems a l ow-dimens ion a l re pr es entatio n may often a pp roximate w el l t he inp u t -outputmap.

Traditional DR ap p roaches  ar e foc used  on the d isc overyoft he inp ut man ifold  a ndn o t the i np ut- ou tput manif o l d.Perfo rm ingan inpu t compression with o uttaking into a cco untt h eass o ci a ted  o u tpu t values may lead  to a high ly co mplex inpu t -ou tp ut mapthat is  diff i cult to  surrogat e. In the  D RSM( d ime nsionality  reducti on for su r rogat e  m odell ing ) appr oa chpropo sed in thi s pap er, we  c apital ise o nthis cla im to try and find an o ptimal  inpu t c ompressio n s c hem e w.r.t.thegeneralisa tio n p erfor man c e of$\wi d eh at{ \ cm}|\ cz,\ b fy$.

A n e st edo p ti misation pr o b l em{#sec :Me t h:DRSM :1_n ested_optim}
---- - -------------- ---- - - --- -
T he g oa l of DRSM is t o o pt i m ise thepa rameters $\ bfw$ ofth e  comp ressio n sche me so t h a tt he aux ilia ryvariables  $\ bf z  = g(\b fx ;\ b fw)$ a re s ui tableto ach i evea n  overall accurat e sur r o gate. The  gene ra l formu l atio n of thisproblem rea ds:

$ $\la bel{e q:DRSM_ ge neral} 
  \acc{\bfwh , \bf
 are_calculated from_the experimental design $\cx$_and associated_model_response $\bfy$._Due_to the high_input dimensionality, a_surrogate $\widehat{\cm}|\cx,\bfy$ may lead_to poor generalisation_performance_or it may not even be computationally tractable. To reduce the dimensionality, the class_of_DR methods_was_introduced_in [Section \[sec:Meth:DR\]]{}. A DR transformation,_expressed by $\cz = g(\cx;_\bfw)$, can_provide a compressed experimental design, $\bfz^{(i)} \in \Rr^m\,,\,_i=1\enu_N$ with $m\ll_M$. The surrogate $\widehat{\cm}|\cz,\bfy$ becomes tractable if $m$ is_sufficiently small. The potential of $\widehat{\cm}|\cz,\bfy$_to achieve satisfactory_generalisation_performance_depends on (i) the_learning capacity of the surrogate itself_and (ii) the assumption that the_input-output map $\bfx \mapsto y$ can be_sufficiently well approximated by a smaller_set of features via the_transformation $g(\cdot)$._This discussion focuses on the_latter and assumes_that the_learning capacity of_the surrogate is adequate. In case_of unstructured inputs,_the importance of each input variable_may_vary depending on_the_output_of interest._In case of_structured_inputs, there_is_typically high correlation between the input_components._Hence, in both families of problems a_low-dimensional representation may often_approximate_well the input-output map.

Traditional_DR approaches are focused on_the discovery of the input manifold_and not_the input-output_manifold. Performing an input compression without taking into account the associated_output values may lead to a_highly complex input-output map_that is_difficult_to surrogate. In_the_DRSM (dimensionality_reduction for surrogate modelling) approach proposed in_this paper,_we capitalise on this claim to_try and find an_optimal_input compression scheme w.r.t. the generalisation_performance of $\widehat{\cm}|\cz,\bfy$.

A nested optimisation problem_{#sec:Meth:DRSM:1_nested_optim}
-----------------------------

The goal of DRSM is_to_optimise_the parameters $\bfw$ of the_compression scheme so that the auxiliary_variables $\bfz =_g(\bfx;\bfw)$ are suitable to achieve an overall_accurate_surrogate. The general formulation of this_problem_reads:

$$\label{eq:DRSM_general}
    \acc{\bfwh,\bf