again provided that the order of differentiation and integration (expectationcan be swapped in the same wayh(this then the hessian matrix of let ( thn be the minimizer of the training loss ln (xi th)rtn (th: = where tn { xn is random training set let rbe the smallest possible crossentropy risktaken over all functionsclearlyr- ln ( )where similar to the supervised learning casewe can decompose the generalization risk`( ( thn ) ( thn )into ( thr( thn rr(thrr( thn (thr(the ln { { ( th approx error statistical error the following theorem specifies the asymptotic behavior of the components of the generp alization risk in the proof we assume that thn -thas theorem approximating the cross-entropy risk it holds asymptotically ( that where er( thn (thtr (thh- (th/( )( (thertn ( thn tr (thh- (th/( ( proofa taylor expansion of ( thn around thgives the statistical error (th (thn th) (thn )( thn th) ( thn (th( thn th)th { ( = where thn lies on the line segment between thand thn for large we may replace (thn with (thasby assumptionb thn converges to ththe matrix (this positive definite because (this strictly convex at thby assumptionand therefore invertible it is important to realize thn is in fact an -estimator of thin particularin the notation of theorem we that have ps sa (th)and (thconsequentlyby that same theoremd ( thn th- - (thf(thh-(th( combining ( with ( )it follows from theorem that asymptotically the expected estimation error is given by ( nextwe consider taylor expansion of rtn (tharound thn rtn ( thn rtn (thrtn ( thn (thb thn )(th thn htn (thn )(thb thn )th { = (