stochastic approximation central difference estimator the behavior of some system ( vehicles on road networkis simulated under certain parameters ( the lengths of the traffic light intervalsand the aim is to choose those parameters optimally ( to maximize the traffic throughputfor each parameter setting the exact value for the objective function is unknown but estimates can be obtained via the simulation in generalsuppose the goal is to minimize function where is unknownbut an estimate of (xcan be obtained for any choice of because the gradient is unknownone cannot directly apply classical optimization methods the stochastic approximation method mimics the classical gradient descent method by replacing deterministic (xgradient with an estimate simple estimator for the -th component of ( (that iss ( )/xi )is the central difference estimator ( ei / ( ei / ( where ei denotes the -th unit vectorand ( +ei / and ( -ei / can be any estimators of ( ei / and ( ei / )respectively the difference parameter should be small enough to reduce the bias of the estimatorbut large enough to keep the variance of the estimator small to reduce the variance in the estimator ( it is important to have ( ei / and ( ei / positively correlated this can for example be achieved by using common random numbers in the simulation common random numbers in direct analogy to gradient descent methodsthe stochastic approximation method produces sequence of iteratesstarting with some xvia (xt )xt+ xt bt ( where is sequence of strictly positive step sizes generic stochastic approximation algorithm for minimizing function is thus as follows algorithm stochastic approximation inputa mechanism to estimate any gradient (xand step sizes outputapproximate optimizer of initialize set while stopping criterion is not met do (xt of at xt obtain an estimated gradient determine step size bt (xt set xt+ xt bt - + robbins-monro return xt (xt is an unbiased estimator of (xt in ( the stochastic approximawhen tion algorithm is referred to as the robbins-monro algorithm when finite differc (xt )as in ( )the resulting algorithm is known as the ences are used to estimate