Title: Two-Armed Bandit Problem, Data Processing, and Parallel Version of the Mirror Descent Algorithm,
Abstract: We consider the minimax setup for the two-armed bandit problem as applied to
data processing if there are two alternative processing methods available with
different a priori unknown efficiencies. One should determine the most
effective method and provide its predominant application. To this end we use
the mirror descent algorithm (MDA). It is well-known that corresponding minimax
risk has the order $N^{1/2}$ with $N$ being the number of processed data. We
improve significantly the theoretical estimate of the factor using Monte-Carlo
simulations. Then we propose a parallel version of the MDA which allows
processing of data by packets in a number of stages. The usage of parallel
version of the MDA ensures that total time of data processing depends mostly on
the number of packets but not on the total number of data. It is quite
unexpectedly that the parallel version behaves unlike the ordinary one even if
the number of packets is large. Moreover, the parallel version considerably
improves control performance because it provides significantly smaller value of
the minimax risk. We explain this result by considering another parallel
modification of the MDA which behavior is close to behavior of the ordinary
version. Our estimates are based on invariant descriptions of the algorithms.
All estimates are obtained by Monte-Carlo simulations. It's worth noting that
parallel version performs well only for methods with close efficiencies. If
efficiencies differ significantly then one should use the combined algorithm
which at initial sufficiently short control horizon uses ordinary version and
then switches to the parallel version of the MDA.