Combining Difficulty Ranking with Multi-Armed Bandits to Sequence
  Educational Content
  As e-learning systems become more prevalent, there is a growing need for them
to accommodate individual differences between students. This paper addresses
the problem of how to personalize educational content to students in order to
maximize their learning gains over time. We present a new computational
approach to this problem called MAPLE (Multi-Armed Bandits based
Personalization for Learning Environments) that combines difficulty ranking
with multi-armed bandits. Given a set of target questions MAPLE estimates the
expected learning gains for each question and uses an exploration-exploitation
strategy to choose the next question to pose to the student. It maintains a
personalized ranking over the difficulties of question in the target set which
is used in two ways: First, to obtain initial estimates over the learning gains
for the set of questions. Second, to update the estimates over time based on
the students responses. We show in simulations that MAPLE was able to improve
students' learning gains compared to approaches that sequence questions in
increasing level of difficulty, or rely on content experts. When implemented in
a live e-learning system in the wild, MAPLE showed promising results. This work
demonstrates the efficacy of using stochastic approaches to the sequencing
problem when augmented with information about question difficulty.
