Width-based Lookaheads with Learnt Base Policies and Heuristics Over the
  Atari-2600 Benchmark
  We propose new width-based planning and learning algorithms inspired from a
careful analysis of the design decisions made by previous width-based planners.
The algorithms are applied over the Atari-2600 games and our best performing
algorithm, Novelty guided Critical Path Learning (N-CPL), outperforms the
previously introduced width-based planning and learning algorithms $\pi$-IW(1),
$\pi$-IW(1)+ and $\pi$-HIW(n, 1). Furthermore, we present a taxonomy of the
Atari-2600 games according to some of their defining characteristics. This
analysis of the games provides further insight into the behaviour and
performance of the algorithms introduced. Namely, for games with large
branching factors, and games with sparse meaningful rewards, N-CPL outperforms
$\pi$-IW, $\pi$-IW(1)+ and $\pi$-HIW(n, 1).
