Combining Offline Models and Online Monte-Carlo Tree Search for Planning
  from Scratch
  Planning in stochastic and partially observable environments is a central
issue in artificial intelligence. One commonly used technique for solving such
a problem is by constructing an accurate model firstly. Although some recent
approaches have been proposed for learning optimal behaviour under model
uncertainty, prior knowledge about the environment is still needed to guarantee
the performance of the proposed algorithms. With the benefits of the Predictive
State Representations~(PSRs) approach for state representation and model
prediction, in this paper, we introduce an approach for planning from scratch,
where an offline PSR model is firstly learned and then combined with online
Monte-Carlo tree search for planning with model uncertainty. By comparing with
the state-of-the-art approach of planning with model uncertainty, we
demonstrated the effectiveness of the proposed approaches along with the proof
of their convergence. The effectiveness and scalability of our proposed
approach are also tested on the RockSample problem, which are infeasible for
the state-of-the-art BA-POMDP based approaches.
