Experience-Driven PCG via Reinforcement Learning: A Super Mario Bros
  Study
  We introduce a procedural content generation (PCG) framework at the
intersections of experience-driven PCG and PCG via reinforcement learning,
named ED(PCG)RL, EDRL in short. EDRL is able to teach RL designers to generate
endless playable levels in an online manner while respecting particular
experiences for the player as designed in the form of reward functions. The
framework is tested initially in the Super Mario Bros game. In particular, the
RL designers of Super Mario Bros generate and concatenate level segments while
considering the diversity among the segments. The correctness of the generation
is ensured by a neural net-assisted evolutionary level repairer and the
playability of the whole level is determined through AI-based testing. Our
agents in this EDRL implementation learn to maximise a quantification of
Koster's principle of fun by moderating the degree of diversity across level
segments. Moreover, we test their ability to design fun levels that are diverse
over time and playable. Our proposed framework is capable of generating
endless, playable Super Mario Bros levels with varying degrees of fun,
deviation from earlier segments, and playability. EDRL can be generalised to
any game that is built as a segment-based sequential process and features a
built-in compressed representation of its game content.
