DeepMind's releases DreamerV3 - first to mine diamonds in Minecraft without help
Mastering Diverse Domains through World Models

Abstract

Developing a general algorithm that learns to solve tasks across a wide range
of applications has been a fundamental challenge in artificial intelligence.
Although current reinforcement learning algorithms can be readily applied to
tasks similar to what they have been developed for, configuring them for new
application domains requires significant human expertise and experimentation.
We present DreamerV3, a general algorithm that outperforms specialized methods
across over 150 diverse tasks, with a single configuration. Dreamer learns a
model of the environment and improves its behavior by imagining future
scenarios. Robustness techniques based on normalization, balancing, and
transformations enable stable learning across domains. Applied out of the box,
Dreamer is the first algorithm to collect diamonds in Minecraft from scratch
without human data or curricula. This achievement has been posed as a
significant challenge in artificial intelligence that requires exploring
farsighted strategies from pixels and sparse rewards in an open world. Our work
allows solving challenging control problems without extensive experimentation,
making reinforcement learning broadly applicable.

Benchmarks

Using fixed hyperparameters across all domains, Dreamer outperforms tuned
expert algorithms across a wide range of benchmarks and data budgets. Dreamer
also substantially outperforms a high-quality implementation of the widely
applicable PPO algorithm. Dreamer overcome the need for expert knowledge and
extensive experimentation when solving new problems, making reinforcement
learning broadly applicable.



Scaling

Due to its robustness, Dreamer shows favorable scaling properties. Notably,
using larger models consistently increases not only its final performance but
also its data-efficiency. Increasing the number of gradient steps further
increases data efficiency. Dreamer thus offers a predictable way for
practitioners to increase performance by increasing computational resources.



Minecraft

Collecting diamonds in the popular videogame Minecraft without human data has
been widely recognized as a milestone for artificial intelligence, because of
the sparse rewards, exploration difficulty, and long time horizons in this
procedurally generated open-world environment. Dreamer is the first algorithm
that collects diamonds in Minecraft from sparse rewards, without expert
demonstrations or curricula, solving this challenge. The video shows the first
diamond that it collects, which happens at 30M environment steps or 17 days of
playtime.





Below, we show uncut videos of runs during which Dreamer collected diamonds. We
find that it succeeds across many starting conditions, which requires searching
the world for trees, swimming across lakes, and traversing mountains. Note that
a reward is provided only for the first diamond per episode, so the agent is
not incentiviced to pick up additional diamonds.






Start Videos






  Import AI
  MarkTechPost
  Analytics India Mag
  Edan Meyer YouTube
  InfoQ
  Papers Read on AI Podcast


