Personal Universes: A Solution to the Multi-Agent Value Alignment
  Problem
  AI Safety researchers attempting to align values of highly capable
intelligent systems with those of humanity face a number of challenges
including personal value extraction, multi-agent value merger and finally
in-silico encoding. State-of-the-art research in value alignment shows
difficulties in every stage in this process, but merger of incompatible
preferences is a particularly difficult challenge to overcome. In this paper we
assume that the value extraction problem will be solved and propose a possible
way to implement an AI solution which optimally aligns with individual
preferences of each user. We conclude by analyzing benefits and limitations of
the proposed approach.
