Did I do that? Blame as a means to identify controlled effects in
  reinforcement learning
  Identifying controllable aspects of the environment has proven to be an
extraordinary intrinsic motivator to reinforcement learning agents. Despite
repeatedly achieving State-of-the-Art results, this approach has only been
studied as a proxy to a reward-based task and has not yet been evaluated on its
own. We show that solutions relying on action-prediction fail to model critical
controlled events. Humans, on the other hand, assign blame to their actions to
decide what they controlled. This work proposes Controlled Effect Network
(CEN), an unsupervised method based on counterfactual measures of blame to
identify effects on the environment controlled by the agent. CEN is evaluated
in a wide range of environments showing that it can accurately identify
controlled effects. Moreover, we demonstrate CEN's capabilities as intrinsic
motivator by integrating it in the state-of-the-art exploration method,
achieving substantially better performance than action-prediction models.
