Exploiting Action Impact Regularity and Exogenous State Variables for   Offline Reinforcement Learning
Offline reinforcement learning-learning a policy from a batch of data-is known to be hard for general MDPs. In this work, we explore a restricted class of MDPs to obtain guarantees for offline reinforcement learning. The key property, which we call Action Impact Regularity (AIR), is that actions primarily impact a part of the state (an endogenous component) with limited impact on the remaining part of the state (an exogenous component). We propose an algorithm that exploits the AIR property, and provide theoretical guarantees for the outputted policy. Finally, we demonstrate that our algorithm outperforms existing offline reinforcement learning algorithms across different data collection policies in simulated and real world environments where the regularity holds.