Resilience engineering is a recent paradigm for the development, analysis and control of systems that interact with their environment and are subject to perturbances or part failures. Resilience engineering has many facets, some of them being well studied in control engineering like fault tolerance or robust control. In this paper, we propose a mathematical model that considers the following aspects relevant to resilience engineering: uncertainty, autonomy, and system-environment interaction. The model extends stochastic hybrid systems with Markov decision processes to capture system autonomy, and game theory to capture the system-environment interaction. For this model, we consider the state-constrained reachability problem as defined for stochastic hybrid systems. We give a characterization of the solutions of this problem in terms of the value function of an ergodic stochastic game.