Runtime-Safety-Guided Policy Repair
  We study the problem of policy repair for learning-based control policies in
safety-critical settings. We consider an architecture where a high-performance
learning-based control policy (e.g. one trained as a neural network) is paired
with a model-based safety controller. The safety controller is endowed with the
abilities to predict whether the trained policy will lead the system to an
unsafe state, and take over control when necessary. While this architecture can
provide added safety assurances, intermittent and frequent switching between
the trained policy and the safety controller can result in undesirable
behaviors and reduced performance. We propose to reduce or even eliminate
control switching by `repairing' the trained policy based on runtime data
produced by the safety controller in a way that deviates minimally from the
original policy. The key idea behind our approach is the formulation of a
trajectory optimization problem that allows the joint reasoning of policy
update and safety constraints. Experimental results demonstrate that our
approach is effective even when the system model in the safety controller is
unknown and only approximated.
