Although the ultimate goal is to learn the optimal policy in Eq. ( ) via learning the optimal state-action value function Q(s, a), the behavioral policy-that is to say the actions taken during training-need not follow the policy that is being learned in Eq. ( ). Indeed, this is undesirable in that the agent will not explore actions which may appear sub-optimal yet may ultimately lead to the global optimal Q(s, a): this is the exploration-exploitation trade off. Q-learning, because it is off-policy (meaning it need not follow Eq. ( ) during training) has the flexibility to follow physics-inspired and potentially more efficient search policies, all the while ensuring it learns the optimal Q(s, a) and π(s).