1. A method for selecting an action to be performed by an agent that interacts with an environment by performing actions selected from a set of actions, the method comprising: receiving a current observation, the current observation being data that characterizes a current state of the environment; for each action in the set of actions: determining, in accordance with a value function representation and from the action and a current state representation for the current state derived from the current observation, a respective value function estimate that is an estimate of a return resulting from the agent performing the action in response to the current observation, wherein the return is a function of future rewards received in response to the agent performing actions to interact with the environment, determining, in accordance with a confidence function representation and from the current state representation and the action, a respective confidence score that is a measure of confidence that the respective value function estimate for the action is an accurate estimate of the return that will result from the agent performing the action in response to the current observation, adjusting the respective value function estimate for the action using the respective confidence score for the action to determine a respective adjusted value function estimate, wherein the adjusted value function estimate p t (s t ,a t ) for the action a t when the environment is in the current state t satisfies: 
 p t ( s t ,a t )=( Q ( s t ,a t )− Q min )× c ( s t ,a t ), and wherein s t is the current state representation, Q(s t ,a t ) is the value function estimate for the action a t , Q min is a predetermined minimal possible value function estimate for any action, and c(s t ,a t ) is the confidence score for the action a t ; selecting an action to be performed by the agent in response to the current observation using the respective adjusted value function estimates; and performing, by the agent, the selected action to interact with the environment.