1. A reinforcement learning system for making an agent learn an action policy for executing a task, the reinforcement learning system is a computer including a processing unit which reads software from a memory, and the reinforcement learning system comprises: an environment recognizing device configured to recognize a first to an nth (n≧2) state variables representing an environment; n numbers of jth (j=1, 2,. .. n) learning devices, each jth learning device configured to calculate a jth reward based on a jth state variable recognized by the environment recognizing device, calculate a jth value according to a jth value function based on the jth state variable, calculate a jth error based on the jth reward and the jth value, and properly modify respectively the jth value function based on the jth error; and an action policy determining device configured to determine an action policy the agent should take based on each jth reward calculated by each jth learning device, wherein each (i+1)th (i=1, 2,. .. n−1) learning device is configured to calculate an (i+1)th reward based on a value of an ith reward function related to an ith state variable and a value of an ith value gradient function which is a temporal differential of an ith value function.