1. A predictive action determination apparatus comprising: a state observation section for observing a state with respect to a predetermined environment and obtaining state data; a state value storage section for storing a state value for each of states of the environment; an environment prediction section for predicting a future state change in the environment, based on the state data obtained by the state observation section; a target state determination section for determining, as a target state, a future state suitable for action determination among future states predicted by the environment prediction section, based on the state value for each of future states stored in the state value storage section; and a first action determination section for determining an action of the apparatus, based on the target state determined by the target state determination section, wherein the environment prediction section predicts a future state change in the environment, which is not influenced by actions of the apparatus.