What is the difference between expected return and value function?
There is a strong relationship between a value function and a return. Namely that a value function calculates the expected return from being in a certain state, or taking a specific action in a specific state. A value function is not a "return function", it is an "expected return function" and that is an important difference.

A return is a measured value (or a random variable, when discussed in the abstract) representing the actual (discounted) sum of rewards seen following a specific state or state/action pair.

Typically there is no need to express an individual return as a "return function", although you may find many formulae in RL for sampling or estimating specific return values in order to calculate targets or errors for the value function.


  A return (or cumulative reward) function effectively adds up the rewards from the current state to the goal state. This implies it's model-based.


If you have a simple MDP, already accurately modelled, where you can calculate expected return directly from that model, then, yes, in theory, that would be a value function. However, this could be more computationally intensive to resolve than dynamic programming (e.g. Policy Iteration or Value Iteration), and in many cases you don't have any such model, but can still apply RL approaches to learn a value function from experience.