If the transition model is available, why would we use sample-based algorithms? Sample-based algorithms, like Monte Carlo Algorithms and TD-Learning, are often presented as useful since they do not require a transition model.
Assuming I do have access to a transition model, are there any reasons one might want to use sample-based methods instead of performing a full Bellman update?
A full Bellman update can be intractable. For instance, if your state space or action space are continuous, the full Bellman update is intractable. You can try to solve this by discretizing, but if your state space is large this will also be intractable.