Why does it make sense to study MDPs with finite state and action spaces? In the standard Markov Decision Process (MDP) formalization of the reinforcement-learning (RL) problem (Sutton &amp; Barto, 1998), a decision maker interacts with an environment consisting of finite state and action spaces.


This is an extract from this paper, although it has nothing to do with the paper's content per se (just a small part of the introduction).

Could someone please explain why it makes sense to study finite state and action spaces? 

In the real world, we might not be able to restrict ourselves to a finite number of states and actions! Thinking of humans as RL agents, this really doesn't make sense.
In addition to the reason outlined in the comment, also note that if the state-space and action-space are both finite and of feasible size, tabular methods can be used, and there are some advantages to them (like the existence of convergence guarantees and generally a smaller number of hyperparameters to tune).