What are features for state-action pairs in RL?
In the cartpole example, a state-action feature could be

$$\begin{bmatrix}
\text{Cart Position}\\
\text{Cart Velocity}\\
\text{Pole Angle}\\
\text{Pole Tip Velocity}\\
\text{Action}
\end{bmatrix}$$

where Action is either left, right, or do nothing. The reward is not part of the feature vector because reward does not describe the state of the agent; it is not an input. It is a (possibly stochastic) signal received from the environment that the agent is trying to predict/control with the use of feature vectors.