How should I take into consideration the number of steps in the reward function?
How would you implement this &quot;Number of Steps&quot; cost?

What the paper is referring to is the reward discounting process which is a standard way of formulating RL problems, either continuous ones, or episodic ones where the goal is to complete a task in the least time (in the episodic version, a fixed cost per time step will also achieve this).
As such, this usually is implemented in the formulation of the value function calculations. The discount factor is usually represented as gamma, $\gamma$.
For Q-learning, the factor should be in the TD target calculation:
$$G_{t:t+1} = R_{t+1} + \gamma \max_{a'}Q(S_{t+1},a')$$
For Monte Carlo control, the factor appears more like this in the calculation of a return:
$$G_t = \sum_{k=0}^{T-t} \gamma^k R_{t+k+1}$$

would it be best to use an exponential functions to discount the reward at the current time step?

Essentially that is what normal discounting is - an exponential decay of future reward. But if you have implemented &quot;normal&quot; Q-learning from equations like those above, it should already be there.