What are the popular approaches to estimating the Q-function?
There's are some solutions to calculating q-values; find the exact values:

Brute-force the action sequence to find max (not pratical)
Do recursion on Bellman equation to get max (the same like action sequence brute-force, not pratical)

Estimate the q-values:

Based on different problems to solve, apply some classic algorithms or human logics to estimate; during estimation, some heuristic tactics may be used
Do a lot of randomisation to find max, including Monte Carlo tree search

Gradually optimise the q-values:
(1) Do optimisation based on Bellman equation (Q-learning):
$$ q_t(s_t,a_t) = q_t(s_t,a_t) + \alpha(r + \gamma\times\max(q_{t+1}(s_{t+1},a_{t+1})) - q_t(s_t,a_t))$$
Bellman equation is true when the temporal difference (the part multiplied with $\alpha$) reaches zero, which means the max of time t+1 reaches exact value.
(2) Do optimisation based on Bellman equation (Q-network), fit the neural network to expected value:
$$r + \gamma\times\max(q_{t+1}(s_{t+1},a_{t+1}))$$