Is Q-Learning ever better than Brute Force?
Reinforcement Learning (RL) algorithms are useful in a setting when the structure of the environment is unknown and/or stochastic. The environment is normally formalized as a Markov Decision Process (MDP) or Partially Observable Markov Decision Process (POMDP) and the goal of an RL agent is then to maximize its cumulative reward (or minimize its regret with respect to the best alternate policy) while behaving in the environment. 

As such it isn't clear how a brute force algorithm would efficiently deal with the stochastic nature of the environment or the fact that this procedure is almost universally assumed to be done online. For more information and a modern overview of RL I recommend this excellent reference by Csaba Szepesvari.