Has the Random Forest algorithm ever been used in Reinforcement Learning applications?
I expect someone somewhere has used a RF estimator inside RL to approximate action values, if only to assess it as a comparison to other function approximators. However, it does look like from a web search that this is not used widely, and I could not find an example either.

The main problem with a RL/RF hybrid with RF as the value estimator is that the random forest base algorithm is not an online one - it works across a finalised data set and processes the whole batch in order to do things like bagging. Even when used as an estimator using experience replay, support for online learning is a desirable feature, and values are typically fed into supervised learning part in small or medium batches. That is because the action values that are learned by the internal estimation function in reinforcement learning are non-stationary. 

Once you have learned the action values for the current policy, and in most RL well before estimates for the values have converged, you change the policy. This changes the expected action values, so your estimator must be able to forget older data and bias towards most recent values. Algorithms that can be made to work online can do this, purely offline ones cannot.

However, there are online Random Forests which have the necessary alterations to work with non-stationary data. I don't see any reason why they could not work. Decision trees in general have been used successfully as estimators in RL - see Reinforcement Learning with Decision Trees. It looks like that in principle it could be done.