In the definition of the state-action value function, what is the random variable we take the expectation of?
I am using the convention of uppercase $X$ for random variable and lowercase $x$ for an individual observation. It is possible your source material did not do this, which might be causing your confusion. However, it is the convention used in Sutton &amp; Barto's Reinforcement Learning: An Introduction.

What I didn't understand what is  here. i.e., what is the random variable on which we are calculating long run rewards?

The random variable is $R_t$, the reward at each time step. The distribution of $R_t$ in turn depends on the distribution of $S_{t-1}$ and $A_{t-1}$ plus the policy and state progression rules. There is no need to include the process that causes the distribution of each $R_t$ in every equation. Although sometimes it is useful to do so, for example when deriving the Bellman equations for value functions.

My guess is policy function. It is averaging long run rewards over all possible policy functions. Is it true?

No, this is not true. In fact it is the more usual assumption that the policy function $\pi(a|s)$ remains constant over the expectation, and this is what the subscript $\pi$ in $\mathbb{E}_{\pi}[...]$ means.
The expectation is over randomness due to the policy $\pi$, plus randomness due to the environment, which can be described by the function $p(r, s'|s,a)$ - the probability of observing reward $r$ and next state $s'$ given starting in state $s$ and taking acion $a$. These two functions combine to create the distribution of $R_t$. It is possible that both functions are deterministic in practice, thus $R_t$ is also deterministic. However, RL theory works on the more general stochastic case, which is also used to model exploratory actions, even if the target policy and environment are deterministic.