What does the symbol $\mathbb E$ mean in these equations?
That's the Expected Value operator. Intuitively, it gives you the value that you would "expect" ("on average") the expression after it (often in square or other brackets) to have. Typically that expression involves some random variables, which means that there may be a wide range of different values the expression may take in any concrete, single event. Taking the expectation basically means that you "average" over all the values the expression could potentially take, appropriately weighted by the probabilities with which certain events occur.

You'll often find assumptions under which the expectation is taken after a vertical line ($\mid$) inside the brackets, and/or in the subscript to the right of the $\mathbb{E}$ symbol. Sometimes, some assumptions may also be left implicit.



For example:

$$\mathbb{E} \left[ R_t \mid s_t = s, a_t = a, \pi \right]$$

may be read in english as "the expected value of the discounted returns from time $t$ onwards ($R_t$), given that the state at time $t$ is $s$ ($s_t = s$), given that our action at time $t$ is $a$ ($a_t = a$), given that we continue behaving according to policy $\pi$ after time $t$".

I would say that, in this case, the expected value also relies on the transition dynamics of the Markov decision process (i.e. the probabilities with which we transition between states, given our actions). This is left implicit.



Second example:

$$V^{\pi}(s) = \mathbb{E}_{a \sim \pi(s)} \left[ Q^{\pi}(s, a) \right]$$

may be read as "$V^{\pi}(s)$ is equal to the expected value of $Q^{\pi}(s, a)$, under the assumption that $a$ is sampled according to $\pi(s)$". 

In theory, something like this could be computed by enumerating over all possible $a$, computing $Q^{\pi}(s, a)$ for every such $a$, and multiplying it by the probability that $\pi(s)$ assigns to $a$. In practice, you could also approximate it by running a large number of experiments in which you sample $a$ from $\pi(s)$, and then evaluate a single concrete case of $Q^{\pi}(s, a)$, and average over all the evaluations.