Understanding GLIE conditions for epsilon greedy approach
For (Q1) we mathematically define $\mathbb{E}[exploit(T)]$ as:
$$\mathbb{E}[exploit(T)] = \sum_{t=1}^T (1-\epsilon_t)$$
where $\epsilon_t$ is the exploration probability at round $t$.
The equation is basically telling us the expected number of times you pull a greedy arm over the time horizon $T$.
A more useful metric would be:
$$\mathbb{E}[explore(T)] = \sum_{t=1}^T \epsilon_t$$
It os obvious that:
$$\mathbb{E}[explore(T)] + \mathbb{E}[exploit(T)] = T$$
First, I would like to gloss over the idea why this formulation produces sublinear regret. Although there are additional assumptions that have clearly not been mentioned in the lecture, I will say it here. There is a paradigm of a 'consistent' algorithm in online learning, which in simple terms mean that the actual best arm will be identified eventually as $T \to \infty$ and by the design of your algorithm, you will be pulling this arm greedily (or with a high probability) after some time giving you a regret better than linear regret.
Now in this case, if you assume the algorithm is 'consistent' it is clear that the number of exploration steps (which is decided by exploration probability) at each step will slowly have a reduced growth as compared to the linear case i.e you are now slowly reducing exploration as your 'consistent' algorithm gurantees you have with high probability identified the best arm. From this definition/intuition given above it is clear that this means that $\mathbb{E}[explore(T)]$ should grow at a slower rate while $\mathbb{E}[exploit(T)]$ should grow at a faster rate with $T$.
From this inutitive idea rises that:
$$\lim_{T\rightarrow\infty}\frac{\mathbb{E}[exploit(T)]}{T}=1 $$
or
$$\lim_{T\rightarrow\infty}\frac{\mathbb{E}[explore(T)]}{T}=0 $$
For (Q2), it is not. It might be in expectation, but in general, it is not. I am unsure of the type of bandits used here so I will not do this calculation. In anyways, whenever sampling is involved, it means you have sampled a trajectory or a particular instance of a joint probability distribution and its expectation can never be equal to one sample (we need $\infty$ samples).(Although I am not clear on this one, the slides are not very clear to me).
For (Q3) Yes. The instructor states an inequality which I could not reproduce (I tried to do it, but it reverses the inequality sign of the instructor, you can try it yourself using the below set of equations and using the fact that $\epsilon_T \leq \epsilon_t, \quad \forall T \geq t$). Now consider the following set of equations:
$$\mathbb{E}[explore(T)] = \sum_{t=1}^T \epsilon_t$$
(a good approximate summation can be found using integrals like this)
$$\mathbb{E}[explore(T)] \approx k \sqrt T$$
where $k$ is a constant.
$$-\mathbb{E}[explore(T)] \approx -k \sqrt T$$
$$T-\mathbb{E}[explore(T)] \approx T - k \sqrt T$$
$$\frac{\mathbb{E}[exploit(T)]}{T} \approx 1-k \frac{1}{\sqrt T}$$
which answers your (Q4) that:
$$\lim_{T \to \infty} \frac{\mathbb{E}[exploit(T)]}{T} \approx \lim_{T \to \infty} 1-k \frac{1}{\sqrt T} = 1$$