What does the term $|\mathcal{A}(s)|$ mean in the $\epsilon$-greedy policy?
This expression: $|\mathcal{A}(s)|$ means

$|\quad|$ the size of

$\mathcal{A}(s)$ the set of actions in state $s$


or more simply the number of actions allowed in the state.
This makes sense in the given formula because $\frac{\epsilon}{|\mathcal{A}(s)|}$ is then the probability of taking each exploratory action in an $\epsilon$-greedy policy. The overall expression is the expected return when following that policy, summing expected results from the exploratory and greedy action.