How to understand the average l2 loss?
I agree that this notation is unclear. I would interpret it as follows: 

Given that the expression is supposed to denote the average norm $|p_i|$ is likely the cardinality of the set $\{p_i\}$. 

In that case the expression would just be the sum over all norms divided by the number of norms, resulting in the average norm. The authors likely use this notation because they didn't want to introduce the number $n$ of $p_{i &lt; n}$. $|\{p_i\}|$ would be clearer, but maybe uglier to typeset.