Size of the language accepted by a Turing Machine
... what does $|L(M)| \leq 10$ mean?


This means that the size of the language accepted by the TM $M$ is less than or equal to $10$. In other words, TM $M$ accepts no more than $10$ strings (inputs), while the length of any accepted input may be greater than $10$.