Minimizing Trimmed Distances
Your loss function is proportional to the $\gamma n$ smallest square differences with respect to $\mu$.

This is minimized by sorting your $x_i$ and choosing the mean of the contiguous substring $x_k, x_{k+1}, \dots, x_{k+\gamma n-1}$with the smallest variance.