Is the set of optimal strategies convex in a single-agent decision choice problem?
The set $Q^*$ is the set of probability distributions over the maximizers of $u$ for each value of $v$. So for a fixed $v$ all the values of $Y$ that have positive probability must give the same utility. That is, if $P_Y(y|v)&gt;0$ and $P_Y(y'|v)&gt;0$ then $u(y,v)=u(y',v)\geq u(\tilde y,v)$ for all $\tilde y\neq y, y'$. Therefore, convex combinations of elements in $Q^*$ must also be collections of probability distributions $P_{Y|V}$ supported over the same elements in $Y$. That is, the convex combination will also be supported over maximizers of $u$ for each $v$. 

In conclusion, even though the set of maximizers is not guaranteed to be convex, the set $Q^*$, of distributions, is a convex set.