Reviewers had questions about why my supervisor wasn't on manuscript
Just reply with the facts: Explain that your supervisor was offered coauthorship, but has chosen not to be a coauthor of the paper.  (If you feel you want to elabore why - e.g. since he felt he did not contribute sufficiently to the paper, or the like - feel free to do so.)
Especially if this is listed under minor comments, take it as such.
In any case, it is generally not up to the referee to judge who should be a coauthor of a paper. What can be demanded is that you offer coauthorships to anyone who made a significant enough contribution to the paper. As long as you did that, you are fine.