What did my referee mean by asking about my "Plan B"?
Don't overthink this.  It sounds like he's just showing a friendly interest in your plans.  No matter how "good" you are, graduate admissions always has some degree of uncertainty, and so everybody should consider what they will do if they don't get admitted.  He just wants to know what that would be for you.

If he really didn't think you were good enough, he wouldn't have agreed to be a reference for you.