Discussing authorship/sole authorship with supervisor
Note: things may vary by field. Single-author publications has never been an important metric in any field I've ever worked in, but can't say for sure about math.

I think your advisor should be a co-author. Here are some things to consider:

Reputation. Even if you did good work, the paper will get more circulation with an established researcher's name on it. So, you probably want him to have authorship, though you should of course be first author.

His contributions. As you say the project "would never have been born without his guidance." It sounds like he did not contribute only offhand comments, but actually proposed a line of research and helped you to follow it. 

Funding. I'm assuming he funded you to do this research. In itself, this doesn't mean he deserves authorship -- but I suspect that a lot has been going on "behind the scenes" to help you with this work. Grant proposals are a lot of work, and he probably put a bit of thought into your general research area before he even hired you. (This may not apply in your particular case, since the advisor didn't fund you.)

The bar for co-authorship is not so high. I have seen questions on this site where people have been given co-authorship for reviewing the grammar! I think this is ridiculous, but adding your advisor seems altogether appropriate. In particular, you will be first author, so it costs you very little -- and it likely means quite a bit to him. In fact, the old supervisor would have a decent case for being added as yet another author. 

Edit: OP has clarified that single-author papers are an important metric in this field. If this is true, the advisor in question surely knows this as well. So, if you want to ask for sole-authorship, you could say something like this:


  I wanted to ask about the authorship for this paper. As you know, single-author papers will be important for my career. So, I wondered if you thought it would be appropriate for me to be the sole author on this paper. 


Note, I still think adding your advisor as a co-author would be more appropriate. Perhaps a "compromise" would be to ask the question in a more neutral way, something like:


  I wanted to ask about the authorship for this paper. You obviously made a lot of contributions and maybe it makes sense for you to be a co-author. On the other hand, this paper might be a good chance for me to get a single-author paper, which as you know, I will need for my career. What do you think?