Should I be concerned when an editor tells me that I have the final say about the paper he sent me to referee?
What you could do is locate a second reviewer, say a colleague, yourself. Explain the situation and the paper (in general terms) and ask if they could give a review. Then write to the editor


  I found that the authors addressed most of my concerns. However, there are still a few points I am unsure of and would like a second opinion on. I have spoken to Researcher B (without disclosing the paper) and he/she is willing to give me some input. May I share the paper with him/her?


Even for mid-tier journals, a single reviewer isn't unheard of, especially when the topic is specialized and as you have said you are an authority on the topic. What usually happens is that the editor doesn't tell you about their plans.