Riding wrong size frame
Frame size is one number. Bike fit is made up of many variables, some of which are fixed to the frame, some (add you've found) aren't. And people aren't all the same proportions (see for example the ape index). 

If the bike is adequately comfortable for the rides you do, it's worth sticking with. It sounds like you might have found yourself an endurance fit, so for short races it could be possible to get more efficiency. If you're not racing (even just a friendly sprint), I doubt you'll notice the difference.

Perhaps your friend is giving you an excuse to buy a new bike, but you don't need it, and in your position I certainly wouldn't get a new one if that meant losing this one that seems to suit you.