Should I patch or do the whole wall?
A patch will be virtually undetectable if done properly. Professional plasterers do often skim the entire wall but there is no reason to texture the entire wall if you do not want to.

Using texture is an easy way to hide imperfections, so is a great idea if you (or your contractor) do not want to spend the time and effort (and $$; time = $$) to patch it properly. Call a different "drywall guy", preferably a reputable professional plaster company and one that doesn't leave unscrewed floating corners like in top pic.