The result was further embarrassment for Michael O'Neill's side after their 3-2 defeat by Luxembourg last month.
Paterson was among Northern Ireland players to miss great chances before the hosts scored two second-half goals.
"If you watched that game you're probably scratching your head wondering how have we lost 2-0. We are," he said.
Media playback is not supported on this device
"People might see the result and say 'Northern Ireland lost 2-0, what a poor result that is' but they don't see how we've played.
"Saying that, we've only got ourselves to blame. We dominated the game for 45 minutes but didn't take our chances and their goalkeeper played very well.
"Then we just switched off for a couple of minutes, conceded, and the game changes.
"In the second half I hit the bar - it's a game of inches - and their second goal was just frustration."
Another night of Northern Ireland ignominy ended with Jonny Evans being sent off for a crude foul, with Mahir Shukurov punishing the foul by netting the free-kick.
After a night that proved a body blow to the team's hopes of a bright finish to their campaign, Paterson still felt able to tentatively look on the bright side.
With two points taken from a possible 12 against Luxembourg and Azerbaijan in the campaign, that might seem a stretch, but the 26-year-old gave it his best.
Media playback is not supported on this device
"There are positives to take, as mad as that sounds," said the Huddersfield striker.
"I don't think we're miles away but it's a fine line. We had four or five good chances, they got one break and scored: that's international football.
"But we are still a young group. We've got to learn quickly and turn results around.
"If we do we have a chance of really pushing in the next campaign, but we have to work harder as a unit."
Manager Michael O'Neill, who cut a frustrated figure in his post-match media duties, now faces the unenviable challenge of lifting a depleted squad for Tuesday's final match in Israel.
Joining Evans in being suspended for that match are defensive talisman Gareth McAuley and midfielder Oliver Norwood, who each picked up their second bookings of the campaign.
Northern Ireland are unlikely to add to their squad in Tel Aviv, leaving O'Neill to sift through his current group for answers.
"We have three players out but we also have men to come in - particularly the likes of Sammy Clingan and Chris Baird," added O'Neill.
"We'll have to look at the centre-back situation because of the players we've lost to suspension and that is self inflicted."
Striker Martin Paterson said Northern Ireland were left dumbfounded by their latest World Cup qualifying humiliation as they lost 2-0 in Azerbaijan.