The Spaniard, 31, lost the first set but fought back to win 6-7 (3-7) 6-3 6-1 6-4 after three hours and 16 minutes.
Nadal saw 13 break points go begging before finally breaking the world number 59 after nearly two hours.
He dropped just five more games on his way to a last-16 meeting with Ukraine's Alexandr Dolgopolov.
"All the matches have been very tight - I am very happy," said Nadal.
"I don't know how many break points I had before I got the first one. Mentally, I fought a lot until that moment, then I think I played better."
Nadal could not land a telling return as the break points came and went against Mayer, and the Argentine took advantage by playing a fine tie-break to take the first set.
I am in the fourth round now. I don't think much about that.
When Nadal finally made the breakthrough at the 14th attempt, at 3-3 in the second set, he screamed "Vamos!" and his family and team celebrated in the stands.
The Spaniard had won all three previous matches against Mayer and went on to dominate after finally breaking.
There was a late flash of resistance when Mayer broke serve with Nadal trying to close out the match, but two games later the world number one sealed victory at 8.40pm local time.
Asked about a potential first-ever meeting with 19-time Grand Slam champion Roger Federer at the US Open in the semi-finals next week, he said: "I am in the fourth round now. I don't think much about that.
"It's a long way away. All the matches have been very tough.
"The only thing I am looking for is the practice tomorrow to keep working hard and focus on the match against Dolgopolov."
Third seed Federer booked his place in the last 16 with a clinical 6-3 6-3 7-5 victory over Spain's Feliciano Lopez.
Top seed Rafael Nadal overcame stubborn resistance from Argentina's Leonardo Mayer to reach the fourth round at the US Open in New York.