For hosts Forfar Athletic it was a second successive home 3-1 defeat after a run of nine straight wins.
Draper put the Welsh champions ahead after 24 minutes only for Stuart Malcolm to reply after the break against the run of play.
Forfar were on top after the break, but two Draper strikes won the tie.
The Welsh side who started their season in the Champions League qualifiers came to Forfar in confident form with 11 straight wins behind them and they looked the more imposing side in the opening 20 minutes.
Alex Darlington, who had scored both goals in a midweek domestic win, had the first opportunity of the match, but goalkeeper Grant Adam got down smartly to save his 20-yard effort.
TNS wasted another chance as Aeron Edwards came into the box unmarked to meet a corner but sent his header back across goal and off-target.
The Welsh side were dominating possession and the Scottish League Two leaders were struggling to link with Jim Lister and Josh Peters, who were trying to provide an attacking threat.
TNS took the lead after 23 minutes, but it was tinged by controversy.
Forfar were convinced they had won a corner, but a goal-kick was awarded that was swiftly taken and a defensive mis-header found its way to  Draper, who controlled the ball and slipped it beyond Adam.
TNS wasted two further chances to add to their lead before Forfar hit back.
Peters forced a save from Paul Harrison in the TNS goal, but from the resultant corner, Malcolm powered a header in off the bar.
Control switched to Forfar after the break as the home side enjoyed better possession and created the better opportunities.
Jim Lister was twice denied in the same move, with Phil Baker blocking his second effort, while goalkeeper Paul Harrison produced a fingertip save to deny Jamie Bain.
Forfar continued to press and a powerful Martyn Fotheringham free-kick just finished the wrong side of the post.
TNS had shown little in an attacking sense in the second half had the inspiration of substitute Robbie Parry to thank for the goal that put them ahead.
He finished his run into the box by shooting past Adam and Draper netted.
Draper, with one New Zealand cap to his name, secured his hat-trick three minutes from the end when Adam misjudged a long ball and the striker took it past him at the edge of the box to secure victory.
Greg Draper grabbed a hat-trick as The New Saints made history by becoming the first non-Scottish side into the Scottish Challenge Cup quarter-finals.