Purpose of test data in binary classification
One of the well known problems of machine learning is overfitting. The main reason for having a test data set is to 


Obtain a realistic measure of how good your model is doing.
Prevent overfiting the validation set.


Now think about the second point, quite often it might occur that you tune up the hyper parameters so that your validation set provides a good measure, but this means you might be overfitting your validation set and when you deploy your shiny model in the real world, boom! poor results are returned.

As per the amount of data to reserve for testing, it depends on how big is your data set, normally the rule of 60% for training, 20% for validation and 20% for test has been used, but if your data set has 5 millions of points, you probably just need 1% for testing (as that will provide you with 5000 examples for testing anyway).

So, yes, your last question makes sense, you will probably be fine taking a small portion of the original data set (again, say 1%) in order to get good results.