Recognising Noise in Simple Classification
The network classifies perfectly with validation data, however I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently. 


You need to train the network on noise well if you want it to be able to recognize it.


  or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold?


You can use spot to get a confidence score for clarification. But training the network on a noise class will work way better

See:


  Gal: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning