That's not to say it doesn't have a key signature. Lets say it has two sharps but it started on G#, ended on Bb and wasn't diatonic. What is it called then?
you'd have to listen to it to tell. just like you'd probably have to look at a picture of a face to be able to tell which way is up. you will intuitively know when you look at it, even if you can't describe how you figured it out