How can I make a durable connection between a PCB and outside of case?
Well, you could buy an LED like this with wires attached, and attach a lockable connector to the wires for the end user to connect.

Or you could use a board mounted LED and then use a light pipe to get the light out of the case.