How does Arduino distinguish between SPI programming data and regular data?
In-system programmer is activated with RESET line. During normal operation reset line is not active and ISP module is disabled.

You can find detailed description of AVR ISP in this document:

http://www.atmel.com/images/doc0943.pdf