How to go from clamshell to sleep
Your MacBook Pro is waking up when you disconnect the Display Port and Thunderbolt connections. This happens because OS X needs to re-evaluate its hardware situation.

Consider using a context-sensitive automation tool like ControlPlane. When you disconnect your display or thunderbolt network, have ControlPlane trigger sleep mode.

Alternatively, try reducing the required period of inactivity before sleeping: System Preferences > Energy Saver