Can I prevent the off button from locking the device? I've noticed that tapping the off button locks the device under ICS.  Are there any tricks for making it only shut down the touchscreen and wait for the timeout to actually lock the device? 

You must tap the off button to disable the screen when inserting the phone into your pocket because otherwise the phone gets spurious input, but frequently you'll want to pull the phone right back out again.
There's an option in Security settings: power button instantly locks screen.