When to NOT use Totem Pole Drivers?
When not to use totem-poles:  


You can't use totem-poles for wired AND (often called wired OR, but most often they're ANDs). If one output is high and the other low you get a short. Always use open collector/open drain drivers for wired AND.  
TTL totem-poles are highly asymmetrical: they can hardly source current, typically 0.4mA versus 16mA when sinking. So don't use them when you need to both source and sink current. CMOS totem-poles are more symmetrical, and don't suffer from this.  
When the load you're driving goes beyond \$V_{CC}\$, like for instance a low-power relay which needs +12V on a TTL output. Sinking current the output will be low as expected, but when the output is high it will see the +12V, which will damage the part.
When the input you connect to uses a wildly different supply voltage. If for example the output would switch between 0 and +1.5V it wouldn't be directly usable as input on a 15V system. By using an open collector the pull-up can be connected to the +15V, and the output levels would be automagically according to that system. Important: Many new low-voltage logic families seem to have clamping diodes on their outputs, even open drain, so that you can't go beyond \$V_{DD}\$. I guess it has to do with the low-voltage process, but it does take away the open drain's most interesting feature.