Can you connect an output pin from one Arduino to the input pin of another? I have a system that uses two Arduino boards.  One is used to control several different functions, and the other is dedicated to driving a 45 second countdown display (comprised of two large 7 segment displays.)  All this second Arduino needs is a single signal to tell it when to start.  I would like to simply connect an output pin (on #1) to an input pin (on #2) and and send a single pulse to get the clock started.  Fo something this simple, I wanted to avoid using I2C.  Can I do this?  If so, is there anything else I need to be aware of (like connecting ground to ground?)
Yes, you can. That is indeed the most easy way of communication.

And like you said, you have to connect the grounds of both arduino's. 

Optionally you could add a pull-up of pull-down resistor, so the countdown doesn't accidentally start if the first arduino is still starting up, or is disconnected.