Video (.mp4) playing in QuickTime but not in Finder Quick Look or iTunes
Conversion by QuickTime Player

QuickTime Player may be swiftly converting the file on opening using OS X's new AVFoundation. This would explain why it can play the file but not QuickLook or iTunes.



It is likely the movie is not in an ideal format for QuickTime Player on OS X 10.9, thus the conversion. With OS X 10.9, Apple dropped native support for many video formats and containers. Where possible legacy formats are converted on opening.

Supported MPEG-4

You may have a valid MPEG-4 container but the contents are not natively supported without conversion.

Have you confirmed the movie's encoding using the Finder's Get Info panel?



Re-Encoding

It is highly likely you will need to re-encode the movie, as suggested in MP4 movie does play in QuickTime but not in iTunes, if you want iTunes for playback.

This questions suggests a slightly different set of flags for ffmpeg to gain iTunes support, Convert a bunch of MKV files to MP4 to read them in iTunes:

./ffmpeg -i &lt;filename&gt;.mkv -c:v copy -c:a aac -b:a 384k -strict -2 &lt;filename&gt;.mp4


Alternatives to iTunes

If you wish to avoid re-encoding to gain iTunes support, consider looking for alternatives to iTunes. VLC and XMBC are both worthy alternatives; they both handle a wide range of video formats.