Is it possible to watch mp4s on an iPod Nano 6th Gen?
The 6th Generation iPod Nano does not support video decoding or playback1. 

However, it can demux video files and play any supported audio streams in those files2. Instead of the video stream, it displays a single keyframe. This is useful for music videos or podcasts, where you might still want to listen to the audio only, but it's not the same thing as full video playback.

Note that the 5th- and 7th-gen both support video decoding and playback3; its omission from the 6th-gen seems to be an anomaly.


1 Sources: everymac.com, apple.com, wikipedia.org.
2 Source: ilounge.com.
3 Sources: 5th-gen, 7th-gen. For those devices, handbrake is a great option, as per @iamkrillin's answer.