What is the difference between mov, mpeg, and dnx hd formats?
MOV is a container i.e. a wrapper for media streams, and doesn't identify the video codec of the stream wrapped inside.

MPEG refers to a set of standards and could refer to MPEG-1 (from 1980s) or MPEG-2 (DVDs or many TV broadcast streams) or MPEG-4 Part 2 (Xvid, DivX..) or Part 10 (H.264 / AVC). Of the MPEG standards, the last one is the latest and best at preserving quality for a given file size.

DNxHD is a proprietary codec by Avid used as an intermediate format for editing on their systems. DNxHD streams are usually stored in MOV or MXF.

Just going by the bare information here, I'd say DNxHD will offer the best quality but also the largest file sizes and playback compatibility isn't broad. Depending on the video codec used in MOV, say, PNG or MJPEG (likely), that may be the best option in terms of quality and compatibility but you should get details first.