mxfenc: Fix possible integer overflows

None of these are likely unless the user is writing a file with two billion
streams or a duration of around two months.

CC: libav-stable@libav.org
Bug-Id: CID 700568 / CID 700569 / CID 700570 /
        CID 700571 / CID 700572 / CID 700573
