title: FFmpeg encoding produces slightly incompatible MKV/MP4 container
body: <p>I've been using the FFmpeg libraries to read and write media files using the C API.</p>
  
  <p>So far, reading seems to be pretty straightforward. I am able to read frames which I can then process, convert to RGB, process, and then convert back to YUV420 to be encoded.</p>
  
  <p>The encoded files play back with VLC media player fine, and Windows Media Player if I have a codec pack installed. However, they do behave strangely: the stock Windows 10 player won't play them, same for Adobe Premiere. Also thumbnailers don't work on it. 
  Basically it seems like nothing other than VLC or FFmpeg itself can play/process the file. I have seen this with both MP4 and MKV, so it is not a format-specific issue.</p>
  
  <p>The problems go away once you remux the file with FFmpeg, for example "ffmpeg -i input.mkv -c copy output.mkv". Everything can play the file correctly.
  Also, the "remuxing.c" sample from the official samples works as well, with the same library version and compilers that I'm using (Visual Studio 2017, FFmpeg compiled with MinGW). It will fix the file and make it playable in all software.</p>
  
  <p>I'm not sure what could be causing this. I also don't understand what the remuxing "fixed". It must be a container issue as the frames aren't touched by remuxing.
  I have analysed the output MKVs with FFprobe -show_packets. It seems to have budged the packet timestamps a little constant factor, and the output stream now has 
  is_avc=true and nal_length_size=4 instead of is_avc=false and nal_length_size=0, but apart from that the files are identical.</p>
  
  <p>Now here's the output of FFprobe with the 3 last test packets, stream info and format info for both streams. As you can see, they are identical except for a couple of field. But something in here must have been "fixed" during remuxing to make it work.</p>
  
  <pre><code>[PACKET]
  codec_type=video
  stream_index=0
  pts=59050
  pts_time=59.050000
  dts=58890
  dts_time=58.890000
  duration=1
  duration_time=0.001000
  convergence_duration=N/A
  convergence_duration_time=N/A
  size=427
  pos=277358
  flags=__
  [/PACKET]
  [PACKET]
  codec_type=video
  stream_index=0
  pts=58970
  pts_time=58.970000
  dts=58970
  dts_time=58.970000
  duration=1
  duration_time=0.001000
  convergence_duration=N/A
  convergence_duration_time=N/A
  size=205
  pos=277792
  flags=__
  [/PACKET]
  [PACKET]
  codec_type=video
  stream_index=0
  pts=59130
  pts_time=59.130000
  dts=59050
  dts_time=59.050000
  duration=1
  duration_time=0.001000
  convergence_duration=N/A
  convergence_duration_time=N/A
  size=268
  pos=278004
  flags=__
  [/PACKET]
  [STREAM]
  index=0
  codec_name=h264
  codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
  profile=Main
  codec_type=video
  codec_time_base=1/2000
  codec_tag_string=[0][0][0][0]
  codec_tag=0x0000
  width=720
  height=576
  coded_width=720
  coded_height=576
  has_b_frames=2
  sample_aspect_ratio=N/A
  display_aspect_ratio=N/A
  pix_fmt=yuv420p
  level=50
  color_range=unknown
  color_space=unknown
  color_transfer=unknown
  color_primaries=unknown
  chroma_location=left
  field_order=progressive
  timecode=N/A
  refs=1
  is_avc=false
  nal_length_size=0
  id=N/A
  r_frame_rate=299/12
  avg_frame_rate=1000/1
  time_base=1/1000
  start_pts=0
  start_time=0.000000
  duration_ts=N/A
  duration=N/A
  bit_rate=N/A
  max_bit_rate=N/A
  bits_per_raw_sample=8
  nb_frames=N/A
  nb_read_frames=N/A
  nb_read_packets=737
  DISPOSITION:default=1
  DISPOSITION:dub=0
  DISPOSITION:original=0
  DISPOSITION:comment=0
  DISPOSITION:lyrics=0
  DISPOSITION:karaoke=0
  DISPOSITION:forced=0
  DISPOSITION:hearing_impaired=0
  DISPOSITION:visual_impaired=0
  DISPOSITION:clean_effects=0
  DISPOSITION:attached_pic=0
  DISPOSITION:timed_thumbnails=0
  TAG:DURATION=00:00:59.211000000
  [/STREAM]
  [FORMAT]
  filename=testEncLeft.mkv
  nb_streams=1
  nb_programs=0
  format_name=matroska,webm
  format_long_name=Matroska / WebM
  start_time=0.000000
  duration=59.211000
  size=278349
  bit_rate=37607
  probe_score=100
  TAG:COMMENT=Slickline Player Export
  TAG:ENCODER=Lavf57.83.100
  [/FORMAT]
  </code></pre>
  
  <p>And the info after remuxing, which works:</p>
  
  <pre><code>[PACKET]
  codec_type=video
  stream_index=0
  pts=59050
  pts_time=59.050000
  dts=58890
  dts_time=58.890000
  duration=1
  duration_time=0.001000
  convergence_duration=N/A
  convergence_duration_time=N/A
  size=427
  pos=277418
  flags=__
  [/PACKET]
  [PACKET]
  codec_type=video
  stream_index=0
  pts=58970
  pts_time=58.970000
  dts=58970
  dts_time=58.970000
  duration=1
  duration_time=0.001000
  convergence_duration=N/A
  convergence_duration_time=N/A
  size=205
  pos=277852
  flags=__
  [/PACKET]
  [PACKET]
  codec_type=video
  stream_index=0
  pts=59130
  pts_time=59.130000
  dts=59050
  dts_time=59.050000
  duration=1
  duration_time=0.001000
  convergence_duration=N/A
  convergence_duration_time=N/A
  size=268
  pos=278064
  flags=__
  [/PACKET]
  [STREAM]
  index=0
  codec_name=h264
  codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
  profile=Main
  codec_type=video
  codec_time_base=1/2000
  codec_tag_string=[0][0][0][0]
  codec_tag=0x0000
  width=720
  height=576
  coded_width=720
  coded_height=576
  has_b_frames=2
  sample_aspect_ratio=N/A
  display_aspect_ratio=N/A
  pix_fmt=yuv420p
  level=50
  color_range=unknown
  color_space=unknown
  color_transfer=unknown
  color_primaries=unknown
  chroma_location=left
  field_order=progressive
  timecode=N/A
  refs=1
  is_avc=true
  nal_length_size=4
  id=N/A
  r_frame_rate=299/12
  avg_frame_rate=1000/1
  time_base=1/1000
  start_pts=0
  start_time=0.000000
  duration_ts=N/A
  duration=N/A
  bit_rate=N/A
  max_bit_rate=N/A
  bits_per_raw_sample=8
  nb_frames=N/A
  nb_read_frames=N/A
  nb_read_packets=737
  DISPOSITION:default=1
  DISPOSITION:dub=0
  DISPOSITION:original=0
  DISPOSITION:comment=0
  DISPOSITION:lyrics=0
  DISPOSITION:karaoke=0
  DISPOSITION:forced=0
  DISPOSITION:hearing_impaired=0
  DISPOSITION:visual_impaired=0
  DISPOSITION:clean_effects=0
  DISPOSITION:attached_pic=0
  DISPOSITION:timed_thumbnails=0
  TAG:DURATION=00:00:59.212000000
  [/STREAM]
  [FORMAT]
  filename=fixedLeft.mkv
  nb_streams=1
  nb_programs=0
  format_name=matroska,webm
  format_long_name=Matroska / WebM
  start_time=0.000000
  duration=59.212000
  size=278409
  bit_rate=37615
  probe_score=100
  TAG:COMMENT=Slickline Player Export
  TAG:ENCODER=Lavf58.12.100
  [/FORMAT]
  </code></pre>
  
  <p>Here is how I'm setting up the output context, for reference: it's pretty standard, following the sample code. </p>
  
  <pre><code>int ret;
  
  avformat_alloc_output_context2(&amp;outputFormatCtx, nullptr, nullptr, outFilePath.c_str());
  
  av_dict_set(&amp;outputFormatCtx-&gt;metadata, "comment", "FFmpeg Export", 0);
  
  if (!outputFormatCtx)
  {
      LOG_AND_THROW("Could not allocate output context");
  }
  
  outputVideoStream = avformat_new_stream(outputFormatCtx, nullptr);
  outputVideoStream-&gt;time_base = AVRational{ 1, AV_TIME_BASE }; // Stream timebase will be used by codec
  
  if (!outputVideoStream)
  {
      LOG_AND_THROW("Failed allocating output stream");
  }
  
  // defaults to "libx264"
  AVCodec *outCodec = avcodec_find_encoder_by_name(selectedCodecName.c_str());
  
  if (!outCodec)
  {
      LOG_AND_THROW("Failed finding output codec");
  }
  
  AVDictionary *opts = nullptr;
  
  if (selectedCodecName == "libx264")
  {
      opts = getX264CodecOptions();
  }
  
  encoderCtx = avcodec_alloc_context3(outCodec);
  
  if (!encoderCtx)
  {
      LOG_AND_THROW("Failed to allocate the encoder context");
  }
  
  encoderCtx-&gt;width = width;
  encoderCtx-&gt;height = height;
  encoderCtx-&gt;pix_fmt = AV_PIX_FMT_YUV420P;
  // time base for the frames we will provide to the encoder
  encoderCtx-&gt;time_base = AVRational{ 1, AV_TIME_BASE };
  // convert framerate from double to rational
  encoderCtx-&gt;framerate = AVRational{ (int)(frameRate * AV_TIME_BASE), AV_TIME_BASE};
  
  // Match encoderCtx time base for the stream
  outputVideoStream-&gt;time_base = encoderCtx-&gt;time_base;
  
  ret = avcodec_open2(encoderCtx, outCodec, &amp;opts);
  
  if (ret &lt; 0)
  {
      LOG_AND_THROW_PARAM("Cannot open video encoder for stream: %d", ret);
  }
  
  // Fill in some params for MP4 stream, details about encoder
  ret = avcodec_parameters_from_context(outputVideoStream-&gt;codecpar, encoderCtx);
  
  if (ret &lt; 0)
  {
      LOG_AND_THROW_PARAM("Failed to copy encoder parameters to output stream: %d", ret);
  }
  
  if (outputFormatCtx-&gt;oformat-&gt;flags &amp; AVFMT_GLOBALHEADER)
  {
      encoderCtx-&gt;flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
  }
  
  av_dump_format(outputFormatCtx, 0, filePath.c_str(), 1);
  
  // End of encoder settings, setting up MP4
  if (!(outputFormatCtx-&gt;oformat-&gt;flags &amp; AVFMT_NOFILE))
  {
      ret = avio_open(&amp;outputFormatCtx-&gt;pb, outFilePath.c_str(), AVIO_FLAG_WRITE);
  
      if (ret &lt; 0)
      {
          LOG_AND_THROW_PARAMSTR("Could not open output file '%s'", outFilePath.c_str());
      }
  }
  
  ret = avformat_write_header(outputFormatCtx, nullptr);
  
  if (ret &lt; 0)
  {
      LOG_AND_THROW_PARAM("Error occurred when opening output file for writing: %d", ret);
  }
  </code></pre>
  
  <p>Can anyone help me figure out why the container is not playing properly?</p>
  
  <p>Thanks in advance.</p>
  
  <p>-James</p>