title: Adding some kind of subtitle to an mp4 with ffmpeg
body: <p>I have a subtitle (it certainly looks like it), but I don't know how to add it into an MP4 I'm creating.
  First a hexdump of the subtitle:</p>
  
  <pre><code>00000000  00 00 00 18 66 74 79 70  69 73 6f 36 00 00 00 00  |....ftypiso6....|
  00000010  69 73 6f 36 64 61 73 68  00 00 00 28 66 72 65 65  |iso6dash...(free|
  00000020  55 53 50 20 62 79 20 43  6f 64 65 53 68 6f 70 11  |USP by CodeShop.|
  00000030  11 11 11 11 11 11 11 11  11 11 11 11 11 11 11 11  |................|
  00000040  00 00 02 1d 6d 6f 6f 76  00 00 00 6c 6d 76 68 64  |....moov...lmvhd|
  00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 01  |................|
  00000060  00 00 00 00 00 01 00 00  01 00 00 00 00 00 00 00  |................|
  00000070  00 00 00 00 00 01 00 00  00 00 00 00 00 00 00 00  |................|
  *
  00000090  00 00 00 00 40 00 00 00  00 00 00 00 00 00 00 00  |....@...........|
  000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000000b0  00 00 00 02 00 00 01 81  74 72 61 6b 00 00 00 5c  |........trak...\|
  000000c0  74 6b 68 64 00 00 00 07  00 00 00 00 00 00 00 00  |tkhd............|
  000000d0  00 00 00 01 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00 01 00 00  |................|
  *
  00000100  00 00 00 00 00 00 00 00  00 00 00 00 40 00 00 00  |............@...|
  00000110  00 00 00 00 00 00 00 00  00 00 01 1d 6d 64 69 61  |............mdia|
  00000120  00 00 00 20 6d 64 68 64  00 00 00 00 00 00 00 00  |... mdhd........|
  00000130  00 00 00 00 00 00 03 e8  00 00 00 00 39 84 00 00  |............9...|
  00000140  00 00 00 35 68 64 6c 72  00 00 00 00 00 00 00 00  |...5hdlr........|
  00000150  73 75 62 74 00 00 00 00  00 00 00 00 00 00 00 00  |subt............|
  00000160  55 53 50 20 53 75 62 74  69 74 6c 65 20 48 61 6e  |USP Subtitle Han|
  00000170  64 6c 65 72 00 00 00 00  c0 6d 69 6e 66 00 00 00  |dler.....minf...|
  00000180  0c 73 74 68 64 00 00 00  00 00 00 00 24 64 69 6e  |.sthd.......$din|
  00000190  66 00 00 00 1c 64 72 65  66 00 00 00 00 00 00 00  |f....dref.......|
  000001a0  01 00 00 00 0c 75 72 6c  20 00 00 00 01 00 00 00  |.....url .......|
  000001b0  88 73 74 62 6c 00 00 00  3c 73 74 73 64 00 00 00  |.stbl...&lt;stsd...|
  000001c0  00 00 00 00 01 00 00 00  2c 73 74 70 70 00 00 00  |........,stpp...|
  000001d0  00 00 00 00 01 68 74 74  70 3a 2f 2f 77 77 77 2e  |.....http://www.|
  000001e0  77 33 2e 6f 72 67 2f 6e  73 2f 74 74 6d 6c 00 00  |w3.org/ns/ttml..|
  000001f0  00 00 00 00 10 73 74 74  73 00 00 00 00 00 00 00  |.....stts.......|
  00000200  00 00 00 00 10 73 74 73  63 00 00 00 00 00 00 00  |.....stsc.......|
  00000210  00 00 00 00 14 73 74 73  7a 00 00 00 00 00 00 00  |.....stsz.......|
  00000220  00 00 00 00 00 00 00 00  10 73 74 63 6f 00 00 00  |.........stco...|
  00000230  00 00 00 00 00 00 00 00  28 6d 76 65 78 00 00 00  |........(mvex...|
  00000240  20 74 72 65 78 00 00 00  00 00 00 00 01 00 00 00  | trex...........|
  00000250  01 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  00000260  18 73 74 79 70 69 73 6f  36 00 00 00 00 69 73 6f  |.stypiso6....iso|
  00000270  36 6d 73 64 68 00 00 00  28 66 72 65 65 55 53 50  |6msdh...(freeUSP|
  00000280  20 62 79 20 43 6f 64 65  53 68 6f 70 11 11 11 11  | by CodeShop....|
  00000290  11 11 11 11 11 11 11 11  11 11 11 11 11 00 00 00  |................|
  000002a0  2c 73 69 64 78 00 00 00  00 00 00 00 01 00 00 03  |,sidx...........|
  000002b0  e8 00 00 00 00 00 00 00  00 00 00 00 01 00 00 0f  |................|
  000002c0  09 00 00 ea 60 90 00 00  00 00 00 00 68 6d 6f 6f  |....`.......hmoo|
  000002d0  66 00 00 00 10 6d 66 68  64 00 00 00 00 00 00 00  |f....mfhd.......|
  000002e0  01 00 00 00 50 74 72 61  66 00 00 00 14 74 66 68  |....Ptraf....tfh|
  000002f0  64 00 02 00 02 00 00 00  01 00 00 00 01 00 00 00  |d...............|
  00000300  14 74 66 64 74 01 00 00  00 00 00 00 00 00 00 00  |.tfdt...........|
  00000310  00 00 00 00 20 74 72 75  6e 00 00 07 01 00 00 00  |.... trun.......|
  00000320  01 00 00 00 70 00 00 ea  60 00 00 0e 99 02 80 00  |....p...`.......|
  00000330  40 00 00 0e a1 6d 64 61  74 3c 3f 78 6d 6c 20 76  |@....mdat&lt;?xml v|
  00000340  65 72 73 69 6f 6e 3d 22  31 2e 30 22 20 65 6e 63  |ersion="1.0" enc|
  00000350  6f 64 69 6e 67 3d 22 75  74 66 2d 38 22 3f 3e 3c  |oding="utf-8"?&gt;&lt;|
  00000360  74 74 20 78 6d 6c 6e 73  3d 22 68 74 74 70 3a 2f  |tt xmlns="http:/|
  00000370  2f 77 77 77 2e 77 33 2e  6f 72 67 2f 6e 73 2f 74  |/www.w3.org/ns/t|
  00000380  74 6d 6c 22 20 78 6d 6c  6e 73 3a 73 6d 70 74 65  |tml" xmlns:smpte|
  00000390  3d 22 68 74 74 70 3a 2f  2f 77 77 77 2e 73 6d 70  |="http://www.smp|
  000003a0  74 65 2d 72 61 2e 6f 72  67 2f 73 63 68 65 6d 61  |te-ra.org/schema|
  000003b0  73 2f 32 30 35 32 2d 31  2f 32 30 31 30 2f 73 6d  |s/2052-1/2010/sm|
  </code></pre>
  
  <p>I honestly have never seen anything like this before...</p>
  
  <p>So the command I'm running now is this:</p>
  
  <blockquote>
    <p>ffmpeg -y -i 'audionld#24000.mp4' -i 'audionld#48000.mp4' -i
    'textnld.mp4' -i 'video.mp4'  -map 0:a -map 1:a -map 2:d -map 3:v
    -metadata:s:a:0 language=nld -metadata:s:a:1 language=nld -acodec copy -bsf:a aac_adtstoasc -vcodec copy -c:d copy  -movflags faststart "test.mp4" 2>&amp;1</p>
  </blockquote>
  
  <p>Which results in this output:</p>
  
  <pre><code>ffmpeg version 3.2.5-1~bpo8+1 Copyright (c) 2000-2017 the FFmpeg developers
    built with gcc 4.9.2 (Debian 4.9.2-10)
    configuration: --prefix=/usr --extra-version='1~bpo8+1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --disable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
    libavutil      55. 34.101 / 55. 34.101
    libavcodec     57. 64.101 / 57. 64.101
    libavformat    57. 56.101 / 57. 56.101
    libavdevice    57.  1.100 / 57.  1.100
    libavfilter     6. 65.100 /  6. 65.100
    libavresample   3.  1.  0 /  3.  1.  0
    libswscale      4.  2.100 /  4.  2.100
    libswresample   2.  3.100 /  2.  3.100
    libpostproc    54.  1.100 / 54.  1.100
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'audionld#24000.mp4':
    Metadata:
      major_brand     : iso6
      minor_version   : 0
      compatible_brands: iso6dash
    Duration: 00:03:06.07, start: 0.000000, bitrate: 64 kb/s
      Stream #0:0(nld): Audio: aac (HE-AAC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 62 kb/s (default)
      Metadata:
        handler_name    : USP Sound Handler
  Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'audionld#48000.mp4':
    Metadata:
      major_brand     : iso6
      minor_version   : 0
      compatible_brands: iso6dash
    Duration: 00:03:06.07, start: 0.000000, bitrate: 95 kb/s
      Stream #1:0(nld): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 93 kb/s (default)
      Metadata:
        handler_name    : USP Sound Handler
  Input #2, mov,mp4,m4a,3gp,3g2,mj2, from 'textnld.mp4':
    Metadata:
      major_brand     : iso6
      minor_version   : 0
      compatible_brands: iso6dash
    Duration: 00:03:05.76, bitrate: 0 kb/s
      Stream #2:0(nld): Data: none (stpp / 0x70707473), 0 kb/s (default)
      Metadata:
        handler_name    : USP Subtitle Handler
  Input #3, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4':
    Metadata:
      major_brand     : iso6
      minor_version   : 0
      compatible_brands: iso6dash
    Duration: 00:03:06.08, start: 0.000000, bitrate: 152 kb/s
      Stream #3:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 416x234 [SAR 1:1 DAR 16:9], 144 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default)
      Metadata:
        handler_name    : USP Video Handler
        encoder         : Elemental H.264
  [mp4 @ 0x7f15299c4520] Could not find tag for codec none in stream #2, codec not currently supported in container
  Could not write header for output file #0 (incorrect codec parameters ?): Invalid argumentStream mapping:
    Stream #0:0 -&gt; #0:0 (copy)
    Stream #1:0 -&gt; #0:1 (copy)
    Stream #2:0 -&gt; #0:2 (copy)
    Stream #3:0 -&gt; #0:3 (copy)
      Last message repeated 1 times
  </code></pre>
  
  <p>This gives the error:
  Could not find tag for codec none in stream #2, codec not currently supported in container
  Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument</p>
  
  <p>So I tried playing with the map like this:</p>
  
  <p>-map 2:s
  ==> Gives me:
  Stream map '2:s' matches no streams.
  To ignore this, add a trailing '?' to the map.</p>
  
  <p>-map 2:s?
  ==> Gives me no error, but the subtitle stream isn't added into the final file.</p>
  
  <pre><code>Stream mapping:
    Stream #0:0 -&gt; #0:0 (copy)
    Stream #1:0 -&gt; #0:1 (copy)
    Stream #3:0 -&gt; #0:2 (copy)
  </code></pre>
  
  <p>Maybe I first have to convert the subtitle into srt or such, but for that I don't find on google what type of file this is.</p>
  
  <p>Any help would be greatly appreciated!</p>
  
  <p>Thanks</p>
  
  <p>---EDIT---</p>
  
  <p>Hereby the code that I am using to convert this in PHP:</p>
  
  <pre><code>  private function convert_TTML_subtitle_file($orig) {
      if ( $orig == '' || !file_exists($orig) ) {
        return false;
      }
  
      $f = file_get_contents($orig);
      $pos = stripos($f, '&lt;tt xmlns="http://www.w3.org/ns/ttml"');
      if ( $pos === false ) {
        return false;
      }
  
      $size = strlen($f);
      $cur  = 0;
      $txt = '';
      $counter = 1;
      while ( $cur &lt; $size ) {
  #      var_dump("cur: $cur");
        $curSize = unpack('N', substr($f, $cur))[1];
  #      var_dump("curSize: $curSize");
        if ( substr($f, $cur+4, 4) == 'mdat' ) {
          $xml = simplexml_load_string( substr($f, $cur+8, $curSize - 8) );
          foreach( $xml-&gt;body-&gt;div-&gt;p as $p ) {
            $txt .= "{$counter}\r\n";
            $counter++;
            $txt .= $p['begin'] . ' --&gt; ' . $p['end'] . "\r\n";
            $txt .= $p-&gt;span . "\r\n\r\n";
          }
        }
        $cur += $curSize;
      }
  
      $fname = tempnam(getcwd(), 'sub');
      file_put_contents($fname, $txt);
      return $fname;
    }
  </code></pre>