title: Encoding and decoding timestamps per frame using libav* into mp4
body: <p>I am writing a program for recording and playing back video together with another program that is connected to a piece of hardware we sell. This other program gathers data from hardware, which contains "timestamp" counters at a fixed sampling rate. </p>
  
  <p>I am looking for a way to encode these timestamps (either raw or converted to unix) into the outputted mp4 file. The main reason being that the video stream might only start getting saved a few minutes into the hardware acquisition</p>
  
  <p>Recording: </p>
  
  <pre><code>                timestamp
  Main program ---------------&gt;  Video program
    |                                 |
    |                                 |
    v                                 v
  Save hardware data               Save frame data
    and timestamp                    and timestamp
  
    |                                 |
    |                                 |
    v                                 v
  Custom data format                .mp4 file
  </code></pre>
  
  <p>Post processing analysis: </p>
  
  <pre class="lang-py prettyprint-override"><code>for i in range(0,datalen):
      hardwareData, timestamp = readHardwareFile()
      frame, timestamp = readMP4()
  
      myData[timestamp].hardware = hardwareData
      mydata[timestamp].video = frame
  
  analyze(myData)
  </code></pre>
  
  <p>The goal being that when a I want to playback in VLC, or with OpenCV, I have access to timestamps for each frame. </p>
  
  <p>I currently save it as a separate text file with timestamp and frame #, but I'm wondering if there is a more standardized way to do this. </p>