anchor.diff:  
   Starting background threads and getting the video data into Java was comparatively straight forward. We used the excellent link:https://github.com/bytedeco/javacv[JavaCV] library, which is a Java wrapper around link:https://opencv.org/[OpenCV] and link:https://www.ffmpeg.org/[FFMpeg] that includes pre-compiled native binaries for all major operating systems. The main challenge was to get the image data across the translation layer back into MATLAB.
   
  -In order to process images at a frame rate of 30 fps in real-time, the total time budget of the main MATLAB thread is 33ms per cycle. Thus, the overhead incurred on the main thread needs to be sufficiently low, i.e., a low number of milliseconds, to leave enough time for the actual processing.
  -
  -While the automated translation from and to Java types tends to be sufficient for most applications, it is very inefficient for large and multi dimensional matrices. For example, a `1080x1920x3` MATLAB image matrix gets translated to a `byte[1080][1920][3]` Java array. This results in separate array objects for every single pixel in the image.  Even for very low resolution images this default conversion is unfortunately nowhere close to fast enough.
  -
  -As an additional complication, Java uses a different memory layout than MATLAB. While Java stores pixel data in row-major order, MATLAB stores images transposed and in column-major order. For example, if the pixel data for a Java image would be laid out as `[RGB][RGB][RGB]...`, the same image would be laid out as `[RRR...][GGG...][BBB...]` in MATLAB.  (`R`, `G`, `B` correspond to red, green, and blue values for one pixel)
  +In order to process images at a frame rate of 30 fps in real-time, the total time budget of the main MATLAB thread is 33ms per cycle. Thus, the overhead incurred on the main thread caused by the acquisition needs to be sufficiently low, i.e., a low number of milliseconds, to leave enough time for actual processing.
  +
  +While the automated translation from and to Java types tends to be sufficient for most applications, it is very inefficient for large and multi dimensional matrices. For example, a `1080x1920x3` MATLAB image matrix gets translated to a `byte[1080][1920][3]` Java array. This results in separate array objects for every single pixel in the image.  Even for very low resolution images this default conversion is unfortunately nowhere close to being fast enough.
  +
  +As an additional complication, Java uses a different memory layout than MATLAB. While Java stores image data in row-major order, MATLAB stores images transposed and in column-major order. For example, if the pixel data for a Java image would be laid out as `[RGB][RGB][RGB]...`, the same image would be laid out as `[RRR...][GGG...][BBB...]` in MATLAB.  (`R`, `G`, `B` correspond to red, green, and blue values for one pixel)
   
   ////
   * Acquiring video on a background thread
  
positive.code: 
  == Motivation
  
  For our booth at ICRA 2014 we put together a demo system in MATLAB that used stereo vision for tracking bean bags, and a robot arm to pick them up. We used two IP cameras that streamed link:https://de.wikipedia.org/wiki/H.264[H.264] video over link:https://en.wikipedia.org/wiki/Real_Time_Streaming_Protocol[RTSP]. While developing the image processing and robot control parts were comparatively simple, it proved to be a challenge to get streaming video data from both cameras into MATLAB fast enough to be useful.
  
  * link:http://www.mathworks.com/hardware-support/ip-camera.html[IP Camera Support] didn't exist at the time and only supported link:https://en.wikipedia.org/wiki/Motion_JPEG[MJPEG] over link:https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol[HTTP]
  * link:http://www.mathworks.com/hardware-support/matlab-webcam.html[USB Webcam Support] only supported USB cameras
  * link:http://www.mathworks.com/help/matlab/ref/imread.html[imread] and link:http://www.mathworks.com/help/matlab/ref/webread.html[webread] were limited to HTTP and too slow for real-time
  
  Since we didn't want to port the code to another language, we instead wrote a small library that enabled us to acquire video data in an efficient manner. The project was later open sourced as link:http://www.github.com/HebiRobotics/HebiCam[HebiCam].
  
  == Technical Background
  
  Most video streams that are sent over the network get compressed in order to save bandwidth (exceptions include GigE cameras). While the cameras often include specialized hardware (link:https://en.wikipedia.org/wiki/Application-specific_integrated_circuit[ASIC]) to do the encoding, the decoding on the computer side tends to happen in Software. Since the decoding is quite CPU intensive, the acquisition needs to happen in a different thread than the processing.
  
  [NOTE]
  Note that Intel has added hardware support for encoding and decoding to their CPUs starting in 2011 (link:https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video[QuickSync]), but there is unfortunately a lack of library support.
  
  Due to MATLAB's single threaded nature, we need to resort to external languages for getting background threads. Out of the box MATLAB provides two interfaces to other languages:  https://www.mathworks.com/help/matlab/matlab_external/introducing-mex-files.html[MEX] for calling C/C++ code, and the https://www.mathworks.com/help/matlab/matlab_external/product-overview.html[Java Interface] for calling Java code. For a variety of reasons, e.g., simplified development and cross-platform deployment, we decided to use the Java interface. The diagram below shows a simplified overview.
  
  [.text-center]
  .System overview for a stereo vision setup
  
negative.code: image = reshape(vector,h,w,c);
  ----
  
  *5. MATLAB Layout as Shared Memory*
  
  This approach is the same as 4. with the difference being that the data is read from shared memory using link:https://mathworks.com/help/matlab/ref/memmapfile.html[memmapfile], thus bypassing the Java translation layer entirely. Shared memory is typically used for inter-process communication, but it can also be used within a single process. Running it in the same process also simplifies synchronization since we can use standard Java locks.
  