anchor.diff:  .Conversion overhead on the MATLAB thread in [ms]
   image::{imagesdir}/streaming/table_performance.svg[width=100%]
   
  -The results show that the default conversion as well as jpeg compression are essentially non-starters for color images. For grayscale images the default conversion works reasonably well due to the fact that the resulting 2D array (`byte[height][width]`) stores the data much more efficiently, but it still is not a great option.
  -
  -Copying the raw pixels from a Java image is significantly faster, but there is still too much overhead caused by the need to change the memory layout.
  -
  -The by far best option is to convert the layout outside of MATLAB and, if possible, to bypass the Java translation layer by using shared memory. This approach scales very well and can transfer even 4K images in real-time.
  +The results show that the default conversion as well as jpeg compression are essentially non-starters for color images. For grayscale images the default conversion works significantly better due to the fact that the data is stored in a much more efficient 2D array (`byte[height][width]`).
   
   [NOTE]
  -Note that we typically use lower resolution images (e.g. 480p) to speed up the processing. We only use higher resolution images when we use MATLAB as a live video display.
  +Note that we currently don't have a good explanation for the ~10x cost increase (rather than ~4x) between 1080p and 4K grayscale. The behavior was the same across computers and different memory settings.
  +
  +When copying the pixels of a Java image we can see another significant performance increase due to the data being stored in a single contiguous array. At this point much of the overhead comes from re-ordering pixels, so by doing the conversion beforehand we can get another 2-3x improvement.
  +
  +Lastly, although accessing shared memory in combination with the locking overhead results in a slightly higher fixed cost, the copying itself is significantly cheaper, resulting in another 2-3x speedup for high resolution images. Overall, the approach scales very well and would even allow streaming of 4K color images from two cameras simultaneously.
   
   == Final Notes
   
  -For us, the biggest lesson learned was that shared memory can be used to efficiently transfer large amounts of data into MATLAB. We also ended up using it quite a bit for inter-process communication, e.g., to exchange data between multiple MATLAB instances. For example, one instance can track a target at a lower rate, while another instance can use its output for robot control at a higher rate.
  -
  -As for our initial motiation, after creating link:https://github.com/HebiRobotics/HebiCam[HebiCam] we were able to develop and reliably run the whole demo entirely in MATLAB. However, during the setup at the conference site we ran into the issue that the colors and lines on the carpet caused the camera calibration to fail repeatedly. We ended up spending more time on getting the cameras calibrated than we spent on the entire initial development of HebiCam. We've since shied away from developing custom stereo vision setups solely for exhibit demos.
  +As for our initial motiation, after creating link:https://github.com/HebiRobotics/HebiCam[HebiCam] we were able to develop and reliably run the whole demo entirely in MATLAB.
  +
  +One lesson learned was that although MATLAB's Java interface can be inefficient for certain cases, there are simple workarounds that can remove most bottlenecks. The most important rule is to avoid converting large multi-dimensional arrays whenever possible.
  +
  +Another lesson learned was that shared memory provides a very efficient way to transfer large amounts of data to and from MATLAB. We also found it useful for inter-process communication between multiple MATLAB instances, e.g., one instance can track a target while another instance can use its output for real-time control. This can be useful to avoid coupling a fast control loop to the (lower) frame rate of a camera or sensor.
   
   ////
   The biggest takeaway for us was that MATLAB supports shared memory. We had used shared memory before for inter-process communication in Java and C++, but we somehow assumed that MATLAB wouldn't support it. We eventually looked into this and found that MATLAB does have support, and that the implementation is actually quite nice. The data definition can even map the memory to an array of structs with multiple variables.
  
positive.code: 
  == Results
  
  All benchmarks were run in MATLAB 2017b on an link:https://www.intel.com/content/www/us/en/products/boards-kits/nuc/kits/nuc6i7kyk.html[Intel NUC6I7KYK]. The performance was measured using MATLAB's link:https://mathworks.com/help/matlab/ref/timeit.html[timeit] function. All measurements are in units of milliseconds. The full benchmark code is available link:https://github.com/HebiRobotics/HebiCam/tree/benchmark[here]. 
  
  The two tables below show the results for converting color (RGB) images as well as grayscale images. The background color of each cell represents a rough classification of the overhead on the main MATLAB thread.
  
  
negative.code: 
  *2. Compressed 1D Array*
  
  A common approach to move image data across distributed components (e.g. link:http://www.ros.org/[ROS]) is to encode the individual images using link:https://en.wikipedia.org/wiki/Motion_JPEG[MJPEG] compression before sending them. Doing this within a single process is obviously wasteful, but we included it because it is common practice in some systems. Since MATLAB does not offer a way to decompress jpeg images in memory, we needed to save to data to file (located on a RAM disk) first.
  
  [source,matlab]
  ----
  