Title: Recovering Dense Tissue Multispectral Signal from in vivo RGB Images,
Abstract: Hyperspectral/multispectral imaging (HSI/MSI) contains rich information
clinical applications, such as 1) narrow band imaging for vascular
visualisation; 2) oxygen saturation for intraoperative perfusion monitoring and
clinical decision making [1]; 3) tissue classification and identification of
pathology [2]. The current systems which provide pixel-level HSI/MSI signal can
be generally divided into two types: spatial scanning and spectral scanning.
However, the trade-off between spatial/spectral resolution, the acquisition
time, and the hardware complexity hampers implementation in real-world
applications, especially intra-operatively. Acquiring high resolution images in
real-time is important for HSI/MSI in intra-operative imaging, to alleviate the
side effect caused by breathing, heartbeat, and other sources of motion.
Therefore, we developed an algorithm to recover a pixel-level MSI stack using
only the captured snapshot RGB images from a normal camera. We refer to this
technique as "super-spectral-resolution". The proposed method enables recovery
of pixel-level-dense MSI signals with 24 spectral bands at ~11 frames per
second (FPS) on a GPU. Multispectral data captured from porcine bowel and
sheep/rabbit uteri in vivo has been used for training, and the algorithm has
been validated using unseen in vivo animal experiments.