Data Visualization

Data Visualization

A free-hand scanning approach allows for flexible, lightweight probes to image intricate anatomies for modalities such as fluorescence lifetime imaging (FLIm) or optical coherence tomography (OCT). We developed a method to generate parametric maps overlaid with a white light image of the surgical field of view, in real time, by combining FLIm parameters with the measurement location obtained from a video of the surgical field. 

While very promising, this approach presents additional challenges compared to widefield imaging systems, namely robust identification of the measurement location, tissue motion during imaging, varying lighting conditions in the surgical field, and sparse sampling of the tissue surface. These challenges limit the coregistration accuracy and interpretability of the acquired imaging data. 

We are developing solutions (“FLImBrush”) for robust localization and visualization of intraoperative free-hand fiber optic fluorescence lifetime imaging (FLIm). FLImBrush builds upon an existing method while employing deep learning-based image segmentation, block-matching based motion correction, and interpolation-based visualization to address the aforementioned challenges. Current results demonstrate that FLImBrush can provide accurate localization of FLIm point-measurements while producing interpretable and complete visualizations of FLIm data acquired from a tissue surface. Each of the main processing steps was shown to be capable of real-time processing (> 30 frames per second), highlighting the feasibility of FLImBrush for intraoperative imaging and surgical guidance. Current findings show the feasibility of integrating FLImBrush into a range of surgical applications including cancer margins assessment during head and neck surgery.

Our group is also exploring stereoscopic reconstruction of spectroscopic data as well as novel mixed reality user interfaces.

 

Funding

NIH (National Institutes of Health): R01CA187427, R03EB026819

 

IP

US10422749B2: Facilitating real-time visualization of tissue features derived from optical signals

 

Publications
M. Marsden, T. Fukazawa, Y. C. Deng, B. W. Weyers, J. Bec, D. Gregory Farwell, and L. Marcu, "FLImBrush: dynamic visualization of intraoperative free-hand fiber-based fluorescence lifetime imaging," Biomed Opt Express 11, 5166-5180 (2020).
D. Gorpas, J. Phipps, J. Bec, D. Ma, S. Dochow, D. Yankelevich, J. Sorger, J. Popp, A. Bewley, R. Gandour-Edwards, L. Marcu, and D. G. Farwell, "Autofluorescence lifetime augmented reality as a means for real-time robotic surgery guidance in human patients," Sci Rep 9, 1187 (2019).
D. Gorpas, D. Ma, J. Bec, D. R. Yankelevich, and L. Marcu, "Real-Time Visualization of Tissue Surface Biochemical Features Derived From Fluorescence Lifetime Measurements," IEEE Trans Med Imaging 35, 1802-1811 (2016).