Monocular Reconstruction during Colonoscopy and Engine Inspection
Author(s)
Chadha, Ishan
Advisor(s)
Chen, Yue
Editor(s)
Collections
Supplementary to:
Permanent Link
Abstract
Monocular reconstruction of surfaces has become possible in recent years with the advent of efficient and accurate algorithms like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), but reconstruction can be difficult to perform in real-time depending on the nature of the environment and the precision required. Alongside this, acquiring ground-truth data in hard-to-reach environments poses additional challenges for validatingthese algorithms. Prior work has tackled this problem using generated datasets; this paper proposes validating this in silico and experimentally. This is accomplished by testing and optimizing state-of-the-art monocular reconstruction algorithms for endoscopy, specifically targeting colorectal endoscopy with the auxiliary application of borescope engine inspection. Proposed algorithms build off of 3DGS and its dynamic extension, EndoGaussian, for storing temporal deformation. Temporal and spatial features are stored in a compressed yet explainable representation called a HexPlane. The introduced opimization also computes an illumination context using the BRDF and light source location, which is particularly applicable to dim, watertight settings where the lighting model of the endoscope or borescope can be of great utility.
The proposed method, Light Encoded Gaussian Splatting (LE-GS), demonstrated an average PSNR of 27.29 on the C3VD dataset, compared to an average PSNR of 25.49 using EndoGaussian. The Simultaneous Localization and Mapping using Gassian Splatting (GS-SLAM) algorithm tested on a 3D printed pipe demonstrated a PSNR of 4.59, demonstrating the potential efficacy of our context-based optimization to reconstruction in medical settings.
Sponsor
Date
2024-07-24
Extent
Resource Type
Text
Resource Subtype
Thesis