Title:
Deep Multi-view Stereo for GTSFM

Thumbnail Image
Author(s)
Liu, Ren
Authors
Advisor(s)
Dellaert, Frank
Advisor(s)
Editor(s)
Associated Organization(s)
Organizational Unit
Series
Supplementary to
Abstract
Current structure-from-motion (SFM) pipelines integrated with multi-view stereo (MVS) module often applies traditional MVS algorithms. These algorithms cannot be well parallelized and can be slow when the data size and the resolution increase. For view synthesis, there are seldom SFM pipelines integrating it. This thesis focuses on how to integrate MVS and view synthesis efficiently into SFM pipelines, especially for the latest deep learning approaches. I first do a thorough survey in both domains, compare the advantages and disadvantages of the latest studies, and select the best-fit approach for our distributed SFM pipeline, Georgia Tech Structure from Motion (GTSFM). We implement the deep multi-view optimizer with PatchmatchNet and integrate it into the working graph of GTSFM. We also design an algorithm to boost a novel deep view synthesis algorithm, Instant-NGP, by forcing the reconstruction region on the overlapping Field-of-Views. This also enables us to extract high-quality dense polygon meshes of foreground objects directly from the reconstructed depth field. Experiments run on DTU and Skydio Crane Mast datasets suggest our MVS approach is more efficient than some popular SFM pipelines with MVS implemented.
Sponsor
Date Issued
2022-05-03
Extent
Resource Type
Text
Resource Subtype
Thesis
Rights Statement
Rights URI