(Georgia Institute of Technology, 2014-09)
Kundu, Abhijit; Li, Yin; Dellaert, Frank; Li, Fuxin; Rehg, James M.
We present an approach for joint inference of 3D scene structure and semantic labeling for monocular video. Starting with monocular
image stream, our framework produces a 3D volumetric semantic + occupancy map, which is much more useful than a series of 2D semantic label
images or a sparse point cloud produced by traditional semantic segmentation and Structure from Motion(SfM) pipelines respectively. We derive
a Conditional Random Field (CRF) model defined in the 3D space, that
jointly infers the semantic category and occupancy for each voxel. Such
a joint inference in the 3D CRF paves the way for more informed priors
and constraints, which is otherwise not possible if solved separately in
their traditional frameworks. We make use of class specific semantic cues
that constrain the 3D structure in areas, where multiview constraints are
weak. Our model comprises of higher order factors, which helps when the
depth is unobservable. We also make use of class specific semantic cues to
reduce either the degree of such higher order factors, or to approximately
model them with unaries if possible. We demonstrate improved 3D structure and temporally consistent semantic segmentation for diffcult, large
scale, forward moving monocular image sequence.
(Georgia Institute of Technology, 2010-12)
Kaess, Michael; Ila, Viorela; Roberts, Richard; Dellaert, Frank
We present a novel data structure, the Bayes tree, that provides an algorithmic
foundation enabling a better understanding of existing graphical model
inference algorithms and their connection to sparse matrix factorization methods.
Similar to a clique tree, a Bayes tree encodes a factored probability density, but
unlike the clique tree it is directed and maps more naturally to the square root information
matrix of the simultaneous localization and mapping (SLAM) problem.
In this paper, we highlight three insights provided by our new data structure. First,
the Bayes tree provides a better understanding of batch matrix factorization in terms
of probability densities. Second, we show how the fairly abstract updates to a matrix
factorization translate to a simple editing of the Bayes tree and its conditional
densities. Third, we apply the Bayes tree to obtain a completely novel algorithm for
sparse nonlinear incremental optimization, that combines incremental updates with
fluid relinearization of a reduced set of variables for efficiency, combined with fast
convergence to the exact solution. We also present a novel strategy for incremental
variable reordering to retain sparsity.We evaluate our algorithm on standard datasets
in both landmark and pose SLAM settings.