Person:

Yezzi, Anthony

Permanent Link

https://hdl.handle.net/1853/71866

Associated Organization(s)

Organizational Unit

School of Electrical and Computer Engineering

Full item page

Publication Search Results

Now showing 1 - 10 of 20

Tracking deforming objects by filtering and prediction in the space of curves

(Georgia Institute of Technology, 2009-12) Sundaramoorthi, Ganesh ; Mennucci, Andrea C. ; Soatto, Stefano ; Yezzi, Anthony

We propose a dynamical model-based approach for tracking the shape and deformation of highly deforming objects from time-varying imagery. Previous works have assumed that the object deformation is smooth, which is realistic for the tracking problem, but most have restricted the deformation to belong to a finite-dimensional group, such as affine motions, or to finitely-parameterized models. This, however, limits the accuracy of the tracking scheme. We exploit the smoothness assumption implicit in previous work, but we lift the restriction to finite-dimensional motions/deformations. To do so, we derive analytical tools to define a dynamical model on the (infinitedimensional) space of curves. To demonstrate the application of these ideas to object tracking, we construct a simple dynamical model on shapes, which is a first-order approximation to any dynamical system. We then derive an associated nonlinear filter that estimates and predicts the shape and deformation of a object from image measurements.
Dynamic shape and appearance modeling via moving and deforming layers

(Georgia Institute of Technology, 2008-08) Jackson, Jeremy D. ; Yezzi, Anthony ; Soatto, Stefano

This model is based on a collection of overlapping layers that can move and deform, each supporting an intensity function that can change over time. We discuss the generality and limitations of this model in relation to existing ones such as traditional optical flow or motion segmentation, layers, deformable templates and deformotion. We then illustrate how this model can be used for inference of shape, motion, deformation and appearance of the scene from a collection of images. The layering structure allows for automatic inpainting of partially occluded regions. We illustrate the model on synthetic and real sequences where existing schemes fail, and show how suitable choices of constants in the model yield existing schemes, from optical flow to motion segmentation, etc.
3-D Reconstruction of Shaded Objects from Multiple Images Under Unknown Illumination

(Georgia Institute of Technology, 2008-03) Jin, Hailin ; Wang, Dejun ; Cremers, Daniel ; Prados, Emmanuel ; Yezzi, Anthony ; Soatto, Stefano

We propose a variational algorithm to jointly estimate the shape, albedo, and light configuration of a Lambertian scene from a collection of images taken from different vantage points. Our work can be thought of as extending classical multi-view stereo to cases where point correspondence cannot be established, or extending classical shape from shading to the case of multiple views with unknown light sources. We show that a first naive formalization of this problem yields algorithms that are numerically unstable, no matter how close the initialization is to the true geometry. We then propose a computational scheme to overcome this problem, resulting in provably stable algorithms that converge to (local) minima of the cost functional. We develop a new model that explicitly enforces positivity in the light sources with the assumption that the object is Lambertian and its albedo is piecewise constant and show that the new model significantly improves the accuracy and robustness relative to existing approaches.
A Variational Approach to Problems in Calibration of Multiple Cameras

(Georgia Institute of Technology, 2007-08) Unal, Gozde ; Yezzi, Anthony ; Soatto, Stefano ; Slabaugh, Gregory G.

This paper addresses the problem of calibrating camera parameters using variational methods. One problem addressed is the severe lens distortion in low-cost cameras. For many computer vision algorithms aiming at reconstructing reliable representations of 3D scenes, the camera distortion effects will lead to inaccurate 3D reconstructions and geometrical measurements if not accounted for. A second problem is the color calibration problem caused by variations in camera responses that result in different color measurements and affects the algorithms that depend on these measurements. We also address the extrinsic camera calibration that estimates relative poses and orientations of multiple cameras in the system and the intrinsic camera calibration that estimates focal lengths and the skew parameters of the cameras. To address these calibration problems, we present multiview stereo techniques based on variational methods that utilize partial and ordinary differential equations. Our approach can also be considered as a coordinated refinement of camera calibration parameters. To reduce computational complexity of such algorithms, we utilize prior knowledge on the calibration object, making a piecewise smooth surface assumption, and evolve the pose, orientation, and scale parameters of such a 3D model object without requiring a 2D feature extraction from camera views. We derive the evolution equations for the distortion coefficients, the color calibration parameters, the extrinsic and intrinsic parameters of the cameras, and present experimental results.
Joint priors for variational shape and appearance modeling

(Georgia Institute of Technology, 2007-06) Jackson, Jeremy D. ; Yezzi, Anthony ; Soatto, Stefano

We are interested in modeling the variability of different images of the same scene, or class of objects, obtained by changing the imaging conditions, for instance the viewpoint or the illumination. Understanding of such a variability is key to reconstruction of objects despite changes in their appearance (e.g. due to non-Lambertian reflection), or to recognizing classes of objects (e.g. cars), or individual objects seen from different vantage points. We propose a model that can account for changes in shape or viewpoint, appearance, and also occlusions of line of sight. We learn a prior model of each factor (shape, motion and appearance) from a collection of samples using principal component analysis, akin a generalization of "active appearance models" to dense domains affected by occlusions. The ultimate goal of this work is stereo reconstruction in 3D, but first we have developed the first stage in this approach by addressing the simpler case of 2D shape/radiance detection in single images. We illustrate our model on a collection of images of different cars and show how the learned prior can be used to improve segmentation and 3D stereo reconstruction.
Integral Invariants for Shape Matching

(Georgia Institute of Technology, 2006-10) Manay, Siddharth ; Cremers, Daniel ; Hong, Byung-Woo ; Soatto, Stefano ; Yezzi, Anthony

For shapes represented as closed planar contours, we introduce a class of functionals which are invariant with respect to the Euclidean group and which are obtained by performing integral operations. While such integral invariants enjoy some of the desirable properties of their differential counterparts, such as locality of computation (which allows matching under occlusions) and uniqueness of representation (asymptotically), they do not exhibit the noise sensitivity associated with differential quantities and, therefore, do not require presmoothing of the input shape. Our formulation allows the analysis of shapes at multiple scales. Based on integral invariants, we define a notion of distance between shapes. The proposed distance measure can be computed efficiently and allows warping the shape boundaries onto each other; its computation results in optimal point correspondence as an intermediate step. Numerical results on shape matching demonstrate that this framework can match shapes despite the deformation of subparts, missing parts and noise. As a quantitative analysis, we report matching scores for shape retrieval from a database.
Multiview Stereo Reconstruction of Dense Shape and Complex Appearance

(Georgia Institute of Technology, 2005-07) Jin, Hailin ; Soatto, Stefano ; Yezzi, Anthony

We address the problem of estimating the three-dimensional shape and complex appearance of a scene from a calibrated set of views under fixed illumination. Our approach relies on a rank condition that must be satisfied when the scene exhibits “specular + diffuse” reflectance characteristics. This constraint is used to define a cost functional for the discrepancy between the measured images and those generated by the estimate of the scene, rather than attempting to match image-to-image directly. Minimizing such a functional yields the optimal estimate of the shape of the scene, represented by a dense surface, as well as its radiance, represented by four functions defined on such a surface. These can be used to generate novel views that capture the non-Lambertian appearance of the scene.
Tracking deformable moving objects under severe occlusions

(Georgia Institute of Technology, 2004-12) Jackson, Jeremy D. ; Yezzi, Anthony ; Soatto, Stefano

We propose a nonlinear model for tracking a slowly deforming and moving contour despite significant occlusions. The contour is represented implicitly, and its motion is described by the action of a finite-dimensional group; we estimate both the implicit representation of the contour (its shape) and its motion. Our contribution consists in defining a generative model that is not subject to arbitrary re-parameterization, choice of (non-unique) key points or control points, and allows enforcing a dynamical model of motion when it is available. Otherwise, our approach allows enforcing simple phenomenological models, for instance low acceleration or low jerk.
A variational framework for reconstructing complex 3-D shape and photometry from multiple images

(Georgia Institute of Technology, 2004-09-15) Yezzi, Anthony ; Soatto, Stefano
Shedding Light on Stereoscopic Segmentation

(Georgia Institute of Technology, 2004-06) Jin, Hailin ; Cremers, Daniel ; Yezzi, Anthony ; Soatto, Stefano

We propose a variational algorithm to jointly estimate the shape, albedo, and light configuration of a Lambertian scene from a collection of images taken from different vantage points. Our work can be thought of as extending classical multi-view stereo to cases where point correspondence cannot be established, or extending classical shape from shading to the case of multiple views with unknown light sources. We show that a first naive formalization of this problem yields algorithms that are numerically unstable, no matter how close the initialization is to the true geometry. We then propose a computational scheme to overcome this problem, resulting in provably stable algorithms that converge to (local) minima of the cost functional. Although we restrict our attention to Lambertian objects with uniform albedo, extensions of our framework are conceivable.