Title:
Harnessing Synthetic Data for Robust and Reliable Vision
Harnessing Synthetic Data for Robust and Reliable Vision
Author(s)
Chattopadhyay, Prithvijit
Advisor(s)
Hoffman, Judy
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
Progress in computer vision has been driven by models trained on large amounts of exemplar data for different tasks. These exemplar data sources intend to capture task-specific information and instance-level variations that a trained model will likely encounter in the wild. However, for conditions where curating lots of labeled real-world data is prohibitively expensive, synthetic data can serve as a cost-effective alternative. Synthetic data sources offer a few key benefits: fast access to labeled task-specific data at scale, labels across varying task complexities, and curation of labeled data across diverse conditions in a controlled manner.
This thesis demonstrates how “controlled variations” in synthetic data can be used to develop robust and reliable vision models. Controlled variations refer to intentional, systematic modifications to synthetic data, designed to either explore specific aspects of model behavior or improve model transfer across distributions. In Chapters 3 and 4, we discuss applying controlled variations internally at the data-engine (simulator) stage to create diverse instances to systematically investigate the robustness of trained vision models. In Chapters 5 and 6, we discuss how applying controlled variations externally as perturbations or data augmentations (to intermediate features or input images) can enable model transfer across changing visual distributions. Finally, in Chapter 7, we discuss how controlled variations applied externally on synthetic data can ensure reliability of predictions made on real data distributions. We conclude by summarizing takeaways and outlining potential future research directions.
Sponsor
Date Issued
2024-07-16
Extent
Resource Type
Text
Resource Subtype
Dissertation