Enabling semantically richer queries over unstructured data
Author(s)
Chunduri, Pramod
Advisor(s)
Editor(s)
Collections
Supplementary to:
Permanent Link
Abstract
Querying unstructured data such as video, audio, and text is critical for domains ranging from traffic surveillance to healthcare and finance. Modern AI models (e.g., vision and language models) unlock significant potential for extracting fine-grained information from such data. However, current data systems that leverage these models primarily focus on efficiently executing semantically simple queries.
This thesis argues that enabling semantically rich queries over unstructured data requires rethinking both the query execution strategies and the query interfaces. To this end, we develop four systems that can efficiently and accurately process semantically rich queries over unstructured data.
First, we present Zeus, a video analytics system that efficiently localizes complex actions in videos using a reinforcement learning (RL)-based query executor. By using accuracy-based rewards during query planning, Zeus substantially improves efficiency while meeting user-specified accuracy targets.
Next, we propose Tracer, an adaptive query processing framework for multi-camera re-identification queries. Tracer uses a recurrent network with a probabilistic search model to optimally select camera feeds to process at each time step. Tracer significantly reduces the cost of re-identification queries on synthetically generated and real-world datasets.
We then introduce SketchQL, a visual query interface that allows users to sketch complex video moments. SketchQL maps these sketches to fine-grained video moments using a transformer model trained on synthetically generated data. SketchQL greatly enhances the usability and accuracy of fine-grained video moment retrieval.
Finally, we present Halo, a long-context question answering (QA) framework designed for domain-augmented queries. Halo incorporates domain knowledge into the QA pipeline via a Domain Hints interface, allowing users to specify structured suggestions that augment the original query. A three-stage execution pipeline applies these hints automatically and optimally, improving both the efficiency and accuracy of long-context QA.
Sponsor
Date
2025-07-29
Extent
Resource Type
Text
Resource Subtype
Dissertation