Title:
Weakly Supervised Learning of Object Segmentations from Web-Scale Video
Weakly Supervised Learning of Object Segmentations from Web-Scale Video
Author(s)
Hartmann, Glenn
Grundmann, Matthias
Hoffman, Judy
Tsai, David
Kwatra, Vivek
Madani, Omid
Vijayanarasimhan, Sudheendra
Essa, Irfan
Rehg, James M.
Sukthankar, Rahul
Grundmann, Matthias
Hoffman, Judy
Tsai, David
Kwatra, Vivek
Madani, Omid
Vijayanarasimhan, Sudheendra
Essa, Irfan
Rehg, James M.
Sukthankar, Rahul
Advisor(s)
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
We propose to learn pixel-level segmentations of objects from
weakly labeled (tagged) internet videos. Specifically, given a large collection of raw YouTube content, along with potentially noisy tags, our
goal is to automatically generate spatiotemporal masks for each object, such as "dog", without employing any pre-trained object detectors.
We formulate this problem as learning weakly supervised classifiers for
a set of independent spatio-temporal segments. The object seeds obtained using segment-level classifiers are further refined using graphcuts
to generate high-precision object masks. Our results, obtained by training on a dataset of 20,000 YouTube videos weakly tagged into 15 classes,
demonstrate automatic extraction of pixel-level object masks. Evaluated
against a ground-truthed subset of 50,000 frames with pixel-level annotations, we confirm that our proposed methods can learn good object
masks just by watching YouTube.
Sponsor
Date Issued
2012-10
Extent
Resource Type
Text
Resource Subtype
Book Chapter
Proceedings
Proceedings