Title:
Bridging distributional discrepancy with temporal dynamics for video understanding
Bridging distributional discrepancy with temporal dynamics for video understanding
Author(s)
Chen, Min-Hung
Advisor(s)
AlRegib, Ghassan
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
Video has become one of the major media in our society, bringing considerable interests in the development of video analysis techniques for various applications. Temporal Dynamics, which characterize how information changes along time, is the key component for videos. However, it is still not clear how temporal dynamics benefit video tasks, especially for the cross-domain case, which is close to real-world scenarios. Therefore, the objective of this thesis is to effectively exploit temporal dynamics from videos to tackle distributional discrepancy problems for video understanding. To achieve this objective, firstly I identified the benefits for exploiting temporal dynamics for videos, including proposing Temporal Segment LSTM (TS-LSTM) and Inception-style Temporal-ConvNet (Temporal-Inception) for general video understanding, and demonstrating that temporal dynamics can help reduce temporal variations for cross-domain video understanding. Since most previous work only evaluates the performance on small-scale datasets with little domain discrepancy, I collected two large-scale datasets for video domain adaptation: UCF HMDB_full and Kinetics-Gameplay to facilitate cross-domain video research, and proposed Temporal Attentive Adversarial Adaptation Network (TA3N) to simultaneously attend, align and learn temporal dynamics across domains. Finally, to utilize temporal dynamics from unlabeled videos for action segmentation, I proposed Self-Supervised Temporal Domain Adaptation (SSTDA) to jointly align cross-domain feature spaces embedded with local and global temporal dynamics by two self-supervised auxiliary tasks, binary and sequential domain prediction, and demonstrated the usefulness of adapting to unlabeled videos across variations.
Sponsor
Date Issued
2020-05-21
Extent
Resource Type
Text
Resource Subtype
Dissertation