Title:
Modeling Natural-Image Spaces for Single-Label Image Classification & Photo-Realistic Style Transfer and Directionally Paired Principal Component Analysis for the Estimation of Coupled Data
Modeling Natural-Image Spaces for Single-Label Image Classification & Photo-Realistic Style Transfer and Directionally Paired Principal Component Analysis for the Estimation of Coupled Data
Author(s)
Fan, Yifei
Advisor(s)
Yezzi, Anthony
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
Despite the tremendous success of machine learning and data-driven approaches in the past decade, machine perception is much less robust and explainable compared to human perception. In this dissertation, we aim at a methodology that provides better explainability of machine-learning algorithms both under the linear-analysis framework and in the deep-learning domain. Under the linear-analysis framework, Principal Component Analysis (PCA) is a classic approach that offers better explainability via dimension reduction. In this dissertation, we extend the classic PCA for coupled yet partially observable test data. The proposed Directionally Paired Principal Component Analysis (DP-PCA) is the optimal linear model that performs dimension reduction and least-square regression between the coupled variable sets in the principal subspace, which leads to the lowest estimation errors at a fast speed and with the least storage requirement.
In the deep-learning domain, we provide a unified explanation of the behaviors of various machine-learning algorithms and the gap between human and machine perception with the proposed conceptual model of natural-image spaces. By formulating classification as a partition of the image space, we further provide a topological view of knowledge in machine perception by defining fundamental concepts, including information, knowledge, beliefs, and truths. We illustrate the benefits of the proposed conceptual image-space model and the topological view of knowledge in two concrete applications: single-label classification and photo stylization.
• For single-label classification, we demonstrate the usage of two types of hidden information in the image space, "poorly justified true beliefs" and "false beliefs," on improving and preserving classification accuracy. The usage of adversarial examples in the hidden information implies that the discrepancy between benign and adversarial examples are not irreconcilable. Through an empirical study on test-time smoothing defense against adversarial attacks, we present a non-monotonic relation between attacks and defenses as well as the large variance in robustness among categories and samples. Following the properties of natural-image spaces, we verify geometric causes of adversarial examples through carefully designed controlled experiments.
• For photo style recognition and transfer, we re-partition the image space with artistic presets and establish a controlled photo style recognition benchmark which disentangles styles from contents. Consequently, the style classifier behaves very differently from a content classifier in various vision tasks, and it can provide useful guidance signals which support global style transfer. By modeling the image space at the object level, we ensemble a content-aware local style transfer pipeline in which the proposed segmentation refinement module removes defects from inaccurate segmentation maps and supports feature blending at various levels.
Sponsor
Date Issued
2020-09-24
Extent
Resource Type
Text
Resource Subtype
Dissertation