(Georgia Institute of Technology, 2019-09-20)
Chen, Shang-Tse
While Artificial Intelligence (AI) has tremendous potential as a defense against real-world cybersecurity threats, understanding the capabilities and robustness of AI remains a fundamental challenge. This dissertation tackles problems essential to successful deployment of AI in security settings and is comprised of the following three interrelated research thrusts. (1) Adversarial Attack and Defense of Deep Neural Networks: We discover vulnerabilities of deep neural networks in real-world settings and the countermeasures to mitigate the threat. We develop ShapeShifter, the first targeted physical adversarial attack that fools state-of-the-art object detectors. For defenses, we develop SHIELD, an efficient defense leveraging stochastic image compression, and UnMask, a knowledge-based adversarial detection and defense framework. (2) Theoretically Principled Defense via Game Theory and ML: We develop new theories that guide defense resources allocation to guard against unexpected attacks and catastrophic events, using a novel online decision-making framework that compels players to employ ``diversified'' mixed strategies. Furthermore, by leveraging the deep connection between game theory and boosting, we develop a communication-efficient distributed boosting algorithm with strong theoretical guarantees in the agnostic learning setting. (3) Using AI to Protect Enterprise and Society: We show how AI can be used in real enterprise environment with a novel framework called Virtual Product that predicts potential enterprise cyber threats. Beyond cybersecurity, we also develop the Firebird framework to help
municipal fire departments prioritize fire inspections. Our work has made multiple important contributions to both theory and practice: our distributed boosting algorithm solved an open problem of distributed learning; ShaperShifter motivated a new DARPA program (GARD); Virtual Product led to two patents; and Firebird was highlighted by National Fire Protection Association as a best practice for using data to inform fire inspections.
(Georgia Institute of Technology, 2019)
Freitas, Scott; Chen, Shang-Tse; Chau, Duen Horng
Deep learning models are being integrated into a wide range of
high-impact, security-critical systems, from self-driving cars to biomedical
diagnosis. However, recent research has demonstrated that
many of these deep learning architectures are highly vulnerable
to adversarial attacks—highlighting the vital need for defensive
techniques to detect and mitigate these attacks before they occur.
To combat these adversarial attacks, we developed UnMask,
a knowledge-based adversarial detection and defense framework.
The core idea behind UnMask is to protect these models by verifying
that an image’s predicted class (“bird”) contains the expected
building blocks (e.g., beak, wings, eyes). For example, if an image is
classified as “bird”, but the extracted building blocks are wheel, seat
and frame, the model may be under attack. UnMask detects such
attacks and defends the model by rectifying the misclassification,
re-classifying the image based on its extracted building blocks. Our
extensive evaluation shows that UnMask (1) detects up to 92.9%
of attacks, with a false positive rate of 9.67% and (2) defends the
model by correctly classifying up to 92.24% of adversarial images
produced by the current strongest attack, Projected Gradient Descent,
in the gray-box setting. Our proposed method is architecture
agnostic and fast. To enable reproducibility of our research, we
have anonymously open-sourced our code and large newly-curated
dataset (~5GB) on GitHub (https://github.com/unmaskd/UnMask).