Organizational Unit:
School of Computational Science and Engineering
School of Computational Science and Engineering
Permanent Link
Research Organization Registry ID
Description
Previous Names
Parent Organization
Parent Organization
Organizational Unit
Includes Organization(s)
ArchiveSpace Name Record
Publication Search Results
Now showing
1 - 4 of 4
-
ItemInteractive Scalable Discovery Of Concepts, Evolutions, And Vulnerabilities In Deep Learning(Georgia Institute of Technology, 2023-12-05) Park, HaekyuDeep Neural Networks (DNNs) are increasingly prevalent, but deciphering their operations is challenging. Such a lack of clarity undermines trust and problem-solving during deployment, highlighting the urgent need for interpretability. How can we efficiently summarize concepts models learn? How do these concepts evolve during training? When models are at risk from potential threats, how do we explain their vulnerabilities? We address these concerns with a human-centered approach, by developing novel systems to interpret learned concepts, their evolution, and potential vulnerabilities within deep learning. This thesis focuses on three key thrusts: (1) Scalable Automatic Visual Summarization of Concepts. We develop NeuroCartography, an interactive system that scalably summarizes and visualizes concepts learned by a large-scale DNN, such as InceptionV1 trained with 1.2M images. A large-scale human evaluation with 244 participants shows that NeuroCartography discovers coherent, human-meaningful concepts. (2) Insights to Reveal Model Vulnerabilities. We develop scalable interpretation techniques to visualize and identify internal elements in DNNs, which are susceptible to potential harms, aiming to understand how these defects lead to incorrect predictions. We develop first-of-its-kind interactive systems such as Bluff that visually compares the activation pathways for benign and attacked images in DNNs, and SkeletonVis that explains how attacks manipulate human joint detection in human action recognition models. (3) Scalable Discovery of Concept Evolution During Training. Our first-of-its-kind ConceptEvo unified interpretation framework holistically reveals the inception and evolution of learned concepts and their relationships during training. ConceptEvo enables powerful new ways to monitor model training and discover training issues, addressing critical limitations of existing post-training interpretation research. A large-scale human evaluation with 260 participants demonstrates that ConceptEvo identifies concept evolutions that are both meaningful to humans and important for class predictions. This thesis contributes to information visualization, deep learning, and crucially, their intersection. We have developed open-source interactive interfaces, scalable algorithms, and a unified framework for interpreting DNNs across different models. Our work impacts academia, industry, and the government. For example, our work has contributed to the DARPA GARD program (Garanteeing AI Robustness against Deception). Additionally, our work has been recognized through a J.P. Morgan AI PhD Fellowship and 2022 Rising Stars in IEEE EECS. NeuroCartography has been highlighted as a top visualization publication (top 1%) invited to SIGGRAPH.
-
ItemUnderstanding, Fortifying and Democratizing AI Security(Georgia Institute of Technology, 2022-04-19) Das, NilakshAs we steadily move towards an AI-powered utopia that could only be imagined in lofty fiction in the recent past, a formidable threat is emerging that endangers the acute capitalization of AI in our everyday lives. A growing body of adversarial machine learning research has revealed that deep neural networks — the workhorse of modern AI applications — are extremely vulnerable to adversarial examples. These are malicious inputs crafted by an attacker that can completely confuse deep neural networks into making incorrect predictions. Therefore, for people to have complete confidence in using AI applications, there is not only an urgent need to develop strong, practical solutions to defend real-world AI cyber-systems; there is also an equally pressing necessity to enable people to interpret AI vulnerabilities and understand how and why adversarial attacks and defenses work. It is also critical that the technologies for AI security be brought to the masses, and AI security research be as accessible and as pervasive as AI itself. After all, AI impacts people from all walks of life. This dissertation addresses these fundamental challenges through creating holistic interpretation techniques for better understanding of attacks and defenses, developing effective and principled defenses for protecting AI across input modalities, and building tools that enable scalable interactive experimentation with AI security and adversarial ML research. This dissertation has a vision of enhancing trust in AI by making AI security more accessible and adversarial ML education more equitable, while focusing on three complementary research thrusts: (1) Exposing AI Vulnerabilities through Visualization & Interpretable Representations. We develop intuitive interpretation techniques for deciphering adversarial attacks. (2) Mitigating Adversarial Examples Across Modalities & Tasks. We develop robust defenses which are generalizable across diverse AI tasks and input modalities. (3) Democratizing AI Security Research & Pedagogy with Scalable Interactive Experimentation. We enable researchers, practitioners and students to perform in-depth security testing of AI models through interactive experimentation. Our work has made a significant impact to industry and society: our research has produced novel defenses that have been tech-transferred to industry; our interactive visualization systems have significantly expanded the intuitive understanding of AI vulnerabilities; and our scalable AI security framework and research tools, becoming available to thousands of students, is transforming AI education at scale.
-
ItemDeveloping Robust Models, Algorithms, Databases and Tools With Applications to Cybersecurity and Healthcare(Georgia Institute of Technology, 2021-12-13) Freitas, ScottAs society and technology becomes increasingly interconnected, so does the threat landscape. Once isolated threats now pose serious concerns to highly interdependent systems, highlighting the fundamental need for robust machine learning. This dissertation contributes novel tools, algorithms, databases, and models—through the lens of robust machine learning—in a research effort to solve large-scale societal problems affecting millions of people in the areas of cybersecurity and healthcare. (1) Tools: We develop TIGER, the first comprehensive graph robustness toolbox; and our ROBUSTNESS SURVEY identifies critical yet missing areas of graph robustness research. (2) Algorithms: Our survey and toolbox reveal existing work has overlooked lateral attacks on computer authentication networks. We develop D2M, the first algorithmic framework to quantify and mitigate network vulnerability to lateral attacks by modeling lateral attack movement from a graph theoretic perspective. (3) Databases: To prevent lateral attacks altogether, we develop MALNET-GRAPH, the world’s largest cybersecurity graph database—containing over 1.2M graphs across 696 classes—and show the first large-scale results demonstrating the effectiveness of malware detection through a graph medium. We extend MALNET-GRAPH by constructing the largest binary-image cybersecurity database—containing 1.2M images, 133×more images than the only other public database—enabling new discoveries in malware detection and classification research restricted to a few industry labs (MALNET-IMAGE). (4) Models: To protect systems from adversarial attacks, we develop UNMASK, the first model that flags semantic incoherence in computer vision systems, which detects up to 96.75% of attacks, and defends the model by correctly classifying up to 93% of attacks. Inspired by UNMASK’s ability to protect computer visions systems from adversarial attack, we develop REST, which creates noise robust models through a novel combination of adversarial training, spectral regularization, and sparsity regularization. In the presence of noise, our method improves state-of-the-art sleep stage scoring by 71%—allowing us to diagnose sleep disorders earlier on and in the home environment—while using 19× less parameters and 15×less MFLOPS. Our work has made significant impact to industry and society: the UNMASK framework laid the foundation for a multi-million dollar DARPA GARD award; the TIGER toolbox for graph robustness analysis is a part of the Nvidia Data Science Teaching Kit, available to educators around the world; we released MALNET, the world’s largest graph classification database with 1.2M graphs; and the D2M framework has had major impact to Microsoft products, inspiring changes to the product’s approach to lateral attack detection.
-
ItemInteractive Scalable Interfaces for Machine Learning Interpretability(Georgia Institute of Technology, 2020-12-01) Hohman, FrederickData-driven paradigms now solve the world's hardest problems by automatically learning from data. Unfortunately, what is learned is often unknown to both the people who train the models and the people they impact. This has led to a rallying cry for machine learning interpretability. But how we enable interpretability? How do we scale up explanations for modern, complex models? And how can we best communicate them to people? Since machine learning now impacts people's daily lives, we answer these questions taking a human-centered perspective by designing and developing interactive interfaces that enable interpretability at scale and for everyone. This thesis focuses on: (1) Enabling machine learning interpretability: User research with practitioners guides the creation of our novel operationalization for interpretability, which helps tool builders design interactive systems for model and prediction explanations. We develop two such visualization systems, Gamut and TeleGam, which we deploy at Microsoft Research as a design probe to investigate the emerging practice of interpreting models. (2) Scaling deep learning interpretability: Our first-of-its-kind Interrogative Survey reveals critical yet understudied areas of deep learning interpretability research, such as the lack of higher-level explanations for neural networks. Through Summit, an interactive visualization system, we present the first scalable graph representation that summarizes and visualizes what features deep learning models learn and how those features interact to make predictions (e.g., InceptionNet trained on ImageNet with 1.2M+ images). (3) Communicating interpretability with interactive articles: We use interactive articles, a new medium on the web, to teach people about machine learning's capabilities and limitations, while developing a new interactive publishing initiative called the Parametric Press. From our success publishing interactive content at scale, we generalize and detail the affordances of Interactive Articles by connecting techniques used in practice and the theories and empirical evaluations put forth by diverse disciplines of research. This thesis contributes to information visualization, machine learning, and more importantly their intersection, including open-source interactive interfaces, scalable algorithms, and new, accessible communication paradigms. Our work is making significant impact in industry and society: our visualizations have been deployed and demoed at Microsoft and built into widely-used interpretability toolkits, our interactive articles have been read by 250,000+ people, and our interpretability research is supported by NASA.