Organizational Unit:

School of Interactive Computing

Permanent Link

https://hdl.handle.net/1853/70783

Parent Organization

Organizational Unit

College of Computing

ArchiveSpace Name Record

https://finding-aids.library.gatech.edu/agents/corporate_entities/1113

Full item page

Publication Search Results

Now showing 1 - 10 of 74

Information Extraction on Scientific Literature under Limited Supervision

(Georgia Institute of Technology, 2023-12-12) Bai, Fan

The exponential growth of scientific literature presents both challenges and opportunities for researchers across various disciplines. Effectively extracting pertinent information from this extensive corpus is crucial for advancing knowledge, enhancing collaboration, and driving innovation. However, manual extraction is a laborious and time-consuming process, underscoring the demand for automated solutions. Information extraction (IE), a sub-field of natural language processing (NLP) focused on automatically extracting structured information from unstructured data sources, plays a crucial role in addressing this challenge. Despite their success, many IE methods often require substantial human-annotated data, which might not be easily accessible, particularly in specialized scientific domains. This highlights the need for adaptable and robust techniques capable of functioning with limited supervision. In this thesis, we study the task of information extraction on scientific literature, particularly addressing the challenge of limited (human) supervision. Specifically, our work has delved into four key dimensions of this problem. First, we explore the potential of harnessing easily accessible resources, like knowledge bases, to develop IE systems without direct human supervision. Second, we examine the use of pre-trained language models to create effective and efficient scientific IE systems, experimenting with various fine-tuning architectures and learning strategies. Next, we investigate the balance between the labor expenditure of human annotation and the computational cost linked with domain-specific pre-training, to achieve optimal performance under the budget constraints. Lastly, we capitalize on the emerging capabilities of large pre-trained language models by showcasing how information extraction can be achieved solely based on a human-crafted data schema. Through these explorations, this thesis aims to lay a solid foundation for the continued advancement of scientific IE under limited supervision.
Lifelong Machine Learning without Lifelong Data Retention

(Georgia Institute of Technology, 2023-12-10) Smith, James Seale

Machine learning models suffer from a phenomenon known as catastrophic forgetting when learning novel concepts from continuously shifting training data. Typical solutions for this continual learning problem require extensive replay of previously seen data, which increases memory costs and may violate data privacy. To address these challenges, we first explore replacing this replay data with alternatives: (i) unlabeled data “from the wild” and (ii) synthetic data generated via model inversion. Our work using this alternative replay data boasts strong performance on replay-free continual learning for image classification. Next, we consider an alternative solution to entirely replace replay data: pre-training. Specifically, we leverage strongly pre-trained models and continuously edit them with prompts and low-rank adapters for both (i) image classification and (ii) natural-language visual reasoning. Finally, we extend the idea of continual learning using pre-trained models to the proposed setting of continual customization of text-to-image diffusion models. We hope that our work on enabling models to learn from evolving data distributions and adapt to new tasks will help unlock the full potential of machine learning in addressing emerging real-world challenges.
Machine Learning for Agile Robotic Control

(Georgia Institute of Technology, 2023-12-06) Wagener, Nolan C.

Roboticists typically exploit structure in a problem, such as by modeling the mechanics of a system, to generate solutions for a given task. However, this structure can limit flexibility and require practitioners to reason about challenging phenomena, such as contacts in mechanics. Data, conversely, provides much more flexibility and, when combined with deep neural networks, has given rise to powerful models in vision and language, all with little hand-engineered structure. While it is tempting to fully forego structure in favor of learning-based methods for robotics, we show how data and learning can be gracefully incorporated in a structured way. In particular, we focus on the control setting, and we demonstrate that robotic control offers a variety of modes that data can be utilized. First, we show that data can be used in a model-based fashion to train a neural network that approximates complex dynamics and which can be used within a model predictive controller (MPC). Then, we show that the MPC process is itself an instance of online learning and demonstrate how to synthesize MPC algorithms from a common online learning algorithm. We apply both of the aforementioned approaches on a real-world aggressive driving task and show that they can accomplish the task. Next, we consider the safe reinforcement learning problem and show that safety interventions can be used as a learning signal to have an agent learn to become safe without needing to execute unsafe actions in the environment. Finally, we consider the simulated humanoid domain and show that pre-collected human motions can act as a strong inductive bias to ground motions learned by the humanoid agent.
Leveraging Low-Dimensional Geometry for Search and Ranking

(Georgia Institute of Technology, 2023-12-06) Fenu, Stefano

There is a substantial body of work on search and ranking in computer science, but less attention has been paid to the question of how to learn geometric data representations that are amenable to search and ranking tasks. Index-based datastructures for search are commonplace, but these discard structural features of the data, often have large memory profiles, and scale poorly with data dimension. Geometric search techniques do exist, but few analogous search datastructures or preprocessing algorithms exist that leverage spatial structure in data to increase search performance. The aim of the research detailed here is to show that leveraging low-dimensional geometry can improve the performance of search and ranking over index-only methods, and that there are dimensionality reduction techniques that can make spatial search algorithms more effective without any additional memory overhead. This work accomplishes these aims by developing methods for: Learning low-dimensional coordinate embeddings explicitly for the purpose of search and ranking; and actively querying and constructing searchable embeddings to minimize user-labeling costs. This dissertation will further provide scalable versions of these algorithms and demonstrate their effectiveness across a broad range of problem domains including visual, text, and educational data. These performance improvements will allow human-in-the-loop search of larger datasets and enable new applications in preference search and ranking.
Controllability and Uncertainty in Generative Models

(Georgia Institute of Technology, 2023-12-06) Ham, Cusuh

This dissertation describes methods for enhancing generative models with either added controllability or expressiveness of uncertainty, demonstrating how a strong prior enables both features. One general approach is to introduce new architectures or training objectives. However, current trends towards massive upscaling of model size, training data, and computational resources can make retraining or fine-tuning difficult and expensive. Thus, another approach is to build upon existing pre-trained models. We consider both types of approaches with an emphasis on the latter. We first tackle the tasks of controllable image synthesis and uncertainty estimation through training-based methods and then switch focus towards computationally-efficient methods that do not require direct updates to the base model's parameters. We conclude by discussing future directions based on the insights from our findings.
Processes and outcomes of systems thinking in an interactive modeling environment

(Georgia Institute of Technology, 2023-09-06) An, Sungeun

Modern society is full of natural, social, technological, and socio-technical systems, and thus systems thinking is an essential skill for prospering in the modern society. Developing interactive environments for supporting learning about complex systems requires a robust understanding how learners engage in systems thinking in various learning contexts. In this interdisciplinary work, I use theories and techniques from cognitive science, learning science, and artificial intelligence to develop an understanding of processes and outcomes of systems thinking for college students in pedagogical learning contexts and unspecified learners in self-directed learning contests in the domain of ecology. To achieve this goal, I present the Virtual Experimentation Research Assistant (VERA; vera.cc.gatech.edu)-- an interactive modeling environment to promote understanding and reasoning about ecological systems. VERA enables learners to access large-scale biological knowledge from the Encyclopedia of Life (EOL), construct conceptual models of ecological systems, run agent-based simulations of these models, and revise the models and simulations as needed to explain ecological phenomena. I have used VERA to complete four studies. The first two field studies were conducted in pedagogical contexts. The first study explored the effects of modeling in acquiring domain knowledge. I found that engaging in ecological modeling using VERA helped college students acquire biological knowledge. I also found that access to large-scale domain knowledge helped them construct more complex models and develop a larger number of hypotheses for a given problem. The second study investigated college students’ behaviors in estimating the parameters for agent-based simulations. I discovered that college students use multiple cognitive strategies for parameter estimation such as systematic search, problem reduction/decomposition, and global/local search. VERA is now accessible through Smithsonian Institution’s EOL website (eol.org), and it is used by thousands of self-directed learners around the world. The third study conducted a fine-grained analysis of self-directed learners' behaviors and models outside pedagogical contexts. I used a variety of learning analytics methods to analyze these behaviors including sequential data mining, hierarchical clustering, and Markov chain models. I found that self-directed learners engage in three types of behaviors: observation, construction, and exploration. The fourth study explored the effects of guided learning and self-exploration on modeling behaviors, model quality, and transfer of learning in a pedagogical context. Using in situ A/B experiments, I found that self-exploration in systems thinking leads to more complex and varied models whereas guidance in systems thinking does not have significant benefits in efficiency and accuracy for transfer of learning. Together these four studies lead to a robust understanding of how adult students learn about systems thinking and how to design interactive modeling environments to support self-directed systems thinking in open and ill-defined problems.
Building and Evaluating Controllable Models for Text Simplification

(Georgia Institute of Technology, 2023-08-17) Maddela, Mounica

Automatic Text Simplification (ATS) aims to improve the readability of texts with simpler grammar and word choices while preserving meaning. ATS is generally treated as a monolingual translation task where the input is a piece of text and the output is a simplified version of the input. One major drawback of the existing methods for ATS is the lack of controllability. ATS is an audience-dependent task and what constitutes simplified text for one group of users may not be acceptable for other groups. An ideal ATS system should be able to control various attributes of the generated text such as syntactic structures, length, readability levels, and word choices. Meanwhile, evaluating ATS systems is as important as building them because efficient automatic evaluation frameworks can accelerate the process of improving existing systems. However, the current automatic evaluation metrics for ATS focus on the semantic content of the simplified text but not the writing style. These metrics tend to favor conservative systems that make minimal changes to the input and inaccurately penalize simplifications that paraphrase the input. An ideal evaluation metric for ATS should not only capture simplification quality but also the different styles of simplification. In this dissertation, I develop controllable simplification systems and diverse automatic metrics for ATS. I propose two controllable approaches for ATS: a sentence simplification system that combines linguistic rules with Transformer models to generate simplified sentences at different readability levels and a lexical simplification system that leverages human judgments of word complexity to replace complex words with simpler phrases. Finally, I propose the first supervised automatic evaluation metric for ATS, LENS, which can capture multiple simplification styles and outperforms the existing metrics in evaluating diverse simplification systems. To train and evaluate LENS, I create SIMPEval, a new training and evaluation dataset for metrics that incorporates different types of simplification operations.
Passive Sensing Frameworks for the Future of Information Workers

(Georgia Institute of Technology, 2023-07-30) Swain, Vedant Das Das

Work sustains our livelihoods and is key to leading a fulfilling life. Improving our effectiveness at work helps us progress toward our goals and reclaim our lives for other activities. Traditionally we have used surveys to understand what makes workers more effective. However, these approaches do not sufficiently reflect workers as a part of a complex ecology --- comprising their daily activities, social dynamics, and the larger community. My thesis posits an alternative and more holistic approach. We can gain a more naturalistic understanding of worker effectiveness by leveraging everyday digital technology dispersed in their ecology as passive sensors. I focus my studies on information workers, a significant portion of white-collar work. This dissertation demonstrates the potential of repurposing everyday digital technology as an ecological lens to explain their performance and wellbeing. I have studied various technology readily available in information work, such as wearables, mobiles, desktops, Bluetooth beacons, WiFi router networks, and social media. My research presents (i) the utility of passively explaining worker wellbeing with behavioral traces and (ii) the acceptability of deploying such technologies for information work. In my studies, I applied statistical modeling and machine learning to show new ways to clarify indicators of worker experiences at the individual, group, and organizational levels. Later, I took a worker-centric perspective to situate such algorithmic inferences in today's work paradigm and describe the methodological and socio-technical challenges. My dissertation contributes to the future of work across multiple dimensions. First, it adds to behavioral computing research by showing computationally efficient and versatile opportunities to model passively collected behavioral traces and provides insight into worker effectiveness. Next, it refines organizational science by providing new opportunities to explain worker experiences by accounting for previously unforeseen behavioral dynamics. Last, it highlights the limits of this approach and provides evidence to suggest how these technologies should (and should not) manifest in the workplace. Collectively, my research aims to help workers by underscoring passive sensing practices that are more holistic, accurate, and humane.
Addressing Computing's Discrimination Problem: A Framework for Anti-Discriminatory Computing

(Georgia Institute of Technology, 2023-07-28) Schlesinger, Arielle Bea

Discrimination is a problem that impacts nearly every aspect of computing. Despite years and resources spent addressing diversity and inclusion, computing has not seen meaningful improvements. From the design and implementation of code to computer science classrooms to technology workplaces, computing's discrimination problem spans the entire tech industry. Decades of minimal progress addressing computing's discrimination problem requires new ways to approach discrimination throughout computing. To address computing's discrimination problem, we must be able to identify and understand the scope of the problem as well as its root causes. Both of the leading professional organizations in computing, the Association for Computing Machinery (ACM) and Institute of Electrical and Electronics Engineers (IEEE), state that a central aim of their work is to support technical advances that are beneficial for people—ACM’s tagline includes "where computing helps solves tomorrow’s problems" and IEEE’s tagline is "Advancing Technology for Humanity". In this dissertation, I discuss how people working in computing can address the root causes of computing's discrimination problem by reversing the systemic tendency to abstract away the complexity of humanity. Specifically, in my dissertation, I identify and cultivate an understanding of the root causes of computing's discrimination problem across various sub-areas of computing. I explain the anti-discriminatory theoretical framework that guides my work so that others can apply and extend this framework. I also report on four research studies across four sub-areas in computing—computer science (broadly), HCI, AI, and programming languages. These studies advance progress towards the goal of identifying and understanding the root causes of computing's discrimination problems. I explore why computing’s discrimination problem has endured and uncover targeted directions for future work to reduce discrimination in computing.
Understanding Social Requirements for Social Media Powered Artificial Intelligence (SOMPAI) for Mental Health Care

(Georgia Institute of Technology, 2023-07-25) Yoo, Dong Whi

In the field of medicine, mental health is unique as it entirely depends on the patient’s ability to express their cognitive and emotional states, symptom progression, and interpersonal relationships. This reliance creates extra challenges for patients and results in less effective evaluations and treatments. Recent advancements in artificial intelligence (AI) have been proposed as a means to develop more objective criteria and evidence for mental health treatments, with AI technologies being viewed as the potential solution to critical issues in mental health care, such as delayed, inaccurate, and ineffective care delivery. To support mental health practices, researchers from computer science, mental health, and other related fields have collaborated on developing AI models. However, despite decades of effort, these advancements have not been successfully integrated into real-world mental health contexts. This discrepancy between AI research and mental health practices can be framed as the socio-technical gap. It represents the intellectual challenge arising from differences between what technologies can provide and what users want to achieve in social contexts. Researchers in Computer-Supported Cooperative Work (CSCW) have pointed out that a socio-technical gap exists because users possess diverse roles, tasks, and procedures in social contexts and can fluidly navigate between them. However, technologies often lack understanding and flexibility in supporting these changes. AI technologies currently face a similar socio-technical gap as they lack social requirements such as nuance, flexibility, and ambiguity tolerance. Emerging research in Human-AI Interaction has revealed more detailed accounts of the social requirements for AI models. These studies propose working closely with AI technology end-users to understand the requirements from their perspectives. Building on these research efforts, this dissertation investigates the social requirements for mental health AI technologies from the standpoint of patients and clinicians, identifying them and envisioning design implications for future AI technologies. Mental health AI technologies utilize various data types, such as electronic health records, sensor-based data, and social media data. This dissertation focuses on social media-powered AI (SoMPAI) as many mental health patients use social media for treatment and recovery purposes, including self-disclosure, help-seeking, and peer support. Social media data can be a valuable source for understanding the mental health of people who are active on social media, particularly adolescents and young adults, who are the primary target of mental health treatment. Mental health patients’ social media data contain a rich amount of written text used in psychoanalysis for several mental disorders. Social media data can also reflect patients’ social activities, which can be valuable to mental health clinicians who must infer patients’ daily social lives from their self-reports during consultations. Therefore, this dissertation aims to understand the social requirements for social media-powered AI from both clinicians’ and patients’ perspectives by utilizing human-centered design approaches and working closely with clinicians and patients to understand their expectations and concerns. This dissertation contributes to several domains: it expands the concept of social requirements and the socio-technical gap in CSCW as they relate to mental health AI technologies; it provides empirical evidence, including the perspectives of mental health patients and clinicians, on expectations and concerns regarding AI models, contributing to recent Human-AI Interaction research; and the design implications of this dissertation will help develop implementable mental health AI technologies that can support current mental health practices.