Organizational Unit:
School of Computational Science and Engineering
School of Computational Science and Engineering
Permanent Link
Research Organization Registry ID
Description
Previous Names
Parent Organization
Parent Organization
Organizational Unit
Includes Organization(s)
ArchiveSpace Name Record
Publication Search Results
Now showing
1 - 4 of 4
-
ItemLearning-based search algorithm design(Georgia Institute of Technology, 2023-07-27) Chen, BinghongClassical search algorithms, such as A* search, evolutionary search, and Monte Carlo tree search, play a central role in solving many combinatorial optimization problems, including robotic path planning, molecule optimization, and playing Go. Conventionally, domain experts design these algorithms analytically by hand. However, such an algorithm design process is not data-driven and cannot benefit from the ever-increasing volumes of data. In this dissertation, we introduce a series of learning-to-search methods for different types of search space. We show that both search efficiency and effectiveness can be improved by learning search algorithms from historical data. Specifically, we focus on addressing challenges in a number of continuous and discrete search spaces with applications in robotics and drug design. High-dimensional continuous space. To search for solutions in high-dimensional continuous space, we resort to sampling-based methods to avoid explicitly discretizing the space. We propose a neural path planner that learns from prior experience to solve new path planning problems. Compared to classical methods, our learning-to-search approach achieves higher sample efficiency in high dimensions and can benefit from prior experience in similar environments. Discrete graph space. Finding graph structures with desired properties is a challenging problem. We present a learning-based evolutionary search algorithm to optimize molecules for desired properties. The proposed algorithm leverages a graph explainable model and the REINFORCE algorithm to generate better molecules on a multi-property molecule optimization benchmark. Discrete decomposable combinatorial space. We present a framework to search for solutions to recursively decomposable problems, based on the AND-OR tree representation that efficiently describes the search space. For the retrosynthesis planning problem, we introduce a learning-based A*-like search algorithm that finds high-quality synthetic routes for target molecules efficiently. The proposed algorithm builds on top of the AND-OR search tree and provides theoretical guarantees similar to the A* algorithm. Continuous molecular conformational space. We present a framework to search for molecular conformers with low energy, based on an Equivariant Transformer Forcefield. This strategy begins with an initial set of conformers, which are subsequently refined through structural optimization. We demonstrate that our ETF-based optimization significantly improves the quality of the conformers generated by state-of-the-art methods, achieving a 45\% reduction in the distance to the reference conformers.
-
ItemKnowledge Reasoning with Graph Neural Networks(Georgia Institute of Technology, 2021-12-15) Zhang, YuyuKnowledge reasoning is the process of drawing conclusions from existing facts and rules, which requires a range of capabilities including but not limited to understanding concepts, applying logic, and calibrating or validating architecture based on existing knowledge. With the explosive growth of communication techniques and mobile devices, much of collective human knowledge resides on the Internet today, in unstructured and semi-structured forms such as text, tables, images, videos, etc. It is overwhelmingly difficult for human to navigate the gigantic Internet knowledge without the help of intelligent systems such as search engines and question answering systems. To serve various information needs, in this thesis, we develop methods to perform knowledge reasoning over both structured and unstructured data. This thesis attempts to answer the following research questions on the topic of knowledge reasoning: (1) How to perform multi-hop reasoning over knowledge graphs? How should we leverage graph neural networks to learn graph-aware representations efficiently? And, how to systematically handle the noise in human questions? (2) How to combine deep learning and symbolic reasoning in a consistent probabilistic framework? How to make the inference efficient and scalable for large-scale knowledge graphs? Can we strike a balance between the representation power and the simplicity of the model? (3) What is the reasoning pattern of graph neural networks for knowledge-aware QA tasks? Can those elaborately designed GNN modules really perform complex reasoning process? Are they under- or over-complicated? Can we design a much simpler yet effective model to achieve comparable performance? (4) How to build an open-domain question answering system that can reason over multiple retrieved documents? How to efficiently rank and filter the retrieved documents to reduce the noise for the downstream answer prediction module? How to propagate and assemble the information among multiple retrieved documents? (5) How to answer the questions that require numerical reasoning over textual passages? How to enable pre-trained language models to perform numerical reasoning? We explored the research questions above and discovered that graph neural networks can be leveraged as a powerful tool for various knowledge reasoning tasks over both structured and unstructured knowledge sources. On structured graph-based knowledge source, we build graph neural networks on top of the graph structure to capture the topology information for downstream reasoning tasks. On unstructured text-based knowledge source, we first identify graph-structured information such as entity co-occurrence and entity-number binding, and then employ graph neural networks to reason over the constructed graphs, working together with pre-trained language models to handle unstructured part of the knowledge source.
-
ItemDeep representation learning on hypersphere(Georgia Institute of Technology, 2020-07-27) Liu, WeiyangHow to efficiently learn discriminative deep features is arguably one of the core problems in deep learning, since it can benefit a lot of downstream tasks such as visual recognition, object detection, semantic segmentation, etc. In this dissertation, we present a unified deep representation learning framework on hypersphere, which inherently introduces a novel hyperspherical inductive bias into deep neural networks. We show that our framework is well motivated from both empirical observations and theories. We discuss our framework from four distinct perspectives: (1) learning objectives on hypersphere; (2) neural architectures on hypersphere; (3) regularizations on hypersphere; (4) hyperspherical training paradigm. From the first three perspectives, we explain how we can utilize the idea of hyperspherical learning to revisit and reinvent corresponding components in deep learning. From the last perspective, we propose a general neural training framework that is heavily inspired by hyperspherical learning. We conduct comprehensive experiments on many applications to demonstrate that our deep hyperspherical learning framework yields better generalization, faster convergence and stronger adversarial robustness compared to the standard deep learning framework.
-
ItemLearning neural algorithms with graph structures(Georgia Institute of Technology, 2020-01-13) Dai, HanjunGraph structures, like syntax trees, social networks, and programs, are ubiquitous in many real world applications including knowledge graph inference, chemistry and social network analysis. Over the past several decades, many expert-designed algorithms on graphs have been proposed with nice theoretical properties. However most of them are not data-driven, and will not benefit from the growing scale of available data. Recent advances in deep learning have shown strong empirical performances for images, texts and signals, typically with little domain knowledge. However the combinatorial and discrete nature of the graph data makes it non-trivial to apply neural networks in this domain. Based on the pros and cons of these two, this thesis will discuss several aspects on how to build a tight connection between neural networks and the classical algorithms for graphs. Specifically: - Algorithm inspired deep graph learning: The existing algorithms provide an inspiration of deep architecture design, for both the discriminative learning and generative modeling of graphs. Regarding the discriminative representation learning, we show how the graphical model inference algorithms can inspire the design of graph neural networks for chemistry and bioinformatics applications, and how to scale it up with the idea borrowed from steady states algorithms like PageRank; for generative modeling, we build an HSMM inspired neural segmental generative modeling for signal sequences; and for a class of graphs, we leverage the idea of attribute grammar for syntax trees to help regulate the deep networks. - Deep learning enhanced graph algorithms: the algorithm framework has procedures that can be enhanced by learnable deep network components. We demonstrate by learning the heuristic function in greedy algorithms with reinforcement learning for combinatorial optimization problems over graphs, such as vertex cover and max cut, and optimal touring problem for real world applications like fuzzing. - Towards Inductive reasoning with graph structures: As the algorithm structure generally provides a good inductive bias for the problem, we take an initial step towards inductive reasoning for such structure, where we make attempts to reason about the loop invariant for program verification and the reaction templates for retrosynthesis structured prediction.