Organizational Unit:
School of Computational Science and Engineering
School of Computational Science and Engineering
Permanent Link
Research Organization Registry ID
Description
Previous Names
Parent Organization
Parent Organization
Organizational Unit
Includes Organization(s)
ArchiveSpace Name Record
Publication Search Results
Now showing
1 - 8 of 8
-
ItemCooperation in Multi-Agent Reinforcement Learning(Georgia Institute of Technology, 2021-12-13) Yang, JiachenAs progress in reinforcement learning (RL) gives rise to increasingly general and powerful artificial intelligence, society needs to anticipate a possible future in which multiple RL agents must learn and interact in a shared multi-agent environment. When a single principal has oversight of the multi-agent system, how should agents learn to cooperate via centralized training to achieve individual and global objectives? When agents belong to self-interested principals with imperfectly-aligned objectives, how can cooperation emerge from fully-decentralized learning? This dissertation addresses both questions by proposing novel methods for multi-agent reinforcement learning (MARL) and demonstrating the empirical effectiveness of these methods in high-dimensional simulated environments. To address the first case, we propose new algorithms for fully-cooperative MARL in the paradigm of centralized training with decentralized execution. Firstly, we propose a method based on multi-agent curriculum learning and multi-agent credit assignment to address the setting where global optimality is defined as the attainment of all individual goals. Secondly, we propose a hierarchical MARL algorithm to discover and learn interpretable and useful skills for a multi-agent team to optimize a single team objective. Extensive experiments with ablations show the strengths of our approaches over state-of-the-art baselines. To address the second case, we propose learning algorithms to attain cooperation within a population of self-interested RL agents. We propose the design of a new agent who is equipped with the new ability to incentivize other RL agents and explicitly account for the other agents' learning process. This agent overcomes the challenging limitation of fully-decentralized training and generates emergent cooperation in difficult social dilemmas. Then, we extend and apply this technique to the problem of incentive design, where a central incentive designer explicitly optimizes a global objective only by intervening on the rewards of a population of independent RL agents. Experiments on the problem of optimal taxation in a simulated market economy demonstrate the effectiveness of this approach.
-
ItemOn Computation and Application of Optimal Transport(Georgia Institute of Technology, 2021-07-28) Xie, YujiaThe Optimal Transport (OT) problem naturally arises in various machine learning problems, where one needs to align data from multiple sources. For example, the training data and application scenarios oftentimes have a domain gap, e.g., the training data is annotated photos collected in the daytime, yet the application scenario is in dark hours. In this case, we need to align the two datasets, so that the annotation information can be shared across them. During my Ph.D. study, I propose scalable algorithms for efficient OT computation, and its novel applications in end-to-end learning. For OT computation, I consider both discrete cases and continuous cases. For the discrete cases, I develop an Inexact Proximal point method for exact Optimal Transport problem (IPOT) with the proximal operator approximately evaluated at each iteration using projections to the probability simplex. The algorithm (a) converges to exact Wasserstein distance with theoretical guarantee and robust regularization parameter selection, (b) alleviates numerical stability issue, (c) has similar computational complexity to Sinkhorn, and (d) avoids the shrinking problem when apply to generative models. Furthermore, a new algorithm is proposed based on IPOT to obtain sharper Wasserstein barycenter. For the continuous cases, I propose an implicit generative learning-based framework called SPOT (Scalable Push-forward of Optimal Transport). Specifically, we approximate the optimal transport plan by a pushforward of a reference distribution, and cast the optimal transport problem into a minimax problem. We then can solve OT problems efficiently using primal dual stochastic gradient-type algorithms. To explore the connections between OT and end-to-end learning, I developed a differentiable top-k operator, and a differentiable permutation step. For the top-k operation, i.e., finding the k largest or smallest elements from a collection of scores, is an important model component used in information retrieval, machine learning, and data mining. However, if the top-k operation is implemented in an algorithmic way, e.g., using bubble algorithm, the resulting model cannot be trained in an end-to-end way using prevalent gradient descent algorithms. This is because these implementations typically involve swapping indices, whose gradient cannot be computed. Moreover, the corresponding mapping from the input scores to the indicator vector of whether this element belongs to the top-k set is essentially discontinuous. To address the issue, we propose a smoothed approximation, namely the SOFT (Scalable Optimal transport-based diFferenTiable) top-k operator. Specifically, our SOFT top-k operator approximates the output of the top-k operation as the solution of an Entropic Optimal Transport (EOT) problem. The gradient of the SOFT operator can then be efficiently approximated based on the optimality conditions of EOT problem. We apply the proposed operator to the k-nearest neighbors and beam search algorithms, and demonstrate improved performance. For the differentiable permutation step, I connect optimal transport to a variant of regression problem, where the correspondence between input and output data is not available. Such shuffled data is commonly observed in many real-world problems. Taking flow cytometry as an example, the measuring instruments may not be able to maintain the correspondence between the samples and the measurements. Due to the combinatorial nature of the problem, most existing methods are only applicable when the sample size is small, and limited to linear regression models. To overcome such bottlenecks, we propose a new computational framework -- ROBOT -- for the shuffled regression problem, which is applicable to large data and complex nonlinear models. Specifically, we reformulate the regression without correspondence as a continuous optimization problem. Then by exploiting the interaction between the regression model and the data correspondence, we develop a hypergradient approach based on differentiable programming techniques. Such a hypergradient approach essentially views the data correspondence as an operator of the regression, and therefore allows us to find a better descent direction for the model parameter by differentiating through the data correspondence. ROBOT can be further extended to the inexact correspondence setting, where there may not be an exact alignment between the input and output data. Thorough numerical experiments show that ROBOT achieves better performance than existing methods in both linear and nonlinear regression tasks, including real-world applications such as flow cytometry and multi-object tracking.
-
ItemLearning dynamic processes over graphs(Georgia Institute of Technology, 2020-07-09) Trivedi, RakshitGraphs appear as a versatile representation of information across domains spanning social networks, biological networks, transportation networks, molecular structures, knowledge networks, web information network and many more. Graphs represent heterogeneous information about the real-world entities and complex relationships between them in a very succinct manner. At the same time, graphs exhibit combinatorial, discrete and non- Euclidean properties in addition to being inherently sparse and incomplete which poses several challenges to techniques that analyze and study these graph structures. There exist various approaches across different fields spanning network science, game theory, stochastic process and others that provide excellent theoretical and analytical tools with interpretability benefits to analyze these networks. However, such approaches do not learn from data and make assumptions about real-world that capture only subset of properties. More importantly, they do not support predictive capabilities critical for decision making applications. In this thesis, we develop novel data driven learning approaches that incorporate useful inductive biases inspired from these classical approaches. The resulting learning approaches exhibit more general properties that go beyond conventional probabilistic assumptions and allow for building transferable and interpretable modules. We build these approaches anchored around two fundamental questions: (i) (Formation Pro- cess) How do these networks come into existence? and (ii) (Temporal Evolution Process) How do real-world networks evolve over time? First, we focus on the challenge of learning in a setting with highly sparse and in- complete knowledge graphs, where it is important to leverage multiple input graphs to sup- port accurate performance for variety of downstream applications such as recommendation, search and question-answering systems. Specifically, we develop a large-scale multi-graph deep relational learning framework that identifies entity linkage as a vital component of data fusion and learns to jointly perform representation learning and graph linkage across multiple graphs with applications to relational reasoning and knowledge construction. Next, we consider networks that evolve over time and propose a generative model of dynamic graphs that is useful to encode evolving network information into low-dimensional representations that facilitate accurate downstream event prediction tasks. Our approach relies on the coevolution principle of network structure evolution and network activities being tightly couple processes and develops a multi time scale temporal point process formulation parameterized by a recurrent architecture comprising of a novel Temporal Attention mechanism. Representation learning is posed as a latent mediation process – observed network processes evolve the state of nodes, while this node evolution governs future dynamics of observed processes and applied to downstream dynamic link prediction tasks and time prediction of future realizations (events) of both observed processes. Finally, we investigate the implication of adopting the optimization perspective of net- work formation mechanisms for building learning approaches for graph structured data. In this work, we first focus on global mechanisms that govern the formation of links in the network and build an inverse reinforcement learning based algorithm to jointly discover latent mechanisms directly from observed data, optimization of which enables a graph construction procedure capable of producing graphs with properties similar to observed data. Such an approach facilitates transfer and generalization properties and has been applied to variety of real-world graphs. In the last part, we consider the settings where the agents forming links are strategic and build a learnable model of network emergence games that jointly discovers the underlying payoff mechanisms and strategic profiles of agents from the data. This approach enables learning interpretable and transferable payoffs while the learned game as a model facilitates strategic prediction tasks, both of which are applied to several real world networks.
-
ItemPoint process modeling and optimization of social networks(Georgia Institute of Technology, 2018-04-05) Farajtabar, MehrdadOnline social media such as Facebook and Twitter and communities such as Wikipedia and Stackoverflow turn to become an inseparable part of today's lifestyle. Users usually participate via a variety of ways like sharing text and photos, asking questions, finding friends, and favoring contents. Theses activities produce sequences of events data whose complex temporal dynamics need to be studied and is of many practical, economic, and societal interest. We propose a novel framework based on multivariate temporal point processes that is used for modeling, optimization, and inference of processes taken place over networks. In the modeling part, we propose a temporal point process model for joint dynamics of information propagation and structure evolution in networks. These two highly intertwined stochastic processes have been predominantly studied separately, ignoring their co-evolutionary dynamics. Our model allows us to efficiently simulate interleaved diffusion and network events, and generate traces obeying common diffusion and network patterns observed in real-world networks. In the optimization part, we establish the fundamentals of intervention and control in networks by combining the rich area of temporal point processes and the well-developed framework of Markov decision processes. We use point processes to capture both endogenous and exogenous events in social networks and formulate the problem as a Markov decision problem. Our methodology helps finding the optimal policy that balances the high present reward and large penalty on low future outcome in the presence of extensive uncertainties. In the inference part, we propose an intensity-free approach for point processes modeling that transforms the nuisance process to the target one. Furthermore, we train our deep neural network model using a likelihood-free approach leveraging Wasserstein distance between point processes.
-
ItemModeling, learning, and inference of high-dimensional asynchronous event data(Georgia Institute of Technology, 2016-05-05) Du, NanThe increasing availability of temporal-spatial events produced from natural and social systems provides new opportunities and challenges for effective modeling the latent dynamics which inherently govern these seemly ``random'' data. In our work, we propose a unified probabilistic framework based on multivariate point processes to better predict `who will do what by when and where?' in the future. This framework comprises a systematic paradigm for modeling, learning, and making inference of large-scale asynchronous high-dimensional event data. With this common framework, we contribute in the following three aspects. Accurate Modeling: we first propose non-parametric and topic-modulated multivariate terminating point processes to capture continuous-time heterogeneous information diffusions. We then develop the low-rank Hawkes process to describe the recurrent temporal interactions among different types of entities. We also build a link between the recurrent neural network and the temporal point process to learn a general representation of the influence from the past event history. Finally, we establish a previously unexplored connection between Bayesian Nonparametrics and temporal point processes to jointly model the temporal data and other type of additional information. Efficient Learning: we develop a robust structure learning algorithm via group lasso, which is able to efficiently uncover sparse heterogeneous interdependent relations specified via vectorized parameters among the dimensions. We also propose an efficient nonnegative matrix rank minimization algorithm, which elegantly inherits the advantages from both the proximal methods and the conditional gradient methods to solve the matrix rank minimization problem under different constraints. Finally, in the data streaming setting, we develop a Bayesian inference algorithm for inferring latent variables and updating the respective model parameters based on both temporal and textual information, which achieves almost constant processing time per data sample. Scalable Inference: another important aspect of our research is to make future predictions by exploiting the learned models. Specifically, based on the terminating processes, we develop the first scalable influence estimation algorithm in continuous-time diffusion networks with provable performance guarantees. Based on the low-rank Hawkes processes, we develop the first time-sensitive recommendation algorithm, which not only can recommend the most relevant item specific to a given moment, but also can predict the next returning time for a user to a designated service. Finally, based on the recurrent point processes, we have derived an analytic solution to shape the overall network activities of users. We show that our method can provide fine-grained control over user activities in a time-sensitive fashion.
-
ItemInfluence modeling in behavioral data(Georgia Institute of Technology, 2015-05-15) Li, LiangdaUnderstanding influence in behavioral data has become increasingly important in analyzing the cause and effect of human behaviors under various scenarios. Influence modeling enables us to learn not only how human behaviors drive the diffusion of memes spread in different kinds of networks, but also the chain reactions evolve in the sequential behaviors of people. In this thesis, I propose to investigate into appropriate probabilistic models for efficiently and effectively modeling influence, and the applications and extensions of the proposed models to analyze behavioral data in computational sustainability and information search. One fundamental problem in influence modeling is the learning of the degree of influence between individuals, which we called social infectivity. In the first part of this work, we study how to efficient and effective learn social infectivity in diffusion phenomenon in social networks and other applications. We replace the pairwise infectivity in the multidimensional Hawkes processes with linear combinations of those time-varying features, and optimize the associated coefficients with lasso regularization on coefficients. In the second part of this work, we investigate the modeling of influence between marked events in the application of energy consumption, which tracks the diffusion of mixed daily routines of household members. Specifically, we leverage temporal and energy consumption information recorded by smart meters in households for influence modeling, through a novel probabilistic model that combines marked point processes with topic models. The learned influence is supposed to reveal the sequential appliance usage pattern of household members, and thereby helps address the problem of energy disaggregation. In the third part of this work, we investigate a complex influence modeling scenario which requires simultaneous learning of both infectivity and influence existence. Specifically, we study the modeling of influence in search behaviors, where the influence tracks the diffusion of mixed search intents of search engine users in information search. We leverage temporal and textual information in query logs for influence modeling, through a novel probabilistic model that combines point processes with topic models. The learned influence is supposed to link queries that serve for the same formation need, and thereby helps address the problem of search task identification. The modeling of influence with the Markov property also help us to understand the chain reaction in the interaction of search engine users with query auto-completion (QAC) engine within each query session. The fourth part of this work studies how a user's present interaction with a QAC engine influences his/her interaction in the next step. We propose a novel probabilistic model based on Markov processes, which leverage such influence in the prediction of users' click choices of suggested queries of QAC engines, and accordingly improve the suggestions to better satisfy users' search intents. In the fifth part of this work, we study the mutual influence between users' behaviors on query auto-completion (QAC) logs and normal click logs across different query sessions. We propose a probabilistic model to explore the correlation between user' behavior patterns on QAC and click logs, and expect to capture the mutual influence between users' behaviors in QAC and click sessions.
-
ItemExtending low-rank matrix factorizations for emerging applications(Georgia Institute of Technology, 2013-08-12) Zhou, KeLow-rank matrix factorizations have become increasingly popular to project high dimensional data into latent spaces with small dimensions in order to obtain better understandings of the data and thus more accurate predictions. In particular, they have been widely applied to important applications such as collaborative filtering and social network analysis. In this thesis, I investigate the applications and extensions of the ideas of the low-rank matrix factorization to solve several practically important problems arise from collaborative filtering and social network analysis. A key challenge in recommendation system research is how to effectively profile new users, a problem generally known as \emph{cold-start} recommendation. In the first part of this work, we extend the low-rank matrix factorization by allowing the latent factors to have more complex structures --- decision trees to solve the problem of cold-start recommendations. In particular, we present \emph{functional matrix factorization} (fMF), a novel cold-start recommendation method that solves the problem of adaptive interview construction based on low-rank matrix factorizations. The second part of this work considers the efficiency problem of making recommendations in the context of large user and item spaces. Specifically, we address the problem through learning binary codes for collaborative filtering, which can be viewed as restricting the latent factors in low-rank matrix factorizations to be binary vectors that represent the binary codes for both users and items. In the third part of this work, we investigate the applications of low-rank matrix factorizations in the context of social network analysis. Specifically, we propose a convex optimization approach to discover the hidden network of social influence with low-rank and sparse structure by modeling the recurrent events at different individuals as multi-dimensional Hawkes processes, emphasizing the mutual-excitation nature of the dynamics of event occurrences. The proposed framework combines the estimation of mutually exciting process and the low-rank matrix factorization in a principled manner. In the fourth part of this work, we estimate the triggering kernels for the Hawkes process. In particular, we focus on estimating the triggering kernels from an infinite dimensional functional space with the Euler Lagrange equation, which can be viewed as applying the idea of low-rank factorizations in the functional space.
-
ItemPersonalized search and recommendation for health information resources(Georgia Institute of Technology, 2012-08-24) Crain, Steven P.Consumers face several challenges using the Internet to fill health-related needs. (1) In many cases, they face a language gap as they look for information that is written in unfamiliar technical language. (2) Medical information in social media is of variable quality and may be appealing even when it is dangerous. (3) Discussion groups provide valuable social support for necessary lifestyle changes, but are variable in their levels of activity. (4) Finding less popular groups is tedious. We present solutions to these challenges. We use a novel adaptation of topic models to address the language gap. Conventional topic models discover a set of unrelated topics that together explain the combinations of words in a collection of documents. We add additional structure that provides relationships between topics corresponding to relationships between consumer and technical medical topics. This allows us to support search for technical information using informal consumer medical questions. We also analyze social media related to eating disorders. A third of these videos promote eating disorders and consumers are twice as engaged by these dangerous videos. We study the interactions of two communities in a photo-sharing site. There, a community that encourages recovery from eating disorders interacts with the pro-eating disorder community in an attempt to persuade them, but we found that this attempt entrenches the pro-eating disorder community more firmly in its position. We study the process by which consumers participate in discussion groups in an online diabetes community. We develop novel event history analysis techniques to identify the characteristics of groups in a diabetes community that are correlated with consumer activity. This analysis reveals that uniformly advertise the popular groups to all consumers impairs the diversity of the groups and limits their value to the community. To help consumers find interesting discussion groups, we develop a system for personalized recommendation for social connections. We extend matrix factorization techniques that are effective for product recommendation so that they become suitable for implicit power-law-distributed social ratings. We identify the best approaches for recommendation of a variety of social connections involving consumers, discussion groups and discussions.