Organizational Unit:
H. Milton Stewart School of Industrial and Systems Engineering
H. Milton Stewart School of Industrial and Systems Engineering
Permanent Link
Research Organization Registry ID
Description
Previous Names
Parent Organization
Parent Organization
Organizational Unit
Includes Organization(s)
Organizational Unit
ArchiveSpace Name Record
Publication Search Results
Now showing
1 - 10 of 1769
-
ItemOn Parameter Efficiency of Neural Language Models(Georgia Institute of Technology, 2024-01-04) Liang, ChenIn recent years, pre-trained neural language models have achieved remarkable capabilities across various natural language understanding and generation tasks. However, the trend of scaling these models to encompass billions of parameters, while enhancing adaptability and emergent capabilities, has brought forth significant deployment challenges due to their massive size. These challenges include constraints in model storage and inference latency for real-world deployment, intensive time and computational costs for task adaptation, and the presence of substantial redundant parameters that affect task adaptability. Motivated by these challenges, this thesis aims to improve the parameter efficiency of these models, seeking to minimize storage requirements, accelerate inference and adaptation, and enhance generalizability. \noindent {\it -- Improving Parameter Utilization in Neural Language Models} \\ While recent studies have identified significant redundancy in pre-trained neural language models, the impact of parameter redundancy on model generalizability remains largely underexplored. We first examine the relationship between parameter redundancy and model generalizability. Observing that removing redundant parameters improves generalizability, we propose an adaptive optimization algorithm for fine-tuning to improve the utilization of the redundant parameters. Experimental results validate increased generalization across various downstream tasks. \noindent {\it -- Model Compression in Neural Language Models} \\ We explore model compression methods, including weight pruning and knowledge distillation, to reduce model storage and accelerate inference. We first develop a reliable iterative pruning method that accounts for uncertainties in training dynamics. Then, we dive into the realm of knowledge distillation, addressing the large teacher-student ``knowledge gap" that often hampers the student's performance. To tackle this, we offer two solutions for producing students for specific tasks by selectively distilling task-relevant knowledge. In scenarios demanding student adaptability across diverse tasks, we propose to reduce the knowledge gap by combining iterative pruning with distillation. Our approaches significantly surpass conventional distillation methods at similar compression ratios. \noindent {\it -- Efficient Task Adaptation in Neural Language Models} \\ While fine-tuning is an essential adaptation method for attaining satisfactory performance on downstream tasks, it is both computation-intensive and time-consuming. To speed up task adaptation, we study the hypernetwork approach, which employs an auxiliary hypernetwork to swiftly generate task-specific weights based on few-shot demonstration examples. We improve the weight generation scheme by exploiting the intrinsic weight structure as an inductive bias, enhancing sample efficiency for hypernetwork training. The method shows superior generalization performance on unseen tasks compared to existing hypernetwork methods.
-
ItemFundamental Limits and Algorithms for Database and Graph Alignment(Georgia Institute of Technology, 2023-12-12) Dai, Osman EmreData alignment refers to a class of problems where given two sets of anonymized data pertaining to overlapping sets of users, the goal is to identify the correspondences between the two sets. If the data of a user is contained in both sets, the correlation between the two data points associated with the user might make it possible to determine that both belong to the same user and hence link the data points. Alignment problems are of practical interest in applications such as privacy and data junction. Data alignment can be used to de-anonymize data, therefore, studying the feasibility of alignment allows for a more reliable understanding of the limitations of anonymization schemes put in place to protect against privacy breaches. Additionally, data alignment can aid in finding the correspondence between data from different sources, e.g. different sensors. The data fusion performed through data alignment in turn can help with variety of inference problems that arise in scientific and engineering applications. This thesis considers two types of data alignment problems: database and graph alignment. Database alignment refers to the setting where each feature (i.e. data points) in a data set is associated with a single user. Graph alignment refers to the setting where data points in each data set are associated with pairs of users. For both problems, we are particularly interested in the asymptotic case where n, the number of users with data in both sets, goes to infinity. Nevertheless our analyses often yield results applicable to the finite n case. To develop a preliminary understanding of the database alignment problem, we first study the closely related problem of planted matching with Gaussian weights of unit variance, and derive tight achievability bounds that match our converse bounds: Specifically we identify different inequalities between log n and the signal strength (which corresponds to the square of the difference between the mean weights of planted and non-planted edges) that guarantee upper bounds on the log of the expected number of errors. Then, we study the database alignment problem with Gaussian features in the low per-feature correlation setting where the number of dimensions of each feature scales as ω(log n): We derive inequalities between log n and signal strength (which, for database alignment, corresponds to the mutual information between correlated features) that guarantee error bounds matching those of the planted matching setting, supporting the claimed connection between the two problems. Then, relaxing the restriction on the number of dimensions of features, we derive conditions on signal strength and dimensionality that guarantee smaller upper bounds on the log of the expected number of errors. The stronger results in the O(log n)-dimensional-feature setting for Gaussian databases show how planted matching, while useful, is not a perfect substitute to understand the dynamics of the more complex problem of database alignment. For graph alignment, we focus on the correlated Erdős–Rényi graph model where the data point (i.e. edge) associated with each pair of users in a graph is a Bernoulli random variable that is correlated with the data point associated with the same pair in the other graph. We study a canonical labeling algorithm for alignment and identify conditions on the density of the graphs and correlation between edges across graphs that guarantees the recovery of the true alignment with high probability.
-
ItemSample complexity of Reinforcement Learning algorithms with a focus on policy space methods(Georgia Institute of Technology, 2023-12-11) Khodadadian, SajadIn this thesis, we develop fast Reinforcement Learning algorithms with finite sample complexity guarantees. The work is divided into two main parts. In the first, we investigate stochastic approximation across various domains to establish finite sample complexity bounds. We study two settings: federated stochastic approximation and two-time-scale linear stochastic approximation with Markovian noise. In the former, we develop a FedSAM algorithm where multiple agents are utilized to solve a fixed-point equation, following a stochastic approximation with Markovian noise. Moreover, we show that FedSAM has linear speedup with respect to the number of agents, while enjoying a constant communication cost. In the latter, we explore two-time-scale linear stochastic approximation with Markovian noise, establishing tight finite-time bounds. The second part delves into finite-time bounds for Reinforcement Learning algorithms, with an emphasis on policy space methods. First, we consider two-time-scale natural actor-critic algorithm with on-policy data. For this algorithm we establish a $\epsilon^{-6}$ sample complexity for convergence to the global optimum. Next, we study two-loop natural actor-critic, and we establish a $\epsilon^{-3}$ sample complexity, improving upon the two-time-scale counterpart. In this case, we consider an off-policy sampling strategy. To enhance the sample complexity of the natural actor-critic, we separate the algorithm into 'Actor' and 'Critic' components. For the Critic, we consider federated TD-learning and TD-learning with Polyak averaging. For the former, we show a linear speedup, and in the latter we establish a tight finite time bound. Furthermore, we establish a tight finite time convergence bound for the TDC algorithm. For the Actor, we demonstrate linear and superlinear convergence rates for the natural policy gradient.
-
ItemConic Reformulation Methods in Revenue Management(Georgia Institute of Technology, 2023-11-29) Shao, HongzhangThis thesis presents a comprehensive study on optimization problems in revenue management, with a particular focus on the development and application of conic programming reformulation techniques. The work is grounded in three distinct but interrelated papers, each addressing a unique aspect of revenue management. The first paper investigates the trade-off between pickup time and idle time in ride-hailing systems. Pickup time refers to the duration from when a car is dispatched to pick up a rider until the rider is picked up. The study reveals that if cars spend less idle time waiting for a dispatch, the mean distance between a rider and the closest available car increases, resulting in longer pickup times. This phenomenon is crucial in ride-hailing as every minute spent on pickup reduces the time available for transporting riders. Despite its importance, existing literature on price optimization and repositioning in ride-hailing systems often overlooks pickup time. This paper presents a novel approach to reformulate a simultaneous price and repositioning optimization problem, considering the distribution of pickup time, into a tractable convex optimization problem. The optimal solution derived from this approach significantly outperforms policies proposed in previous studies, as demonstrated in simulations. The second paper delves into the complexities of identifying the ideal product pricing and assortment, a critical aspect of revenue management. The challenge lies in concurrently establishing prices and assortments while navigating a resource network with finite capacity. Existing research on the static joint optimization of price and assortment often overlooks resource constraints. Our study, however, addresses the revenue management problem with resource constraints and price boundaries, where prices and product assortments must be collaboratively determined over time. We demonstrate that under the Markov chain (MC) choice model (which subsumes the multinomial logit (MNL) model), the choice-based joint optimization problem can be transformed into a tractable convex conic optimization problem. For static joint optimization without resource constraints, we reveal that the optimal price for a product remains consistent across all optimal solutions with positive sales. With the same price vector, an assortment can be optimal if it both contains and is contained by optimal assortments. Interestingly, optimal assortments in such problems are closed under union but may not be closed under intersection. In the context of revenue management problems, we prove that an optimal solution with a constant price vector is possible, even in the presence of resource constraints. This finding suggests that there is no need to continuously adjust prices throughout the planning period. The third paper addresses a fundamental problem in revenue management: finding the optimal choice of product attributes. These attributes significantly influence both the market share and profit margin of a product. The decision-maker is tasked with choosing the optimal vector of attributes for each product to maximize total profit, revenue, or market share. However, existing literature on product line design with multiple attributes often results in intractable optimization problems. In contrast, studies on pricing problems under discrete choice models typically assume that price is the only attribute to be chosen for each product, and the methods used in such literature cannot be generalized to solve optimization problems with multiple product attributes. In this paper, we introduce a method to reformulate static multi-attribute optimization problems and multi-stage fluid optimization problems with resource constraints and upper and lower bounds on attributes as tractable convex conic optimization problems. Our results apply to optimization problems under the multinomial logit (MNL) model, the Markov chain (MC) choice model, and with certain conditions, the nested logit (NL) model. This method also provides a unified approach to solve pricing problems under discrete choice models and can reproduce many existing results established under different methods. Overall, this thesis underscores the potential of conic programming reformulation techniques in solving complex optimization problems in revenue management, providing a unified approach that can be applied across various models and scenarios.
-
ItemDesign and Analysis of Stochastic Processing and Matching Networks(Georgia Institute of Technology, 2023-08-21) Jhunjhunwala, Prakirt RajStochastic Processing Networks (SPNs) and Stochastic Matching Networks (SMNs) play a crucial role in various engineering domains, encompassing applications in Data Centers, Telecommunication, Transportation, and more. As these networks become increasingly complex and integral to modern systems, designing efficient decision-making policies while obtaining strong performance guarantees on throughput and delay has become a pressing research area. This thesis addresses the multifaceted challenges prevalent in today's stochastic networks and investigates their impact on system performance. Major design considerations are thoroughly examined, including scalability, customer abandonment, multiple bottlenecks, and adherence to Service Level Agreements (SLAs). Each of these factors heavily influences the system delay and queue length. In Chapter 2, we focus on establishing bounds for the tail probabilities of queue lengths in queueing systems. The results help provide strict SLA guarantees for large-scale systems. As obtaining exact steady-state distributions is often infeasible, the study provides exponentially decaying bounds in Many-Server Heavy-Traffic regimes, where the load on the system approaches the capacity simultaneously as the system size grows large. Unlike other approaches, the derived bounds are not limited to asymptotic cases and remain applicable even for finite values of load and system size. The method uses an exponential Lyapunov function to bound the Moment-Generating Function (MGF) of queue lengths, and the application of Markov's inequality contributes to the derivation of the tail bounds. To demonstrate our methodology, we primarily use a load balancing system operating under the Join-the-Shortest Queue policy (JSQ), and we obtain tail bounds applicable in non-asymptotic large-scale regimes as well as non-asymptotic Large Deviations regimes. In Chapter 3, we again look at a Load Balancing system operating under the Join-the-Shortest Queue policy (JSQ), but with an additional aspect of customer abandonments. In particular, we characterize the `distribution of appropriately centered and scaled steady-state queue length' (or limiting distribution) as the abandonment rate becomes very small. Our work encompasses the case when the system sees heavy traffic as well as the case when the system is overloaded. As the system load increases, we observe that the limiting distribution undergoes a phase transition from exponential to a truncated-normal and finally to a normal distribution. The chapter employs the Transform method to establish results about the limiting Moment Generating Function (MGF) of queue lengths. Afterward, in Chapter 4, we focus our study on understanding the performance of SPNs with multiple bottlenecks, for which the problem becomes significantly more challenging. For this, we use the Input-Queued Switch (IQ-Switch) model, which models a data center network and serves as a representative of SPNs with multiple bottlenecks. Prior literature has established that the well-studied MaxWeight policy provides superior throughput and mean queue length performance. Even though the MaxWeight algorithm results in small queue lengths, the complexity of implementing it is high, which is practically undesirable. We show that several classes of low time-complexity algorithms have similar mean queue lengths to MaxWeight when the system load is very high. Moving ahead, in Chapter 5, we aim to go beyond the mean queue length and provide strict SLA or tail guarantees for an SPN with multiple bottlenecks. We tackle this problem by studying the steady-state queue length distribution. For the case of IQ-Switch, finding `the complete joint distribution of queue length vector in heavy traffic' (or limiting joint distribution) was posed as an open problem in prior literature. Our work solves the open problem for IQ-switch (under a particular conjecture) operating under the MaxWeight scheduling algorithm and other low-complexity algorithms considered in Chapter 4. For IQ-Switch, under uniform traffic and heavy load condition, we provide the limiting distribution in terms of a non-linear combination of independent and exponentially distributed random variables. We do this by establishing a functional equation on the Laplace transform of the limiting joint distribution using the Transform method, which can be solved to obtain the result. Finally, in Chapter 6, we study the queueing dynamics of an SMN using the exciting example of a quantum network. This system is much harder to analyze than an SPN, as the effective service rate depends on the system state. We aimed to provide performance guarantees on the queue length like in previous chapters. However, we soon realized that even the fundamental problem of finding the stability conditions for an SMN is not entirely answered. Thus, in this chapter, we characterize the stability conditions for a class of quantum networks under the MaxWeight policy. Interestingly, we find that the stability region of the quantum network is defined as the convex hull of the achievable throughput of suitably designed sub-networks.
-
ItemExploiting Problem Structure for Faster Optimization: A Trilinear Saddle Point Approach(Georgia Institute of Technology, 2023-07-28) Zhang, ZheOptimization is vital in operations research, encompassing model fitting and decision-making. The exponential growth of data holds promise for realistic models and intelligent decision-making. However, the sheer volume and the exceptional dimension of big data make computations prohibitively expensive and time-consuming. In this thesis, we propose a trilinear saddle point approach to tackle some challenges in big-data optimization. By effectively leveraging problem structure, our approach significantly improves computation complexities for a few important problem classes in stochastic programming and non-linear programming. This offers valuable insights into the intrinsic computational hardness. In Chapter Two, we consider a distributionally robust two-stage stochastic optimization problem with discrete scenario support. While much research effort has been devoted to tractable reformulations for DRO problems, especially those with continuous scenario support, few efficient numerical algorithms are developed, and most of them can neither handle the nonsmooth second-stage cost function nor the large number of scenarios $K$ effectively. We fill the gap by reformulating the DRO problem as a trilinear min-max-max saddle point problem and developing novel algorithms that can achieve an $O(1/\epsilon)$ iteration complexity which only mildly depends on the scenario number . The major computations involved in each iteration of these algorithms can be conducted in parallel if necessary. Besides, for solving an important class of DRO problems with the Kantorovich ball ambiguity set, we propose a slight modification of our algorithms to avoid the expensive computation of the probability vector projection. Finally, preliminary numerical experiments are conducted to demonstrate the empirical advantages of the proposed algorithms. In Chapter Three, we study the convex nested stochastic composite optimization (NSCO) problem, which finds applications in reinforcement learning and risk-averse optimization. Existing NSCO algorithms exhibit significantly worse stochastic oracle complexities compared to those without nested structures, and they require all outer-layer functions to be smooth. To address these challenges, we propose a stochastic trilinear (multi-linear) saddle point formulation that enables the design of order-optimal algorithms for general convex NSCO problems. When all outer-layer functions are smooth, we propose a stochastic sequential dual (SSD) method to achieve an oracle complexity of $O(1/\epsilon^2)$ ($O(1/\epsilon)$) when the problem is non-strongly (strongly) convex. In cases where there are structured non-smooth or general non-smooth outer-layer functions, we propose a nonsmooth stochastic sequential dual (nSSD) method, achieving an oracle complexity of $O(1/\epsilon^2)$. Notably, we prove that this $O(1/\epsilon^2)$ complexity is unimprovable even under a strongly convex setting. These results demonstrate that the convex NSCO problem shares similar oracle complexities as those without nested compositions, except for strongly convex and outer-non-smooth problems. In Chapter Four, we investigate the communication complexity of convex risk-averse optimization over a network. The problem generalizes the well-studied risk-neutral finite-sum distributed optimization problem and its importance stems from the need to handle risk in an uncertain environment. For algorithms in the literature, there exists a gap in communication complexities for solving risk-averse and risk-neutral problems. To address this gap, we utilize a trilinear saddle point reformulation to design two distributed algorithms: the distributed risk-averse optimization (DRAO) method and the distributed risk-averse optimization with sliding (DRAO-S) method. The single-loop DRAO method involves solving potentially complex subproblems, while the more sophisticated DRAO-S method requires only simple computations. We establish lower complexity bounds to show their communication complexities to be unimprobvable, and conduct numerical experiments to illustrate the encouraging empirical performance of the DRAO-S method. In Chapter Five, we utilize the trilinear saddle point approach to develop new complexity results for classic nonlinear function-constrained optimization. We introduce the single-loop Accelerated Constrained Gradient Descent (ACGD) method, which modifies Nesterov's celebrated Accelerated Gradient Descent (AGD) method by incorporating a linearly-constrained descent step. Lower complexity bounds are provided to establish the tightness of ACGD's complexity bound under a specific optimality regime. To enhance efficiency for large-scale problems, we propose the ACGD with Sliding (ACGD-S) method. ACGD-S replaces computationally demanding constrained descent steps with basic matrix-vector multiplications. ACGD-S shares the same oracle complexity as ACGD and achieves an unimprovable computation complexity measured by the number of matrix-vector multiplications. These advancements offer insights into complexity and provide efficient solutions for nonlinear function-constrained optimization, catering to both general and large-scale scenarios.
-
ItemDecomposition Algorithms for Certain Integer Problems over Networks(Georgia Institute of Technology, 2023-07-28) Li, YijiangInteger optimization stands as a fundamental and widely embraced tool in addressing many real-world problems. Among the abundant applications of integer optimization, numerous network-related problems arise and exhibit significant influence in many areas such as transportation, energy systems, and supply chains. The intricate nature of these problems often results in complicated large-scale formulations that are computationally expensive to solve directly. Instead, decomposition is a more computationally tractable means in many cases. In this thesis, we focus on a few integer problems over networks. In Chapter 2, we investigate the airport flight-to-gate assignment problem, where the goal is to minimize the total delays by optimally assigning each scheduled flight to a compatible gate. We provide a column generation approach for solving this problem. We decompose the pricing problem such that each gate is the basis for an independent pricing problem to be solved and use a combination of an approximation algorithm based on the submodularity of the underlying set and dynamic programming algorithms to solve the independent pricing problems. We also design and employ a rolling horizon method and block decomposition algorithm to solve the large-sized instances. Finally, we perform extensive computational experiments to validate the performance of our approach. In Chapter 3, we focus on the gas network design problem. Gas networks are used to transport natural gas, which is an important resource for both residential and industrial customers throughout the world. The gas network design problem is a challenging nonlinear and non-convex optimization problem. We propose a decomposition framework to solve this problem. In particular, we utilize a two-stage procedure that involves a convex reformulation of the original problem. We conduct experiments on a benchmark network to validate and analyze the performance of our framework. In Chapter 4, we combine the water network design and operation problems. In general, the design problems consider pipe sizing and placements of pump stations, while the operation problems are multiple time period problems that account for temporal changes in supply and demand and consider the scheduling of the installed pump stations. We propose two methods to obtain good candidate primal solutions. One method is a similar decomposition framework that is used in Chapter 3 while the other method is based on a time decomposition. We conduct computational experiments on networks that closely resemble real-world networks. In Chapter 5, we consider the resiliency of infrastructure networks. The infrastructure systems generally consist of multiple types of infrastructure facilities that are interdependent. In the event of natural disaster, some of the infrastructure nodes can be damaged and disabled creating failures and such failures can propagate to other facilities that depend on the disabled facilities creating a cascade of failures and eventually a potential system collapse. We propose a bilevel interdiction model to study this problem of cascading failures in an interdependent infrastructure network with a probabilistic dependency graph. We utilize a Benders type decomposition algorithm to solve the resulting formulation. Computational experiments are performed using synthetic networks to validate the performance of this algorithm.
-
ItemStochastic Service Network Design with Different Relay Patterns for Hyperconnected Relay Transportation(Georgia Institute of Technology, 2023-06) Li, Jingze ; Liu, Xiaoyue ; Dahan, Mathieu ; Montreuil, BenoitHyperconnected relay transportation enables using a relay system of short-haul drivers to deliver long-haul shipments collectively, which helps address root causes of trucker shortage issues by transforming working conditions with potentials of daily returning home, accessing consistent schedules, and facilitating load matching. This Paper investigates hyperconnected relay transportation as a sustainable solution to trucker shortage issues through a logistics platform. We propose a two-stage programming model to optimize consistent working schedules for short-haul drivers while minimizing transportation costs. The first stage involves opening services and contracting truckers under demand uncertainty, where each service has a service route and approximate service schedules adhering to USA federal short-haul hour-of-service regulations. The second stage assigns hauling capacities to open services and manages commodity shipping or outsourcing given the demand realization. We extend the model formulation to account for various operational patterns (e.g., freight loading and unloading or hauler swapping) and schedule consistency requirements (e.g., weekly or daily consistency). A scenario-based approach is employed to solve the model for a case study of automotive delivery in the Southeast USA region. The experimental results validate the proposed approach, and further explore the impact of stochastic demands, operational patterns, consistent schedules, and hauling capacities on hyperconnected service network design. This research aims to offer practical guidance to practitioners in the trucking industry.
-
ItemHyperconnected Logistic Service Networks: Bidding-Based Design Framework(Georgia Institute of Technology, 2023-06) Kwon, Simon ; Montreuil, Benoit ; Dahan, Mathieu ; Klibi, WalidIn hyperconnected urban logistics, all components and stakeholders are connected on multiple layers through standardized interfaces and open networks to achieve seamless responsiveness, efficiency, resilience, and sustainability. Key for high performance is achieving coordination and cooperation of urban stakeholders. In this Paper, we introduce the design of hyperconnected logistic service networks where associated logistic activities to move flows within an urban city are outsourced to third-party logistic service providers (3PL) via a bidding process to create service networks that are highly responsive and flexible at robustly responding to customer demand. We propose a framework for designing such networks that leverages a reverse combinatorial auction mechanism in which a logistic orchestrator serves as the auctioneer, putting out the logistic activities for auction and a set of participating service providers serve as bidders. We describe the design components of hyperconnected service networks and positions them into a comprehensive 3-stage design-making framework. Finally, we identify promising future research avenues for each stage in the proposed framework.
-
ItemHyperconnected Urban Synchromodality: Synergies between Freight and People Mobility(Georgia Institute of Technology, 2023-06) Labarthe, Olivier ; Klibi, Walid ; Montreuil, Benoit ; Deschamps, Jean-ChristopheThis Paper investigates the opportunity to exploit an on-demand freight transshipment service in urban areas. This contribution attempts at first to focus on the feasibility to connect people and freight mobility with a joint usage of transportation options. It builds on the hyperconnectivity principles enabled by the Physical Internet (PI) manifesto for city logistics. To this end, this Paper proposes an effective solution approach for optimizing multimodal on-demand transshipment. The approach considers multiple mobility options such as on-demand delivery services, cargo bikes, tramways, and buses to transship goods from an urban logistic hub to another. The hyperconnected synchromodal mobility solution is proposed as an alternative option to classical pickup and deliverybased transportation. The proposal is first characterized in link with the interconnectivity needs and then its operability is modeled as a new transportation approach. The proposed solution aims to increase the sustainability of cities by reducing congestion levels, the impact of logistics moves, as well as carbon emissions in urban areas. An illustrative case is provided to demonstrate how the novel hyperconnected synchromodal transportation system could operate, and to provide an evaluation of the economic and sustainability benefits of such system in an urban context.