Title:
Hierarchical Reinforcement Learning Framework for Stochastic Spaceflight Campaign Design
Hierarchical Reinforcement Learning Framework for Stochastic Spaceflight Campaign Design
Author(s)
Takubo, Yuji
Chen, Hao
Ho, Koki
Chen, Hao
Ho, Koki
Advisor(s)
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
This paper develops a hierarchical reinforcement learning architecture for multimission spaceflight campaign design under uncertainty, including vehicle design, infrastructure deployment planning, and space transportation scheduling. This problem involves a high-dimensional design space and is challenging especially with uncertainty present. To tackle this challenge, the developed framework has a hierarchical structure with reinforcement learning and network-based mixed-integer linear programming (MILP), where the former optimizes campaign-level decisions (e.g., design of the vehicle used throughout the campaign, destination demand assigned to each mission in the campaign), whereas the latter optimizes the detailed mission-level decisions (e.g., when to launch what from where to where). The framework is applied to a set of human lunar exploration campaign scenarios with uncertain in situ resource utilization performance as a case study. The main value of this work is its integration of the rapidly growing reinforcement learning research and the existing MILP-based space logistics methods through a hierarchical framework to handle the otherwise intractable complexity of space mission design under uncertainty. This unique framework is expected to be a critical steppingstone for the emerging research direction of artificial intelligence for space mission design.
Sponsor
Date Issued
2021-12
Extent
Resource Type
Text
Resource Subtype
Post-print