Federated Reinforcement Learning Has Linear Speedup
Author(s)
Einhorn, Michael F.
Advisor(s)
Editor(s)
Collections
Supplementary to:
Permanent Link
Abstract
Whether Federated Reinforcement Learning has linear speedup is an important question for determining how to allocate resources when training models. Previous work has shown that Federated Supervised Learning has linear speedup, and that Federated Reinforcement Learning works, but have not measured the speedup for Federated Reinforcement Learning empirically. We tested the relationship between speedup, the learning rate, the number of clients, and the synchronization frequency, partially confirming prior theoretical predictions for Federated Tabular Q Learning. These results empirically showed that Federated Tabular Q Learning and Federated PPO have linear speedup.
Sponsor
Date
Extent
Resource Type
Text
Resource Subtype
Undergraduate Research Option Thesis