(Georgia Institute of Technology, 2020-12-07)
Choudhary, Anirudh
Building clinical decision support systems, which includes diagnosing patient’s disease states
and formulating a treatment plan, is an important step toward personalized medicine. The counterfactual
nature of clinical decision-making is a major challenge for machine learning-based treatment
recommendation, i.e., we can only observe the outcome of the clinician’s actions while the
outcome of alternative treatment options is unknown. The thesis is an attempt to formulate robust
counterfactual learning frameworks for efficient offline policy evaluation and policy learning using
observational data. We focus on the offline data scenario and leverage historically collected Electronic
Health Records, since online policy testing can potentially adversely impact the patient’s
well-being. The problem is compounded by the inherent uncertainty in clinical decision-making
due to heterogeneous patient contexts, the presence of significant variability in patient-specific
predictions, smaller datasets, and limited knowledge of the clinician’s intrinsic reward function
and environment dynamics. This motivates the need to tackle uncertainty and enable improved
clinical policy generalization via context-based policy learning. We propose counterfactual frameworks
to tackle the highlighted challenges under two learning scenarios: contextual bandits and
dynamic treatment regime. In the bandit setting, we focus on effectively tackling the model uncertainty
inherent in inverse propensity weighting methods and highlight our approach’s efficacy
on oral anticoagulant dosing task. In dynamic treatment regime, we focus on sequential treatment
interventions and consider the problem of imitating the clinician’s policy for sepsis management.
We formulate it as a multi-task problem and propose meta-Inverse Reinforcement Learning framework
to jointly adapt policy and reward functions to diverse patient groups, thus enabling improved
policy generalization.