Reducing Human Labor Cost in Deep Learning for Natural Language Processing

Deep learning has fundamentally changed the landscape of natural language processing (NLP). The success of deep learning techniques relies on huge amounts of manually labeled data in many applications. However, large amounts of labeled data are usually prohibitive or expensive to obtain. In addition, accurately evaluating the NLP models also requires expansive human evaluation. This dissertation focuses on reducing such human labor costs in deep learning for NLP. We develop novel frameworks for training deep learning models with limited/noisy annotation and a novel framework for estimating human evaluation scores. Specifically, 1. we propose a new transfer learning framework for robust and efficient fine-tuning for pre-trained models to attain better generalization performance on tasks with limited data; 2. we propose a two-stage self-training based transfer learning framework for training on weakly labeled data; 3. we propose a new reliable human-free automatic evaluation framework for dialog systems based on recent advances of off-policy evaluation in reinforcement learning.

Date

2021-05-01

Resource Type

Text

Resource Subtype

Dissertation

Full item page