ChatHF: Collecting Rich Human Feedback from Real-time Conversations
Loading...
Author(s)
Li, Andrew Larry
Advisor(s)
Editor(s)
Collections
Supplementary to:
Permanent Link
Abstract
We introduce ChatHF, an interactive annotation framework for chatbot evaluation, which integrates configurable annotation within a chat interface. ChatHF can be flexibly configured to accommodate various chatbot evaluation tasks, for example detecting offensive content, identifying incorrect or misleading information in chatbot responses, and chatbot responses that might compromise privacy. It supports post-editing of chatbot outputs and supports visual inputs, in addition to an optional voice interface. ChatHF is suitable for collection and annotation of NLP datasets, and Human-Computer Interaction studies, as demonstrated in case studies on image geolocation and assisting older adults with daily activities.
Sponsor
Date
2024-12-16
Extent
Resource Type
Text
Resource Subtype
Thesis