Title:
An Empirical Study of the Effect of Agent Competence on User Performance and Perception
An Empirical Study of the Effect of Agent Competence on User Performance and Perception
Files
Author(s)
Catrambone, Richard
Stasko, John T.
Xiao, Jun
Stasko, John T.
Xiao, Jun
Advisor(s)
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
We studied the role of the competence of a user interface agent/assistant that helped users to learn and use a new text editor. Participants in the study made a set of prescribed changes to a document via the editor with the aid of one of four interface agents. Participants could ask questions out loud to the agent and the agent would respond using a synthesized voice; the agent would also make proactive suggestions. The agents varied in the quality of responses and suggestions made. One group of participants were provided with a help screen as well as the agent. We focused on assessing the relation between users' objective performance, interaction style, and subjective experience. Results revealed that the perceived utility of the agent was influenced by the types of errors made by the agent, while participants' subjective impressions of the agent related to the perceptions of its representation. In addition, allowing participants to choose their preferred assistance style(s) (agent vs. online-help) improved objective performance. We correlate quantitative findings with qualitative interview data and discuss implications for the design and the implementation of systems with interface agents.
Sponsor
Date Issued
2004
Extent
325572 bytes
Resource Type
Text
Resource Subtype
Technical Report