Title:
Impact of composite population priors on computer adaptive test proficiency estimates
Impact of composite population priors on computer adaptive test proficiency estimates
Author(s)
Morrison, Kristin M.
Advisor(s)
Embretson, Susan E.
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
Testing has existed for thousands of years and has evolved from all examinees receiving the same test to adaptive testing, in which the test is tailored to the individual examinee. These adaptive testing designs have shown to be improvements over fixed-length, conventional tests in terms of proficiency measurement, reduced testing time, and faster scoring. A popular method of test administration is computer adaptive testing (CAT) using expected a posteriori (EAP) estimation. This Bayesian estimation approach utilizes previous information known about the examinee to obtain more precise estimates of the individual’s ability. An appropriate prior will generally increase estimation precision, decrease outlier influences, and provide an estimate for all possible response patterns. An inappropriate prior, however, may result in biased estimates (Embretson & Reise, 2000). Previous studies have used collateral information (i.e., additional information) concerning the examinee, such as demographic variables, age, grade, or previous test scores, to aid in estimation. Several studies have used previous test scores (Matteucci & Veldkamp, 2013; Veldkamp & Matteucci, 2013; van der Linden, 1999), but none have looked directly at priors based on group membership. This study examined the influence of various group priors, such as composite priors (i.e., priors created from combining groups) and individual priors (i.e., priors specific to the group), on estimation in CAT designs. Results of the study show group-specific priors perform best; however, it is impossible to know true group membership. Thus, results of the study support the use of priors based on the population because priors based on demographics may adversely impact some high ability groups.
Sponsor
Date Issued
2017-04-24
Extent
Resource Type
Text
Resource Subtype
Dissertation