Title:
The Impact of Joint Modeling of Response Accuracy and Response Time on Parameter Predictability and Ability Estimation
The Impact of Joint Modeling of Response Accuracy and Response Time on Parameter Predictability and Ability Estimation
Author(s)
Pezeshki, Maryam
Advisor(s)
Embretson, Susan E.
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
To maintain test quality, a large supply of items is always desired. Generating items based on known cognitive complexity features has potential to circumvent the traditional steps in item development required before operational testing; that is, identifying item writers, training them on test blueprints to develop new items, qualitatively reviewing the developed items, evaluating the reviewed items, and finally, estimating and evaluating the empirical properties of items. Automatic item generation can result in a reduction in cost and labor, including empirical tryout, if the generated new items possess predictable item parameters which are required in an accurate estimation of the trait level. The effect of different levels of parameter predictability on the accuracy of trait level estimation is not clear. On the other hand, adding response time as a collateral source of information may have a mitigating effect on the lower predictability of item parameters on person estimation accuracy. Using a hierarchical model of response accuracy and response time, the present study aims, first, to investigate the impact of varying parameter predictability levels on the trait level estimation accuracy under three different estimation comparisons. Second, the impact of adding response time as a collateral source of information on the accuracy of trait level estimation is also examined. Results show smaller Root Mean Squared Error (RMSE) when using true item difficulty to estimate the trait level. Further, more accuracy is achieved with item family models representing known parameters compared to other item predictability conditions. Moreover, higher correlations between response accuracy and response time person estimates resulted in more accurate trait estimation. Implications for item generation and response processes aspect of validity are presented.
Sponsor
Date Issued
2023-03-28
Extent
Resource Type
Text
Resource Subtype
Dissertation