Equivalent design problems, an experimental study

A current standard for testing advances in engineering design is to present participants with a design problem and evaluate their performance across metrics. This approach, however, cannot be used in repeated measures testing, as a design problem cannot be given repeatedly to a participant without biasing results, therefore necessitating equivalent design problems. This study provides a foundation for creating these equivalent design problems by investigating four design problems for equivalency using between-subjects and within-subjects testing. These problems have been modified for greater similarity and include peanut, corn, alarm, and coconut design problems. All design problems were given during between-subjects testing and the peanut and corn problems were also tested within-subjects. The within-subjects analysis revealed correlations on three of five metrics tested, including quantity which is the most frequent metric in the field [1, 2]. This indicates that the problems used are close to equivalent and may be used in repeated measures studies for these metrics. Additionally, the between-subjects analysis revealed that the design problems do not show group mean equivalency, meaning between-subjects analysis is insufficient in assessing equivalency and within-subjects analysis should be the standard in future studies of design problem equivalency. Three problem characteristics [3] were also investigated in the between-subjects analysis for their impact on metric scores. While trends emerged, such as higher difficulty leads to fewer and lower quality solutions, more research needs to be done to understand how manipulating these problem characteristics and couplings of characteristics can be used to make equivalent design problems.

Date

2017-05-09

Resource Type

Text

Resource Subtype

Thesis

Full item page