Title:
Exploring the Robustness of the Surprisingly Popular Signal
Exploring the Robustness of the Surprisingly Popular Signal
Author(s)
Sukernek, Justin
Advisor(s)
Thomas, Rick P.
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
A large portion of the decision-making literature is concerned with forecasting the future, often using wisdom of the crowd as the basis for successful forecasts. However, crowd wisdom can be limited when the consensus is incorrect. Bayesian truth serum and the Surprisingly Popular algorithm, two novel methodologies in this space, offer solutions to this limitation by leveraging social sensing to calculate the 'surprisingly popular signal' at the respondent and question level, respectively. In this dissertation, I present three experiments that compare the three methodologies across forecasting, consumer decision-making, and general knowledge. In all three experiments, SP yielded the highest accuracy when utilizing a subsample of the most knowledgeable participants, a finding that is coherent with the existing literature. Experiment two incorporated social influence, uncovering a positive effect of disagreement on BTS scores and bidirectional effects of social influence on respondents' perceptions of how others would answer. Furthermore, two of the experiments demonstrate evidence of the BTS's ability to identify subsamples of participants that increase SP's accuracy, performing a similar function to domain knowledge. Finally, a process-based simulation of knowledge and social influence on SP and BTS is conducted, corroborating empirical findings. Overall, results provide promising evidence of SP's effectiveness across all task contexts, as well as some evidence for a potential new application for BTS.
Sponsor
Date Issued
2023-04-18
Extent
Resource Type
Text
Resource Subtype
Dissertation