Professional article
Open access

Using Bayesian hierarchical parameter estimation to assess the generalizability of cognitive models of choice

Published inPsychonomic Bulletin & Review (PB&R), vol. 22, no. 2, p. 391-407
Publication date2015

To be useful, cognitive models with fitted parameters should show generalizability across time and allow accurate predictions of future observations. It has been proposed that hierarchical procedures yield better estimates of model parameters than do nonhierarchical, independent approaches, because the formers' estimates for individuals within a group can mutually inform each other. Here, we examine Bayesian hierarchical approaches to evaluating model generalizability in the context of two prominent models of risky choice—cumulative prospect theory (Tversky & Kahneman, 1992) and the transfer-of-attention-exchange model (Birnbaum & Chavez, 1997). Using empirical data of risky choices collected for each individual at two time points, we compared the use of hierarchical versus independent, nonhierarchical Bayesian estimation techniques to assess two aspects of model generalizability: parameter stability (across time) and predictive accuracy. The relative performance of hierarchical versus independent estimation varied across the different measures of generalizability. The hierarchical approach improved parameter stability (in terms of a lower absolute discrepancy of parameter values across time) and predictive accuracy (in terms of deviance; i.e., likelihood). With respect to test–retest correlations and posterior predictive accuracy, however, the hierarchical approach did not outperform the independent approach. Further analyses suggested that this was due to strong correlations between some parameters within both models. Such intercorrelations make it difficult to identify and interpret single parameters and can induce high degrees of shrinkage in hierarchical models. Similar findings may also occur in the context of other cognitive models of choice.

  • Bayesian inference
  • Parameter estimation
  • Bayesian modeling
  • Decision making
  • Math modeling
  • Model evaluation
NoteThe online version of this article contains supplementary material: http://scheibehenne.de/ScheibehennePachur2014.onlineSupplement.pdf
Citation (ISO format)
SCHEIBEHENNE, Benjamin, PACHUR, Thorsten. Using Bayesian hierarchical parameter estimation to assess the generalizability of cognitive models of choice. In: Psychonomic Bulletin & Review (PB&R), 2015, vol. 22, n° 2, p. 391–407. doi: 10.3758/s13423-014-0684-4
Main files (1)
Article (Published version)
Secondary files (1)

Technical informations

Creation09/22/2015 9:06:00 PM
First validation09/22/2015 9:06:00 PM
Update time03/14/2023 11:38:19 PM
Status update03/14/2023 11:38:19 PM
Last indexation10/18/2023 11:43:34 PM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack