.png){kind=icon}
.png){kind=icon}
Return
Does Every Study? Implementing Ordinal Constraint in Meta-Analysis
.png){kind=icon}
PRE
AI
DOI：10.1037/met0000428.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The most prominent goal when conducting a meta-analysis is to estimate the true effect size across a set of studies. This approach is problematic whenever the analyzed studies have qualitatively different results; that is, some studies show an effect in the predicted direction while others show no effect and still others show an effect in the opposite direction. In case of such qualitative differences, the average effect may be a product of different mechanisms, and therefore uninterpretable. The first question in any meta-analysis should therefore be whether all studies show an effect in the same, expected direction. To tackle this question a model with ordinal constraints is proposed where the ordinal constraint holds each study in the set. This every study model is compared with a set of alternative models, such as an unconstrained model that predicts effects in both directions. If the ordinal constraints hold, one underlying mechanism may suffice to explain the results from all studies, and this result could be supported by reduced betweenstudy heterogeneity. A major implication is then that average effects become interpretable. We illustrate the model comparison approach using Carbajal et al.'s (2021) meta-analysis on the familiar-word-recognition effect, show how predictor analyses can be incorporated in the approach, and provide R-code for interested researchers. As common in meta-analysis, only surface statistics (such as effect size and sample size) are provided from each study, and the modeling approach can be adapted to suit these conditions. Translational Abstract The most prominent goal when conducting a meta-analysis is to estimate the true effect size across a collection of studies. This approach is problematic whenever the analyzed studies have qualitatively different results, i.e. some studies show an effect in the expected direction while others show no effect and still others show an effect in the opposite direction. In case of such qualitative differences, the average effect may be a product of a mixture of psychological mechanisms. The first question in any meta-analysis should therefore be whether all studies show an effect in the same, expected direction. To tackle this question a model with multiple ordinal constraints is proposed - one constraint for each study in the set. This every study model is compared to a set of alternative models, such as an unconstrained model that predicts effects in both directions. If the ordinal constraints hold, one underlying mechanism may suffice to explain the results from all studies. A major implication is then that average effects become interpretable. We illustrate the model comparison approach using Carbajal et al.'s (2021) metaanalysis on the familiar-word-recognition effect, show how additional variables can be incorporated in the approach, and provide Rcode for interested researchers.
Keywords:
meta-analysis
Bayesian inference
order-constrained inference
Journal
IF:
7.8
Papers:
1.3K
Citations:
2.1W
