arrow
Return

Increasing Transparency Through a Multiverse Analysis

delete2016-09-29
delete750
delete
OA
AI
S
Steegen, Sara
T
Tuerlinckx, Francis
A
Andrew Gelman
V
Vanpaemel, Wolf *
DOI:10.1177/1745691616658637delete
deleteOriginal
deleteShare
deleteSave
View PDF
Abstract

Abstract

En 中文
Empirical research inevitably includes constructing a data set by processing raw data into a form ready for statistical analysis. Data processing often involves choices among several reasonable options for excluding, transforming, and coding data. We suggest that instead of performing only one analysis, researchers could perform a multiverse analysis, which involves performing all analyses across the whole set of alternatively processed data sets corresponding to a large set of reasonable scenarios. Using an example focusing on the effect of fertility on religiosity and political attitudes, we show that analyzing a single data set can be misleading and propose a multiverse analysis as an alternative practice. A multiverse analysis offers an idea of how much the conclusions change because of arbitrary choices in data construction and gives pointers as to which choices are most consequential in the fragility of the result.
Keywords:
multiverse analysis
arbitrary choices
data processing
good research practices
transparency
selective reporting
AI Summary

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.

Journal

Perspectives on Psychological Science cover
Perspectives on Psychological Science
IF:
8.4
Papers:
1.5K
Citations:
1.8W

Organization

C
Columbia University
Scholars:
7.1W
Papers: 6.4W
Citations: 263
K
KU Leuven
Scholars:
5.7W
Papers: 5.2W
Citations: 8.1W
Cited Papers

Cited Papers

err2003-01-01
err0
PREAI
errHilda Pantin; J. Douglas Coatsworth; Daniel J. Feaster; Frederick L. Newman; Ervin Briones; Guillermo Prado; Seth J. Schwartz; José Szapocznik
errShare
errSave
Atherosclerosis, vascular amyloidosis and brain hypoperfusion in the pathogenesis of sporadic Alzheimer's disease
err2013-07-19
err0
errOAAI
errWalter Kalback; Chera Esh; Eduardo M. Castaño; Afroza Rahman; Tyler Kokjohn; Dean C. Luehrs; Lucia Sue; Raquel Cisneros; Francoise Gerber; Claudia Richardson; Bernd Bohrmann; Douglas G. Walker; Thomas G. Beach; Alex E. Roher
errShare
errSave
Synthesis and application of β-carbolines as novel multi-functional anti-Alzheimer’s disease agents
err2017-01-01
err0
errOAAI
errWilliam Horton; Abha Sood; Swarada Peerannawar; Nandor Kugyela; Aditya Kulkarni; Rekha Tulsan; Chris D. Tran; Jessica Soule; Harry LeVine; Béla Török; Marianna Török
errShare
errSave
Estimating the reproducibility of psychological science
errSCIENCE
IF45.8
err2015-08-28
err4.2K
errOAAI
errAarts, Alexander A.; Anderson, Joanna E.; Anderson, Christopher J.; Attridge, Peter R.; Attwood, Angela; Axt, Jordan; Babel, Molly; Bahnik, Stepan; Baranski, Erica; Barnett-Cowan, Michael; Bartmess, Elizabeth; Beer, Jennifer; Bell, Raoul; Bentley, Heather; Beyan, Leah; Binion, Grace; Borsboom, Denny; Bosch, Annick; Bosco, Frank A.; Bowman, Sara D.; Brandt, Mark J.; Braswell, Erin; Brohmer, Hilmar; Brown, Benjamin T.; Brown, Kristina; Bruening, Jovita; Calhoun-Sauls, Ann; Callahan, Shannon P.; Chagnon, Elizabeth; Chandler, Jesse; Chartier, Christopher R.; Cheung, Felix; Christopherson, Cody D.; Cillessen, Linda; Clay, Russ; Cleary, Hayley; Cloud, Mark D.; Cohn, Michael; Cohoon, Johanna; Columbus, Simon; Cordes, Andreas; Costantini, Giulio; Alvarez, Leslie D. Cramblet; Cremata, Ed; Crusius, Jan; DeCoster, Jamie; DeGaetano, Michelle A.; Della Penna, Nicolas; den Bezemer, Bobby; Deserno, Marie K.; Devitt, Olivia; Dewitte, Laura; Dobolyi, David G.; Dodson, Geneva T.; Donnellan, M. Brent; Donohue, Ryan; Dore, Rebecca A.; Dorrough, Angela; Dreber, Anna; Dugas, Michelle; Dunn, Elizabeth W.; Easey, Kayleigh; Eboigbe, Sylvia; Eggleston, Casey; Embley, Jo; Epskamp, Sacha; Errington, Timothy M.; Estel, Vivien; Farach, Frank J.; Feather, Jenelle; Fedor, Anna; Fernandez-Castilla, Belen; Fiedler, Susann; Field, James G.; Fitneva, Stanka A.; Flagan, Taru; Forest, Amanda L.; Forsell, Eskil; Foster, Joshua D.; Frank, Michael C.; Frazier, Rebecca S.; Fuchs, Heather; Gable, Philip; Galak, Jeff; Galliani, Elisa Maria; Gampa, Anup; Garcia, Sara; Gazarian, Douglas; Gilbert, Elizabeth; Giner-Sorolla, Roger; Gloeckner, Andreas; Goellner, Lars; Goh, Jin X.; Goldberg, Rebecca; Goodbourn, Patrick T.; Gordon-McKeon, Shauna; Gorges, Bryan; Gorges, Jessie; Goss, Justin; Graham, Jesse; Grange, James A.; Gray, Jeremy; Hartgerink, Chris; Hartshorne, Joshua; Hasselman, Fred; Hayes, Timothy; Heikensten, Emma; Henninger, Felix; Hodsoll, John; Holubar, Taylor; Hoogendoorn, Gea; Humphries, Denise J.; Hung, Cathy O. -Y.; Immelman, Nathali; Irsik, Vanessa C.; Jahn, Georg; Jaekel, Frank; Jekel, Marc; Johannesson, Magnus; Johnson, Larissa G.; Johnson, David J.; Johnson, Kate M.; Johnston, William J.; Jonas, Kai; Joy-Gaba, Jennifer A.; Kappes, Heather Barry; Kelso, Kim; Kidwell, Mallory C.; Kim, Seung Kyung; Kirkhart, Matthew; Kleinberg, Bennett; Knezevic, Goran; Kolorz, Franziska Maria; Kossakowski, Jolanda J.; Krause, Robert Wilhelm; Krijnen, Job; Kuhlmann, Tim; Kunkels, Yoram K.; Kyc, Megan M.; Lai, Calvin K.; Laique, Aamir; Lakens, Daniel; Lane, Kristin A.; Lassetter, Bethany; Lazarevic, Ljiljana B.; LeBel, Etienne P.; Lee, Key Jung; Lee, Minha; Lemm, Kristi; Levitan, Carmel A.; Lewis, Melissa; Lin, Lin; Lin, Stephanie; Lippold, Matthias; Loureiro, Darren; Luteijn, Ilse; Mackinnon, Sean; Mainard, Heather N.; Marigold, Denise C.; Martin, Daniel P.; Martinez, Tylar; Masicampo, E. J.; Matacotta, Josh; Mathur, Maya; May, Michael; Mechin, Nicole; Mehta, Pranjal; Meixner, Johannes; Melinger, Alissa; Miller, Jeremy K.; Miller, Mallorie; Moore, Katherine; Moeschl, Marcus; Motyl, Matt; Mueller, Stephanie M.; Munafo, Marcus; Neijenhuijs, Koen I.; Nervi, Taylor; Nicolas, Gandalf; Nilsonne, Gustav; Nosek, Brian A.; Nuijten, Michele B.; Olsson, Catherine; Osborne, Colleen; Ostkamp, Lutz; Pavel, Misha; Penton-Voak, Ian S.; Perna, Olivia; Pernet, Cyril; Perugini, Marco; Pipitone, R. Nathan; Pitts, Michael; Plessow, Franziska; Prenoveau, Jason M.; Rahal, Rima-Maria; Ratliff, Kate A.; Reinhard, David; Renkewitz, Frank; Ricker, Ashley A.; Rigney, Anastasia; Rivers, Andrew M.; Roebke, Mark; Rutchick, Abraham M.; Ryan, Robert S.; Sahin, Onur; Saide, Anondah; Sandstrom, Gillian M.; Santos, David; Saxe, Rebecca; Schlegelmilch, Rene; Schmidt, Kathleen; Scholz, Sabine; Seibel, Larissa; Selterman, Dylan Faulkner; Shaki, Samuel; Simpson, William B.; Sinclair, H. Colleen; Skorinko, Jeanine L. M.; Slowik, Agnieszka; Snyder, Joel S.; Soderberg, Courtney; Sonnleitner, Carina; Spencer, Nick; Spies, Jeffrey R.; Steegen, Sara; Stieger, Stefan; Strohminger, Nina; Sullivan, Gavin B.; Talhelm, Thomas; Tapia, Megan; te Dorsthorst, Anniek; Thomae, Manuela; Thomas, Sarah L.; Tio, Pia; Traets, Frits; Tsang, Steve; Tuerlinckx, Francis; Turchan, Paul; Valasek, Milan; van 't Veer, Anna E.; Van Aert, Robbie; van Assen, Marcel; van Bork, Riet; van de Ven, Mathijs; van den Bergh, Don; van der Hulst, Marije; van Dooren, Roel; van Doorn, Johnny; van Renswoude, Daan R.; van Rijn, Hedderik; Vanpaemel, Wolf; Echeverria, Alejandro Vasquez; Vazquez, Melissa; Velez, Natalia; Vermue, Marieke; Verschoor, Mark; Vianello, Michelangelo; Voracek, Martin; Vuu, Gina; Wagenmakers, Eric-Jan; Weerdmeester, Joanneke; Welsh, Ashlee; Westgate, Erin C.; Wissink, Joeri; Wood, Michael; Woods, Andy; Wright, Emily; Wu, Sining; Zeelenberg, Marcel; Zuni, Kellylynn
errShare
errSave
Money, Status, and the Ovulatory Cycle
err2014-02-01
err69
PREAI
errDurante, Kristina M.; Griskevicius, Vladas; Cantu, Stephanie M.; Simpson, Jeffry A.
errShare
errSave
Bayes Factor Approaches for Testing Interval Null Hypotheses
err2011-12-01
err409
PREAI
errMorey, Richard D.; Rouder, Jeffrey N.
errShare
errSave
Scientific Utopia: I. Opening Scientific Communication
err2012-07-01
err199
errOAAI
errNosek, Brian A.; Bar-Anan, Yoav
errShare
errSave
researcher View more