arrow
Return

Pitfalls in using ML to predict cognitive function performance

delete2025-10-29
delete0
delete
OA
AI
G
Gianna Kuhles *
S
Sami Hamdan
S
Stefan Heim
S
Simon B. Eickhoff
K
Kaustubh R. Patil
J
Julia A. Camilleri
S
Susanne Weis
DOI:10.1038/s41598-025-24325-9delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
Machine learning analyses are widely used for predicting cognitive abilities, yet there are pitfalls that need to be considered during their implementation and interpretation of the results. Hence, the present study aimed at drawing attention to the risks of erroneous conclusions incurred by confounding variables illustrated by a case example predicting executive function (EF) performance by prosodic features. Healthy participants (n = 231) performed speech tasks and EF tests. From 264 prosodic features, we predicted EF performance using 66 variables, controlling for confounding effects of age, sex, and education. A reasonable prediction performance was apparently achieved for EF variables of the Trail Making Test. However, in-depth analyses revealed indications of confound leakage, leading to inflated prediction accuracies, due to a strong relationship between confounds and targets. These findings highlight the need to control confounding variables in ML pipelines and caution against potential pitfalls in ML predictions.

Journal

Scientific Reports cover
Scientific Reports
IF:
3.9
Papers:
28.3W
Citations:
83.5W

Organization

M
Medical Faculty
Scholars:
915
Papers: 400
Citations: 1
Cited Papers

Cited Papers

Predicting the progression of mild cognitive impairment using machine learning: A systematic, quantitative and critical review
err2021-01-01
err49
errOAAI
errAnsart, Manon; Epelbaum, Stephane; Bassignana, Giulia; Bone, Alexandre; Bottani, Simona; Cattai, Tiziana; Couronne, Raphael; Faouzi, Johann; Koval, Igor; Louis, Maxime; Thibeau-Sutre, Elina; Wen, Junhao; Wild, Adam; Burgos, Ninon; Dormont, Didier; Colliot, Olivier; Durrleman, Stanley
errShare
errSave
Leakage and the reproducibility crisis in machine-learning-based science
err2023-09-01
err111
errOAAI
errKapoor, Sayash; Narayanan, Arvind
errShare
errSave
Heterogeneity of executive function revealed by a functional random forest approach across ADHD and ASD
err2020-01-01
err31
errOAAI
errCordova, Michaela; Shada, Kiryl; Demeter, Damion, V; Doyle, Olivia; Miranda-Dominguez, Oscar; Perrone, Anders; Schifsky, Emma; Graham, Alice; Fombonne, Eric; Langhorst, Beth; Nigg, Joel; Fair, Damien A.; Feczko, Eric
errShare
errSave
SpEx: a German-language dataset of speech and executive function performance
err2024-04-24
err0
errOAAI
errCamilleri, Julia A.; Volkening, Julia; Heim, Stefan; Mochalski, Lisa N.; Neufeld, Hannah; Schlothauer, Natalie; Kuhles, Gianna; Eickhoff, Simon B.; Weis, Susanne
errShare
errSave
errShare
errSave
Development Trends of White Matter Connectivity in the First Years of Life
err2011-09-23
err0
errOAAI
errPew-Thian Yap; Yong Fan; Yasheng Chen; John H. Gilmore; Weili Lin; Dinggang Shen
errShare
errSave
Voice Analysis for Neurological Disorder Recognition-A Systematic Review and Perspective on Emerging Trends
err2022-07-07
err24
errOAAI
errHecker, Pascal; Steckhan, Nico; Eyben, Florian; Schuller, Bjoern W.; Arnrich, Bert
errShare
errSave
Identifying Key Predictors of Cognitive Dysfunction in Older People Using Supervised Machine Learning Techniques: Observational Study
err2020-09-16
err7
errOAAI
errRankin, Debbie; Black, Michaela; Flanagan, Bronac; Hughes, Catherine F.; Moore, Adrian; Hoey, Leane; Wallace, Jonathan; Gill, Chris; Carlin, Paul; Molloy, Anne M.; Cunningham, Conal; McNulty, Helene
errShare
errSave
no more