arrow
Return

Challenges in Obtaining Valid Causal Effect Estimates With Machine Learning Algorithms

delete2021-07-15
delete51
delete
OA
AI
A
Ashley I. Naimi *
A
Alan Mishler
E
Edward H. Kennedy
DOI:10.1093/aje/kwab201delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
Unlike parametric regression, machine learning (ML) methods do not generally require precise knowledge of the true data-generating mechanisms. As such, numerous authors have advocated for ML methods to estimate causal effects. Unfortunately, ML algorithms can perform worse than parametric regression. We demonstrate the performance of ML-based singly and doubly robust estimators. We used 100 Monte Carlo samples with sample sizes of 200, 1,200, and 5,000 to investigate bias and confidence-interval coverage under several scenarios. In a simple confounding scenario, confounders were related to the treatment and the outcome via parametric models. In a complex confounding scenario, the simple confounders were transformed to induce complicated nonlinear relationships. In the simple scenario, when ML algorithms were used, double-robust estimators were superior to singly robust estimators. In the complex scenario, single-robust estimators with ML algorithms were at least as biased as estimators using misspecified parametric models. Doubly robust estimators were less biased, but coverage was well below nominal. The use of sample splitting, inclusion of confounder interactions, reliance on a richly specified ML algorithm, and use of doubly robust estimators was the only explored approach that yielded negligible bias and nominal coverage. Our results suggest that ML-based singly robust methods should be avoided.
Keywords:
causal inference
doubly robust estimation
epidemiologic methods
machine learning
nonparametric methods
semiparametric theory

Journal

American Journal of Epidemiology cover
American Journal of Epidemiology
IF:
4.8
Papers:
9.9K
Citations:
3.7W

Organization

E
Emory University
Scholars:
5.0W
Papers: 4.2W
Citations: 5.7W