Researchers' practice of trying many specifications and discarding those that fail, then reporting only the surviving result, invalidates the standard reliability measures (confidence intervals, 95% significance), because what appears in the paper is the last of hundreds of attempts rather than a first effort—a point Roberts draws from Ed Leamer.
causalpending
Speaker
Russ RobertsEvidence Quote
“the traditional methods measures of reliability confidence intervals... those really just go out the window totally when what we're seeing in the paper is not the researchers first effort but the last effort preceded by hundreds sometimes of regressions”
Created: 6/17/2026, 10:31:42 AM
My Notes
Loading notes...