There is an 'iron law of resistance' whereby traditional experts (wine tasters, baseball scouts, physicians) resist yielding their status to statistical number-crunchers, not purely on the merits but due to ego and dislike of outsiders telling them what to do.

Semmelweis's discovery that doctors moving from the morgue to childbirth without washing hands killed thousands of women was resisted partly because of his arrogance and poor marketing of the findings, illustrating that statistics presented poorly fail to persuade even when correct.

Physicians still fail to wash hands enough between patients despite scrubbing before surgery, and routinized clean-hands projects that force handwashing at fixed checkpoints have been shown to save lives.

Randomized testing of store layouts (planograms) shows that placing higher-markup toothbrushes at chest-high prime viewing level increases purchases, demonstrating that controlled experiments reveal causal merchandising effects.

Capital One combines historical data mining (correlating customer attributes to predict products before an operator even answers, and computing counter-offers for rate negotiations) with massive randomized experiments on mailings to identify which promotions cause more new accounts.

The law of large numbers ensures that randomly assigned groups are statistically identical not just on average but in their full distributions, so any difference in outcomes can be attributed to the one variable being manipulated—making randomized results transparent and not requiring trust in the statistician.

Evidence-based medicine has produced a systematic grading of the quality of evidence behind treatments, which creates competition among researchers to fill gaps and lets physicians who lack statistical training still rely on evidence grades rather than pure intuition.

Before roughly 1990 physicians rarely looked up patient-specific information because it was too time-intensive and they couldn't read the statistics; the internet now gives every physician a massive library, and graded treatments let even statistically unskilled doctors calibrate how much discretion to exercise.

Ayres predicts the next EBM revolution will strip physicians of their 'front-end' discretion over diagnosis and test ordering, as digital medical records let data determine which classes of patients should receive which tests based on when medical benefit exceeds cost.

A hospital mandate requiring pneumonia patients to get antibiotics within four hours leads to over-testing—every patient who might have pneumonia, including those with mere colds, gets an expensive chest x-ray—illustrating that data-derived rules can have hidden costs even when the underlying finding is sound.

In any institution with many frontline employees, roughly half will be below average, so designing the institution around the top 10% is irrational; routinizing decisions and removing some discretion is therefore likely to improve outcomes.

Regression output can report not just a point prediction but its precision (e.g., a 95% confidence range), so when data quality is low or the outcome is inherently random, the statistics themselves disclose that the prediction is imprecise.

Even simple economic facts are elusive: claims that US living standards have stagnated since the late 1970s rely on real average hourly earnings, but inflation measurement excludes fringe benefits and demographic composition effects, so even this 'simple' data is highly controversial.

Measurement difficulties limit where statistical analysis can work: when you can't measure the outcome you care about (standard of living, happiness), lack a large enough comparable historical dataset, or can't run a randomized experiment, statistical prediction cannot answer the question.

The central claim is relative, not absolute: statistical prediction is not invariably accurate or precise, but in case after case it does better than human prediction—and counterintuitively, the more subtle and complex the prediction (more than ten causal variables), the worse humans do relative to even crude statistics, because humans cannot bring themselves to put the right weights on the big variables.

In a 2002 study, a crude statistical algorithm using only structural factors (was the government a party, region of origin, whether the lower court was liberal/conservative) predicted Supreme Court justices' affirm/reverse votes better than 83 legal experts, because the experts—though many had clerked on the Court—couldn't bring themselves to weight known patterns heavily enough (e.g., that the Court tends to reverse the Ninth Circuit).

LoJack (a hidden radio chip enabling police to track stolen cars) reduces car theft rates across an area, not just for equipped vehicles, and—unlike concealed handguns—there is no plausible theoretical mechanism by which LoJack could increase crime, since it is never used as an offensive weapon.

Whether concealed weapons deter or increase crime is fundamentally an empirical question that theory cannot resolve, because theory is ambiguous about the net effect and cannot tell you the magnitude—so conflicting empirical studies (Ayres vs. Lott) are the only path to an answer.

John Lott helped change the norms of data sharing in economics, contributing to a shift in which leading journals now require empirical articles to post their data publicly or explain why not—building on Ed Leamer's earlier call to 'take the con out of econometrics.'

Friedman's 'A Monetary History of the United States' changed the economics profession's view of inflation—shifting it from multiple competing explanations toward the consensus that changes in the money supply cause inflation—through laboriously meticulous (though statistically unsophisticated) analysis.

Heckman's work with Donohue showed the 1964 Civil Rights Act had a dramatic and independent statistical impact on hiring of African Americans in southern textile industries, beyond pre-existing trends of progress—an example of careful microeconomics establishing a causal effect.

Multiple alternative approaches increasingly show that women who would have preferred to abort but did not take less good care of those children later, visible not only in the children's criminality but in a variety of other factors—supporting the Donohue-Levitt abortion thesis though it remains disputed.

The clash of competing studies, especially in contentious areas, drives progress toward eventual consensus by drawing attention and accumulating studies; emerging consensus suggests concealed-weapons laws and the death penalty (vs. life imprisonment) have little impact either way, while the immigration-wages question remains unresolved.

Statistical analysis should be made contestable—Ayres argues businesses adopting number-crunching should hire 'statistical auditors' and allow people to re-analyze the same questions with alternative assumptions, mirroring the academic clash-of-studies model.

In contentious empirical debates (death penalty, guns, abortion), findings tend to confirm the researcher's prior bias because the vast range of possible regressions and techniques lets researchers keep crunching until they get a result that confirms their hypothesis, discarding the many that don't.

A good test for intellectual honesty is whether you can name a statistical study you believe but don't like; if you only believe results you like, you're a biased consumer of evidence.

Randomized trials are far less vulnerable to author bias than regressions because it is easier to 'cook the books' on regressions; the Moving to Opportunity experiment exemplifies this—its funders expected vouchers letting poor families move to middle-class neighborhoods to dramatically improve life outcomes, but preliminary results across ~40 life-chance measures over years show little improvement.

Sophisticated statistical techniques like two-stage least squares (instrumental variables) are designed to correct for reverse causation and confounding, but they are often applied to data that is simply not good enough for the task, so we can never control for all factors and may falsely attribute correlations to an unobserved variable.

Researchers' practice of trying many specifications and discarding those that fail, then reporting only the surviving result, invalidates the standard reliability measures (confidence intervals, 95% significance), because what appears in the paper is the last of hundreds of attempts rather than a first effort—a point Roberts draws from Ed Leamer.

With only non-experimental data and temporal orderings, causal conclusions rest on attaching compelling narratives to correlations—'correlation is in the data but causation is in the mind of the observer'—making such inference literature and wisdom (faith-based decision-making swayed by advocates' rhetoric) rather than science (quoting Leamer).

Roberts proposes that to address bias, empirical work might be restricted to actual experiments, to cases where statistical techniques give less scope for researcher creativity, or conducted as 'empirical tournaments' where both ideological sides analyze data simultaneously and in the open.

Walmart shifts pop-tart inventory upward ahead of forecasted hurricanes because data mining of past consumer responses revealed that people demand the non-refrigerated, gooey comfort food after hurricanes.

Welcome and introduction to EconTalk and the guest Ian Ayres.

Statistical analysis of growing conditions (rainfall, temperature) via multivariate regression predicts Bordeaux wine quality more accurately than traditional expert tasters, because the wine is unpalatable and imprecise to judge in its immature early months.