Mind the Gap(s)

How eighty years of manufactured uncertainty pushed the scientists out of the room

Near the end of the film, Dr. Strangelove, with the doomsday machine already triggered, the men in the War Room turn to who will live underground and for how long. Within a minute they have arrived at a new and urgent national security problem: the mineshaft gap. The Soviets, someone reasons, will surely build more of them.

The joke works because nothing in that scene is checkable. The gap is invented in the room, in real time, by men with no instrument for measuring the thing they have just decided to fear. It is the most accurate two minutes of political science ever committed to film, and it describes a mechanism that has been running, more or less continuously, since 1949.

Three meanings of “gap” run through the last eighty years. There is mind the gap — the space between the platform and the train, the space you fall into. There is the god of the gaps — the thing we install where our knowledge doesn’t reach, which shrinks as knowledge advances and which we defend all the harder for that. And there is the artefact of Cold War politics: the bomber gap, the missile gap, the WMD stockpile, the AI overhang.

The argument is this. For a brief period in the middle of the twentieth century, scientists sat inside the rooms where states made decisions. They are not there now. The usual explanations — taxpayer revolt, culture war, the decadence of the universities — are secondary. The primary mechanism is that governments repeatedly asked technical people to estimate what a rival could do, the estimates were wrong in politically convenient directions, and the profession was billed for failures manufactured downstream of the science. The gap machine ate its own instrument panel.

I. Before: science as a private virtue

Between the wars, serious research in the English-speaking world was funded largely by private wealth: Rockefeller money underwrote Palomar and, under Warren Weaver, effectively invented molecular biology. Scientists were a self-governing guild adjacent to power rather than inside it, and they liked the arrangement — patronage came without portfolio. State science existed, but it was built for war or for primary production. Our own DSIR, founded in 1926, ran on that imperial logic for two-thirds of a century: designed to serve pasture, timber and radar, not to sit at the Cabinet table.

II. 1940–45: the room opens

Vannevar Bush walked into the Oval Office in June 1940 with a single sheet of paper proposing a body to mobilise American science. Roosevelt approved it in fifteen minutes, and for the duration of the war a small number of physicists and engineers had direct access to the president. What bought that access was delivery: radar, the proximity fuze, penicillin at scale, and — decisively for the argument if not for the ethics — the bomb.

The underrated part is operational research, where scientists stopped supplying devices and started contradicting generals. Blackett’s group worked out that convoy losses fell as convoys grew larger, since the escort perimeter scales with radius while the tonnage inside scales with area. Not new physics — arithmetic applied to institutional assumptions, and the moment the technical adviser became a threat as well as an asset.

The cautionary tale arrives early. Henry Tizard and Frederick Lindemann had been friends; by 1940 they were enemies, and Lindemann had Churchill’s ear. His 1942 paper on area bombing estimated the German workers who could be dehoused per ton of bombs. Tizard reckoned it high by a factor of five, Blackett by rather more, and postwar survey confirmed the critics. It did not matter. Lindemann was in the room and Tizard was not.

Proximity to power beat correctness in 1942, before the golden age had properly started. Everything after is a variation.

III. 1945–65: the bargain, and what it actually was

Science, The Endless Frontier is remembered as the founding document of the postwar settlement: government supplies money and forbearance, scientists supply discovery, and discovery reliably produces health, prosperity and security. The myth is tidier than the record. Bush largely lost. He wanted one foundation run by scientists and insulated from Congress; Harley Kilgore wanted political accountability. The fight delayed the National Science Foundation until 1950, by which time the Atomic Energy Commission, the Office of Naval Research and the NIH had occupied the ground. What emerged was mission-driven and far more political than the Endless Frontier framing suggests. Bush lost the architecture and won the language, which is the sort of victory that looks total for about thirty years and then collapses.

Sputnik supplied the accelerant. Eisenhower elevated his advisers into PSAC and gave James Killian walk-in rights; Congress funded American mathematics and science teaching on explicitly military grounds. Roughly 1958 to 1963 is the high-water mark. Killian, then Kistiakowsky, then Wiesner — not advising the room from outside but in it, daily, with standing to say that number is wrong.

It lasted about five years.

IV. 1965–80: the expulsion

The conventional account has the fiscal backlash coming first, with Proxmire as its herald. The sequence is the other way around, and the correction changes what the story is about. PSAC scientists used their access to oppose things the White House wanted. They were sceptical about the supersonic transport and hostile to anti-ballistic missile deployment, and — worse — they said so in public and testified to Congress. Nixon abolished PSAC in 1973. The room did not close because science had become expensive. It closed because the advice had become inconvenient, and the advisers had a constituency of their own.

William Proxmire’s Golden Fleece Award, from 1975, supplied the method rather than the motive: any research programme can be reduced to a punchline by quoting its abstract without its rationale. The technique is cheap and structurally unanswerable, since the defence requires more attention than the attack.

Beneath it ran the material fact: the Sputnik cohort finished their doctorates into the market of 1970–71, and there were no jobs. Stagflation made the Endless Frontier bargain look less like a covenant than an unaudited invoice.

V. 1980–2008: from public good to marketable asset

Bayh-Dole in 1980 let universities take title to inventions from federally funded work. Sensible policy — and it changed the terms permanently: science stopped being defended as a commons and started being defended as a pipeline. Once the justification is commercial the assessment becomes commercial, and a project that cannot show a licensing pathway argues from a weaker position than it did in 1965.

The clarifying event is the Superconducting Super Collider, cancelled in 1993 with a couple of billion dollars already in the ground — two years after the Soviet Union ceased to exist, while the more expensive and less defensible Space Station survived by a single vote. If you want evidence that the postwar bargain was always a security bargain in the clothes of a knowledge bargain, that is your exhibit. The Human Genome Project became the successor template on new terms: dated, milestoned, auditable. Big science survived by becoming a project plan, and what was lost was the standing to say we do not yet know what this is for.

Austerity after 2008 exported the fight to countries that had not had it, and climate and COVID converted scientific institutions into identity markers, at which point the findings became almost incidental to their reception. The New Zealand variant is distinctive: we imported the managerialism rather than the culture war. DSIR was broken up in 1992 into Crown Research Institutes required to behave like companies; the Performance-Based Research Fund arrived in 2003 and taught a generation to optimise for countable outputs; the current consolidation into Public Research Organisations is the fourth or fifth restructure of a system never given long enough to compound. Our failure mode is not denialism. It is the belief that a research system is a portfolio of contracts, and that the interval between reorganisations can be shorter than the interval required to do the work.

VI. The gap machine

Here is the argument, and I think it is under-argued elsewhere. The most damaging thing that happened to scientific authority in government was not budget cuts or Proxmire or Twitter. It was the repeated, public failure of state technical forecasting about rivals — and the fact that the discipline was blamed for outcomes produced by the politics of the estimate rather than by the estimate itself.

Joe-1, August 1949. American estimates put a Soviet nuclear device in the mid-1950s. It arrived in 1949. The irony is almost never noticed: the detection was itself a triumph of applied physics — atmospheric sampling and radiochemistry on filter papers, a programme that existed because scientists insisted on it against institutional indifference. Good science caught bad estimation, and the lesson drawn was that the scientists had been wrong about the Russians.

The bomber gap and the missile gap, 1955–61. A flypast at Tushino was read as evidence of Bison bomber production at rates the Soviets never achieved. Estimates of Soviet ICBM inventories ran to the hundreds, and Kennedy campaigned on them. CORONA imagery from 1960 dissolved both; the actual force in 1961 was a handful of unreliable missiles on exposed pads. Both were overestimates, both useful to somebody, and both were resolved not by argument but by a satellite — an instrument that made the estimate checkable and thereby ended the debate.

Hold that thought, because what follows is what happens when a question is not checkable.

Team B, 1976. Under pressure from critics of détente, DCI George H. W. Bush authorised an external panel to re-examine the CIA’s own raw intelligence and produce a competing assessment. Its most revealing error concerned Soviet anti-submarine warfare: finding no evidence of a non-acoustic detection system, the panel reasoned that the absence was itself suggestive, since such a system would be concealed if it existed. Not finding it was consistent with its existence — the god of the gaps with a security clearance, unfalsifiable by construction, the inversion of what assessment is for.

The precedent matters more than the errors. Team B established that a technical estimate could be contested by convening an alternative panel with a different prior. After 1976 the estimate is no longer a finding. It is a position, and positions have sponsors. Once that is true, the scientist in the room is no longer the person who knows; they are one of the parties.

Iraq, 2002. The load-bearing case, precisely because the expertise was in the building and was overruled. The October 2002 National Intelligence Estimate concluded that Iraq was reconstituting its nuclear programme, resting substantially on a shipment of aluminium tubes characterised as centrifuge rotor components. The specialists at Oak Ridge — the people who actually build and operate enrichment cascades — said the tubes were the wrong dimensions, wall thickness and alloy, and matched an artillery rocket Iraq had reverse-engineered. State’s intelligence bureau dissented on the nuclear case, Air Force intelligence on the drones. The dissents were right, and they were relegated to boxed text and footnotes. The headline judgements were written to support a decision already taken.

What happened next is the crux. When the stockpiles were not found, the lesson drawn was the experts were wrong. The actual lesson was the experts were ignored — hands-on technical competence outranked by narrative competence. The distinction was never made, because making it would have required assigning blame to individuals rather than to a profession, and professions absorb blame more quietly.

And then the polarity flipped. Since roughly 2010 the failures have been underestimates: the Chinese hypersonic test in 2021 that the Chairman of the Joint Chiefs compared to Sputnik; SMIC producing 7-nanometre silicon in 2023, after an export-control regime designed on the assumption that it could not; DeepSeek in early 2025, and the discovery that frontier capability was less capital-bound than the consensus assumed.

Same erosion of credibility, opposite direction — and the inversion tells you what the gap machine runs on. In the Cold War the reward structure favoured fear: overestimating Soviet capability funded programmes, and nobody was punished for caution about an adversary. Today it favours dismissal. Underestimating a competitor flatters the incumbent industrial position, justifies existing policy, and costs nothing until the moment it costs everything. The gap is manufactured downstream of the science, in the direction the incentives point. The scientists get billed for it either way.

VII. The doom loop

Put the mechanism together and it is a circuit, not a sequence. Exclusion degrades assessment: when PSAC is abolished and Team B convened, the technical voice weakens and the estimate becomes a negotiated document. Degraded assessment fails publicly, because reality is eventually checkable even when the estimate is not. Public failure is attributed to expertise as such, which justifies further exclusion. The room, each time round, becomes more confident and less informed.

Iraq is the clean demonstration because you can name who was right and point to where they were sitting. Oak Ridge was right. State’s analysts were right. Both were downstream of the drafting. The failure was not epistemic; it was architectural.

VIII. Where the scientists actually went

The honest complication is that scientific advice has not disappeared. It has proliferated: an Office of Science and Technology Policy, chief science advisers across the Anglosphere including our own, national academies, the IPCC, standing committees on everything.

What changed is the form. The adviser in the room became advice as a document — and documents can be received, thanked, filed and cited selectively. A person with standing can say you are about to do something stupid, and I will be in the record saying it. A report cannot interrupt. The proliferation of advisory bodies is not evidence that the room is open; it is what results from the room being closed, in the way a building’s fire escapes multiply as its exits are sealed.

The further problem should worry us most. The frontier capability has left government entirely. On artificial intelligence, on compute, on launch, the state is no longer the assessor of the technology; it is the customer, and often a junior one. The next gap — and it will probably concern AI — will be estimated for governments by firms holding positions in the outcome. Team B at least had to argue with the CIA. The successor arrangement has no counterparty at all.

IX. Coda: the small country problem

Return to Tizard and Lindemann, because the question was never whether governments listen to scientists. They listen constantly, to whichever scientist is already in the room. The question is whether anyone there can tell which one is right, and whether anyone has the standing to say the sentence that ends careers: we do not know.

That sentence is what the gap machine exists to prevent being spoken. A gap is what you install where the knowledge isn’t, and its appeal over an admission of ignorance is that a gap has a budget line.

For New Zealand this is not an abstraction. We have no independent capacity to estimate what a competitor can do, so we consume other people’s estimates, importing their errors and incentives without the compensating benefit of their capabilities. Iraq 2003 is what that looks like when it fails.

But it may also be the beginning of an advantage. A country too small to manufacture its own gaps has an unusually strong interest in the actual state of the world. We have no domestic constituency for overestimating a rival and no industrial position to protect by underestimating one. That asset is worth nothing unless we build the standing for technical people to use it: research organisations that outlast electoral cycles, advisers with access rather than reporting lines, and a tolerance for the sentence about not knowing.

Eighty years ago a man walked into an office with one sheet of paper and was believed. We have spent the intervening decades building elaborate machinery to ensure that this can never happen again, and calling the machinery accountability.

Sources and further reading to add before publication: Bush, Science, The Endless Frontier (1945); Zachary, Endless Frontier (1997); Wang, In Sputnik’s Shadow (2008) on PSAC; Cahn, Killing Détente (1998) on Team B; the Silberman-Robb WMD Commission report (2005); Stokes, Pasteur’s Quadrant (1997) on the linear model.