Combining Experiments, Large Language Models, and Theory to Discover Quantum Materials

TL;DR

• The question: can today's (or tomorrow's) AI zero-shot the next big scientific breakthrough? It has to be zero-shot, because a breakthrough by definition cannot be in the training set.
• His answer is no — and the position he's arguing against is what he calls thinkism: locking yourself (or an AGI, or a swarm of agents) in a room with every textbook, paper and patent in the field and thinking very hard.
• The history of superconductivity is the evidence. The 1911 discovery, MgB2, and the 1986 cuprates were each found by building a new capability, by brute-force trial and error, or by looking for something else entirely — never by deriving from theory.
• So agents aren't useless — they belong in specific slots of the loop scientists have used for centuries: simulation (ML force fields), characterization, literature search, and design of experiments.
• His advice to science students inverts the doom narrative: go deep on fundamentals and on experimental methods and measurement tools. New instruments create new data, and only then does AI's thinkism skill have something to work with.

Key Points

Framing: can AGI zero-shot a breakthrough? (~00:34–00:35)

He pins the question down immediately: a breakthrough can't be in the training set, so it has to be produced without a previous demonstration — zero-shot by construction.

Restated: can thinkism ever be enough for advancing physics? Thinkism being the idea that you lock yourself in a room with all the textbooks, all the previous experiments, papers and patents in the field, and think really hard — whether you're a human, an AGI, or a bunch of agents.

His answer: no, because the universe is too complex for any intelligence to think its way into a scientific advance. The alternative is what we've done for centuries — iterate with the universe: hypothesis, attempt, failure, learn from the failure, try again.

Three superconductivity breakthroughs, none from thinking harder (~00:35–00:39)

1911, the discovery itself. Nobody reasoned their way to "cool mercury far enough and electrons will feel an attractive force through phonons." What happened is that Kamerlingh Onnes's lab built a new capability — cooling below any temperature previously recorded, around 4 K, eventually reaching 1.8 K — and mercury happens to superconduct at 4 K. It was a side project of the main effort, which was liquefying helium. Onnes won a Nobel a few years later, and Cubuk underlines the detail that matters: the Nobel was not for superconductivity, it was for liquefying helium.

1957/58, BCS theory. Bardeen and collaborators explained conventional superconductivity. In principle, humans could then use that understanding to find many more superconductors — but that isn't what happened. From 1958 to roughly 2000, the biggest advance in conventional superconductivity (the SOTA, in deep-learning language) was MgB2, and it was found by trying tens of thousands of materials until one turned out to be an excellent BCS superconductor.

1986, the cuprates. Karl Alexander Müller and a collaborator found the first high-temperature, unconventional superconductor. But read his Nobel lecture and you discover he was actually looking for a conventional superconductor, because he had no concept of what an unconventional one would even be.

He scopes the claim carefully: this holds for complex systems — materials science, chemistry, solid-state physics — and explicitly not for math and theoretical computer science, which he considers a different regime. On the complex-systems side, most things were found by accident or by intelligent trial and error.

Where the agents go (~00:39–00:41)

His conclusion isn't pessimistic: we're fine, we just have to work out how to use agents inside the scientific method we already have.

The loop is: observations → hypothesis → experiment → result → (it probably didn't work) → understand why it failed → try again. AI slots into several places at once:

• Simulation, in the middle of the loop, has already been completely rewritten by machine learning. Force fields approximate the ground-state quantum mechanics, and he puts it strongly: even if another AI winter arrived, he can't imagine ever going back to force fields without machine learning. AlphaFold is the other example. But he flags the limit — force fields and AlphaFold are tools; they don't hand you the next drug or the next superconductor.
• Characterization: making sense of what the experiment actually produced. Technical, but exactly the kind of thing AI can automate and scale. He notes that Berkeley has produced frontier work here, using robotic arms to try many powder syntheses in the lab.
• Literature search: reading everything and surfacing which ideas are worth trying next.
• Design of experiments: given everything measured so far, choosing the next experiment.

The honest caveat: it's never as clean as the slide. Reality is a spaghetti plot where everything interacts with everything else — but you can still see the whole method accelerate if AI is inserted deliberately and carefully.

Advice to science students (~00:41–00:42)

Don't look at LLM progress and get discouraged; conclude the opposite. It's an exciting time to do science.

His concrete advice: go all-in on the fundamentals. If you're a physicist, study the theory of superconductivity and solid-state chemistry; if you're a biologist, learn what actually matters in biology. And take experimentation seriously, because a lot of historical progress came from someone inventing a new experimental method, technique, or measurement tool — all of which you can still contribute to.

The loop closes on itself: as you create new experimental methods and tools, AI sees more data, applies its thinkism skill to make sense of it faster and at larger scale, and your next experiment gets both more successful and more intelligent.

Quotes

"The universe is too complex for any intelligence, whether it's humans or machines, to think their way into the scientific advance." (~00:35)

The thesis of the talk.

"Nobel Prize was not given to superconductor discovery — it was given to the fact that he could liquefy helium." (~00:37)

The prize went to the new capability, not the discovery it enabled — which is exactly the causal direction he's arguing for.

"He was actually trying to find a conventional superconductor, because he didn't even know what an unconventional superconductor would be." (~00:38)

"This universe is too complex for thinkism alone to innovate." (~00:41)

His closing slide.