Panel: Frontier Research

Topic 1: Where do rewards come from after coding? (~00:54–00:59)

Igor Babuschkin's opening question: every new direction shares one property — it's getting harder to generate the rewards that power RL. How does each of you think about that in your own domain?

Ed Chi: reward hacking shows up as model instability; the answer is more top-down

• The dominant paradigm for the last couple of years used correctness as the feedback signal, and reward functions were easy to write when correctness was built into the question.
• But many questions we want to ask — science included — have answers that aren't correct or incorrect, but somewhere in between.
• The hard part isn't just approximating them. The RL process is extremely good at finding loopholes in a reward function that isn't black and white — reward hacking — and in practice it manifests as model instabilities during algorithm development.
• He ties it back to his talk: using correctness alone is like slapping your kid when they get something wrong and offering no explanation. "It's a very dumb way of teaching."
• His read on the field: distillation and SFT coming back into vogue is top-down processing returning. Some people find it distasteful — it feels like using human brains to teach machine brains, less emergent — but he believes we need more top-down processing injected into AI training.

Ekin Doğuş Çubuk: two problems with furnaces, and why context is always incomplete

Igor's framing: "How are things looking with the furnace? Is it generating rewards?"

• Problem one: you often don't know whether it worked. You take the thing out of the furnace and it isn't obvious; it might have partially worked.
• Problem two, and the deeper one: how much of the relevant variables fit in the context. The fields where LLMs and RL made huge impact share two properties — a clear yes or no, and all the needed context being available to the model.
• Physics fails the second test: we have more atoms than we can track. His analogy is thermodynamics, which was a big deal in the 19th century precisely because we realized we could replace 10²³ atoms per mole with a handful of variables like pressure, energy, and volume.
• So you give up on providing all the relevant context and ask instead what you can provide, and whether it's sufficient. Practically: add as much metrology and characterization as possible — even more than before — so the LLM gets maximum context about what happened in the furnace or the thin-film chamber.

Richard Socher: metacognition, and the blank space where subjective functions should be

• He reconnects his four-layer stack (human knowledge → scientific measurement → simulation → real experiments) and the hope of resolving some of the inner loops there.
• Then the thing he's been chewing on lately: metacognition. If you truly have something superintelligent, it wouldn't robotically do exactly what you asked.
• The gap he points at: there's essentially no research on subjective functions, only the objective functions we all work on.
• How could those emerge? By giving AI an environment that isn't easily reward-hackable — and he names reward hacking as his number-one safety and alignment concern.
• His early-sign example: Wispr Flow writes what you mean rather than what you say. Most AI today does the opposite — tell it to make a benchmark number go higher and it does exactly that, "but you didn't mean by hacking Hugging Face." He expects this to get resolved eventually.

Topic 2: Are you actually seeing acceleration? (~00:59–01:06)

Richard Socher: agent count, not headcount

• "It is kind of nuts. We have a fairly small team, but they act as if they're like an order of magnitude more people."
• "It's much less about headcount these days than it is about agent count." Wielding hundreds or thousands of agents takes a lot of compute.
• His forecast: forward-leaning organizations spending more on compute than on headcount is very likely, especially at the frontier.

Ekin Doğuş Çubuk: the bottleneck is understanding what you made

Socher asks him back: how much robotic process automation is possible now, and is the bottleneck intelligence or mechanics?

• On the experimental side, the biggest bottleneck is characterization and analysis. Mixing things and trying a thousand at once isn't hard; understanding what you actually made is.
• That's good news, because AI is good at exactly this: you can show it how to do XRD refinement, how to use Rietveld refinement tools, how to put simulations in the loop. Doing characterization analysis well isn't as hard as discovering something new — it's mostly using the tools correctly.
• The upshot for science: we can try many more things and analyze them intelligently.
• Two more areas: theorists and experimentalists all write code, and LLMs have completely revolutionized that; and on the simulation side, graph neural networks permanently changed force fields over the last decade — we can now model a remarkable fraction of the periodic table without doing quantum chemistry. (His aside: you could argue even Einstein's first paper was a force-field paper.)

Ed Chi: the harness is evolved prompt engineering — and together they've plateaued

Asked whether Google is using agents to accelerate itself, he says yes for coding work, but wants to reframe the question from a slightly odd angle:

• Agent harnesses are prompt engineering, evolved. The path: prompts → putting more into the prompt → assembling prompts programmatically → LangChain-style prompt chaining → wrapping a whole program around prompt generation and sequencing → agent harnesses and coding harnesses.
• The tension underneath the question has been live for three or four years: how much belongs in the model versus in the programmable framework we now call the harness (previously just "programmable LLM frameworks").
• People in the audience are evaluating all pairs right now, and the pattern is clear: strong model, weaker harness; weak model, stronger harness.
• An etymological detour: "harness" comes from horses. We measure engines in horsepower because of the industrial revolution, which made us all engineers tending engines — and an engine mount is basically a harness. So prompt engineering and the harness are the structure that gets the engine to do what we want.
• Then the heaviest claim of the panel: people running these combinations are realizing the two together only reach a particular performance level, and there's a plateau nobody seems able to get over right now — which he considers the industry's major current bottleneck. (Igor: "That's the frontier." Ed Chi: "That's literally the frontier right now.")

Richard Socher, jamming on that: stuck below "anthropic bounds"

• The whole field is stuck below what he calls anthropic bounds — not just the company. We score models with ELO from human judgments; we build benchmarks from human-designed labels, classes, and tasks.
• All of those carry implicit and sometimes very explicit human ceilings. You can get 100 out of 100; humans get maybe 95 and you land 5% above the human baseline — and then where do you go?
• The only ways out are simulations with verifiers, where superhuman capability is actually reachable, and open-ended evolutionary environments.

Topic 3: What should researchers without frontier compute do? (~01:06–01:18)

Richard Socher: go find low-hanging fruit in other fields — economics, for example

• Blunt version: pre-training, mid-training, post-training, and RL training are probably not the best research areas for a university PhD student. But AI's application areas are wildly underexplored.
• He points at his 2018 two-level RL paper, the AI Economist: a lower level of economic agents optimizing their own utility (some willing to work 100-hour weeks, some only 10, some unmotivated), collecting resources, blocking each other, building monopolies; and an upper-level AI economist setting taxes and subsidies. That lets you simulate billions of years of taxation and subsidization — essentially political philosophy — optimizing whatever objective you specify (productivity × equality, sustainability, the middle class, whatever politicians care about).
• The paper never had its GPT moment. Nobody took the idea and scaled it up massively.
• So the whole field of economics is still pre-AI. He cites a very famous Berkeley economist's well-known formula, which is provably correct only in a one-step economy with no adaptation, no adjustment, and no temporal sequences.
• If AI can simulate people better and better, especially in aggregate statistics, there's enormous low-hanging fruit for PhD students applying AI to economics — and that's one example among many underexplored field combinations. He's glad venture capital is now interested in chemistry, physics, and biology too.

Ekin Doğuş Çubuk: master the fundamentals nobody else has

• The caveat first: predicting the future is hard. If in four years robots are as dexterous, intelligent, creative, and stylish as humans, this answer gets complicated.
• But today it's easy: robots aren't dexterous, aren't creative, and have no sense of interestingness. So students should study the fundamentals.
• Very practically: when a scientist applies to Periodic Labs, whether they've used agentic tools to run simulations doesn't make much difference — that's learnable quickly. What's unique, and increasingly rare, is someone who genuinely understands the physics or the experiments they're studying.
• His old joke: at Google he hired interns from physics labs to do physics research and they'd be much stronger at ML than at physics. Back then he also worried that simulation tooling is all Fortran, C, and C++ while the kids only know Python — but he considers that resolved: even though it's not fully here, you can see agents will translate a Fortran package into Python or JAX.
• What machines still can't do: be creative about fundamentals, produce a true innovation in our understanding of the universe, run experiments, or invent new experiments. So the big alpha is understanding the fundamentals of the application domain and connecting that to machine learning.

Ed Chi: chain of thought was paid for on a personal credit card

The best story of the panel, prefaced with "I hope this doesn't get me fired from Google":

• Before chain of thought, he and Denny Zhou were both at Google Brain. Denny complained that the people controlling resources inside Google Brain made it very hard for him to do research — precisely the situation in Igor's question.
• Ed's advice at the time: "I don't know. Go think about how humans think, and maybe we can figure out a way to inject these processes into the model."
• Then one day Denny asked whether he could run a bunch of experiments on GPT — because at Google they were supposed to use PaLM, but he couldn't get chips or checkpoints for it.
• "Who will pay for it?" Ed pulled out his credit card. This was before ChatGPT, so there were no free resources; you bought an API key. Denny racked up a couple thousand dollars of API charges, and they wrote the paper that became chain of thought.
• The comparison is the point: Google was already spending billions training large language models, and a new top-down idea — next-idea prediction rather than next-token prediction — cost under $10,000 in total. The paper was around January 2022.
• To the objection that it was three or four years ago and impossible in 2026: where were you when Peter Steinberger was working on OpenClaw? A lot of people only started paying attention to agentic programming frameworks when he put the pieces together, and that didn't cost billions either. ("He was just some dude. I was flying in from Taiwan last night so I didn't get to attend his talk.")
• He places the complaint in a longer arc: many mature fields went through this. Experimental physicists complain about not having a linear accelerator; when he did supercomputing in the 90s he complained about not having a Cray Y-MP's 2.6 GFLOPS.
• A quiz for the room: why was the US investing in supercomputers in 1990? Answer: nuclear simulations, because of a test-ban treaty — we couldn't blow things up anymore, so we had to invent a new way of understanding physics.
• The lesson: we're very good at finding new ways to look at old problems. When everybody is working on RL, you should be thinking about what the next idea is.

Richard Socher: rejection, resilience, and "not everything is a transformer"

• In 2010 his first neural-nets-for-NLP papers were desk rejected — "we don't do neural nets in NLP conferences" — without anyone looking at the experiments.
• Later they submitted a prompt-engineering paper where one neural network answered any and all kinds of questions, and reviewers literally wrote that it made no sense: you should never have one model answering different kinds of questions, they should all be different models. Unfathomable at the time, so it was rejected. That was decaNLP; the ICLR submission is still public on OpenReview. The first GPT paper cited it five times.
• His observation: there's now almost a monoculture, mirroring 2010's anti-neural-net monoculture with the sign flipped — now it's almost all neural nets.
• So it's worth thinking about completely different approaches again. Not everything is a transformer; the transformer is one sample from a very large equivalence class, and there are likely better classes of algorithms unexplored. Compute substrates too — the brain has more FLOPS per watt than any hardware we have.
• "There's no excuse for researchers in AI and outside of AI not to still do academic work."

Igor Babuschkin: the tools are democratizing

His closing addition: frontier research keeps getting more expensive, but the tools keep getting more democratized. You can start from a powerful open-weight model today, and post-training and RL tooling is democratizing too — he names the River AI API and Tinker — so genuinely frontier work is possible on a limited budget.

Topic 4: Predictions for the next 12 months (~01:19–01:24)

Ed Chi: personalization gets a real attempt, and induction is what's missing

• He admits he predicted personalization would be big at last year's summit, and it's been surprisingly slow.
• The issue has mostly been data — the sparsity of personalization data — and integration.
• Prediction: these things usually take three tries, and a good try will probably land within the next 12 months. Two blockers: data/integration, and inductive inference.
• Today's models lean heavily deductive: you bought a bicycle, therefore you'll need tires. But recommendation and personalization tasks are deeply ambiguous and need induction — you bought a turntable, so maybe you like classical music, or jazz.
• He's a serious vinyl collector, and says one reason he came up to Berkeley is that Rasputin isn't far away, so he can walk over and peruse the aisles. That's exactly the induction he wants: Google Maps should be telling him Rasputin is a five-minute walk away.

Ekin Doğuş Çubuk: induct from math and code to fields where pure theory goes far

• "Speaking of induction" — where LLMs and agents have already done something incredibly different is math, theoretical computer science, and programming.
• So inducting forward, the next step should be fields where pure theory goes a long way: maybe particle theory (nodding to Igor's own background), maybe astrophysics. It could also be computational physics, where some very hard problems aren't necessarily about the real universe but matter in their own right.
• He's honest that he doesn't know whether it's a 12-month thing — predicting the future is genuinely hard — but the trend suggests that's the next step.
• (Igor adds that mathematics results are coming out daily now, with more and more theorems proven — a promising bet.)

Richard Socher: cheats with three, plus a negative prediction

• One: more proactive consumer interfaces that capture more of your context and make recommendations before you ask. A few startups are growing here, and more is coming.
• Two: faster clinical trials in biology, with more AI-developed drugs and proteins succeeding at higher rates in later-stage trials.
• Three: the first baby forms of recursive self-improvement. He admits the cheat — the best predictions are the ones you're going to work on yourself and try to make true. He says it's feasible, it's happening, and they're confident; the rest is a question of how much compute you give it and how hard the tasks and environments are.
• The negative prediction: despite being extremely excited and optimistic about AI, he considers hard-takeoff scenarios — everyone loses their jobs, the AI kills everyone — highly unrealistic sci-fi, and predicts none of the doomsday scenarios will happen at all.
• His closing line: "Just like GPT was too dangerous to release to the world, there will be no open source model that will be too dangerous to release to the world."

Quotes

"It's much less about headcount these days than it is about agent count." (Socher, ~01:00)

"The two together can only reach a particular performance level, and there is a plateau that we see we can't seem to go over at the current moment — and that is actually the major bottleneck currently facing the industry." (Ed Chi, ~01:05)

Model plus harness has topped out; he thinks that ceiling is the real frontier problem.

"The whole field of AI is kind of stuck below what I call anthropic bounds — not just the company." (Socher, ~01:05)

A pun with a serious argument behind it: ELO and human-labeled benchmarks both encode a human ceiling.

"The problem with physics is, we have more atoms than we can track." (Çubuk, ~00:57)

"And I pulled out my credit card. … I think we ultimately spent less than $10,000 to come up with the next idea." (Ed Chi, ~01:13–01:14)

What chain of thought cost, against the billions Google was spending on training at the time.

"Not everything is a transformer. Transformer is just one sample in a very large equivalence class." (Socher, ~01:17)

"Just like GPT was too dangerous to release to the world, there will be no open source model that will be too dangerous to release to the world." (Socher, ~01:24)

The last words of the session.