End-to-End Security Research with a Language Model

TL;DR

• Prediction vs. reality. Two years ago a room of frontier-lab researchers estimated 2–5 years at best (5+ on average) until agents do more than 90% of our coding, and more than 10 years until agents do more than 90% of our general research. They were somewhat wrong on coding and very wrong on research.
• The evidence is a cryptanalysis paper published that week. The model found a new attack on 7-round (reduced) AES, cutting the best known complexity from the 2013 result of 2^99 to roughly 2^89 — empirically 200–800× faster. He was emphatic that AES itself is not broken; breaking AES would break the internet. Same approach also produced attacks on HAWK (a post-quantum algorithm) and LEA.
• The shape of the shift, and the new bottleneck. A human sits and thinks for weeks, talks to people, then implements; the model generates many ideas and implements all of them to see which works. "Ideas are cheap, execution is everything" may be inverting. Concretely: from the prompt "go beat AES," the model produced 3,000+ ideas, 2,000+ of them actually applicable, ~200 genuinely novel, and exactly four made it into the paper. Most of the team's time now goes to verifying results.

Key Points

The two questions from two years ago (~03:01–03:04)

At an event much like this one two years ago, Nasr and a group of researchers from top frontier labs did what researchers do and tried to predict the future. Two questions:

1. How long until LLMs and agents do more than 90% of our coding? Best case 2–5 years; more than 5 years on average.
2. How long until agents do more than 90% of our general research? More than 10 years.

His verdict: on coding they were wrong, though maybe not by that much. On research they were possibly very, very wrong.

He pinned the definition down first. By "LLM research" he does not mean using an LLM as a tool — many conferences, including security venues that are extremely conservative, now allow that to some degree. He means full end-to-end research: having the idea, forming a hypothesis, developing a methodology, testing it, and going beyond. He acknowledged plenty of disagreement in the room, and that many people think it's impossible — he was one of them two years ago.

The paper: 7-round AES (~03:05–03:10)

"This week we published this work showing Claude can do cryptographic analysis" — meaning finding flaws in cryptographic algorithms. One result is a new attack on the symmetric algorithm AES. The paper has two authors: himself and Nicholas Carlini. He deliberately put their CVs on screen: Carlini comes from systems security and more recently machine-learning security; Nasr from network security, now machine-learning security. "Two of us plus a copy of the internet plus a bunch of coding agents shouldn't essentially be able to come up with an attack on cryptographic systems." They both like cryptography and have tried to break simpler things before — and are, in his words, very weak at it.

AES background: standardized by NIST in 2001, an iterative algorithm with 10 rounds. It takes an input and a key and has four main parts — substitution (the S-box maps an input to another set of outputs), a permutation that moves bits around, and adding a function of the key — repeated roughly ten times.

Then the disclaimer he gave twice: they did not break AES. If someone breaks AES, it breaks the internet. This is academic work on reduced-round AES — seven rounds instead of the full ten, which is what academics study because it is more tractable and improvement is plausible.

The complexity ladder:

• Brute force on 7-round AES: about 2^128 operations — longer than the time until our sun goes nova.
• The best existing algorithm, from 2013: about 2^99 — call it 200,000 to 2,000,000 years.
• And not for lack of trying: this is a fairly well-studied problem.
• Their result: about 2^89, depending on how you count, and empirically 200 to 800 times faster. His own gloss: "so maybe in around a thousand years — I don't know how much that matters."

The mechanism, compressed into two minutes: attacks on crypto algorithms typically look for a relationship between input and output without breaking the algorithm itself. The AES S-box uses a specific formula — it inverts the input in a finite field, then scales and shifts it. His analogy: if you take an object and scale and shift it, the ratios within the shape stay the same, just moved and zoomed. The model recognized that the S-box has this property and used it to construct the algorithm. The method is very complicated; he pointed interested people to the paper.

And it wasn't only AES — they also produced attacks on other cryptosystems, including HAWK (one of the post-quantum algorithms) and LEA (used in other countries). Again: none of these attacks has changed anything about the internet. These are academic achievements.

How the model does research: 3,000 ideas for four results (~03:10–03:12)

He contrasted two working styles:

• If he did it himself: sit somewhere and think for a few weeks, come up with a few ideas, go talk to a few people, converge on a few, then implement them.
• The model does it differently: generate a lot of ideas, and instead of going to talk to people, implement all of them to see which one actually works.

Hence the inversion: "We used to say ideas were cheap and execution is everything. Now maybe ideas are more important, and execution the model can do very, very fast."

AES has a complication: execution itself is infeasible — you cannot simply run a 2^89 algorithm. So they built a harness that takes a research idea, consults a few other agents on whether the idea is good, then implements it and iterates on it. His honest caveat: you need a harness right now; maybe not in the future, but at least today you do.

The search numbers were specific. Starting from a single instruction — "go beat AES" — the tree of ideas unfolded into:

• more than 3,000 ideas generated;
• more than 2,000 actually applicable to the problem;
• around 200 that were genuinely novel;
• and only four that made it into the paper.

The cost was a lot of compute.

"Maybe you're just bad at crypto" — and the new bottleneck (~03:12–03:13)

He raised the obvious objection himself: maybe they simply are bad at cryptography, which is why they find this impressive. He conceded it might be true — he has no cryptography background, so what impresses him may not impress anyone else. But they showed the results to actual cryptographers, who were interested in them.

The second-order objection: maybe humans in general are bad at crypto. His answer is that they have looked well beyond crypto at other security research — extending from his own network-security background into privacy and elsewhere — with promising results in all of them. And notably, most of their time now goes into verifying the results.

His closing point returned to the nature of research: we used to say you need a novel idea and a good idea at the same time, and one prevailing belief was that LLMs might be good at each but not at both simultaneously. Maybe, he suggested, that is starting to move.

Quotes

"Two of us plus a copy of the internet plus a bunch of coding agents shouldn't essentially be able to come up with an attack on cryptographic systems." (~03:06)

He undercuts his own credentials first, so the source of the result is unambiguous.

"We didn't break AES. If someone breaks AES, it breaks the internet." (~03:07)

The disclaimer he made twice, and the most important sentence in the talk.

"Maybe we had this saying before that ideas were cheap and the execution is everything. Now maybe ideas is more important, and execution the model can do very, very fast." (~03:10)

The scarce resource in research is changing places.

"We are spending most of our time verifying the results." (~03:13)

The new bottleneck is not production, it is verification.