Panel: Agentic AI in Capital Markets

TL;DR

• The defining difference from software is the cost of a mistake. In a software firm a bad AI output gets caught by a unit test; in finance it costs money and can spill over into society. Quant firms have historically been early adopters, but agentic systems demand more caution.
• Trust has to be re-earned, not just earned. Models change, data changes, markets shift — what you trusted six months ago may not be trustworthy today (Ali Nazari).
• Bottlenecks don't disappear, they move. When a frontier model hands you 30 plausible research directions at once, the hard problem shifts from generating ideas to deciding which one deserves your time.
• Scaling laws may simply not transfer to finance. Li Deng's team spent years trying to scale and never saw emergent capability; what matters instead is systemic design — especially evaluation and feedback on the output side, because the market reacts to your orders and you change the environment you are trying to predict.
• The next compute explosion is a balance-sheet problem. Two Sigma has 1,800 employees, but a quarter of a million agents behind them each generating compute demand forces a real ROI conversation: "you dig a lot of dry wells" seeking alpha — the question is whether you're failing fast enough.
• The talent bar has already flipped. Two Sigma dropped coding tests ("we'd be testing LLMs and not people"); Vatic went from discarding any CV without a PhD to having more than half its people without one, and prefers senior undergraduates as interns because they do agentic work faster than full-timers.

Key Points

Opening: wow moments and disappointments (~03:51–03:58)

Moderator Bradley Olson asked each panelist to pair their introduction with a recent "wow moment" — or a moment when AI fell short.

• Jeff Wecker (runs engineering at Two Sigma; previously partner and first Chief Data Officer at Goldman Sachs; before that rebuilt Bridgewater's investment engine) tried an LLM for parenting advice a few years ago and found it "absolutely terrible"; a recent retry was genuinely impressive. "I don't know if the LLMs have gotten better or I got a lot worse."
• Jen Allum (co-head of one of D. E. Shaw's AI teams) picked something unglamorous: AI as an executive coach. D. E. Shaw is known for precise communication, an area she wants to improve, and she finds the feedback timely, actionable, and both qualitative and quantitative — provided you give it the right context.
• Ali Nazari (heads deep learning research at Susquehanna; previously founded a deep-learning hedge fund; before that an information theorist) chose a disappointment. A month ago he handed a frontier model a research problem and got back roughly 30 plausible solutions, each of which would have taken him months or quarters to reach alone. Then it hit him: the hardest part hadn't gone away — it had moved. His time now goes into deciding which of those 30 ideas is worth pursuing rather than into having ideas at all.
• Li Deng (~18 years at Microsoft, then Citadel, now Vatic Investments) offered three: 1. 2010: inspired by Geoffrey Hinton, he looked seriously at neural networks for the first time; the first two or three experiments cut error rates by a third to a half. He immediately abandoned every other tool (he had spent his first decade at Microsoft converting all his ML tooling to Bayesian networks) and was training 13–14 layer networks as early as 2010 — while most people at Microsoft refused to believe the results. 2. ChatGPT: his first reaction was that something must have gone wrong — exactly the reaction others had had to his speech recognition results ("you probably mixed training and test data"). But ChatGPT was a shipping product being genuinely used, so it couldn't be an artifact. He had to ask a former direct report, by then leading a reasoning team at Google, to explain that a class of decoding techniques worked unlike anything in traditional ML. 3. Agentic AI: not just pattern matching, but genuine reasoning, composition, and knowledge management.

The adoption curve in finance turns on the cost of mistakes (~03:58–04:03)

Jeff Wecker framed Two Sigma as "a technology firm that invests": of 1,800 employees, just under 1,000 are in engineering and another 400–500 in modeling. Engineers absorb new tools fast — Claude became available in February 2025, they were using it within weeks and had it broadly available within a month to six weeks. What excites him is getting everyone at the firm to think of themselves as AI-first, and the metrics are unambiguous: earlier and faster adopters are measurably more productive across code branches, written work, and published output. People with no technical background are now running Claude in auto mode or the firm's in-house workbenches in auto mode, and "the rate of innovation now is as high as it's been in recent memory."

Ali Nazari added the constraint that makes finance different. Adoption is real — every engineer and researcher at SIG uses AI tools — but the cost of mistakes is the dividing line. In a software company a bad AI output gets caught by unit tests, fixed, and forgotten. In finance it costs money, and in some cases carries risk well beyond the firm. Quant finance has been an early adopter of statistical learning and ML, but agentic systems require care. His framing:

Some things I use AI for constantly; for others I want a human in the loop. And trust isn't just something you gain — it has to be regained, because models change, data changes, and markets shift. What you trusted six months ago may not be trustworthy now.

Jen Allum, asked how she encourages experimentation before committing to scale, argued that firms like hers are underappreciated as innovation businesses: the relentless need to innovate means the DNA and the institutional history of experimentation are already there. As one of the original quant funds and a federated business with multiple strategies, D. E. Shaw can run experiments across the organization and let value surface bottom-up. Her operating principle is to optimize for rate of learning and rate of discovery: be maximalist in policy, build and buy and everything in between, let lots of experiments run, see what catches fire — and only then decide where to scale it, for whom, and how.

Noise, evaluation, and the agentic self-improvement loop (~04:04–04:07)

Li Deng mapped the quant workflow onto the tech one: extract features (what quants call alpha) → model the forecast → experiment and evaluate (back-testing, the analogue of validation) → go live (paper or online trading, the analogue of testing). The difference is noise: the input signal is noisy and the output signal is noisier still.

Part of why he left Microsoft for Wall Street was to understand this. He expected the problem to be essentially unsupervised — the signal being so noisy you might as well ignore it — but found it to be a mix: pure unsupervised learning gets you nowhere, there is a little supervision, and it is extremely noisy. The first time he looked at financial data, after residualizing returns, he assumed he'd been given the wrong file: it looked like pure random noise or a random walk. The signal is there, but you're extracting a very small amount of it.

He referenced Oriol Vinyals' earlier talk on recursive self-improvement pipelines: "we went through them all," but his team weights evaluation far more heavily — because when the evaluation output is itself extremely noisy, you don't even know what the right signal to predict is, let alone how to give feedback that improves your features.

On scaling he was blunt: the work of the past few years has not been about making the model bigger and waiting for LLM-style emergent capability. "We tried quite a bit, but we don't know how far we'd have to go — because of the nature of the noisy input and output, we actually gave up on that part." The real focus is making the whole system close a proper self-loop, and agentic methods are the key lever for minimizing the human effort alpha engineers have to pour in.

Compute cost: the explosion is imminent (~04:07–04:09)

Jeff Wecker traced the modeling history: at the firm's founding 26 years ago it was mostly low-dimensionality statistical models, then low-to-mid ML, then ensembles, then training their own transformers. GPT-3.5 was the inflection where publicly available models finally became usable, because the frontier labs handed everyone such a head start on training that it collapsed the cost.

Agent-centric workflows change the shape of the problem again:

We have 1,800 employees today. But if behind those 1,800 employees there are a quarter of a million agents doing operations, all making their own compute demand, we really have to think about the return on investment of that.

He conceded that "when you're seeking alpha, you dig a lot of dry wells — there's no doubt about it," but insisted on examining the maturity model of alpha investments and asking whether you are failing fast enough to hold back the compute spend.

Talent and hiring in the AI era (~04:09–04:14)

Jen Allum: D. E. Shaw has always hired for curiosity, lifelong learning, critical thinking, and systems thinking, and those qualities matter more now, not less. Because decades-long careers at the firm are common, what she's really screening for is adaptability — the expectation that someone will have multiple careers inside one organization. Building a team now means weighting those qualities alongside functional expertise and thinking a step ahead to the next role, and the one after that.

Ali Nazari: the constants hold (smart, curious, asks the right questions, works independently) and matter more than before. What changed is the interview process, which was designed for an era when implementation was the hardest part. With AI doing much of the implementation, they now test whether a candidate adapts, uses the tools properly, and — critically — can detect when an AI system is confidently wrong. His open question for the industry:

Judgment traditionally develops by trying something, making a mistake, and learning from it. If AI does most of the experimentation for us, how does the next generation develop judgment?

Li Deng gave the sharpest numbers. Two years ago the team was mostly Ivy League PhDs in math, physics, and some CS; a CV without a doctorate was rejected unread. Today more than half the team doesn't hold a PhD, and for internships they actively want senior undergraduates — who often do agentic work faster than full-timers. "The whole talent profile has changed quite a bit in just two years."

Jeff Wecker wanted one thing on the record for the audience: "we are hiring." But the job is different:

We've eliminated coding tests, of course — because we'd be testing LLMs and not people.

He argued the scientist / computer scientist / engineer archetype becomes more important going forward, with the load-bearing skills being: decomposing a problem cleanly into components, prompting to get the outcomes you actually want, and designing agents capable of exercising judgment to accelerate the generation of features, models, order-placement logic, or portfolio construction models. He's looking for people with the cognitive depth to be inquisitive, learn the business, and then bring their scientific training to bear on deconstructing what's newly possible.

Designing a quant firm from scratch (~04:14–04:17)

Ali Nazari split quant trading into research (creating knowledge) and trading (using it), said he'd weight research far more heavily, and named three concrete changes:

1. Every experiment — failures included — becomes organizational knowledge. Failures especially: people keep them in their heads, and the knowledge walks out the door when they leave.
2. Evaluation becomes part of the research loop, not something bolted on at the end.
3. Human and AI researchers go through the same research process and are evaluated identically, regardless of whether the output was AI- or human-generated, with knowledge from both flowing into the organization.

I look at research not as a collection of tasks, but as a process of continuous learning. I think that's the big distinction.

Jen Allum took the partnership angle: D. E. Shaw builds a fair amount in-house but also partners widely, and tries to be a good partner. Delivering a product into a regulated space raises hard requirements around enterprise readiness, security, and IP protection. Echoing Ali's point about the know-how these firms already possess, she sees room for much sharper thinking from product builders about what the actual value proposition is — what a new product would be, and how it would be genuinely additive to a firm like theirs.

Wildcard: a widely held belief about AI that's wrong (~04:17–04:21)

Jeff Wecker: the wrong belief is that you'll need fewer people. Some roles and jobs won't exist, granted. But the idea that you won't need technologists, scientists, and mathematicians in a world where systemic decision support is the future will turn out false — and the likely consequence is a whole host of new roles created to absorb the shift in the workforce.

The idea that there won't be opportunities for people in the future just doesn't make a lot of sense. It hasn't been true with any other major invention over human history. The world's changed, the markets grow, and the opportunity for more people to make an impact continues to grow.

Li Deng picked scaling. Earlier speakers in robotics and foundation models had talked about a first, second, and possibly third wave of scaling; he thinks that may hold for robotics but does not believe it holds in finance. His team spent years trying to scale and never reached the point where emergent capability appeared. What matters in finance is the systematic view of the whole system — especially the integration between ML components on the output side, evaluation and feedback — because feedback in financial markets is a categorically different problem from tech, and that design carries far more weight in the agentic era.

He closed with a story. When he moved from Microsoft to Citadel seven years ago, his boss — a billionaire — told him personally that they were hiring him but not for the systems-building expertise that had made his name in speech recognition. In tech, a translation or recognition system you build works today and keeps working for a while; you adapt every few years (a new president shifts how language is used, so the language model has to change; coding standards eventually shift and break a trained system).

But in financial markets, every time you put an order into the market, people react differently — you change the environment, far more than in the high-tech world.

The conclusion: the traditional high-tech way of thinking about scaling, and much of what the audience had heard that day, may mostly not apply to them.

Quotes

"The hardest part didn't disappear. It just moved." (~03:54)

Ali Nazari, after a frontier model returned 30 plausible research directions at once: the bottleneck relocated from having ideas to triaging them.

"Trust is something that needs to be gained … and trust needs to be regained. Model change, data change, market change, market shift … something that you were trusting six months ago might not be trustable right now." (~04:02)

The core stance of finance toward agentic systems: trust is not a certification, it decays.

"We have 1,800 employees today, but if behind those 1,800 employees there are a quarter of a million agents doing operations, all making their own compute demand, we really have to think about the implications, return on investment of that." (~04:08)

Jeff Wecker's warning about what agentic workflows do to a compute budget.

"We've eliminated coding tests, of course — because we'd be testing LLMs and not people." (~04:13)

The whole disruption of technical hiring in one line.

"I look at research as not a collection of tasks, [but] a process of continuous learning." (~04:16)

Ali Nazari's answer to designing a quant firm from scratch.

"In financial markets, every time you put an order into the market, people react differently — it changes the environment, much more so than the high-tech world." (~04:21)

Li Deng on why tech's scaling intuitions don't port: your own actions alter the environment you're predicting.