Scale Compute to the Signal

TL;DR

• The compute ceiling at the edge is biological, not engineering. Wearables have roughly 1–1.5 W to work with because human skin starts burning past 43°C. No number of gigafactories moves that wall — and the resulting edge/cloud asymmetry is the fertile ground for next-generation hybrid architectures.
• Three power tiers, one rule: escalate only when the task calls for it. Tier one processes signals where they originate (wake-word detectors, event cameras) at minimal energy; tier two runs algorithms or tool-calling workflows on-device at orders of magnitude more power; tier three invokes data-center-class compute at orders of magnitude beyond that. Escalation isn't just compute-expensive — the cost of the radio has not fallen anywhere near as fast as the cost of transporting a bit.
• Three calls to action: (1) the industry needs a composable runtime that can reason "this task needs no camera or microphone, just the radio and one API call" and budget power accordingly; (2) benchmarks should measure joules per task, not only parameter count and accuracy; (3) silicon developers should keep pushing resting power down.

Key Points

Inverting the question: what if there is no more compute to give? (~00:23:05–00:26:20)

Rahul Bakshi leads an applied science team at Amazon responsible for custom silicon — the latest generation, AZ3, powers the Echo devices launched last fall. His days are consequently spent on edge compute, edge devices, and edge-scale processing.

The dominant question for the last few years has been how to give agents more compute, and it's a good question because compute unlocks capability. But he flipped it on stage: what if, because of physics, there simply isn't more compute to give? His second theme: a great many edge agents don't need frontier-level intelligence at all, and working inside a tightly defined power budget opens up genuinely interesting optimizations.

One lever for more compute is more data centers — Amazon, the other hyperscalers, and the neoclouds are all building gigafactories. But at the other end sits a real power ceiling. He used wearables as the extreme case because the constraints there are hardest: a smartwatch, smart glasses, or a ring typically has about 1 to 1.5 watts to work with.

The reason is biological: human skin cannot tolerate more than 43°C without starting to burn and becoming deeply uncomfortable (~00:25:51). That is the hard edge-side ceiling — and, in his framing, the asymmetry between edge and cloud is exactly where next-generation hybrid architectures will be invented.

Three power tiers and the escalation rule (~00:26:20–00:29:00)

Accept the ceiling and what runs on edge-scale devices sorts into three power profiles:

1. Most efficient: process the signal where it originates. Tiny wake-word detectors and event cameras live here — detect at the source, burn minimal energy.
2. Next: decide what to do with that signal. Running algorithms or tool-calling workflows still happens on-device but costs orders of magnitude more compute and power.
3. Most expensive: data-center-class compute, invoked when the task needs context that isn't on the device or requires world knowledge — orders of magnitude beyond tier two again.

The rule that falls out: escalate only when the task calls for it (~00:27:20).

He then offered two independent data points that land on the same conclusion (~00:27:35):

• Earlier this year, Berkeley's function calling leaderboard (BFCL) showed a small language model matching frontier-level performance on the function-calling task. The benchmark tracks the primitives — does the generated JSON carry the right functions, the right APIs, the right parameters? The point is not that small models have caught up generally, but that for small, well-defined task sets there is now a pathway to frontier-level intelligence at edge scale.
• The second data point is the counterintuitive one (~00:28:37): over the last decade the cost of transporting a single bit across wireless networks has fallen many-fold, but the cost of the radio has not fallen by anything like the same factor. Escalating to the next tier is expensive in both energy and battery life, and should happen only when necessary.

Three calls to action (~00:29:00–00:30:40)

The cascaded tier architecture itself is well established — Amazon's, Apple's, and Meta's assistants have detected wake words and escalated only when necessary for more than a decade. What's missing is a composable runtime (~00:29:22), and that was his first ask of the room.

His example was concrete: you ask your assistant "hey, what time's my flight, is it late?" If the runtime were smart enough to know that this activity requires no perception sensors at all — no camera, no microphone, just the radio and one API call — then a power budget could be constructed for it and function calling could be given a corresponding dynamic schedule.

The second ask went to the benchmarking community (~00:30:16): Berkeley already has the function-calling benchmark, and there's an opportunity to extend it to include joules per task, not just parameter counts and accuracy. Energy is a frontier too.

The third went to the silicon developers in the audience (~00:30:33): push the frontier on lowering resting power so more efficient edge agents become buildable.

Quotes

"What if there wasn't more compute to give to our agents because of physics limitations?" (~00:23:45)

The talk's pivot: compute is not an unbounded input.

"Human skin cannot tolerate more than 43 degrees centigrade of heat without starting getting burned." (~00:25:51)

The wearable power ceiling isn't an engineering compromise; it's a physiological constant.

"The takeaway is escalate only when the task calls for it." (~00:27:20)

Edge agent design in one sentence.

"What is missing is a composable runtime." (~00:29:22)

The models are good enough; the missing layer is the one that budgets sensors, radio, and compute.