Innovation and token supply

KV Cache Optimization

DeepSeek's latent attention compresses stored keys and values; sparse attention selects relevant tokens, while newer designs reuse cache across layers and use lower-precision storage.

Inference Optimizations

DeepInfer, vLLM, and SGLang combine tuned kernels, paged memory, prefix reuse, and scheduling to serve more tokens on the same hardware.

Reasoning Theater

Models can keep producing reasoning after an answer is internally settled; one study cut up to 80% of tokens on selected tests. OpenAI and Anthropic offer adaptive thinking, while open models have room to improve.

Intelligence Compression

Older pruning work removed up to 90% of nodes in task-specific networks without major accuracy loss. Monosemantic feature research suggests a path to leaner models, though frontier gains remain unproven.

Specialized Models

System One decision models such as Jev handle broad classes of structured decisions without generating prose; the developer reports much lower cost and higher throughput.

Compute Optimization

Even non-AI workloads have room to improve: Cloudflare says algorithmic and pipeline changes deferred about 2.5 years of hardware expansion.

Context Optimization

Retrieval from large corpora still brings in irrelevant material. Better indexing and filtering can reduce the context sent through a model.

On Device AI

As models become smaller, more inference can move onto devices and the edge instead of the cloud.

Continuous Learning

Major releases often start new training runs; smaller updates may continue from checkpoints. Continual learning is still early because new training can erase old skills.

Intelligence Supply=Compute Capacity×Yield×(Unit Price − Unit Cost)

If innovation trends and spending plans continue, we could see an exponential increase in supply

13x × (1 + 70%) × (1 + 25%) = 27.625x per year

Yield Improvements13x

Multiplier each year

Compute capacity70%

Annual increase

Training to Inference25%

Additional supply shift