Skip to content
Pioneer Page PP markPPPIONEERPAGE

Pages from the advancing edge of computing.

Foundation models

Apple paper uses convex analysis to speed up RAG retrieval search

Researchers describe training neural networks to approximate support functions for faster and more accurate vector matching in retrieval systems.

Apple paper uses convex analysis to speed up RAG retrieval search
Depth
1 outlet · newest source 426h ago

Key points

  • Apple published an ICML 2026 paper on amortizing maximum inner product search.
  • The method trains small neural networks offline to route queries to best matches.
  • Standard centroid routing can cause errors because it is not an exact approach.

Apple Machine Learning Research published a paper at ICML 2026 titled Amortizing Maximum Inner Product Search with Learned Support Functions. The authors describe training small neural networks offline for a fixed database of keys.The technique uses small AI models to quickly find the most relevant document for a user's question.The authors leverage the convexity of the support function to train ICNNs that approximate optimal routing. These networks learn how to route queries toward their best match faster. The approach trades an upfront amortization cost for faster and more accurate retrieval at inference time.

Evidence: confirmedFaster Retrieval for Better, More Efficient RAG

aifleet-apple

How we got here

Maximum inner product search sits at the core of recommendation systems and retrieval-augmented generation. As the number of keys grows to millions and vector dimensions reach thousands, the computational cost of running an exhaustive comparison grows dramatically. Numerous approximate methods have been proposed to trade off accuracy for speed using clustering and quantization.

What is new

The new work leverages the mathematical structure of the support function of a set of keys. This function is convex and can be approximated by input convex neural networks. The gradient of that support function at any query returns the best-matching key itself. The researchers aim to mimic the ground-truth support function for a much lower computational cost.

Reading between the lines

Baseline methods often assume that the set of queries will resemble the keys. In RAG, queries arise from short user-generated questions while keys represent long curated text. This makes keys and queries substantially different. Most production MIPS use cases target existing users with a stable query distribution.

The open questions

The source material states that centroid routing is not an exact approach and can result in errors. The new method relies on the assumption that queries follow a stable, well-characterized distribution. It is unclear how the method performs when query distributions shift significantly or when keys are not fixed. The trade-off between upfront training cost and inference speed remains specific to the described setup.

Why it matters

  • Evaluators can test whether input convex neural networks reduce retrieval latency compared to standard clustering.
  • The distinction between short queries and long keys highlights a structural mismatch in existing RAG benchmarks.
  • Amortizing costs offline may offer a path to more efficient retrieval for stable user bases.
  • The convex analysis approach provides a new theoretical framework for understanding MIPS geometry.

Topics: Apple, RAG, ICML, vector search, machine learning

Updated

The strongest doubt

Counterpoint from the sources

The source notes that centroid routing is not exact and can result in errors, implying the new method must overcome these inherent geometric limitations.

Ask the article

Local keyword search of this article; no generated answers.

Did this article give you the evidence you needed?

No vote saved on this device.
Your choice stays on this device. No public tally.

Terms in this story

Maximum inner product search

A method to find the key vector with the largest inner product with a query.

Support function

A convex function mapping a query to its largest inner product with any key in a set.

Input convex neural networks

Neural networks specifically designed to approximate convex functions.

Questions readers ask

What conference featured this Apple research?

The paper was presented at ICML 2026.

How does the new method handle query routing?

It uses small neural networks trained offline to approximate the support function and route queries.

Why are standard MIPS methods problematic for RAG?

They assume queries resemble keys, but RAG queries are short while keys are long and curated.

Connected stories

This storyApple memory costs surge as AI demand tightens supply through 2027Apple memory costs surge as AI dApple engineering AI daily cap drops from $300 to $200Apple engineering AI daily cap dApple launches 200,000-person study to evaluate conversational AI health tools with OpenEvidenceApple launches 200,000-person stAMD earnings test AI revenue claims before Friday jobs dataAMD earnings test AI revenue cla

Reading receipt

  • Apple published an ICML 2026 paper on amortizing maximum inner product search.
  • The method trains small neural networks offline to route queries to best matches.
  • Standard centroid routing can cause errors because it is not an exact approach.

Still open: How does performance degrade when query distributions are not stable?

Still open: What is the exact magnitude of the upfront amortization cost for large databases?

Time spent: 0:00 · Depth: Full

Sources

  1. Apple Machine Learning Research

Related stories

Background reading