Key points
- Apple published an ICML 2026 paper on amortizing maximum inner product search.
- The method trains small neural networks offline to route queries to best matches.
- Standard centroid routing can cause errors because it is not an exact approach.
Apple Machine Learning Research published a paper at ICML 2026 titled Amortizing Maximum Inner Product Search with Learned Support Functions.confirmed The authors describe training small neural networks offline for a fixed database of keys.The technique uses small AI models to quickly find the most relevant document for a user's question.The authors leverage the convexity of the support function to train ICNNs that approximate optimal routing. These networks learn how to route queries toward their best match faster. The approach trades an upfront amortization cost for faster and more accurate retrieval at inference time.
How we got here
Maximum inner product search sits at the core of recommendation systems and retrieval-augmented generation. As the number of keys grows to millions and vector dimensions reach thousands, the computational cost of running an exhaustive comparison grows dramatically. Numerous approximate methods have been proposed to trade off accuracy for speed using clustering and quantization.
What is new
The new work leverages the mathematical structure of the support function of a set of keys. This function is convex and can be approximated by input convex neural networks. The gradient of that support function at any query returns the best-matching key itself. The researchers aim to mimic the ground-truth support function for a much lower computational cost.
Reading between the lines
Baseline methods often assume that the set of queries will resemble the keys. In RAG, queries arise from short user-generated questions while keys represent long curated text. This makes keys and queries substantially different. Most production MIPS use cases target existing users with a stable query distribution.
The open questions
The source material states that centroid routing is not an exact approach and can result in errors. The new method relies on the assumption that queries follow a stable, well-characterized distribution. It is unclear how the method performs when query distributions shift significantly or when keys are not fixed. The trade-off between upfront training cost and inference speed remains specific to the described setup.
Why it matters
- Evaluators can test whether input convex neural networks reduce retrieval latency compared to standard clustering.
- The distinction between short queries and long keys highlights a structural mismatch in existing RAG benchmarks.
- Amortizing costs offline may offer a path to more efficient retrieval for stable user bases.
- The convex analysis approach provides a new theoretical framework for understanding MIPS geometry.
Topics: Apple, RAG, ICML, vector search, machine learning
Updated
The strongest doubt
Counterpoint from the sources
The source notes that centroid routing is not exact and can result in errors, implying the new method must overcome these inherent geometric limitations.
Ask the article
Local keyword search of this article; no generated answers.Did this article give you the evidence you needed?
Your choice stays on this device. No public tally.Terms in this story
Maximum inner product search
A method to find the key vector with the largest inner product with a query.
Support function
A convex function mapping a query to its largest inner product with any key in a set.
Input convex neural networks
Neural networks specifically designed to approximate convex functions.
Questions readers ask
What conference featured this Apple research?
The paper was presented at ICML 2026.
How does the new method handle query routing?
It uses small neural networks trained offline to approximate the support function and route queries.
Why are standard MIPS methods problematic for RAG?
They assume queries resemble keys, but RAG queries are short while keys are long and curated.
Connected stories
Reading receipt
- Apple published an ICML 2026 paper on amortizing maximum inner product search.
- The method trains small neural networks offline to route queries to best matches.
- Standard centroid routing can cause errors because it is not an exact approach.
Still open: How does performance degrade when query distributions are not stable?
Still open: What is the exact magnitude of the upfront amortization cost for large databases?
