03 — AGENT · 2026
Anvil — LLM Quantization Agent
1.8× faster
time-to-first-token · better perplexity
llama.cpp · Arm KleidiAI · Snapdragon 695
The problem
There are 2.7×10⁸ per-layer quantization configurations for Qwen2.5-1.5B. Naive search is expensive, and KleidiAI does not accelerate Q4_K_M — the most-shipped format.
The approach
A surrogate-guided planner over per-layer configs that matched a greedy search's 60-evaluation result in roughly 9 on-device evaluations.
The result
Anvil's config gave 1.8× faster time-to-first-token and better perplexity (9.522 vs 9.605) at +48% size — built for the Arm Create: AI Optimization Challenge 2026.
Code private / patent-pendingAll work