OnlineSPEC
A research framework that treats speculative decoding as an online learning problem, continuously updating the draft model from feedback produced during verification to keep speedups high as a workload shifts.
OnlineSPEC formulates speculative decoding as online learning and shows a theoretical link between the algorithm's regret bound and the resulting speedup, then implements the idea as three methods: Ens-EAGLE and Ens-EAGLE-3, which combine several draft models with different learning rates in an online ensemble; Opt-Hydra, which adds optimism-based momentum updates to the Hydra draft head; and Online-LR, which applies online direct-preference optimization to lookahead reasoning drafting.
The repository is organized as a multi-project workspace with shared modules, one directory per method plus a common package, so each approach can be run and reproduced independently.
It is written in Python and accompanies an ICML 2026 paper.