NobleBlocks

Research indexPreprint

Speculative Decoding for Multi-Sample Inference

Li, Shi, Feng and 8 others

arXiv (Cornell University)7 Mar 2025Open access

DOI10.48550/arxiv.2503.05330

Free to read at the publisher

You can read the abstract here, or the full text at the publisher.

Read at the publisher

Abstract

We propose a novel speculative decoding method tailored for multi-sample reasoning scenarios, such as self-consistency and Best-of-N sampling. Our method exploits the intrinsic consensus of parallel generation paths to synthesize high-quality draft tokens without requiring auxiliary models or external databases. By dynamically analyzing structural patterns across parallel reasoning paths through a probabilistic aggregation mechanism, it identifies consensus token sequences that align with the decoding distribution. Evaluations on mathematical reasoning benchmarks demonstrate a substantial improvement in draft acceptance rates over baselines, while reducing the latency in draft token construction. This work establishes a paradigm shift for efficient multi-sample inference, enabling seamless integration of speculative decoding with sampling-based reasoning techniques.

Discussion

Topics

More

Suggest a correction or report