SSSD: Simply-Scalable Speculative Decoding 事件

SSSD: Simply-Scalable Speculative Decoding arXiv:2411.05894v3 Announce Type: replace Abstract: Speculative Decoding has emerged as a popular technique for accelerating inference in Large Language Models. However, most existing approaches yield only modest improvements in production serving systems. Methods that achieve substantial speedups typically rely on an additional trained draft model or auxiliary model components, increasing deployment and maintenance complexity. This added complexity re