Qrita: High-performance Top-k and Top-p using Pivot-based Truncation and Selection 事件

Qrita: High-performance Top-k and Top-p using Pivot-based Truncation and Selection arXiv:2602.01518v2 Announce Type: replace Abstract: Despite their importance in model sampling, efficient implementation of Top-k and Top-p algorithms for large vocabularies remains a significant challenge. Existing approaches often rely on sorting, which incurs significant computation and memory overhead on GPUs, or on stochastic approaches that alter the algorithm's output. In this work, we propose Qrita, an ef