MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM 事件

MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM arXiv:2602.20191v2 Announce Type: replace-cross Abstract: Dynamic runtime latency and memory constraints necessitate flexible large language model (LLM) deployment, where an LLM can be inferred with various quantization precisions based on available computational resources. Recent work on such any-precision quantization either relies on hardware-inefficient vector quantization or induces additional scaling factors when