Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillatio 事件

Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillatio arXiv:2606.05682v1 Announce Type: new Abstract: Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments. Quantization-aware distillation (QAD) helps recover accuracy lost under low bit quantization by training a quantized student to match the output distribution of a frozen higher precisio