Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization 事件

Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization arXiv:2605.29327v1 Announce Type: new Abstract: Efficient Distillation (EDistill) compresses large language models (LLMs) by structured pruning parameters and tuning lightweight modules with high training efficiency. Although these EDistilled LLMs achieve state-of-the-art (SOTA) performance on general ability benchmarks relative to similarly sized LLMs, we identify a severe degradation in the