Balancing Knowledge Distillation for Imbalance Learning with Bilevel Optimization 事件

Balancing Knowledge Distillation for Imbalance Learning with Bilevel Optimization arXiv:2605.17839v3 Announce Type: replace-cross Abstract: Knowledge distillation transfers knowledge from a high capacity teacher to a compact student using a mixture of hard and soft losses. On imbalanced data, a fixed weighting between hard and soft losses becomes brittle the learning process. Recent studies try to reweight these components in long-tailed settings. However, most of these methods do not adapt wei