Data and Evaluation Closed-Loop for Model Capability Enhancement 文章

ArXiv CS.AI2026-06-30PAPERen作者: Zhixuan Li, Jiangan Yuan, Han Xu

详细信息

来源站点
ArXiv CS.AI
作者
Zhixuan Li, Jiangan Yuan, Han Xu
文章类型
PAPER
语言
en
发布日期
2026-06-30

摘要

arXiv:2606.28471v1 Announce Type: new Abstract: Model capability is the central variable in LLM pre-training, yet is never observed directly: data shapes it prospectively, while evaluation reveals it only retrospectively, compressing samples, prompts, decoding, and scoring rules into one noisy score. Practical optimization runs this backward: a failure is observed first, and the engineer must infer the corpus fix. The two sides speak incompatible vocabularies -- benchmark names and per-sample correctness versus data sources, domains, and quality labels -- so this inference is usually intuition, not method. We close this gap with the \emph{capability slice}: a group of evaluation samples sharing background condition, task type, solving operation, and output constraint -- precise enough to localize a single weakness yet stable enough to survive aggregation, unlike a benchmark name, too coarse, or a single sample, too noisy.