Improving Labeling Consistency with Detailed Constitutional Definitions and AI-Driven Evaluation 事件

Improving Labeling Consistency with Detailed Constitutional Definitions and AI-Driven Evaluation arXiv:2605.24247v1 Announce Type: new Abstract: Many automated labeling pipelines classify inputs into categories defined by a written specification, content moderation being a prominent use case. Simple category definitions are not detailed enough for labelers to produce the accurate, consistent golden labels these pipelines require. One solution is to write a prescriptive definition that settles e