LLM as a Meta-Judge: Synthetic Data for NLP Evaluation Metric Validation 事件

LLM as a Meta-Judge: Synthetic Data for NLP Evaluation Metric Validation arXiv:2603.09403v2 Announce Type: replace Abstract: Validating evaluation metrics for NLG typically relies on expensive and time-consuming human annotations, which predominantly exist only for English datasets. We propose LLM as a Meta-Judge, a scalable framework that utilizes LLMs to generate synthetic evaluation datasets via controlled semantic degradation of real data, replacing human judgment. We validate our approach