Statistically Reliable LLM-Based Ranking Evaluation via Prediction-Powered Inference 事件

Statistically Reliable LLM-Based Ranking Evaluation via Prediction-Powered Inference arXiv:2606.05308v1 Announce Type: cross Abstract: With PRECISE, we extended Prediction-Powered Inference to produce bias-corrected estimates of ranking evaluation metrics by combining a small human-labeled set with a large LLM-judged set. PPI is provably unbiased regardless of the LLM judge's error profile. We make it applicable to hierarchical metrics like Precision@K, where annotations are per-document but th