Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch 事件

Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch arXiv:2511.17826v2 Announce Type: replace-cross Abstract: Deterministic inference is increasingly critical for large language model (LLM) applications such as LLM-as-a-judge evaluation, multi-agent systems, and Reinforcement Learning (RL). However, existing LLM serving frameworks exhibit non-deterministic behavior: identical inputs can yield different outputs when system configurations (e.g., tensor