Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models 事件

Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models arXiv:2603.29123v2 Announce Type: replace Abstract: The next-token prediction (NTP) objective trains language models to predict a single token at each step, even though many continuations can express the same meaning. For example, in the sentence ``this sticker can be placed here'', positioned, attached, or put are all plausible alternatives. While standard NTP training treats these alternatives as mutually ex