Scaling Pre-training to One Hundred Billion Data for Vision Language Models 事件

Scaling Pre-training to One Hundred Billion Data for Vision Language Models arXiv:2502.07617v2 Announce Type: replace Abstract: We provide an empirical investigation of the potential of pre-training vision-language models on an unprecedented scale: 100 billion examples. We find that model performance tends to saturate at this scale on many common Western-centric classification and retrieval benchmarks, such as COCO Captions. Nevertheless, tasks of cultural diversity achieve more substantial gai