Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers 文章

ArXiv CS.AI2026-05-27NEWSen作者: Tim Tsz-Kit Lau, Weijie Su
查看原文 →
关系图谱

Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers · 相关技术

相关技术

NATMulti-layer perceptronsODEPLAneural network递归自我改进UnifiedUCTSwiGLUStraight-Through EstimatorSpatial Pivot-Aligned Coordinate-free Embedding (SPACE)SPANarrative Abstraction BenchmarkMoEMLPISPHISGroupGated Linear UnitsGEOForFOLEvaluation Driven DevelopmentAdamANNICE