SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices 事件

SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices arXiv:2606.07098v1 Announce Type: new Abstract: We present SigmaScale, a method for learning auxiliary scaling matrices $S$ to aid truncated Singular Value Decomposition (SVD) based Large Language Model (LLM) compression. Instead of deriving scaling matrices analytically, SigmaScale optimizes two sets of vectors that define diagonal row and column scaling transformations under an activation-aware comp