How Does Sequence Modeling Architecture Influence Base Capabilities of Pre-trained Language Models? Exploring Key Architecture Design Principles to Avoid Base Capabilities Degradation - Lu, Xin; Zhao, Yanyan; Wei, Si; Wang, Shijin; Qin, Bing; Liu, Ting (Advances in Neural Information Processing Systems, 2025)