💡 8 sentences to nail Long Context — get the interview core points in one page (see §2-§9 for full derivations). LongRoPE (Ding 2024 ICML): evolutionary search for an independent scaling factor ...
最后有一个细节,就是MLA的最终版本,还将Q的输入也改为了低秩投影形式,这与减少KV Cache无关 ...