From Dev.to
Zero Forgetting in LLMs? We Benchmarked Null-Space Projection on SmolLM and Hit the Plasticity Wall
Why orthogonal projections eliminate catastrophic forgetting in autoregressive Transformers—and the mathematical reason behind the 62% downstream plasticity penalty. Fine-tuning Large Language Models...