← authors · overview

• Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce For Pruning Llms To High Sparsity
  2023 · 176 citations
• Mix-ln: Unleashing The Power Of Deeper Layers By Combining Pre-ln And Post-ln
  2024 · 30 citations
• Ffn-skipllm: A Hidden Gem For Autoregressive Decoding With Adaptive Feed Forward Skipping
  2024 · 26 citations
• Junk DNA Hypothesis: Pruning Small Pre-trained Weights Irreversibly And Monotonically Impairs "difficult" Downstream Tasks In Llms
  2023 · 13 citations
• Outlier-weighed Layerwise Sampling For LLM Fine-tuning
  2024 · 11 citations