Key papers

• Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs (2026)
  Yidu Wu et al.
  5.01
• GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries (2026)
  Miguel P. Bento et al.
  4.39
• MoEITS: A Green AI approach for simplifying MoE-LLMs (2026)
  Luis Balderas et al.
  1.83
• Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs (2025)
  Kairun Zhang et al.
  1.50
• Thunder-LLM: Efficiently Adapting LLMs to Korean with Minimal Resources (2025)
  Jinpyo Kim et al.
  1.28
• CMoE: Fast Carving of Mixture-of-Experts for Efficient LLM Inference (2025)
  Zehua Pei et al.
  1.28
• Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts (2025)
  Weigao Sun et al.
  1.28
• SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity (2025)
  Samir Khaki et al.
  1.28
• Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs (2025)
  Liang Zeng et al.
  1.28
• Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs (2025)
  Yehui Tang et al.
  1.22
• xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference (2025)
  Maximilian Beck et al.
  1.11
• Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization (2025)
  Taishi Nakamura et al.
  1.06
• Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs (2024)
  Yeonhong Park et al.
  —
• JetMoE: Reaching Llama2 Performance with 0.1M Dollars (2024)
  Yikang Shen et al.
  —
• LLMem: Estimating GPU Memory Usage for Fine-Tuning Pre-Trained LLMs (2024)
  Taeho Kim et al.
  —
• LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (2024)
  Tong Zhu et al.
  —
• FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation (2024)
  Liqun Ma and Mingjie Sun and Zhiqiang Shen
  —
• LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale (2024)
  Jaehong Cho et al.
  —
• LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference (2024)
  Zhiwen Mo et al.
  —
• PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning (2024)
  Qibin Wang et al.
  —
• LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training (2024)
  Xiaoye Qu et al.
  —
• GMoE: Empowering LLMs Fine-Tuning via MoE Graph Collaboration (2024)
  Ting Bai et al.
  —
• Gradient Weight-normalized Low-rank Projection for Efficient LLM Training (2024)
  Jia-Hong Huang et al.
  —
• Approximating Two-Layer Feedforward Networks for Efficient Transformers (2023)
  Róbert Csordás et al.
  —
• H2O Open Ecosystem for State-of-the-art Large Language Models (2023)
  Arno Candel et al.
  —
• Efficient LLM inference solution on Intel GPU (2023)
  Hui Wu et al.
  —
• Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models (2024)
  Keisuke Kamahori et al.
  —
• Pre-training Small Base LMs with Fewer Tokens (2024)
  Sunny Sanyal et al.
  —
• Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients (2024)
  Aashiq Muhamed et al.
  —
• SlimLM: An Efficient Small Language Model for On-Device Document Assistance (2024)
  Thang M. Pham et al.
  —