Key papers

• Making Open-Source Text LLM Watermarks Durable Against Merging (2026)
  Luisa Scharff et al.
  5.01
• GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries (2026)
  Miguel P. Bento et al.
  4.39
• CausalMix: Data Mixture as Causal Inference for Language Model Training (2026)
  Zinan Tang et al.
  2.00
• HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better (2026)
  Gengluo Li et al.
  2.00
• Enhancing In-context Panoramic Generation via Geometric-aware Pretraining (2026)
  Haoran Feng et al.
  2.00
• Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs (2025)
  Kairun Zhang et al.
  1.50
• RePro: Training Language Models to Faithfully Recycle the Web for Pretraining (2025)
  Zichun Yu et al.
  1.50
• Benchmarking Optimizers for Large Language Model Pretraining (2025)
  Andrei Semenov et al.
  1.44
• CMoE: Fast Carving of Mixture-of-Experts for Efficient LLM Inference (2025)
  Zehua Pei et al.
  1.28
• FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language (2025)
  Guilherme Penedo et al.
  1.28
• Technical Report: Full-Stack Fine-Tuning for the Q Programming Language (2025)
  Brendan R. Hogan et al.
  1.28
• Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs (2025)
  Yehui Tang et al.
  1.22
• Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization (2025)
  Taishi Nakamura et al.
  1.06
• FinerWeb-10BT: Refining Web Data with LLM-Based Line-Level Filtering (2025)
  Erik Henriksson et al.
  1.00
• Train No Evil: Selective Masking for Task-Guided Pre-Training (2020)
  Yuxian Gu et al.
  —
• CPM: A Large-scale Generative Chinese Pre-trained Language Model (2020)
  Zhengyan Zhang et al.
  —
• ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021)
  Shuohuan Wang et al.
  —
• Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning (2022)
  Haokun Liu et al.
  —
• OpenBA: An Open-sourced 15B Bilingual Asymmetric seq2seq Model Pre-trained from Scratch (2023)
  Juntao Li et al.
  —
• Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs (2024)
  Yeonhong Park et al.
  —
• LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (2024)
  Tong Zhu et al.
  —
• The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale (2024)
  Guilherme Penedo et al.
  —
• FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation (2024)
  Liqun Ma and Mingjie Sun and Zhiqiang Shen
  —
• LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference (2024)
  Zhiwen Mo et al.
  —
• PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning (2024)
  Qibin Wang et al.
  —
• Gradient Weight-normalized Low-rank Projection for Efficient LLM Training (2024)
  Jia-Hong Huang et al.
  —
• Approximating Two-Layer Feedforward Networks for Efficient Transformers (2023)
  Róbert Csordás et al.
  —
• Ziya2: Data-centric Learning is All LLMs Need (2023)
  Ruyi Gan et al.
  —
• Pre-training Small Base LMs with Fewer Tokens (2024)
  Sunny Sanyal et al.
  —
• OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework (2024)
  Sachin Mehta et al.
  —
• Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients (2024)
  Aashiq Muhamed et al.
  —
• SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding (2024)
  Hao Li et al.
  —