Key papers

• Lora: Low-rank Adaptation Of Large Language Models (2021)
  Edward J. Hu, Yelong Shen, Phillip Wallis, et al.
  48.60
• Zephyr: Direct Distillation Of LM Alignment (2023)
  Lewis Tunstall, Edward Beeching, Nathan Lambert, et al.
  38.56
• Efficient Streaming Language Models With Attention Sinks (2023)
  Guangxuan Xiao, Yuandong Tian, Beidi Chen, et al.
  37.76
• Training Language Models To Follow Instructions With Human Feedback (2022)
  Long Ouyang, Jeff Wu, Xu Jiang, et al.
  36.92
• Llama: Open And Efficient Foundation Language Models (2023)
  Hugo Touvron, Thibaut Lavril, Gautier Izacard, et al.
  36.83
• Qlora: Efficient Finetuning Of Quantized Llms (2023)
  Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, et al.
  36.23
• Minicpm: Unveiling The Potential Of Small Language Models With Scalable Training Strategies (2024)
  Shengding Hu, Yuge Tu, Xu Han, et al.
  36.01
• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022)
  BigScience Workshop: Teven Le Scao, Angela Fan, Christopher Akiki, et al.
  33.79
• Layerskip: Enabling Early Exit Inference And Self-speculative Decoding (2024)
  Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, et al.
  32.46
• Prefix-tuning: Optimizing Continuous Prompts For Generation (2021)
  Xiang Lisa Li, Percy Liang
  32.38
• Omniquant: Omnidirectionally Calibrated Quantization For Large Language Models (2023)
  Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, et al.
  31.92
• The Fineweb Datasets: Decanting The Web For The Finest Text Data At Scale (2024)
  Guilherme Penedo, Hynek Kydlíček, Loubna Ben Allal, et al.
  31.68
• Training Compute-optimal Large Language Models (2022)
  Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, et al.
  31.52
• SPHINX: The Joint Mixing Of Weights, Tasks, And Visual Embeddings For Multi-modal Large Language Models (2023)
  Ziyi Lin, Chris Liu, Renrui Zhang, et al.
  31.35
• Step-dpo: Step-wise Preference Optimization For Long-chain Reasoning Of Llms (2024)
  Xin Lai, Zhuotao Tian, Yukang Chen, et al.
  31.31
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (2025)
  DeepSeek-AI et al.
  31.18
• Deepseek LLM: Scaling Open-source Language Models With Longtermism (2024)
  Deepseek-Ai, :, Xiao Bi, et al.
  30.12
• Mobilellm: Optimizing Sub-billion Parameter Language Models For On-device Use Cases (2024)
  Zechun Liu, Changsheng Zhao, Forrest Iandola, et al.
  28.23
• Magpie: Alignment Data Synthesis From Scratch By Prompting Aligned Llms With Nothing (2024)
  Zhangchen Xu, Fengqing Jiang, Luyao Niu, et al.
  28.04
• Ultrafeedback: Boosting Language Models With Scaled AI Feedback (2023)
  Ganqu Cui, Lifan Yuan, Ning Ding, et al.
  27.73
• Is DPO Superior To PPO For LLM Alignment? A Comprehensive Study (2024)
  Shusheng Xu, Wei Fu, Jiaxuan Gao, et al.
  27.53
• Magicoder: Empowering Code Generation With Oss-instruct (2023)
  Yuxiang Wei, Zhe Wang, Jiawei Liu, et al.
  27.43
• The Reversal Curse: Llms Trained On "A Is B" Fail To Learn "B Is A" (2023)
  Lukas Berglund, Meg Tong, Max Kaufmann, et al.
  27.32
• Efficient Large Language Models: A Survey (2023)
  Zhongwei Wan, Xin Wang, Che Liu, et al.
  27.30
• Agentgym: Evolving Large Language Model-based Agents Across Diverse Environments (2024)
  Zhiheng Xi, Yiwen Ding, Wenxiang Chen, et al.
  27.20
• Internlm2 Technical Report (2024)
  Zheng Cai, Maosong Cao, Haojiong Chen, et al.
  27.03
• Longalign: A Recipe For Long Context Alignment Of Large Language Models (2024)
  Yushi Bai, Xin Lv, Jiajie Zhang, et al.
  26.93
• Xlstm: Extended Long Short-term Memory (2024)
  Maximilian Beck, Korbinian Pöppel, Markus Spanring, et al.
  26.88
• Culturax: A Cleaned, Enormous, And Multilingual Dataset For Large Language Models In 167 Languages (2023)
  Thuat Nguyen, Chien van Nguyen, Viet Dac Lai, et al.
  26.68
• SOLAR 10.7B: Scaling Large Language Models With Simple Yet Effective Depth Up-scaling (2023)
  Dahyun Kim, Chanjun Park, Sanghoon Kim, et al.
  26.34
• Learning From Mistakes Makes LLM Better Reasoner (2023)
  Shengnan An, Zexiong Ma, Zeqi Lin, et al.
  26.19
• Saullm-7b: A Pioneering Large Language Model For Law (2024)
  Pierre Colombo, Telmo Pessoa Pires, Malik Boudiaf, et al.
  26.00
• Mammoth: Building Math Generalist Models Through Hybrid Instruction Tuning (2023)
  Xiang Yue, Xingwei Qu, Ge Zhang, et al.
  25.94
• SVD-LLM: Truncation-aware Singular Value Decomposition For Large Language Model Compression (2024)
  Xin Wang, Yu Zheng, Zhongwei Wan, et al.
  25.83
• Direct Nash Optimization: Teaching Language Models To Self-improve With General Preferences (2024)
  Corby Rosset, Ching-An Cheng, Arindam Mitra, et al.
  25.73
• Evoprompt: Connecting Llms With Evolutionary Algorithms Yields Powerful Prompt Optimizers (2023)
  Qingyan Guo, Rui Wang, Junliang Guo, et al.
  25.70
• Eagle: Exploring The Design Space For Multimodal Llms With Mixture Of Encoders (2024)
  Min Shi, Fuxiao Liu, Shihao Wang, et al.
  25.55
• When Scaling Meets LLM Finetuning: The Effect Of Data, Model And Finetuning Method (2024)
  Biao Zhang, Zhongtao Liu, Colin Cherry, et al.
  25.34
• Teaching Large Language Models To Reason With Reinforcement Learning (2024)
  Alex Havrilla, Yuqing Du, Sharath Chandra Raparthy, et al.
  25.08
• Toward Self-improvement Of Llms Via Imagination, Searching, And Criticizing (2024)
  Ye Tian, Baolin Peng, Linfeng Song, et al.
  24.80
• Continual Learning Of Large Language Models: A Comprehensive Survey (2024)
  Haizhou Shi, Zihao Xu, Hengyi Wang, et al.
  24.79
• Layoutllm: Layout Instruction Tuning With Large Language Models For Document Understanding (2024)
  Chuwei Luo, Yufan Shen, Zhaoqing Zhu, et al.
  24.66
• Self-play With Execution Feedback: Improving Instruction-following Capabilities Of Large Language Models (2024)
  Guanting Dong, Keming Lu, Chengpeng Li, et al.
  24.59
• Imagebind-llm: Multi-modality Instruction Tuning (2023)
  Jiaming Han, Renrui Zhang, Wenqi Shao, et al.
  24.37
• Federatedscope-llm: A Comprehensive Package For Fine-tuning Large Language Models In Federated Learning (2023)
  Weirui Kuang, Bingchen Qian, Zitao Li, et al.
  24.26
• Lm-cocktail: Resilient Tuning Of Language Models Via Model Merging (2023)
  Shitao Xiao, Zheng Liu, Peitian Zhang, et al.
  23.85
• Chatglm-math: Improving Math Problem-solving In Large Language Models With A Self-critique Pipeline (2024)
  Yifan Xu, Xiao Liu, Xinghan Liu, et al.
  23.84
• Llama-moe: Building Mixture-of-experts From Llama With Continual Pre-training (2024)
  Tong Zhu, Xiaoye Qu, Daize Dong, et al.
  23.81
• Disc-lawllm: Fine-tuning Large Language Models For Intelligent Legal Services (2023)
  Shengbin Yue, Wei Chen, Siyuan Wang, et al.
  23.76
• Discovering The Gems In Early Layers: Accelerating Long-context Llms With 1000x Input Token Reduction (2024)
  Zhenmei Shi, Yifei Ming, Xuan-Phi Nguyen, et al.
  23.61
• Synthetic Data (almost) From Scratch: Generalized Instruction Tuning For Language Models (2024)
  Haoran Li, Qingxiu Dong, Zhengyang Tang, et al.
  23.56
• Faithful Logical Reasoning Via Symbolic Chain-of-thought (2024)
  Jundong Xu, Hao Fei, Liangming Pan, et al.
  23.51
• Instruction Tuning For Large Language Models: A Survey (2023)
  Shengyu Zhang, Linfeng Dong, Xiaoya Li, et al.
  23.46
• MMC: Advancing Multimodal Chart Understanding With Large-scale Instruction Tuning (2023)
  Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, et al.
  23.28
• SPHINX-X: Scaling Data And Parameters For A Family Of Multi-modal Large Language Models (2024)
  Dongyang Liu, Renrui Zhang, Longtian Qiu, et al.
  23.22
• A Paradigm Shift In Machine Translation: Boosting Translation Performance Of Large Language Models (2023)
  Haoran Xu, Young Jin Kim, Amr Sharaf, et al.
  23.20
• A Rank Stabilization Scaling Factor For Fine-tuning With Lora (2023)
  Damjan Kalajdzievski
  23.01
• LESS: Selecting Influential Data For Targeted Instruction Tuning (2024)
  Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, et al.
  22.96
• Zhongjing: Enhancing The Chinese Medical Capabilities Of Large Language Model Through Expert Feedback And Real-world Multi-turn Dialogue (2023)
  Songhua Yang, Hanjie Zhao, Senbin Zhu, et al.
  22.87
• A Survey On Self-evolution Of Large Language Models (2024)
  Zhengwei Tao, Ting-En Lin, Xiancai Chen, et al.
  22.85