Key papers

• Training Language Models To Follow Instructions With Human Feedback (2022)
  Long Ouyang, Jeff Wu, Xu Jiang, et al.
  36.92
• Llama: Open And Efficient Foundation Language Models (2023)
  Hugo Touvron, Thibaut Lavril, Gautier Izacard, et al.
  36.83
• React: Synergizing Reasoning And Acting In Language Models (2022)
  Shunyu Yao, Jeffrey Zhao, Dian Yu, et al.
  36.63
• SPHINX: The Joint Mixing Of Weights, Tasks, And Visual Embeddings For Multi-modal Large Language Models (2023)
  Ziyi Lin, Chris Liu, Renrui Zhang, et al.
  31.35
• The Dawn Of Lmms: Preliminary Explorations With Gpt-4v(ision) (2023)
  Zhengyuan Yang, Linjie Li, Kevin Lin, et al.
  30.34
• Eagle: Exploring The Design Space For Multimodal Llms With Mixture Of Encoders (2024)
  Min Shi, Fuxiao Liu, Shihao Wang, et al.
  25.55
• Layoutllm: Layout Instruction Tuning With Large Language Models For Document Understanding (2024)
  Chuwei Luo, Yufan Shen, Zhaoqing Zhu, et al.
  24.66
• MA-LMM: Memory-augmented Large Multimodal Model For Long-term Video Understanding (2024)
  Bo He, Hengduo Li, Young Kyun Jang, et al.
  24.50
• Imagebind-llm: Multi-modality Instruction Tuning (2023)
  Jiaming Han, Renrui Zhang, Wenqi Shao, et al.
  24.37
• A Systematic Survey Of Prompt Engineering In Large Language Models: Techniques And Applications (2024)
  Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, et al.
  24.27
• Lm-cocktail: Resilient Tuning Of Language Models Via Model Merging (2023)
  Shitao Xiao, Zheng Liu, Peitian Zhang, et al.
  23.85
• Cobra: Extending Mamba To Multi-modal Large Language Model For Efficient Inference (2024)
  Han Zhao, Min Zhang, Wei Zhao, et al.
  23.43
• You Only Look At Screens: Multimodal Chain-of-action Agents (2023)
  Zhuosheng Zhang, Aston Zhang
  23.42
• MMC: Advancing Multimodal Chart Understanding With Large-scale Instruction Tuning (2023)
  Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, et al.
  23.28
• SPHINX-X: Scaling Data And Parameters For A Family Of Multi-modal Large Language Models (2024)
  Dongyang Liu, Renrui Zhang, Longtian Qiu, et al.
  23.22
• Llava-phi: Efficient Multi-modal Assistant With Small Language Model (2024)
  Yichen Zhu, Minjie Zhu, Ning Liu, et al.
  22.96
• Visrag: Vision-based Retrieval-augmented Generation On Multi-modality Documents (2024)
  Shi Yu, Chaoyue Tang, Bokai Xu, et al.
  22.87
• Quantifying Language Models' Sensitivity To Spurious Features In Prompt Design Or: How I Learned To Start Worrying About Prompt Formatting (2023)
  Melanie Sclar, Yejin Choi, Yulia Tsvetkov, et al.
  22.72
• Ovis: Structural Embedding Alignment For Multimodal Large Language Model (2024)
  Shiyin Lu, Yang Li, Qing-Guo Chen, et al.
  22.55
• Fine-tuning Multimodal Llms To Follow Zero-shot Demonstrative Instructions (2023)
  Juncheng Li, Kaihang Pan, Zhiqi Ge, et al.
  22.33
• AMBER: An Llm-free Multi-dimensional Benchmark For Mllms Hallucination Evaluation (2023)
  Junyang Wang, Yuhang Wang, Guohai Xu, et al.
  22.17
• List Items One By One: A New Data Source And Learning Paradigm For Multimodal Llms (2024)
  An Yan, Zhengyuan Yang, Junda Wu, et al.
  21.63
• Eyes Wide Shut? Exploring The Visual Shortcomings Of Multimodal Llms (2024)
  Shengbang Tong, Zhuang Liu, Yuexiang Zhai, et al.
  21.34
• PPTC Benchmark: Evaluating Large Language Models For Powerpoint Task Completion (2023)
  Yiduo Guo, Zekai Zhang, Yaobo Liang, et al.
  21.19
• Rethinking Machine Unlearning For Large Language Models (2024)
  Sijia Liu, Yuanshun Yao, Jinghan Jia, et al.
  21.08
• Groundinggpt:language Enhanced Multi-modal Grounding Model (2024)
  Zhaowei Li, Qi Xu, Dong Zhang, et al.
  20.91
• Language Models Are Homer Simpson! Safety Re-alignment Of Fine-tuned Language Models Through Task Arithmetic (2024)
  Rishabh Bhardwaj, Do Duc Anh, Soujanya Poria
  20.43
• Frozen Transformers In Language Models Are Effective Visual Encoder Layers (2023)
  Ziqi Pang, Ziyang Xie, Yunze Man, et al.
  19.32
• From Persona To Personalization: A Survey On Role-playing Language Agents (2024)
  Jiangjie Chen, Xintao Wang, Rui Xu, et al.
  19.12
• INCLUDE: Evaluating Multilingual Language Understanding With Regional Knowledge (2024)
  Angelika Romanou, Negar Foroutan, Anna Sotnikova, et al.
  19.08
• Making Llama SEE And Draw With SEED Tokenizer (2023)
  Yuying Ge, Sijie Zhao, Ziyun Zeng, et al.
  18.81
• Codi-2: In-context, Interleaved, And Interactive Any-to-any Generation (2023)
  Zineng Tang, Ziyi Yang, Mahmoud Khademi, et al.
  18.73
• Longagent: Scaling Language Models To 128k Context Through Multi-agent Collaboration (2024)
  Jun Zhao, Can Zu, Hao Xu, et al.
  18.61
• From GPT-4 To Gemini And Beyond: Assessing The Landscape Of Mllms On Generalizability, Trustworthiness And Causality Through Four Modalities (2024)
  Chaochao Lu, Chen Qian, Guodong Zheng, et al.
  18.43
• Navgpt-2: Unleashing Navigational Reasoning Capability For Large Vision-language Models (2024)
  Gengze Zhou, Yicong Hong, Zun Wang, et al.
  18.43
• How Easy Is It To Fool Your Multimodal Llms? An Empirical Analysis On Deceptive Prompts (2024)
  Yusu Qian, Haotian Zhang, Yinfei Yang, et al.
  18.29
• Worldgpt: Empowering LLM As Multimodal World Model (2024)
  Zhiqi Ge, Hongzhe Huang, Mingze Zhou, et al.
  18.08
• Draw-and-understand: Leveraging Visual Prompts To Enable Mllms To Comprehend What You Want (2024)
  Weifeng Lin, Xinyu Wei, Ruichuan An, et al.
  18.04
• Unleashing The Potential Of Prompt Engineering For Large Language Models (2023)
  Banghao Chen, Zhaofeng Zhang, Nicolas Langrené, et al.
  17.87
• Pink: Unveiling The Power Of Referential Comprehension For Multi-modal Llms (2023)
  Shiyu Xuan, Qingpei Guo, Ming Yang, et al.
  17.64
• Mmevalpro: Calibrating Multimodal Benchmarks Towards Trustworthy And Efficient Evaluation (2024)
  Jinsheng Huang, Liang Chen, Taian Guo, et al.
  17.18
• Mm-embed: Universal Multimodal Retrieval With Multimodal Llms (2024)
  Sheng-Chieh Lin, Chankyu Lee, Mohammad Shoeybi, et al.
  17.06
• Luminate: Structured Generation And Exploration Of Design Space With Large Language Models For Human-ai Co-creation (2023)
  Sangho Suh, Meng Chen, Bryan Min, et al.
  16.90
• Llms Meet Multimodal Generation And Editing: A Survey (2024)
  Yingqing He, Zhaoyang Liu, Jingye Chen, et al.
  16.87
• Llms Are Few-shot In-context Low-resource Language Learners (2024)
  Samuel Cahyawijaya, Holy Lovenia, Pascale Fung
  16.84
• Gemini In Reasoning: Unveiling Commonsense In Multimodal Large Language Models (2023)
  Yuqing Wang, Yun Zhao
  16.81
• Prompt Highlighter: Interactive Control For Multi-modal Llms (2023)
  Yuechen Zhang, Shengju Qian, Bohao Peng, et al.
  16.80
• Symdpo: Boosting In-context Learning Of Large Multimodal Models With Symbol Demonstration Direct Preference Optimization (2024)
  Hongrui Jia, Chaoya Jiang, Haiyang Xu, et al.
  16.66
• Archer: Training Language Model Agents Via Hierarchical Multi-turn RL (2024)
  Yifei Zhou, Andrea Zanette, Jiayi Pan, et al.
  16.61
• Chat Vector: A Simple Approach To Equip Llms With Instruction Following And Model Alignment In New Languages (2023)
  Shih-Cheng Huang, Pin-Zu Li, Yu-Chi Hsu, et al.
  16.55
• From CLIP To DINO: Visual Encoders Shout In Multi-modal Large Language Models (2023)
  Dongsheng Jiang, Yuchen Liu, Songlin Liu, et al.
  16.49
• Kam-cot: Knowledge Augmented Multimodal Chain-of-thoughts Reasoning (2024)
  Debjyoti Mondal, Suraj Modi, Subhadarshi Panda, et al.
  16.39
• Red Teaming Visual Language Models (2024)
  Mukai Li, Lei Li, Yuwei Yin, et al.
  16.37
• Chartllama: A Multimodal LLM For Chart Understanding And Generation (2023)
  Yucheng Han, Chi Zhang, Xin Chen, et al.
  16.36
• Language Agents With Reinforcement Learning For Strategic Play In The Werewolf Game (2023)
  Zelai Xu, Chao Yu, Fei Fang, et al.
  16.23
• Investigating The Catastrophic Forgetting In Multimodal Large Language Models (2023)
  Yuexiang Zhai, Shengbang Tong, Xiao Li, et al.
  16.05
• Prompt A Robot To Walk With Large Language Models (2023)
  Yen-Jen Wang, Bike Zhang, Jianyu Chen, et al.
  15.96
• HMT: Hierarchical Memory Transformer For Efficient Long Context Language Processing (2024)
  Zifan He, Yingqi Cao, Zongyue Qin, et al.
  15.94
• Eyes Closed, Safety On: Protecting Multimodal Llms Via Image-to-text Transformation (2024)
  Yunhao Gou, Kai Chen, Zhili Liu, et al.
  15.78
• When Large Language Model Agents Meet 6G Networks: Perception, Grounding, And Alignment (2024)
  Minrui Xu, Dusit Niyato, Jiawen Kang, et al.
  15.57