Key papers

• Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts (2026)
  Yingfa Chen et al.
  1.94
• HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers (2026)
  Guozhen Zhang et al.
  1.94
• Grouped Query Experts: Mixture-of-Experts on GQA Self-Attention (2026)
  Vishesh Tripathi et al.
  1.94
• STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation (2026)
  Ying Shen et al.
  1.89
• Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing (2026)
  Zeyue Tian et al.
  1.83
• Residual Stream Duality in Modern Transformer Architectures (2026)
  Yifan Zhang
  1.78
• Graph-Aware Isomorphic Attention for Adaptive Dynamics in Transformers (2025)
  Markus J. Buehler
  1.28
• ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer (2025)
  Lin Yueyu et al.
  1.28
• Multimodal Representation Alignment for Image Generation: Text-Image Interleaved Control Is Easier Than You Think (2025)
  Liang Chen et al.
  1.28
• UniHDSA: A Unified Relation Prediction Approach for Hierarchical Document Structure Analysis (2025)
  Jiawei Wang et al.
  1.28
• In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer (2025)
  Zechuan Zhang et al.
  1.28
• Native-Resolution Image Synthesis (2025)
  Zidong Wang et al.
  1.28
• Diagonal Batching Unlocks Parallelism in Recurrent Memory Transformers for Long Contexts (2025)
  Danil Sivtsov et al.
  1.28
• LaTtE-Flow: Layerwise Timestep-Expert Flow-based Transformer (2025)
  Ying Shen et al.
  1.28
• Cut2Next: Generating Next Shot via In-Context Tuning (2025)
  Jingwen He et al.
  1.28
• Gated Associative Memory: A Parallel O(N) Architecture for Efficient Sequence Modeling (2025)
  Rishiraj Acharya
  1.28
• HunyuanOCR Technical Report (2025)
  Hunyuan Vision Team et al.
  1.28
• Vcc: Scaling Transformers to 128K Tokens or More by Prioritizing Important Tokens (2023)
  Zhanpeng Zeng et al.
  —
• Supervised Pretraining Can Learn In-Context Reinforcement Learning (2023)
  Jonathan N. Lee et al.
  —
• From Sparse to Soft Mixtures of Experts (2023)
  Joan Puigcerver et al.
  —
• RMT: Retentive Networks Meet Vision Transformers (2023)
  Qihang Fan et al.
  —
• PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis (2023)
  Junsong Chen et al.
  —
• UniAudio: An Audio Foundation Model Toward Universal Audio Generation (2023)
  Dongchao Yang et al.
  —
• BitNet: Scaling 1-bit Transformers for Large Language Models (2023)
  Hongyu Wang et al.
  —
• Hiformer: Heterogeneous Feature Interactions Learning with Transformers for Recommender Systems (2023)
  Huan Gui et al.
  —
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces (2023)
  Albert Gu et al.
  —
• VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generation (2023)
  Jinguo Zhu et al.
  —
• VideoPoet: A Large Language Model for Zero-Shot Video Generation (2023)
  Dan Kondratyuk et al.
  —
• BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data (2024)
  Mateusz Łajszczak et al.
  —
• On the Scalability of Diffusion-based Text-to-Image Generation (2024)
  Hao Li et al.
  —
• InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD (2024)
  Xiaoyi Dong et al.
  —
• Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory (2024)
  Xueyan Niu et al.
  —
• OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation (2024)
  Kepan Nan et al.
  —
• The Llama 3 Herd of Models (2024)
  Abhimanyu Dubey et al.
  —
• xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations (2024)
  Can Qin et al.
  —
• Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planning (2024)
  Gang Liu et al.
  —
• Differential Transformer (2024)
  Tianzhu Ye et al.
  —
• Mogo: RQ Hierarchical Causal Transformer for High-Quality 3D Human Motion Generation (2024)
  Dongjie Fu
  —
• GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training (2024)
  Renqiu Xia et al.
  —
• DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation (2024)
  Minghong Cai et al.
  —