OpenWebText

Dataset Card for "openwebtext" Dataset Summary An open-source replication of the WebText dataset from OpenAI, that was used to train GPT-2. This distribution was created by Aaron Gokaslan and Vanya Cohen of Brown University. Supported Tasks and Leaderboards More Information Needed Languages More Information Needed Data

Papers using OpenWebText (4)

Simplified and Generalized Masked Diffusion for Discrete Data2024 · 3 citesCoDAR: Continuous Diffusion Language Models are More Powerful Than You Think2026Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking2025Think While You Generate: Discrete Diffusion with Planned Denoising2024 · 1 cites