CommonCrawl

CommonCrawl is a dataset that contains a vast archive of web pages, used to evaluate the performance of large language models on tasks related to HTML understanding, such as parsing and automating web-based tasks.

Papers using CommonCrawl (3)

AICC: Parse HTML Finer, Make Models Better -- A 7.3T AI-Ready Corpus Built by a Model-Based HTML Parser2025FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language2025Understanding HTML with Large Language Models2022 · 3 cites