C4

The C4 dataset is a large-scale benchmark used to evaluate federated learning algorithms, containing diverse text data for training language models.

Papers using C4 (3)

DiLoCo: Distributed Low-Communication Training of Language Models2023 · 7 citesLeveraging Function Space Aggregation for Federated Learning at Scale2023Asynchronous Local-SGD Training for Language Modeling2024