← all papers · overview

Can Large Language Models Invent Algorithms To Improve Themselves?: Algorithm Discovery For Recursive Self-improvement Through Reinforcement Learning

·2024

Abstract

Large Language Models (LLMs) have achieved remarkable capabilities, yet their improvement methods remain fundamentally constrained by human design. We present Self-Developing, a framework that enables LLMs to autonomously discover, implement, and refine their own improvement algorithms. Our approach employs an iterative cycle where a seed model generates algorithmic candidates as executable code, evaluates their effectiveness, and uses Direct Preference Optimization to recursively improve increasingly sophisticated improvement strategies. We demonstrate this framework through model merging, a practical technique for combining specialized models. Self-Developing successfully discovered novel merging algorithms that outperform existing human-designed algorithms. On mathematical reasoning benchmarks, the autonomously discovered algorithms improve the seed model's GSM8k performance by 6% and exceed human-designed approaches like Task Arithmetic by 4.3%. Remarkably, these algorithms exhibit

Related papers

Ranked by semantic similarity — how closely each paper's abstract matches this one (100% = near-identical topic).