Model Cascading For Code: A Cascaded Black-box Multi-model Framework For Cost-efficient Code Completion With Self-testing

Abstract

The rapid advancement of large language models (LLMs) has significantly improved code completion tasks, yet the trade-off between accuracy and computational cost remains a critical challenge. While using larger models and incorporating inference-time self-testing algorithms can significantly improve output accuracy, they incur substantial computational expenses at the same time. Furthermore, serve