Abstract
Robotic tabletop cleaning is applicable in various environments ranging from domestic to industrial settings, yet it still faces challenges in the cluttered scenario where wastes of diverse types exist. Moreover, to deal with different waste, such as liquid spills or fine crumbs, different cleaning tools might be required, which further complicates the task. Inspired by significant advancements in vision language models (VLMs), in this paper, we propose RoboCleaner, a VLM-powered multi-agent framework for tabletop cleaning that leverages the collaborative intelligence of multiple agents. Specifically, the proposed framework consists of three VLM-powered agents: a Planning Agent for decision-making, an Execution Agent for precise operation and a Reflection Agent for outcome evaluation and providing feedback for iterative improvement. Through the collaboration of these agents, our framework is capable of handling a wide range of challenging cleaning tasks. Extensive experiments are conducted demonstrating that the proposed RoboCleaner can achieve high task success rates and operational efficiency within cluttered environments. Also, we have observed the emergent problem-solving capabilities of the proposed framework, which further validates the robustness and adaptability of the framework. Note to Practitioners—The motivation of this work originates from the need to develop a robust, autonomous solution for efficiently managing tabletop cleaning in complex environments, ranging from domestic to industrial settings. The proposed RoboCleaner is supposed to enhance the operational efficiency by reducing the time and resources required for manual cleaning tasks. Traditional approaches, including rule-based and learning-based systems, often fail to adapt to the diverse and unpredictable nature of cluttered environments, thereby limiting their effectiveness in practical scenarios. To address this, our system proposes to utilize a multi-agent framework integrated with advanced vision language models, allowing adaptive selection of cleaning tools based on availability. This adaptability minimizes the requirement for human oversight, thereby enhancing both efficiency and reliability. The proposed method is practically applicable across various tabletop cleaning scenarios, with the cleaning strategy effectively managed through the multi-agent framework and vision language models, positioning RoboCleaner as a valuable asset for automated cleaning tasks in diverse settings.