Starling: An I/o-efficient Disk-resident Graph Index Framework For High-dimensional Vector Similarity Search On Data Segment

Abstract

High-dimensional vector similarity search (HVSS) is gaining prominence as a powerful tool for various data science and AI applications. As vector data scales up, in-memory indexes pose a significant challenge due to the substantial increase in main memory requirements. A potential solution involves leveraging disk-based implementation, which stores and searches vector data on high-performance devices like NVMe SSDs. However, implementing HVSS for data segments proves to be intricate in vector databases where a single machine comprises multiple segments for system scalability. In this context, each segment operates with limited memory and disk space, necessitating a delicate balance between accuracy, efficiency, and space cost. Existing disk-based methods fall short as they do not holistically address all these requirements simultaneously. In this paper, we present Starling, an I/O-efficient disk-resident graph index framework that optimizes data layout and search strategy within the se

Stats

• citations47
• S2 citations—
• github stars0
• HF likes0
• heat score12.61
• arxiv keywang2024starling

Related papers

• Reveal Hidden Pitfalls And Navigate Next Generation Of Vector Similarity Search From Task-centric Views (2025)0.00
• Leanvec: Searching Vectors Faster By Making Them Fit (2023)0.00
• Passing The Baton: High Throughput Distributed Disk-based Vector Search With Batann (2025)0.00
• Gleanvec: Accelerating Vector Search With Minimalist Nonlinear Dimensionality Reduction (2024)0.00
• Billion-scale Similarity Search Using A Hybrid Indexing Approach With Advanced Filtering (2025)4.52
• Zoom: Ssd-based Vector Search For Optimizing Accuracy, Latency And Memory (2018)0.00
• Semantic Vector Encoding And Similarity Search Using Fulltext Search Engines (2017)6.77
• Approximate Vector Set Search Inspired By Fly Olfactory Neural System (2024)0.00