Exharmony: Authorship And Citations For Benchmarking The Reviewer Assignment Problem
2025 · Sajad Ebrahimi, Sara Salamat, Negar Arabzadeh, et al.
Abstract
The peer review process is crucial for ensuring the quality and reliability of scholarly work, yet assigning suitable reviewers remains a significant challenge. Traditional manual methods are labor-intensive and often ineffective, leading to nonconstructive or biased reviews. This paper introduces the exHarmony (eHarmony but for connecting experts to manuscripts) benchmark, designed to address these challenges by re-imagining the Reviewer Assignment Problem (RAP) as a retrieval task. Utilizing the extensive data from OpenAlex, we propose a novel approach that considers a host of signals from the authors, most similar experts, and the citation relations as potential indicators for a suitable reviewer for a manuscript. This approach allows us to develop a standard benchmark dataset for evaluating the reviewer assignment problem without needing explicit labels. We benchmark various methods, including traditional lexical matching, static neural embeddings, and contextualized neural embeddi
Authors
(none)
Tags
Stats
Related papers
- Self-supervised Contrastive BERT Fine-tuning For Fusion-based Reviewed-item Retrieval (2023)5.84
- Benchmarking Tool-Use Reliability: A Compositional Evaluation Of Hybrid Architectures (2026)0.00
- Effective Distributed Representations For Academic Expert Search (2020)6.34
- Domain-adaptive And Scalable Dense Retrieval For Content-based Recommendation (2026)0.00
- Beyond Benchmarks: Evaluating Embedding Model Similarity For Retrieval Augmented Generation Systems (2024)0.00
- Divide And Conquer: Towards Better Embedding-based Retrieval For Recommender Systems From A Multi-task Perspective (2023)7.16
- From BM25 To Corrective RAG: Benchmarking Retrieval Strategies For Text-and-table Documents (2026)0.00
- RAG Playground: A Framework For Systematic Evaluation Of Retrieval Strategies And Prompt Engineering In RAG Systems (2024)0.00