LLMJudge

'LLMJudge' is a benchmark dataset used to evaluate the relevance judgments and self-reported confidence scores of large language models (LLMs) prompted to simulate Big Five personality traits in the context of web search decisions.

Papers using LLMJudge (4)

Judging with Personality and Confidence: A Study on Personality-Conditioned LLM Relevance Assessment2026Criteria-Based LLM Relevance Judgments2025JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment2024 · 2 citesBest in Tau@LLMJudge: Criteria-Based Relevance Evaluation with Llama32024