Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.

Ranking LLMs on agentic tasks

Vivaria is METR's tool for running evaluations and conducting agent elicitation research.

one click to open multi AI sites ｜ 一键打开多个 AI 站点，查看 AI 结果

Code scanner to check for issues in prompts and LLM calls

Benchmark evaluating LLMs on their ability to create and resist disinformation. Includes comprehensive testing across major models (Claude, GPT-4, Gemini, Llama, etc.) with standardized evaluation metrics.

Cost-of-Pass: An Economic Framework for Evaluating Language Models

Adaptive Testing Framework for AI Models (Psychometrics in AI Evaluation)

JudgeGPT - (Fake) News Evaluation, a research project

LLM-as-a-judge for Extractive QA datasets

ROC methodology explained with R-examples

FROC methodology explained with R-examples

RJafroc quick start for those already familiar with windows jafroc

Installation files for Windows JAFROC software

CLI tool to evaluate LLM factuality on MMLU benchmark.

ROC/FROC datasets from my collaborations

Repository for the LWDA'24 presentation on 'Psychometric Profiling of GPT Models for Bias Exploration', featuring conference materials including the poster, paper, slides, and references.

UQLM (Uncertainty Quantification for Language Models) is a Python package for UQ-based LLM hallucination detection

Automatic multi-metric evaluation of human-bot dialogues using LLMs (Claude, GPT-4o) across different datasets and settings. Built for the Artificial Intelligence course at the University of Salerno.