OpenJudge

OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards

npx skills add agentscope-ai/OpenJudge -g -y

19 skills

00 Academic Router

Use when the user wants help with academic papers or citations but it's unclear which...

agentscope-ai/OpenJudge1k

00 Arena Router

Use when the user wants to compare or benchmark multiple LLMs/agents arena-style but...

agentscope-ai/OpenJudge1k

01 Auto Arena

Automatically evaluate and compare multiple AI models or agents without pre-existing...

agentscope-ai/OpenJudge1k

01 Graders And Pipeline

Build custom LLM evaluation pipelines using the OpenJudge framework.

agentscope-ai/OpenJudge1k

01 Paper Review

Review academic papers for correctness, quality, and novelty using OpenJudge's...

agentscope-ai/OpenJudge1k

02 Bib Verify

Verify a BibTeX file for hallucinated or fabricated references by cross-checking every...

agentscope-ai/OpenJudge1k

02 Rl Reward

Build RL reward signals using the OpenJudge framework.

agentscope-ai/OpenJudge1k

03 Ref Hallucination Arena

Benchmark LLM reference recommendation capabilities by verifying every cited paper...

agentscope-ai/OpenJudge1k

Align Human

Use when the user has a judge/grader and human-labeled data, and wants to measure how...

agentscope-ai/OpenJudge1k

Bootstrap

Use when the user has nothing, no traces, no labels, no eval set, and needs to build...

agentscope-ai/OpenJudge1k

Claude Authenticity

Detect whether an API endpoint is backed by genuine Claude (not a wrapper, proxy, or...

agentscope-ai/OpenJudge1k

Eval Report

Use when the user has run multiple evaluation skills and wants a comprehensive analysis...

agentscope-ai/OpenJudge1k

Find Skills Combo

Discover and recommend **combinations** of agent skills to complete complex...

agentscope-ai/OpenJudge1k

Meta Eval

Use when the user wants to build an evaluation system for an LLM/agent application but...

agentscope-ai/OpenJudge1k

Metric Design

Use when the user has evaluation principles or a dataset but needs help choosing the...

agentscope-ai/OpenJudge1k

Mmx CLI

Generate text, images, video, speech, and music via the MiniMax AI platform.

agentscope-ai/OpenJudge1k

Prompt Regression

Use when the user has changed a prompt (system prompt, RAG template, agent instruction...

agentscope-ai/OpenJudge1k

Rag Eval

Use when the user has a RAG (Retrieval-Augmented Generation) system and wants to...

agentscope-ai/OpenJudge1k

Redteam

Use when the user wants to test their LLM/agent application for safety and security...

agentscope-ai/OpenJudge1k