# Aamas Experiments

Use when designing or auditing AAMAS experiments - self-play and population-based...

- Repository: https://github.com/brycewang-stanford/Awesome-Journal-Skills (1346 stars)
- Category: Design
- License: MIT
- Last updated: 2026-09-27
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill aamas-experiments -g -a claude-code -y`
Codex: `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill aamas-experiments -g -a codex -y`
Cursor: `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill aamas-experiments -g -a cursor -y`
Gemini CLI: `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill aamas-experiments -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add brycewang-stanford/Awesome-Journal-Skills`

## Description

Use when designing or auditing AAMAS experiments - self-play and population-based training, opponent selection, equilibrium and regret metrics, game-theoretic simulations, ablations, seeds, hyperparameters, compute, and claim-to-evidence fit - with emphasis on experiments that probe the interaction rather than chase a single-agent leaderboard.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# AAMAS Experiments

Use this before submission when the empirical or simulation story is not yet locked. At AAMAS
the experiment exists to test the *interaction* claim, not to top a benchmark.

## Experiment audit

- Map each empirical claim to a game, a self-play run, a population sweep, an ablation, or a
  deviation test.
- Choose opponents deliberately: self-play alone rarely suffices; include held-out opponents,
  population sets, or classical strategies as the claim requires.
- Separate simulations that validate a solution concept (where the equilibrium is known) from
  real or applied studies that show practical multiagent behavior.
- Report uncertainty for stochastic results over both seeds and opponents: standard errors,
  confidence intervals, or paired tests.
- Report the environment, number of agents, training regime, evaluation protocol, metrics,
  hyperparameter ranges, chosen settings, seeds, hardware, software versions, and runtime.
- Add ablations for the interaction mechanism (communication, reward sharing, the payment rule),
  not just cosmetic variants.
- Audit for the mismatch between the strategic claim and the setup: an equilibrium claim tested
  against only one fixed opponent, or a cooperation claim that hides a reward-shaping constant.

## What experiments are for at this venue

- The strongest design shows the interaction under stress: agents that *can* deviate, opponents
  the method did not train against, and populations that vary in size or composition.
- One experiment that lets agents try to exploit the mechanism and fails to profit is worth more
  than five extra environments where nothing strategic is tested.
- Reviewers, often game theorists, check whether the metric matches the claim: convergence to a
  named solution concept, exploitability, social welfare, or regret - not just episodic return.

## Interaction-validation design table

The rest: https://github.com/brycewang-stanford/Awesome-Journal-Skills/blob/HEAD/AAMAS-Skills/skills/aamas-experiments/SKILL.md