# Ab Test Setup

When the user wants to plan, design, or implement an A/B test or experiment.

- Repository: https://github.com/LeoYeAI/openclaw-marketing-skills (1053 stars)
- Category: Design
- License: see repository
- Last updated: 2026-06-02
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add LeoYeAI/openclaw-marketing-skills --skill ab-test-setup -g -a claude-code -y`
Codex: `npx skills add LeoYeAI/openclaw-marketing-skills --skill ab-test-setup -g -a codex -y`
Cursor: `npx skills add LeoYeAI/openclaw-marketing-skills --skill ab-test-setup -g -a cursor -y`
Gemini CLI: `npx skills add LeoYeAI/openclaw-marketing-skills --skill ab-test-setup -g -a gemini-cli -y`

## Description

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," or "how long should I run this test." Use this whenever someone is comparing two approaches and wants to measure which performs better. For tracking implementation, see analytics-tracking. For page-level conversion optimization, see page-cro.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# A/B Test Setup

You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results.

## Initial Assessment

**Check for product marketing context first:**
If `.agents/product-marketing-context.md` exists (or `.agents/product-marketing-context.md` in older setups), read it before asking questions. Use that context and only ask for information not already covered or specific to this task.

Before designing a test, understand:

1. **Test Context** - What are you trying to improve? What change are you considering?
2. **Current State** - Baseline conversion rate? Current traffic volume?
3. **Constraints** - Technical complexity? Timeline? Tools available?

---

## Core Principles

### 1. Start with a Hypothesis
- Not just "let's see what happens"
- Specific prediction of outcome
- Based on reasoning or data

### 2. Test One Thing
- Single variable per test
- Otherwise you don't know what worked

### 3. Statistical Rigor
- Pre-determine sample size
- Don't peek and stop early
- Commit to the methodology

### 4. Measure What Matters
- Primary metric tied to business value
- Secondary metrics for context
- Guardrail metrics to prevent harm

---

## Hypothesis Framework

### Structure

```
Because [observation/data],
we believe [change]
will cause [expected outcome]
for [audience].
We'll know this is true when [metrics].
```

### Example

**Weak**: "Changing the button color might increase clicks."

**Strong**: "Because users report difficulty finding the CTA (per heatmaps and feedback), we believe making the button larger and using contrasting color will increase CTA clicks by 15%+ for new visitors. We'll measure click-through rate from page view to signup start."

---

## Test Types

The rest: https://github.com/LeoYeAI/openclaw-marketing-skills/blob/HEAD/skills/ab-test-setup/SKILL.md