Runbook Creation

Create effective operational runbooks, standard operating procedures, and troubleshooting guides that any on-call engineer can follow under pressure.

Runbook Template, Full Structure

# Runbook: [Service / Process Name]

**Owner:** [Team or individual]
**Last Reviewed:** YYYY-MM-DD
**Version:** X.Y
**Severity if unavailable:** SEV[1-4]

---

## Overview

Brief description of the service, why this runbook exists, and when to
use it.

## Prerequisites

- [ ] Required access / IAM role: [details]
- [ ] Tools installed: [kubectl, aws-cli, psql, etc.]
- [ ] VPN connected to [environment]
- [ ] Communication channel open: [Slack #channel]

## Procedure

### Step 1, [Action Name]

[Explanation of what this step does and why.]

```bash
# command here
```

**Expected output:** [describe what success looks like]

### Step 2, [Action Name]

```bash
# command here
```

**Expected output:** [description]

*(Continue with numbered steps...)*

## Verification

How to confirm the procedure succeeded:

- [ ] [Check 1, e.g., health endpoint returns 200]
- [ ] [Check 2, e.g., no errors in logs for 5 minutes]
- [ ] [Check 3, e.g., metrics return to baseline]

## Rollback

If the procedure fails or causes unexpected issues:

### Rollback Step 1
```bash
# rollback command
```

### Rollback Step 2
```bash
# rollback command
```

## Troubleshooting

| Symptom | Likely Cause | Resolution |
|---------|-------------|------------|
| [symptom 1] | [cause] | [fix] |
| [symptom 2] | [cause] | [fix] |

## Escalation

If unresolved after [X] minutes:
- **Primary:** @[team-lead], [phone/Slack]
- **Secondary:** @[manager], [phone/Slack]

## Related Runbooks

- [Link to related runbook 1]
- [Link to related runbook 2]

## Change Log

| Date | Author | Change |
|------|--------|--------|
| YYYY-MM-DD | [Name] | Initial version |

Example Runbook, Database Failover

# Runbook: PostgreSQL Database Failover