What do I need to use A/B Test Readout?

The hypothesis; Results; Guardrail metrics; Pre-registered decision rule.

Is A/B Test Readout free to use?

Yes — run it free in the browser playground (no API key needed for the sponsored trial, or bring any provider key), or install it into Claude Code, Cursor, ChatGPT and more via npx pm-claude-skills. MIT-licensed and open source.

A/B Test Readout

Q: How do I know the A/B Test Readout output is good?

The skill verifies itself against its own quality checks: Distinguishes statistical from practical significance; Checks guardrail metrics, not just the primary; Flags peeking, power, novelty, and segment-driven wins; Recommendation follows from the evidence, with a monitoring/next-test step.

🔵 Stable🕐 updated 2026-06-21 🔷 SkillSpec L3 pm-data

Analyse a finished A/B test and write the readout — the result, whether it's statistically and practically significant, what it means, and the ship/no-ship call. Use when asked to analyse experiment results, write an A/B test readout, interpret test data, or decide whether to ship a variant. Produces a clear verdict with the lift and confidence, segment cuts, the risks (peeking, novelty, sample), and a recommendation. Distinct from planning a test — this reads results.

📚 Based on Frequentist A/B test inference (statistical & practical significance)

▶ Run it free — no key needed 📝 Grade your existing draft View SKILL.md ↗

What to give it

▸The hypothesis — and the primary metric

▸Results — control vs variant: conversions/rate, sample size per arm, duration

▸Guardrail metrics — (revenue, retention, latency, complaints) that mustn't regress

▸Pre-registered decision rule — (what would count as a win) if one exists

✅ The bar it holds itself to

Every skill in this library self-verifies — these are this skill's own quality checks, straight from its definition.

✓Distinguishes statistical from practical significance

✓Checks guardrail metrics, not just the primary

✓Flags peeking, power, novelty, and segment-driven wins

✓Recommendation follows from the evidence, with a monitoring/next-test step

✓Doesn't declare a winner on an underpowered or peeked result

⚠️ What it refuses to do

"B won by 8%!" with no significance or sample size

Calling a result early (peeking) and shipping

Ignoring a guardrail regression because the primary went up

A statistically significant but practically meaningless lift treated as a win

Install

npx pm-claude-skills add --agent claude   # or codex · cursor · gemini · hermes
# or one-line MCP (every skill, any client):
claude mcp add pm-skills -- npx -y pm-claude-skills-mcp

Start with

Related skills

🔌 Embed this skill

Drop this on your blog, docs, or site — it renders a "Run this skill" card:

<div data-pm-skill="ab-test-readout"></div>
<script src="https://mohitagw15856.github.io/pm-claude-skills/embed.js" async></script>

💬 Discussion

A/B Test Readout is one of 599 open-source professional AI agent skills — all SkillSpec L3. Try them all in the browser · ⭐ Star on GitHub · Browse the full catalog