ER
Experiment Readout
🔵 Stable🕐 updated 2026-06-28
🔷 SkillSpec L3
⚙ ships an executable helper
pm-dataeng
Analyse a finished A/B test and write an honest results readout with real statistics. Use when asked to read out an A/B test, analyse experiment results, check if a result is statistically significant, or decide ship/no-ship from test data. Produces a readout — the computed lift, p-value & confidence interval, a significance verdict, guardrail check, and a clear ship / no-ship / iterate recommendation. Includes a stdlib significance calculator.
What to give it
▸The metric & data — for a conversion test: users and conversions per variant (control vs. treatment). For a continuous metric: mean, SD, and n per variant.
▸The hypothesis — what you expected and the minimum effect that matters.
▸Guardrail metrics — what shouldn't get worse (revenue, latency, retention).
▸Test setup — planned sample size/duration, and whether it ran to plan (for the peeking check).
✅ The bar it holds itself to
Every skill in this library self-verifies — these are this skill's own quality checks, straight from its definition.
✓Lift, p-value, and a confidence interval are computed (not just "higher")
✓Statistical significance AND practical significance are both assessed
✓Guardrail metrics are checked, not just the primary
✓Validity is checked: ran to planned n, no peeking, no sample-ratio mismatch
✓An inconclusive result is reported honestly, not spun into a win
✓The recommendation is explicit (ship/no-ship/iterate/re-run)
⚠️ What it refuses to do
Do not call significance by eye — compute the p-value and CI; a higher number isn't a result
Do not ignore the confidence interval — a CI spanning zero (or huge) means you don't actually know the effect
Do not confuse statistical with practical significance — a tiny significant lift may not be worth shipping
Do not trust a peeked/early-stopped test — stopping when it looks good inflates false positives massively
Do not spin a null result — "no detectable difference" is honest and often the right call
Install
npx pm-claude-skills add --agent claude # or codex · cursor · gemini · hermes
# or one-line MCP (every skill, any client):
claude mcp add pm-skills -- npx -y pm-claude-skills-mcpRelated skills
🔌 Embed this skill
Drop this on your blog, docs, or site — it renders a "Run this skill" card:
<div data-pm-skill="experiment-readout"></div>
<script src="https://mohitagw15856.github.io/pm-claude-skills/embed.js" async></script>💬 Discussion
Experiment Readout is one of 599 open-source professional AI agent skills — all SkillSpec L3.
Try them all in the browser · ⭐ Star on GitHub · Browse the full catalog