Free to read · no account needed

9 min · skills

Test a skill before you trust it

Goal: Before you trust a skill, run good and bad cases (including privacy/invention red lines)—write expected behavior like an acceptance test.

Best for: before you share a skill with teammates or put it on autopilot.

Treat a skill like a mini product: three fixed examples (happy path, messy input, red-line refusal). Save inputs and what “good” looks like.

Why beginners skip this

Trusting a skill without a test feels like saved time—the first accident lands on teammates. Fixture pass before share.

Step 1 — Build three fixtures

Build three fixtures: happy path, messy/incomplete input, and a case that should refuse or ask—do not only test the happy path.

Skill purpose: [p]
Invent 3 test fixtures:
1) Happy path input + good output checklist
2) Messy input + how skill should respond
3) Red-line input + expected refusal/ask
Keep fixtures short.
Example

Filled prompt

Live task this week (not a toy): apply this guide's Goal — run a tiny test suite on a skill so bad instructions fail in private, not with a client.
Fill with your real artifact names/files. Constraints: no invented facts; label Unknowns; stop before send/delete/pay.
Return the working prompt + one pass/fail check I can mark in a minute.

What good output looks like

  • Fixtures short enough to paste
  • Red-line expects refusal or redaction
  • Good checklist is observable

Step 2 — Run and score

Score 1–5 on followed steps, respected never-do, and usable output; change one instruction at a time.

Score this skill run (1–5 each): followed steps · respected never-do · usable output.
Input: [fixture]
Output: [paste]
List top 3 instruction patches.
Example

Filled prompt

Known-good: last week’s sanitized status note (no fake metrics).
Known-bad: “run a tiny test suite on a skill so bad”
Skill must format the good and refuse/ask on the bad. Show expected behaviors.

What good output looks like

  • Calls out invention
  • Patches say “only use dates present in notes”
  • Does not rewrite the whole skill blindly

Step 3 — Freeze a “known good” example

Pin one input/output pair in the skill as Few-shot. Re-run after every edit.

Turn this known-good pair into a few-shot block I can paste into the skill.
Input: …
Output: …
Add one line: “run a tiny test suite on a skill so bad”
Example

Filled prompt

Known-good weekly update pair → few-shot block + “run a tiny test suite on a skill so bad”
Context: live job for skills-test-a-skill-before-trust — “run a tiny test suite on a skill so bad” Keep never-invent / draft-only.

What good output looks like

  • Skill purpose is one job for “run a tiny test suite on a skill so bad instructio” — not a company brain
  • Done + golden example present and tested
  • Never-send / never-delete honored
  • Pass/fail: fail-case run once (skills-test-a-skill-before-trust #1)

Known-good / known-bad

A skill that only sees happy paths will fail loudly in production. Keep one sample that should succeed and one that should trigger a refusal or “run a tiny test suite on a skill so bad” Re-run both after every version bump.

Version after fail

When known-bad slips through, bump the version and add the case as an example—not a silent edit.

Keep it real

Run this guide’s Practice now on work you will actually ship this week—not a fictional scenario. Dated notes beat perfect theory.

Do not bookmark-only

Finish Practice now, then tick this guide on your path. Titles unread are not skills.

Close the loop

Write one line: next time I trust a skill with no test, I refuse share.

Red flag

Trusting before three fixtures; failing in front of customers—kill bad instructions in private.

Save this card

Card: three fixtures (happy / messy / refuse) · score · private kill. Trust only after all three pass.

Before you start

Open the skill card (or blank one-pager). Purpose must be one job for “run a tiny test suite on a skill so bad.” Golden example slot cannot stay empty. Guide: skills-test-a-skill-before-trust. Keep the real job visible the whole time; a blank box means stop and fetch the artifact first.

How to use this week

This week write a fixture for a real skill: sample input + expected output + fail case; no share until it passes.

Pitfalls

PitfallFix
Testing only the happy pathInclude messy + refuse
Changing five rules at onceOne change per retest
No saved fixturesKeep a tests.md
Example

Filled prompt

Temptation: skip the habit for skills-test-a-skill-before-trust to save five minutes.
Job still: run a tiny test suite on a skill so bad
Smallest correct pass: Done visible · checkable artifact · Unknowns [?].
Boundaries: draft only; no silent send; no invented metrics.
Return: artifact + refuse-line said aloud.

What good output looks like

  • Names the shortcut refused (specific to skills-test-a-skill-before-trust)
  • Still delivers checkable output for “run a tiny test suite on a skill so bad”
  • Unknowns labeled; draft-only
  • Pass/fail: refuse-line dated (skills-test-a-skill-before-trust #4)

Practice now

Flaky ban: rerun thrice; flaky = not trusted (Asia/Shanghai).

Timeout case: one test asserts a timeout/refuse path (Asia/Shanghai).

Refuse fixture: committed test that must refuse (Asia/Shanghai).

Edge name: name the edge case before you run it (Asia/Shanghai).

Refuse-case drill: one test must hit the refuse path on purpose (Asia/Shanghai).

Skill-test drill: run three cases (happy/edge/refuse) before trusting auto-invoke (Asia/Shanghai).

Thin-spot drill: rewrite the Save card with real nouns from your work (repo, doc, inbox). Generic placeholders fail the drill.

Tonight prepare known-good / known-bad / boundary fixtures for one skill and score them—no share if it fails.

Today card (copy) — skills-test-a-skill-before-trust: 1. Artifact I will produce: ___ 2. Pass/fail check: ___ 3. Refuse-line if tempted to skip verify/send: ___ (date: Asia/Shanghai) 4. One never-do from this guide: ___

Extra pass: do the card on a real file today—reading alone does not count.

One sentence takeaway

Fixture: sample input + expected output + fail case; no share until pass.

Next

Versioning · Few-shot that helps

← Back to home · Learn · Tools · Compare · Start