A skill getting better
The same request should get a better result next month than it did this month — and you should be able to see why. In WilsonOS that happens to skills: the reusable capability an agent runs, such as the one that writes the mysteries in Writing the stories. This walkthrough follows that skill across five batches, and then shows how a proposed change is tested before it is allowed to replace what works.
Make the story writer pass every test on the first attempt, so the owner reads finished work rather than drafts.
What you can do
Section titled “What you can do”- See every batch a skill produced, with what passed, what failed and why.
- Compare the current version of a skill with a candidate on the same brief, scored on criteria you can read.
- Tell the grader what each criterion means, show it examples of what a score looks like, and give it a reference answer, in words or a picture.
- Overrule the grader on any round and record why.
- Promote a better version yourself — or keep the current one.
Five batches, five versions
Section titled “Five batches, five versions”-
Batch 1 — acceptance — Workbench
The first full run of the skill at version 49, to see whether it could produce acceptable cases at all.
-
Batch 2 — gates calibrated on the real books — Workbench
The tests themselves were calibrated against cases already in print, so “passes” means “as good as what we sell”. Four cases accepted. This is the batch the gate record comes from: two stories failed their first attempt by drawing too much attention to a planted clue.
-
Batch 3 — testing the scorer — Workbench
A batch run to check the scoring itself: does a high score mean what the owner would say on reading it?
-
Batch 4 — version 49.7 — Workbench
The first batch under the revised skill, with the lessons of batches 2 and 3 written into it.
-
Batch 5 — version 49.7.1 — Workbench
Three accepted, two escalated on how well a child would grasp the key clue — and, for the first time, every case passed the cold solve and the fair-play check on the first attempt, with no broken keystone clue. The escalations produced version 49.7.2.
Each step is a version of the skill with a reason attached, and each reason is a specific failure from the batch before. Nothing changed because one story was good or one story was bad.
Testing a change before it replaces what works
Section titled “Testing a change before it replaces what works”A proposed version does not replace the current one because it looks better. It is run against the current version on the same brief, and both outputs are scored.
A · current version
- Cold solve — passed
- Fair play — passed
- Clue grasp — 3 of 5
- Craft and delight — 4.0
B · candidate
- Cold solve — passed
- Fair play — passed
- Clue grasp — 4 of 5
- Craft and delight — 4.1
Grader: B wins on clue grasp, no regressions. Your call
-
Same challenge, both versions — Skills, Showdown
The current version and the candidate get the identical brief, once or over several rounds and inputs, with the same reference images, documents and style.
-
Scored on stated criteria — Skills, Showdown
A grader scores each run one to five on criteria written down before the run, each with a line on what it means for this job, calibrated by examples you give and, if you have one, a reference answer. It looks at the pictures a visual skill makes, not just the text. A criterion that does not fit the request is marked not applicable rather than forced. You can override any score, or overrule the grader’s call on a round and say why. Regressions — anything the candidate does worse — are listed, not averaged away.
-
You promote — Skills, Showdown
If the candidate wins without regressions, you promote it. The old version and the reasons are kept.
What to notice
Section titled “What to notice”- The evidence is the work. Every version change points back to a batch and a specific failure in it.
- The tests were tested. Batch 3 checked the scorer before anyone relied on its numbers.
- A good week does not rewrite the skill. A candidate has to beat the current version on the same brief, and you make the call.
Where it stands today
Section titled “Where it stands today”The Skill Showdown — current against candidate, scored, promoted by you — is available. How much a skill may do on its own is set by hand today; a trust ladder that moves with its record is Planned. Details and evidence: What works today.
Related
Sources: brands/code-pineapple/pipeline/minute-mystery-batch-1 … batch-5 (task records), skills/minute-mystery-writer, docs/wilsonos-app/feature-matrix.md (Skill Showdown), docs/tasks/Complete/585-showdown-editor-and-one-library.md (judge brief, image judging, N/A) · Last verified 2026-10-14

