Skip to content
Describes how WilsonOS is designed to work.What works today →

A skill getting better

The same request should get a better result next month than it did this month — and you should be able to see why. In WilsonOS that happens to skills: the reusable capability an agent runs, such as the one that writes the mysteries in Writing the stories. This walkthrough follows that skill across five batches, and then shows how a proposed change is tested before it is allowed to replace what works.

Goal

Make the story writer pass every test on the first attempt, so the owner reads finished work rather than drafts.

Who does the work
Scout runs the batches; each batch's results become the evidence for the next version of the minute-mystery-writer skill.
Where you see it
WorkbenchSkills — Showdown
What came back
Five batches and five skill versions. Batch 5 was the first in which every case passed the cold solve and the fair-play check on the first attempt.
  • See every batch a skill produced, with what passed, what failed and why.
  • Compare the current version of a skill with a candidate on the same brief, scored on criteria you can read.
  • Tell the grader what each criterion means, show it examples of what a score looks like, and give it a reference answer, in words or a picture.
  • Overrule the grader on any round and record why.
  • Promote a better version yourself — or keep the current one.
  1. Batch 1 — acceptance — Workbench

    The first full run of the skill at version 49, to see whether it could produce acceptable cases at all.

  2. Batch 2 — gates calibrated on the real books — Workbench

    The tests themselves were calibrated against cases already in print, so “passes” means “as good as what we sell”. Four cases accepted. This is the batch the gate record comes from: two stories failed their first attempt by drawing too much attention to a planted clue.

  3. Batch 3 — testing the scorer — Workbench

    A batch run to check the scoring itself: does a high score mean what the owner would say on reading it?

  4. Batch 4 — version 49.7 — Workbench

    The first batch under the revised skill, with the lessons of batches 2 and 3 written into it.

  5. Batch 5 — version 49.7.1 — Workbench

    Three accepted, two escalated on how well a child would grasp the key clue — and, for the first time, every case passed the cold solve and the fair-play check on the first attempt, with no broken keystone clue. The escalations produced version 49.7.2.

Each step is a version of the skill with a reason attached, and each reason is a specific failure from the batch before. Nothing changed because one story was good or one story was bad.

Testing a change before it replaces what works

Section titled “Testing a change before it replaces what works”

A proposed version does not replace the current one because it looks better. It is run against the current version on the same brief, and both outputs are scored.

Skill Showdown — current vs candidateSame brief · one round

A · current version

  • Cold solve — passed
  • Fair play — passed
  • Clue grasp — 3 of 5
  • Craft and delight — 4.0

B · candidate

  • Cold solve — passed
  • Fair play — passed
  • Clue grasp — 4 of 5
  • Craft and delight — 4.1

Grader: B wins on clue grasp, no regressions. Your call

Promote BKeep ARun another round
Illustration, not a screenshot. The Skill Showdown view runs a current version and a candidate on the same brief, scores both, and leaves the promotion to you. The scores here are made up to show the shape.
  1. Same challenge, both versions — Skills, Showdown

    The current version and the candidate get the identical brief, once or over several rounds and inputs, with the same reference images, documents and style.

  2. Scored on stated criteria — Skills, Showdown

    A grader scores each run one to five on criteria written down before the run, each with a line on what it means for this job, calibrated by examples you give and, if you have one, a reference answer. It looks at the pictures a visual skill makes, not just the text. A criterion that does not fit the request is marked not applicable rather than forced. You can override any score, or overrule the grader’s call on a round and say why. Regressions — anything the candidate does worse — are listed, not averaged away.

  3. You promote — Skills, Showdown

    If the candidate wins without regressions, you promote it. The old version and the reasons are kept.

  • The evidence is the work. Every version change points back to a batch and a specific failure in it.
  • The tests were tested. Batch 3 checked the scorer before anyone relied on its numbers.
  • A good week does not rewrite the skill. A candidate has to beat the current version on the same brief, and you make the call.

The Skill Showdown — current against candidate, scored, promoted by you — is available. How much a skill may do on its own is set by hand today; a trust ladder that moves with its record is Planned. Details and evidence: What works today.

Sources: brands/code-pineapple/pipeline/minute-mystery-batch-1 … batch-5 (task records), skills/minute-mystery-writer, docs/wilsonos-app/feature-matrix.md (Skill Showdown), docs/tasks/Complete/585-showdown-editor-and-one-library.md (judge brief, image judging, N/A) · Last verified 2026-10-14