GitHub Copilot Train-the-Trainer

Agent Skills

Test selection, non-selection, and visible failure

Session M3-05 | 3 hours

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Agenda

Segment Time
Skill directory and loading model 8 min
Required frontmatter and description 8 min
Selection and visible failure 10 min
Validation versus live invocation 10 min
Pinned interview comparison and plan handoff 18 min
Access check and lab handoff 6 min
Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

A skill has two jobs

  1. Select the right task.
  2. Guide the work after selection.

A broad description can select the wrong task even when the procedure is good.
The procedure also needs failure rules so it does not claim success after a
required check fails.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Repository skill structure

.github/
└── skills/
    └── api-design/
        └── SKILL.md

The directory uses lowercase words and hyphens. The instruction file is named
exactly SKILL.md.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Required frontmatter

---
name: api-design
description: Use for creating or reviewing REST routes under src/api/ when acceptance criteria, a reviewer, and a verified focused test command are available. Do not use for documentation, UI, deployment, or generated clients.
---

Match name to the directory. Use description to tell Copilot when to select the skill.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Test the positive trigger

Review GET /api/tasks/:id under src/api/.
Check identifier handling and the existing not-found response.

Required context:

  • acceptance criteria;
  • named paths;
  • reviewer;
  • focused test command that runs in this project.

Expected status: applied.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Test a non-trigger

Rewrite the installation section in README.md.

Expected:

{
  "status": "not_applicable",
  "skill": "api-design",
  "reason": "Task is outside REST route work under src/api/."
}

A skill should not run on tasks outside its scope.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Matching work can still be blocked

Add POST /api/tasks under src/api/.
Acceptance criteria exist.
No reviewer is assigned.

The trigger matches. The precondition does not.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Make the failure visible

{
  "status": "blocked",
  "skill": "api-design",
  "reason": "Missing precondition: human reviewer."
}

Do not invent a reviewer, skip the check, or return success.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

A reviewable skill body

Preconditions
Procedure
Validation
Failure behavior
Maintenance

Each section answers a different question. Keep the procedure short enough that a
human can apply it as a checklist.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Validate the files

npm run validate:skill

The validator checks:

  • SKILL.md exists;
  • frontmatter has name and description;
  • name matches the directory;
  • required sections are present.
Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Test all three outcomes

npm test

The suite checks that:

  • a route task applies the skill;
  • a documentation task does not;
  • a missing reviewer blocks;
  • valid and invalid directories are distinguishable.
Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Run the two demo scenarios

npm run demo

The output shows:

  1. correct application to named API files;
  2. a visible missing-reviewer failure.

The local runner is a teaching harness, not a replacement skill platform.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

What local tests and live invocation show

Check What it shows
Local directory validator Your file has the required structure
Harness unit tests The fixed scenario contract behaves as expected
/skills info and actual invocation Copilot discovered and used the skill

Live CLI access is required. Stop if a required skill cannot load.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Inspect the pinned interview skill

Matt Pocock's grill-me is a community entry point that invokes grilling.

  • Check the recorded revision and SHA-256 hashes.
  • Read both files and the retained MIT license.
  • Load only the reviewed project-local copy.
  • Keep permissions and the agreed usage limit in place.

No marketplace installer or live download is needed for this lab.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Compare the same request twice

Use separate fresh Interactive sessions:

Run Skill
Local /decision-interview from Session M7-02
Community /grill-me with its grilling dependency

Use the same request, owner decisions, and late visibility constraint.
Record actual loading and changes to the decisions.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Interview first, plan second

Keep Plan mode off while decisions are unsettled.

  1. Finish the interview and review the decisions.
  2. Save the accepted brief with open questions.
  3. Start a fresh /plan session from that brief.
  4. Name two decisions that change plan steps or acceptance checks.

Do not implement or create live work items during this comparison.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Review candidates before adaptation

Check:

  • source and license;
  • files and scripts;
  • dependencies and network use;
  • data flow;
  • repository fit;
  • owner and retirement condition.

Do not run an unfamiliar script just to inspect it.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Review recurring session corrections

/chronicle improve

This subcommand scopes history to the current repository or working directory.
Check a recommendation against an actual correction.

Choose instructions, a skill, or neither. Cancel the apply step; suggestions may
start selected. Do not reindex or inspect unrelated history for this lab.

Too little history is a valid result. This uses the last five minutes of the
45-minute comparison block.

Session M3-05: Agent Skills Ecosystem
GitHub Copilot Train-the-Trainer

Lab handoff

  1. Create api-design/SKILL.md.
  2. Validate the directory and frontmatter.
  3. Run the trigger scenario.
  4. Run the non-trigger scenario.
  5. Run the missing-reviewer scenario.
  6. Finish with the two-scenario demo.
  7. Check live API-skill invocation and the missing-reviewer stop.
  8. Compare both live interviews and trace decisions into /plan.
  9. Check scoped Chronicle suggestions without auto-applying them.

Submit actual outputs. The harness or prepared transcripts cannot replace
live loading, either interview, or the plan handoff.

Session M3-05: Agent Skills Ecosystem