# Skill Optimizer

> Optimización auto-evolutiva de skills mediante un optimizador en espacio de texto basado en el paper SkillOpt, con validación obligatoria y versionado atómico.

Fuente: https://skillsagentes.com/skills/garrytan/gbrain/skill-optimizer
Markdown: https://skillsagentes.com/skills/garrytan/gbrain/skill-optimizer.md

Repositorio: https://github.com/garrytan/gbrain
Autor: garrytan
Licencia: MIT
Actualizado: hace 5 días
Coste de contexto: 21 tok instalada, 2.4k tok al activarse, 3k tok con todos los archivos del bundle
Bundle: 3 archivos, 12 KB
Permisos que pide: ninguno declarado

## Instalación

Un skill son archivos markdown: los mismos archivos valen para cualquier agente y lo único que cambia es el directorio de destino, es decir la bandera `--agent`. Añade `-g` para instalarlo en todos los proyectos de la máquina.

```bash
# Claude Code
npx -y skills add garrytan/gbrain --skill skill-optimizer --agent claude-code
# Cursor
npx -y skills add garrytan/gbrain --skill skill-optimizer --agent cursor
# Codex
npx -y skills add garrytan/gbrain --skill skill-optimizer --agent codex
# Gemini CLI
npx -y skills add garrytan/gbrain --skill skill-optimizer --agent gemini
# Windsurf
npx -y skills add garrytan/gbrain --skill skill-optimizer --agent windsurf
# Cline
npx -y skills add garrytan/gbrain --skill skill-optimizer --agent cline
```

## Qué hace

- Runs SkillOpt, a validation-gated text-space optimizer that rewrites a skill's SKILL.md body against a benchmark
- Bootstraps a starter benchmark (~15 tasks with rule judges) from an existing SKILL.md or routing-eval.jsonl
- Gates every candidate edit with median-of-3 + epsilon=0.05 validation before rewriting SKILL.md
- Version-tracks proposals, history, rejected edits, and audit trail per run

## Cuándo usarla

- Improving an existing skill's execution quality against a benchmark
- Bootstrapping a benchmark file for a new skill that has none
- Re-tuning a skill after switching target models

## Cuándo no

- Bundled/plugin-shipped skills cannot be mutated in place without --allow-mutate-bundled AND a --held-out set of >=5 disjoint tasks

## Qué la activa

- "Optimiza esta skill contra el benchmark"
- "Haz que la skill sea mejor"
- "Ejecuta skillopt para skills/foo"
- "Genera un benchmark inicial para esta skill y luego optimízala"
- "Vuelve a ajustar la skill tras cambiar de modelo"

## Antes de instalar

- Requiere el CLI gbrain, un directorio skills/ propio y editable, y revisar/reforzar manualmente los jueces generados antes de optimizar.

## Archivos

- SKILL.md — 10 KB
- routing-eval.jsonl — 578 B
- skillopt-benchmark.jsonl — 2 KB

## SKILL.md

Reproducido tal cual desde garrytan/gbrain bajo MIT. Esta sección es el documento original y está en inglés.

# Skill Optimizer

Self-evolving skill optimization. Treats SKILL.md as the trainable parameters
of a frozen agent. Validation-gated, budget-capped, atomic-versioned.

Based on SkillOpt (arXiv 2605.23904, Microsoft Research, May 2026).

## When to invoke this skill

The user wants to:
- Improve an existing skill's execution quality against a benchmark
- Bootstrap a benchmark file for a new skill
- Re-tune a skill after switching target models

## Iron Law

- **Validation gating is MANDATORY.** Every candidate must clear median-of-3
  + epsilon=0.05 margin against the sel-set before SKILL.md gets rewritten.
- **Frontmatter mutation is FORBIDDEN.** The optimizer only edits the body.
  Routing surface (`triggers:`, `brain_first:`) stays invariant.
- **Bundled skills require explicit opt-in AND an independent held-out set.**
  Skills shipping with gbrain cannot be auto-mutated. To rewrite one in place
  the user passes BOTH `--allow-mutate-bundled` AND `--held-out <path>` with
  at least 5 benchmark-disjoint tasks; without the held-out set the run
  hard-refuses (exit 2). Drop `--allow-mutate-bundled` (or pass `--no-mutate`,
  the default for the dream-cycle phase) to write proposed.md for review
  instead — no held-out needed for review-only output.
- **Bootstrap output requires human review.** Both `--bootstrap-from-skill`
  and `--bootstrap-from-routing` write a sentinel; you must review + STRENGTHEN
  the generated judges, delete the sentinel, and re-run with
  `--bootstrap-reviewed` before optimization can use the file.

## The pipeline

```
gbrain skillopt <skill-name> [flags]
  │
  ├── Pre-flight gates
  │     ├── working tree clean (or --force)
  │     ├── benchmark valid + D_sel >= 5 (D17)
  │     ├── cost preflight (D3) — refuses over --max-cost-usd
  │     └── per-skill DB lock (D14)
  │
  ├── Baseline eval on D_sel (sets best_sel_score)
  │
  ├── for epoch in 1..N:
  │     for step in 1..steps_per_epoch:
  │       ├── forward pass: rollouts on D_train batch
  │       ├── backward pass: reflect × 2 (failures + successes per D7)
  │       ├── rank + clip via LR cosine schedule
  │       ├── apply edits (body-only per D5, tagged result per D9)
  │       ├── validation gate: median-of-3 + epsilon=0.05 (D12)
  │       └── if accept: commit via D8 history-intent-first
  │     │
  │     └── slow update (D6) if no improvement this epoch
  │
  └── Final test eval on D_test → run receipt
```

## Where your skills live

Plugin-bundled skills ship as a read-only snapshot; you cannot write a
benchmark or optimization artifacts next to them. Run skillopt against your
own writable skills directory: pass `--skills-dir <path>`
(`gbrain skillopt X --skills-dir ~/my-project/skills`), or run from a
workspace that contains a `skills/` tree and the CLI auto-detects it. Every
`skills/X/...` path below refers to that writable directory (your own
authored skills, skillify-scaffolded skills, or a brain-resident skillpack),
not the plugin snapshot.

## Starting a benchmark from the skill itself (the common case)

**The user will NOT hand-write a benchmark, and you shouldn't start from a blank
file either.** When the user says "make skill X better" and
`skills/X/skillopt-benchmark.jsonl` doesn't exist, generate a starter from the
SKILL.md directly:

1. **Generate the starter.** Run:
   ```
   gbrain skillopt X --bootstrap-from-skill
   ```
   One LLM call reads `skills/X/SKILL.md`, infers what the skill produces and what
   "good" looks like, and writes ~15 tasks (each with rule judges) to
   `skills/X/skillopt-benchmark.jsonl` plus a `# BOOTSTRAP_PENDING_REVIEW`
   sentinel. No `routing-eval.jsonl` is needed. Tune the count with
   `--bootstrap-tasks N` (max 50).
2. **Review AND STRENGTHEN the judges.** This is YOUR job and it is load-bearing.
   The generated rule checks are weak drafts — the model tends to emit generic
   `contains`, loose `max_chars`, or invented headings. Read each task, fix soft
   checks, add the must-haves the skill actually requires (real section names,
   real length ceilings, `min_citations` where sources are expected,
   `tool_called`/`tool_not_called` for tools the skill genuinely uses). A thin
   benchmark optimizes for a thin definition of quality — do not rubber-stamp.
3. **Delete the sentinel line** (`# BOOTSTRAP_PENDING_REVIEW`, the last line).
4. **Run the optimizer with `--split 1:1:1`:**
   ```
   gbrain skillopt X --bootstrap-reviewed --split 1:1:1
   ```
   The 1:1:1 split is REQUIRED for a 15-task starter — the default `4:1:5` makes
   the validation set `floor(15/10)=1`, below the `D_sel >= 5` floor, and the
   optimizer refuses with `d_sel_too_small`. (4:1:5 needs ~50 tasks.) Add
   `--dry-run` first to preview cost.

Benchmark line shape (what the generator writes, one per line):
```
{"task_id":"x-001","task":"<user prompt>","judge":{"kind":"rule","checks":[{"op":"max_chars","arg":1800},{"op":"contains","arg":"agenda"}]}}
```

Rule-check vocabulary you'll strengthen with: `contains`, `regex`,
`section_present`, `max_chars`, `min_citations`, `tool_called`, `tool_not_called`.
Rule judges are deterministic and free, but shallow for skills whose quality is
sequencing, privacy, refusal boundaries, or file placement — for those, hand-add
richer checks (or an `llm` judge) during review.

**Fallback — author freehand.** If the generated starter is poor (rare, but
possible for very behavior-shaped skills), discard it and write the JSONL
yourself: read the SKILL.md, write ~15 realistic tasks covering the boring middle,
attach >=2 rule checks each, save to `skills/X/skillopt-benchmark.jsonl`, run with
`--split 1:1:1`. The full walkthrough lives at
https://github.com/garrytan/gbrain/blob/master/docs/tutorials/improving-skills-with-skillopt.md.

## Decision tree

| Situation | Action |
|---|---|
| Skill has no benchmark | `gbrain skillopt foo --bootstrap-from-skill` → review + strengthen the judges → delete sentinel → `gbrain skillopt foo --bootstrap-reviewed --split 1:1:1` (see section above) |
| Skill has a `routing-eval.jsonl` and you want a head start | `gbrain skillopt foo --bootstrap-from-routing` → review the generated tasks → `--bootstrap-reviewed` (routing tasks test dispatch; tighten them into quality tasks before trusting) |
| Iterating on an existing skill | `gbrain skillopt foo --benchmark skills/foo/skillopt-benchmark.jsonl` |
| Costly run, want preview | Add `--dry-run` |
| Bundled skill (skills/ in gbrain repo) | Default writes proposed.md; to commit in place add `--allow-mutate-bundled` AND `--held-out <path>` (>=5 benchmark-disjoint tasks) — else it hard-refuses. Plugin installs cannot mutate bundled skills in place (the snapshot is read-only): copy the SKILL.md into your own skills dir to iterate, or stick with the default proposed.md review flow |
| Want to review changes before applying | Add `--no-mutate` (writes proposed.md, no held-out needed) |
| Guard against benchmark overfitting | Add `--held-out <path>` — a candidate that beats the benchmark but regresses on the held-out set is refused |
| Mid-run crash | `gbrain skillopt foo --resume <run-id>` |

## Output Format

When invoked, this skill produces:

- Updated `skills/<name>/SKILL.md` (when mutation is allowed)
- `skills/<name>/skillopt/best.md` — pointer copy of current best
- `skills/<name>/skillopt/versions/vNNNN_eN_sN.md` — per-step snapshots
- `skills/<name>/skillopt/history.json` — append-only run record
- `skills/<name>/skillopt/rejected.json` — bounded LRU of rejected edits
- `~/.gbrain/audit/skillopt-YYYY-Www.jsonl` — ISO-week-rotated audit trail

## Anti-Patterns

- **Don't bypass the validation gate.** The median-of-3 + epsilon=0.05 is
  load-bearing; without it, the optimizer accepts noise as improvement.
- **Don't optimize bundled skills without `--allow-mutate-bundled` AND
  `--held-out`.** They ship with gbrain and are load-bearing for downstream
  agents. In-place mutation requires both flags (held-out >=5 benchmark-disjoint
  tasks); without the held-out set the run hard-refuses and points you at
  proposed.md.
- **Don't use bootstrap output without strengthening it.** Both
  `--bootstrap-from-skill` and `--bootstrap-from-routing` have the optimizer
  model invent success criteria — generic and weak by default. Review and
  tighten the judges before SkillOpt optimizes against them, or it trains the
  skill toward benchmark artifacts instead of real quality.
- **Don't skip `--split 1:1:1` on a ~15-task starter.** The default `4:1:5`
  split drops the validation set below the `D_sel >= 5` floor and the run
  aborts with `d_sel_too_small`.

## Contract

`runSkillOpt(opts)` returns:
```
{
  outcome: 'accepted' | 'no_improvement' | 'aborted' | 'errored',
  receipt: {
    run_id, skill_sha8, benchmark_sha8, models, cost,
    baseline_sel_score, best_sel_score,   // real measured baseline (no longer hardcoded 0)
    baseline_test_score, test_score,      // final held-out test-split eval
  },
  finalText: string,
  mutatedSkillFile: boolean,
  proposedPath?: string
}
```

## Related skills

- `skillify` — scaffolds a new skill (use BEFORE skillopt)
- `skillpack-check` — audits skill conformance (item 13 surfaces skillopt status)
- `conventions/quality.md` — output quality standards skillopt enforces via judges

## Dónde encaja

- Categoría: [Herramientas para desarrolladores](https://skillsagentes.com/categorias/herramientas-desarrollo.md) — Skills que cambian cómo tu agente escribe, revisa y despliega código.
- Creador: [garrytan](https://skillsagentes.com/creators/garrytan.md) — 134 skills en el directorio
- [Todas las skills](https://skillsagentes.com/skills.md)
- [Ranking de instalaciones](https://skillsagentes.com/ranking.md)

## Otras skills del mismo repositorio

- [Setup](https://skillsagentes.com/skills/garrytan/gbrain/setup.md): Configura GBrain con auto-aprovisionamiento de Supabase o PGLite, inyección en AGENTS.md y primera importación.
- [Maintain](https://skillsagentes.com/skills/garrytan/gbrain/maintain.md): Chequeos de salud del brain: aplicación de back-links, auditoría de citas, validación de filing, detección de info obsoleta, páginas huérfanas y benchmarks.
- [Schema Unify](https://skillsagentes.com/skills/garrytan/gbrain/schema-unify.md): Migra un brain de gbrain-base a la taxonomía de 14 tipos canónicos de gbrain-base-v2 usando gbrain onboard --check y el handler Minion unify-types.
- [Retrieval Reflex](https://skillsagentes.com/skills/garrytan/gbrain/retrieval-reflex.md): Cuándo y qué recuperar: abre la página del brain de una entidad relevante antes de responder desde memoria.
- [Minion Orchestrator](https://skillsagentes.com/skills/garrytan/gbrain/minion-orchestrator.md): Skill unificado de Minions para jobs deterministas de shell y orquestación de subagentes LLM: cola durable, observable y controlable, más la doctrina de ejecución durable para operaciones largas.

---

Skills Agentes · [Índice de páginas en markdown](https://skillsagentes.com/sitemap.md) · [Inicio](https://skillsagentes.com/index.md)