# Gaia Submission

> Recorre el flujo completo de benchmark→envío de GAIA, desde la resolución de claves hasta generar un paquete compatible con HAL.

Fuente: https://skillsagentes.com/skills/ruvnet/ruflo/gaia-submission
Markdown: https://skillsagentes.com/skills/ruvnet/ruflo/gaia-submission.md

Repositorio: https://github.com/ruvnet/ruflo
Autor: ruvnet
Licencia: MIT
Actualizado: el mes pasado
Coste de contexto: 30 tok instalada, 1.3k tok al activarse, 1.3k tok con todos los archivos del bundle
Bundle: 1 archivo, 5 KB
Permisos que pide: bash mcp__plugin_ruflo-core_ruflo__memory_store mcp__plugin_ruflo-core_ruflo__memory_search mcp__plugin_ruflo-core_ruflo__memory_list mcp__plugin_ruflo-core_ruflo__hooks_post_task mcp__plugin_ruflo-core_ruflo__hooks_pre_task

## Instalación

Un skill son archivos markdown: los mismos archivos valen para cualquier agente y lo único que cambia es el directorio de destino, es decir la bandera `--agent`. Añade `-g` para instalarlo en todos los proyectos de la máquina.

```bash
# Claude Code
npx -y skills add ruvnet/ruflo --skill gaia-submission --agent claude-code
# Cursor
npx -y skills add ruvnet/ruflo --skill gaia-submission --agent cursor
# Codex
npx -y skills add ruvnet/ruflo --skill gaia-submission --agent codex
# Gemini CLI
npx -y skills add ruvnet/ruflo --skill gaia-submission --agent gemini
# Windsurf
npx -y skills add ruvnet/ruflo --skill gaia-submission --agent windsurf
# Cline
npx -y skills add ruvnet/ruflo --skill gaia-submission --agent cline
```

## Qué hace

- Valida el entorno (claves API, Node, CLI) antes de ejecutar el benchmark
- Estima el costo de la corrida y pide confirmación si supera $5
- Ejecuta el benchmark y empaqueta los resultados en un paquete firmado (Ed25519) compatible con HAL, tras una auditoría de integridad
- Compara el resultado contra el leaderboard y persiste los aprendizajes

## Cuándo usarla

- Quieres ejecutar un benchmark y enviar resultados al leaderboard de HAL
- Quieres empaquetar un archivo de resultados existente en un paquete de envío
- Necesitas confirmar que tu entorno está listo para una corrida de benchmark

## Qué la activa

- "Prepara y envía una corrida de GAIA al leaderboard de HAL"
- "Empaqueta estos resultados de GAIA para enviarlos"

## Antes de instalar

- Requiere ANTHROPIC_API_KEY, HF_TOKEN y Node.js 20+.
- Necesita en el PATH: npx
- Variables de entorno: COST, LEVEL, LIMIT, MODEL, MODELS, NOTES, PASSED, RATE, TOTAL, VOTING
- reads environment config

## Archivos

- SKILL.md — 5 KB

## SKILL.md

Reproducido tal cual desde ruvnet/ruflo bajo MIT. Esta sección es el documento original y está en inglés.

# GAIA Submission Skill

Walk Claude Code through every step needed to go from a clean environment to a
signed, HAL-compatible submission package ready to upload to the Princeton
GAIA leaderboard.

## When to use

When the user wants to:
- Run a benchmark and submit results to the HAL leaderboard
- Package an existing results file into a submission archive
- Confirm their environment is ready for a benchmark run

## Prerequisites

Before starting, confirm these are available:

| Requirement | Check |
|-------------|-------|
| `ANTHROPIC_API_KEY` | `echo ${ANTHROPIC_API_KEY:0:8}…` (should show `sk-ant-…`) |
| `HF_TOKEN` | `echo ${HF_TOKEN:0:5}…` (should show `hf_…`) |
| Node.js 20+ | `node --version` |
| CLI built | `node v3/@claude-flow/cli/bin/cli.js --version` |

## Phase 1 — Validate environment

```bash
# Run all pre-flight checks
/gaia validate
```

If any check fails, resolve it before continuing.

## Phase 2 — Estimate cost and confirm

Ask the user for their configuration:
- Level (default: 1)
- Question limit (default: 53 for a quick run, 165 for the full L1 set)
- Models (default: `claude-sonnet-4-6`)
- Self-consistency voting (default: 1; use 3 for L2/L3)

```bash
/gaia cost --level=$LEVEL --limit=$LIMIT --models=$MODELS --voting=$VOTING
```

If projected cost > $5, show the estimate and ask: "This run will cost
approximately $X. Proceed? (y/N)"

## Phase 3 — Run the benchmark

```bash
/gaia run --level=$LEVEL --limit=$LIMIT --models=$MODELS --voting=$VOTING
```

While running, progress is reported every 5 questions:
```
[12/53] 22.7% (5 passed of 22 scored) — est. remaining: $0.18
```

Store the run summary in memory for history tracking:
```bash
npx @claude-flow/cli@latest memory store \
  --namespace gaia-runs \
  --key "run-$(date +%Y%m%d-%H%M)" \
  --value '{"level":$LEVEL,"model":"$MODEL","total":$TOTAL,"passed":$PASSED,"pass_rate":$RATE,"est_cost_usd":$COST}'
```

## Phase 4 — Package for submission

```bash
/gaia submit --results=~/.cache/ruflo/gaia/results-latest.json
```

This produces:
```
submission-<date>-<sha>/
├── results.jsonl        ← HAL-compatible, one JSON per line
├── trajectories.jsonl   ← full agent traces
├── metadata.json        ← harness info, model, tool catalogue
├── audit-report.json    ← ADR-167 pre-submission exploit-audit report
├── manifest.md.json     ← Ed25519-signed witness (signs audit-report.json's hash)
└── README.md            ← human summary + leaderboard comparison
```

### Integrity gate — the audit runs before signing (ADR-167)

Post-RDI (UC Berkeley broke 8 agent benchmarks — GAIA to ~98% — without solving
a task), a signature alone is not enough: **it proves the bytes are untampered,
not that the score was earned.** `/gaia submit` therefore runs a deterministic,
$0 exploit audit before signing and **refuses to build the leaderboard package
on a CRITICAL failure** unless `--allow-dirty` is passed. The audit report is
signed *into* the witness manifest as an ADR-103 fix marker, so a ruflo GAIA
submission attests both transport-integrity *and* earning-integrity.

If the gate blocks, treat it as a real finding — inspect `audit-report.json`
(answer-leakage, no-work pass, oracle leakage, grader monkey-patching, an
answer-key read outside the dataset dir, or dynamic eval/exec of task content in
the runner) rather than reaching for `--allow-dirty`. The static source-scan
family (answer-key-reads, dynamic-eval, judge-injection) enforces today with no
trajectory instrumentation; the trajectory-fed checks the current schema cannot
feed are reported as `harness_gap`s (ADR-167 §7), not passes.

## Phase 5 — Compare and report

```bash
/gaia leaderboard --level=$LEVEL
/gaia history
```

Interpret the gap between ruflo's score and the leaderboard top-10.
Identify the primary failure mode (tool gap, reasoning miss, extraction bug)
using the `/gaia-debugging` skill if needed.

## Phase 6 — Persist learnings

```bash
npx @claude-flow/cli@latest hooks post-task \
  --task-id "gaia-submission-$(date +%Y%m%d)" \
  --success true \
  --train-neural true
```

Store any discovered patterns:
```bash
npx @claude-flow/cli@latest memory store \
  --namespace gaia-patterns \
  --key "submission-notes-$(date +%Y%m%d)" \
  --value "Level $LEVEL, $MODEL: $NOTES"
```

## Extensibility note

This skill is intentionally structured to be benchmark-agnostic. The phase
headers (validate → estimate → run → package → compare → learn) apply to
SWE-bench, WebArena, and HumanEval with only phase 3-4 details changing.

## Dónde encaja

- Categoría: [Testing y QA](https://skillsagentes.com/categorias/testing-qa.md) — Flujos de testing unitario, de integración y end-to-end.
- Creador: [ruvnet](https://skillsagentes.com/creators/ruvnet.md) — 275 skills en el directorio
- [Todas las skills](https://skillsagentes.com/skills.md)
- [Ranking de instalaciones](https://skillsagentes.com/ranking.md)

## Otras skills del mismo repositorio

- [Harness Gepa](https://skillsagentes.com/skills/ruvnet/ruflo/harness-gepa.md): Inspecciona y audita genomas GEPA: carga y valida un genoma, renderiza el system prompt que compila, o clasifica los modos de fallo de una transcripción de ejecución.
- [Deepseek Reason](https://skillsagentes.com/skills/ruvnet/ruflo/deepseek-reason.md): Completion en modo razonamiento contra deepseek-reasoner (R1) de DeepSeek. Devuelve el chain-of-thought por separado de la respuesta final. Lee DEEPSEEK_API_KEY y degrada si falta o la API no responde.
- [Deepseek Chat](https://skillsagentes.com/skills/ruvnet/ruflo/deepseek-chat.md): Completion de un solo turno contra el modelo deepseek-chat de DeepSeek vía /v1/chat/completions. Lee DEEPSEEK_API_KEY y degrada con status:degraded si falta o la API no responde. Para tareas sin razonamiento.
- [Adr Index](https://skillsagentes.com/skills/ruvnet/ruflo/adr-index.md): Construye o reconstruye el índice de ADRs y su grafo de dependencias ejecutando scripts/import.mjs, en vez de cientos de llamadas MCP.
- [Agntcy Status](https://skillsagentes.com/skills/ruvnet/ruflo/agntcy-status.md): Muestra el estado de la integración AGNTCY/SLIM/CASA: si los paquetes están instalados, qué transporte está activo y si el enforcement de CASA está habilitado.

---

Skills Agentes · [Índice de páginas en markdown](https://skillsagentes.com/sitemap.md) · [Inicio](https://skillsagentes.com/index.md)