# Archive Crawler

> Archivista universal para archivos personales (Dropbox/B2/Gmail-takeout/disco local). Filtra contenido de alto valor y lo muestra de forma interactiva; exige un allow-list scan_paths explícito en gbrain.yml.

Fuente: https://skillsagentes.com/skills/garrytan/gbrain/archive-crawler
Markdown: https://skillsagentes.com/skills/garrytan/gbrain/archive-crawler.md

Repositorio: https://github.com/garrytan/gbrain
Autor: garrytan
Licencia: MIT
Actualizado: hace 3 meses
Coste de contexto: 75 tok instalada, 2.7k tok al activarse, 2.8k tok con todos los archivos del bundle
Bundle: 2 archivos, 11 KB
Permisos que pide: ninguno declarado

## Instalación

Un skill son archivos markdown: los mismos archivos valen para cualquier agente y lo único que cambia es el directorio de destino, es decir la bandera `--agent`. Añade `-g` para instalarlo en todos los proyectos de la máquina.

```bash
# Claude Code
npx -y skills add garrytan/gbrain --skill archive-crawler --agent claude-code
# Cursor
npx -y skills add garrytan/gbrain --skill archive-crawler --agent cursor
# Codex
npx -y skills add garrytan/gbrain --skill archive-crawler --agent codex
# Gemini CLI
npx -y skills add garrytan/gbrain --skill archive-crawler --agent gemini
# Windsurf
npx -y skills add garrytan/gbrain --skill archive-crawler --agent windsurf
# Cline
npx -y skills add garrytan/gbrain --skill archive-crawler --agent cline
```

## Qué hace

- Escanea árboles de archivos personales (Dropbox, B2, Gmail takeout, discos locales) buscando escritura, ideas y relaciones de valor
- Aplica un filtro de 'oro' que muestra contenido valioso e ignora ruido (configs, binarios, recibos, spam)
- Crea una página manifiesto en projects/<archive-slug>/STATUS.md con inventario y estado de triage por ítem
- Muestra ítems de a uno, captura la reacción exacta del usuario y crea páginas del cerebro para lo que valga la pena
- Se niega a ejecutarse si no hay un allow-list explícito archive-crawler.scan_paths: en gbrain.yml

## Cuándo usarla

- El usuario pide explorar o rastrear un archivo personal antiguo (Dropbox, B2, disco duro, volcado de Gmail)
- Se quiere encontrar contenido de alto valor entre grandes cantidades de archivos personales viejos

## Qué la activa

- "Rastrea mi archivo de Dropbox en busca de escritos viejos"
- "Encuentra oro en mi volcado de disco duro"
- "Escanea mi Gmail takeout por ideas y correspondencia importante"
- "Mina mis archivos viejos en busca de ideas de negocio"

## Antes de instalar

- Requiere definir explícitamente archive-crawler.scan_paths en gbrain.yml antes de ejecutar; sin eso el skill se rehúsa a correr.
- Necesita en el PATH: python3
- makes network requests

## Archivos

- SKILL.md — 10 KB
- routing-eval.jsonl — 666 B

## SKILL.md

Reproducido tal cual desde garrytan/gbrain bajo MIT. Esta sección es el documento original y está en inglés.

# archive-crawler — The Universal Archivist

> **Convention:** see [conventions/quality.md](../conventions/quality.md) for
> citation rules, exact-phrasing requirements when capturing the user's
> reactions, and back-link enforcement.
>
> **Convention:** see [_brain-filing-rules.md](../_brain-filing-rules.md) —
> this skill is **schema-generic**: it reads the user's filing rules from
> the rules JSON instead of hardcoding any specific era / archive layout.

## Safety gate (REQUIRED, no exceptions)

archive-crawler refuses to run unless `archive-crawler.scan_paths:` is
explicitly set in `gbrain.yml`. This is a deliberate safety fence against
the agent over-scoping a scan and ingesting sensitive content (tax PDFs,
medical records, credentials).

```yaml
# gbrain.yml — the allow-list is mandatory
archive-crawler:
  scan_paths:
    - ~/Documents/writing/
    - ~/Dropbox/Archive/
    - /mnt/backup/old-letters/
  # Optional deny-list inside the allow-list:
  # deny_paths:
  #   - ~/Documents/finances/
  #   - ~/Documents/medical/
```

If `scan_paths` is empty or missing, the skill exits with:

```
archive-crawler: refusing to run. No `archive-crawler.scan_paths:` allow-list
in gbrain.yml. Add explicit paths the agent is permitted to scan, then re-run.
This is a safety fence — the agent will not infer what's safe to read.
```

This contract is enforced by `src/core/storage-config.ts` (mirrors the
`db_tracked` / `db_only` allow-list pattern from v0.22.11 storage tiering).

## What this is

Generic engine for exploring any tree of personal content within an
explicit allow-list. Works on local mounts, Dropbox API targets,
Backblaze B2, Gmail takeouts (`.mbox`), and similar archives. Filters
for "gold" (the user's own writing, ideas, relationships) and surfaces
it interactively for review. Skips noise (system files, configs, binary
blobs).

## Concepts

### Source

A source is any tree of files to explore. Sources have:

- **type**: `local` | `dropbox` | `backblaze` | `gmail-takeout` | `mbox` | `pst`
- **root**: filesystem path, Dropbox path, B2 prefix, mbox path
- **manifest**: a brain page tracking progress at
  `projects/<archive-slug>/STATUS.md`

### Manifest

Every archive exploration gets a manifest brain page that tracks:

1. **Tree inventory** — folders / files / sizes / types
2. **Triage status** — each item: `⬜ unseen` / `👀 reviewed` /
   `✅ ingested` / `⏭️ skip` / `🔥 high-signal`
3. **User reactions** — exact quotes when they react (per
   conventions/quality.md exact-phrasing rule)
4. **Priority queue** — what to explore next, ranked
5. **Session log** — timestamped record of what was shown per session

### Gold filter

Before showing anything to the user, apply the gold filter:

| Keep (show) | Skip (note existence, don't show) |
|-------------|-----------------------------------|
| Personal writing (journals, letters, reflections, essays) | System files, configs, package.json, node_modules |
| Conversations (IM logs, email threads with substance) | Binary blobs (images / video) |
| Ideas, theses, frameworks | Receipts, invoices, tax docs |
| Relationship material (letters to / from people who matter) | Spam, newsletters, mailing-list bulk |
| Creative work (poetry, stories, code with soul) | Corrupted / null files |
| Origin stories (first versions of things that became important) | |
| Emotional content (anger, love, grief, discovery) | |

## Protocol

### Phase 1: Inventory

When pointed at a new source:

1. **Confirm scan_paths is set** (safety gate). Exit if not.
2. **Map the tree** — list folders + files + sizes + date ranges.
3. **Classify folders** — group by likely content type (writing, email,
   code, photos, docs, system).
4. **Create manifest** — write `projects/<archive-slug>/STATUS.md` with
   the full inventory.
5. **Propose priority queue** — rank folders by likely gold density.
6. **Present to user** — show the map and proposed order. Let them
   override.

### Phase 2: Crawl

Work through folders in priority order:

1. **Read before showing** — open each candidate file, apply the gold
   filter, skip noise.
2. **Show one at a time** — present gold items individually for review.
3. **Capture exact reaction** — track the user's response in the
   manifest using their exact words (per conventions/quality.md).
4. **Ingest if worth keeping** — create a brain page immediately.
5. **Update manifest** — mark item status after each interaction.
6. **Never re-show** — check the manifest before presenting anything.

### Phase 3: Ingest

When an item is worth keeping, file it by **primary subject** per
`_brain-filing-rules.md`:

- User's own writing / ideas / origin-story content → `originals/<slug>.md`
- Reflections / personal-life content → `personal/<slug>.md`
- Product / business ideas → `ideas/<slug>.md`
- Letters or threads about a specific person → `people/<person>/timeline`
  back-link plus the letter at `personal/<slug>.md` or `originals/<slug>.md`

**The skill is schema-generic.** It does NOT bake in any specific
era-folder structure (e.g., `originals/archive/` for pre-2003,
`originals/yc-era/` for post-2019, etc.). The user's filing rules from
`_brain-filing-rules.json` are read at runtime; the agent decides per-page
where content lands within those sanctioned directories.

Brain page format:

```markdown
---
title: "[Title or first line]"
type: original
source_type: "[local|dropbox|backblaze|gmail-takeout|mbox|pst]"
source_path: "[path within the allow-listed scan_paths]"
date: "YYYY-MM-DD"  # date from the file metadata or content
people: ["person-1", "person-2"]
tags: ["tag-1", "tag-2"]
---

# [Title]

[Summary: what it is, when it's from, why it matters]

**User's reaction:** [exact quote, no paraphrasing]

## Context

[Cross-links to people, concepts, projects.]

---

[Raw source material below the line — full text]
```

## File-type handlers

### Plain text / HTML / Markdown
Read directly. Strip HTML tags for display.

### `.mbox` (email archives)

```python
import mailbox
mbox = mailbox.mbox('/path/to/file.mbox')
for msg in mbox:
    body = ''
    if msg.is_multipart():
        for part in msg.walk():
            if part.get_content_type() == 'text/plain':
                body = part.get_payload(decode=True).decode('utf-8', errors='replace')
                break
    else:
        body = msg.get_payload(decode=True).decode('utf-8', errors='replace')
    # Apply gold filter
```

### `.doc` / `.docx`

```bash
# .docx (modern)
python3 -c "
import zipfile, xml.etree.ElementTree as ET
with zipfile.ZipFile('/path/to/file.docx') as z:
    tree = ET.parse(z.open('word/document.xml'))
    print(''.join(t.text or '' for t in tree.iter('{http://schemas.openxmlformats.org/wordprocessingml/2006/main}t')))
"

# .doc (legacy, requires antiword or catdoc)
antiword /path/to/file.doc 2>/dev/null || catdoc /path/to/file.doc 2>/dev/null
```

### `.pst` (Outlook archives)

```bash
# Validate first; many PSTs are null bytes
python3 -c "
with open('/path/to/file.pst', 'rb') as f:
    print('Valid PST' if f.read(4) == b'!BDN' else 'CORRUPT/NULL')
"
# If valid:
readpst -o /tmp/pst-output /path/to/file.pst
```

### `.zip` / `.tar` / `.tar.gz`

Extract to a temp dir, then recurse through the extracted tree.

### Images

Note existence + metadata (filename, size, date). Don't show unless the
user asks. Flag scans / portraits as potentially personal.

## Manifest template

```markdown
---
title: "[Archive Name] — Ingestion Status"
type: project
created: YYYY-MM-DD
updated: YYYY-MM-DD
source_type: "[local|dropbox|...]"
scan_paths: ["paths from gbrain.yml"]
---

# [Archive Name] — Ingestion Status

## Source
- **Type:** [local|dropbox|...]
- **Allow-listed paths:** [from gbrain.yml]
- **Total files:** [N]
- **Total size:** [X GB]
- **Date range:** [earliest] — [latest]

## Inventory

### [Folder 1]
| Item | Type | Size | Status | Reaction |
|------|------|------|--------|----------|
| file1.txt | text | 2KB | ✅ ingested | 🔥 "exact quote" |
| file2.doc | doc | 15KB | ⏭️ skip | — |
| file3.html | html | 4KB | ⬜ unseen | — |

### [Folder 2]
...

## Priority Queue
1. [Highest priority — why]
2. [Next — why]
...

## Session Log

### YYYY-MM-DD — [Session topic]
- Reviewed: [list]
- Reactions: [exact quotes]
- Ingested: [brain pages created]
- Next: [what's queued]
```

## Anti-Patterns

- ❌ Running without `archive-crawler.scan_paths:` set. Hard refusal.
  This is the safety contract — never bypass.
- ❌ Hardcoding era-specific filing paths (e.g., `originals/archive/`,
  `originals/yc-era/`). Read filing rules at runtime instead.
- ❌ Re-showing items already marked in the manifest. The user's time
  is the scarcest resource.
- ❌ Paraphrasing reactions. Exact words only.
- ❌ Wrapping found content in lessons or takeaways. Let stories breathe.
- ❌ Skipping back-links when content references people / companies who
  have brain pages. Iron Law per conventions/quality.md.

## Related skills

- `skills/voice-note-ingest/SKILL.md` — same exact-phrasing pattern for
  audio capture
- `skills/idea-ingest/SKILL.md` — single-link-or-article ingest with
  the same primary-subject filing rule
- `skills/conventions/quality.md` — citations, back-links, voice

## Contract

This skill guarantees:

- Routing matches the canonical triggers in the frontmatter.
- Output written under the directories listed in `writes_to:` (when applicable).
- Conventions referenced (`quality.md`, `brain-first.md`, `_brain-filing-rules.md`) are followed.
- Privacy contract preserved: no real names, no fork-specific filesystem path literals, no upstream-fork references.

The full behavior contract is documented in the body sections above; this section exists for the conformance test.

## Output Format

The skill's output shape is documented inline in the body sections above (see "Output", "Brain page format", or equivalent). The literal section header here exists for the conformance test (`test/skills-conformance.test.ts`).

## Dónde encaja

- Categoría: [Productividad](https://skillsagentes.com/categorias/productividad.md) — Planificación, toma de notas y automatización de flujos personales.
- Creador: [garrytan](https://skillsagentes.com/creators/garrytan.md) — 134 skills en el directorio
- [Todas las skills](https://skillsagentes.com/skills.md)
- [Ranking de instalaciones](https://skillsagentes.com/ranking.md)

## Otras skills del mismo repositorio

- [Setup](https://skillsagentes.com/skills/garrytan/gbrain/setup.md): Configura GBrain con auto-aprovisionamiento de Supabase o PGLite, inyección en AGENTS.md y primera importación.
- [Maintain](https://skillsagentes.com/skills/garrytan/gbrain/maintain.md): Chequeos de salud del brain: aplicación de back-links, auditoría de citas, validación de filing, detección de info obsoleta, páginas huérfanas y benchmarks.
- [Schema Unify](https://skillsagentes.com/skills/garrytan/gbrain/schema-unify.md): Migra un brain de gbrain-base a la taxonomía de 14 tipos canónicos de gbrain-base-v2 usando gbrain onboard --check y el handler Minion unify-types.
- [Retrieval Reflex](https://skillsagentes.com/skills/garrytan/gbrain/retrieval-reflex.md): Cuándo y qué recuperar: abre la página del brain de una entidad relevante antes de responder desde memoria.
- [Minion Orchestrator](https://skillsagentes.com/skills/garrytan/gbrain/minion-orchestrator.md): Skill unificado de Minions para jobs deterministas de shell y orquestación de subagentes LLM: cola durable, observable y controlable, más la doctrina de ejecución durable para operaciones largas.

---

Skills Agentes · [Índice de páginas en markdown](https://skillsagentes.com/sitemap.md) · [Inicio](https://skillsagentes.com/index.md)