Runbook — remote crawl → grade → prioritized report

A complete worked example: take a domain from zero to a prioritized SEO report using the remote Screaming Frog crawler. Nothing runs locally. Substitute your own domain for example.com.

Prereq: the remote MCP is connected (server pulse-screamingfrog, tools mcp__<name>__* visible). Setup is in SKILL.md → “Setup”. If the tools aren’t there, the MCP isn’t wired — fix that first; there is no local fallback crawler.

1. List before you crawl (cache-first)

list_crawl_records(domain: "example.com")
# → { records: [{ crawled_at, age_days: 0.2, verdict: "success", config: "full-js", archive_bytes: 28938568 }, … ] }

If a recent success archive exists and the user didn’t ask for a fresh crawl, skip to step 4 — a full crawl is expensive.

2. Queue the crawl

sf_crawl(domain: "example.com", config_name: "full-js")
# → { started: true, pid: 222436, concurrent_slot: "1/4", note: "…runs in background; poll sf_crawl_progress" }

full-js renders JavaScript (right for modern sites); use text-only for static/server-rendered sites when you want speed. A cache hit returns { started: false, cached_crawl } — that’s fine, go to step 4.

3. Poll until settled (back off — don’t spin)

sf_crawl_progress()
# → { crawls: [{ domain: "example.com", running: true, progress: { completed: 180, waiting: 40, percent: "50" } }] }
# repeat every ~15s (small) / 30s (large) / 60s+ (big JS) until running: false

4. Confirm the verdict (never grade before this)

crawl_verdict(domain: "example.com")
# → { results: [{ complete: true, verdicts: ["✓ example.com success (1309710 bytes)"], summary: { successful: "1/1" } }] }

Capture the verdict word. Only success is a crawl you grade. blocked / paused / failed → still fetch it if you want to look, but pass --crawl-verdict <that> so the grader returns BLOCKED, not a misleading REVIEW/FAIL, and tell the user (offer a force: true re-crawl).

5. Fetch + grade (one step)

get_download_link(domain: "example.com")
# → { download_url: "https://…/downloads/<uuid>", expires_in_hours: 12 }

node scripts/nkpfrog.mjs --fetch "<download_url>" \
  --url https://example.com --crawl-verdict success

--fetch downloads the .tar.gz, unpacks it, descends into the dated subfolder, and grades — no manual curl/tar, no zip (the archive is gzipped tar). Manual equivalent if you prefer:

curl -fsSL "<download_url>" -o crawl.tar.gz && tar -xf crawl.tar.gz -C ./sf-export
node scripts/nkpfrog.mjs ./sf-export --url https://example.com --crawl-verdict success

Representative output:

nkpfrog — Screaming Frog crawl audit
site:    https://example.com
verdict: REVIEW
crawled: 294 URLs
issues:  0 critical · 4 high · 8 medium · 26 low  (SEO-adjusted priority)

── PRIORITIZED ISSUES (38) ──
[HIGH]  8 URL(s)  Structured Data: Rich Result Validation Errors
[HIGH]  5 URL(s)  Directives: Noindex
[MEDIUM] 30 URL(s)  Page Titles: Over 60 Characters
[LOW]  253 URL(s)  Security: Missing Content-Security-Policy Header
...

What the priority column did: issues_overview_report.csv is re-ranked by the SEO Priority Framework (SKILL.md), not SF’s raw labels. The 253-URL “Missing Content-Security-Policy” sits in LOW even though it’s the highest-volume finding — security headers don’t affect rankings. A 5-URL accidental Noindex is promoted to HIGH because it blocks indexation. Volume ≠ priority.

6. Deliverables (optional)

# an action plan instead of a flat list — findings grouped + routed to the other toolkit skills
node scripts/nkpfrog.mjs ./sf-export --url https://example.com --enhanced

# slice one report out — "I just want the sitemap, plain text" / "the media report"
node scripts/nkpfrog.mjs ./sf-export --report sitemap
node scripts/nkpfrog.mjs ./sf-export --report images --format tsv --out media.tsv
node scripts/nkpfrog.mjs ./sf-export --list-reports        # everything sliceable in this archive

The grade run writes report.html (self-contained, theme-aware), report.txt, and nkpfrog.json to --out (default ~/Desktop/toolkit-reports/<domain>/nkpfrog).

Interpreting & going deeper

Once you have the prioritized list, drill in — the export already contains the per-URL detail:

You want to…	Do this
Understand what an issue means / how to fix it	`references/deep/issues-reference*.md`
Read a particular CSV’s columns	`references/deep/csv-exports*.md`
The URLs behind a finding	`--report <slug>` (e.g. `directives_noindex`), or open the `issues_reports/<slug>.csv`
Which pages link to a broken/noindexed URL	open the issue’s `_inlinks.csv`, or run klink*
Per-page depth on a finding	route to orbeetle / klink / abra / oranguru / porygon / axew / rapidash (or just `--enhanced`)
Confirm analytics coverage across the crawl	feed the URL list to ampharos; audit GTM with beru

Presenting findings (recommended structure)

Crawl overview — total URLs, verdict.
Critical / High — indexation blockers and ranking-impacting items, with URL counts.
Medium — crawl-efficiency and on-page items.
Quick wins — low-effort, visible impact (missing meta descriptions, oversized images).
Excluded from SEO scope — security headers and pure-UX items; mention but don’t rank as SEO priorities.