Site crawl (Screaming Frog) · Playbook
Screaming Frog crawl remediation — high-level pointers (local fallback)
nkpfrog tells you what the crawl found and its SEO-adjusted priority. The detailed, current house
remediation procedures — how to batch-fix redirects, the canonical policy, which findings block indexing —
live in the team wiki (set OUTLINE + OUTLINE_API_URL, scope with NKPFROG_WIKI_COLLECTION, and the
report links them directly). This file is the minimal high-level fallback for when the wiki isn’t
configured: direction only, deliberately not step-by-step, so there’s nothing to drift out of sync.
- Indexing-breakers first —
noindexon pages that should rank, blocked-by-robots, canonical pointing elsewhere. These silently remove pages from search; fix before cosmetic issues regardless of count. - Response codes — 5xx and broken internal links hurt users and crawl budget; 3xx chains should be single-hop 301s to the final URL (compose abra for the full chain).
- Redirects — map old→new once, permanent (301), no loops, no chains; verify after a migration.
- On-page at scale — missing/duplicate titles & meta, thin content, missing H1 across many URLs is a template problem, not a per-page one — fix the template (compose orbeetle / oranguru per page).
- Priority is SEO-adjusted — security-header findings are demoted, indexing/redirect/response issues promoted, vs. Screaming Frog’s native priority. Trust the nkpfrog priority column over SF’s.
- A blocked/paused crawl is not gradeable — if the crawl didn’t complete cleanly, the findings are artifacts of the block, not real defects. Re-crawl before acting.
Full playbook → the team wiki’s crawl-remediation reference (batch procedures, canonical policy, agent-safety notes).