Crawl progressPro

A website crawl as a radial tree: one ring per depth of links, live counts and a log that says why pages were skipped.

Markdown

On this page

ProA live crawl of a docs site, ring by ring, with a log that says why pages were skipped.

Installation

$ pnpm dlx shadcn@latest add @chatcn-pro/crawl-progress

Usage

tsx

import { CrawlProgress } from "@/components/ui/chatcn/pro/crawl-progress"

<CrawlProgress root="docs.example.com" pages={pages} status="crawling" onPause={pause} />

The start page sits in the middle and every page lands on the ring for its depth, inside the arc of the page that linked it, so a section that fans out or dead ends is visible at a glance. Fetched pages fill in ink, queued ones are hollow, skips are amber diamonds and failures red squares, so the map still reads in grayscale. Point at a depth row to light its ring. Skips are grouped by reason, and the log keeps the reason next to every page.

Examples

States

ProFinished, queued, paused and an unreachable site.

Props

Prop	Type	Notes
`root`	`string`	Where the crawl started
`pages`	`CrawlPage[]`	`{ id, url, depth, parent?, state, reason?, at? }`; state is `queued`, `fetching`, `fetched`, `skipped` or `failed`
`status`	`"idle" \| "crawling" \| "paused" \| "done" \| "error"`	Default crawling
`error`, `onRetry`	`string`, `() => void`	Say how to fix it
`maxDepth`	`number`	Rings to draw, default the deepest page
`onPause`, `onResume`, `onStop`	`() => void`	Each adds its control
`logLimit`	`number`	Log lines kept, default 40
`labels`	`Partial<CrawlProgressLabels>`	Every string

Keyboard

Key	Action
Tab	Pause or resume, stop, then the log
Arrow keys	Scroll the log once it has focus

Accessibility

The map is an image named with the fetched share and depth; the counts, depth rows and log carry the same facts as text.
The status is a polite live region. The log is a log region that stays quiet while crawling and announces once paused or finished.
Every log line names its state in words before the URL.

Motion

Marks change fill in 150 ms as pages are fetched, and depth bars grow with a 300 ms width transition. Nothing travels across the map, so a crawl of thousands of pages stays calm and reduced motion needs no change.