Browser tool

A browser agent at work: the steps it takes (open, click, type, scroll) with the running one live, and a screenshot thumbnail with the cursor where it last acted.

Markdown

On this page

An agent navigates, clicks, types and scrolls, with a live screenshot and cursor.

The plan streams in as the tool input, then progress arrives as preliminary output: which step runs and what is on screen. Select the thumbnail to see the screenshot full size.

Installation

$ pnpm dlx shadcn@latest add https://chatcn.dev/r/tool-browser.json

Usage

tsx

import { ToolBrowser } from "@/components/ui/chatcn/tool-browser"

<ToolBrowser
  state="input-available"
  steps={[
    { kind: "navigate", url: "https://preview-482.palletapp.dev/harbor-goods" },
    { kind: "click", target: "Settings", x: 0.09, y: 0.49 },
    { kind: "type", target: "Team name", text: "Harbor Goods" },
  ]}
  screenshot={screenshotUrl}
/>

tsx

import { ToolBrowserPart, toolBrowserRenderers } from "@/components/ui/chatcn/tool-browser"

// One part straight from AI SDK or the chatcn contract
<ToolBrowserPart part={part} onApprove={approve} onDeny={deny} />

// Or map tool names to renderers for a whole message
renderToolPart(part, { ...toolBrowserRenderers })

The part reader takes a list of steps (or actions) or a single action such as { action: "click", element: "Sign in" }, and understands common names from Playwright, browser-use and Stagehand (goto, fill, press, scroll_down). Send url, title, screenshot and steps with live statuses as output; mark it preliminary while the session runs so the header keeps its live state. It handles browser, browser_action, browser_use and the single action Playwright tools (browser_click, browser_type, ...).

Examples

Every state

While the plan streams, every step waits. Running, earlier steps are done and the current one shimmers. A click that spends money asks first. Errors mark the step that failed and say how to fix it.

Streaming, running, approval, done, error and denied.

Props

Prop	Type	Notes
`steps`	`BrowserStep[]`	The actions in order. Labels are written from the fields
`url`	`string`	The current page. Defaults to the last navigate step
`pageTitle`	`string`	Shown in the full size view
`screenshot`	`string \| ReactNode`	An image URL or data URL, or an element sized to fill its box
`screenshotAlt`	`string`	Defaults to Screenshot of <url>
`cursor`	`{ x, y }`	0 to 1. Defaults to the current step's point
`aspectRatio`	`number`	Defaults to 16 / 10
`loading`	`boolean`	The page loading bar. Defaults to on while a navigate step runs
`state`	`ToolDisplayState`	The ToolCall lifecycle: streaming, running, approval, output, error, denied, cancelled
`approval`, `decision`		Approval request and the user's answer, as on ToolCall
`onApprove`, `onDeny`, `onRetry`		Same callbacks as ToolCall
`errorText`	`ReactNode`	Shown on `output-error`
`variant`	`"card" \| "inline"`	Inline sits on a trace rail
`open`, `defaultOpen`, `onOpenChange`		Opens by itself while running and on error

BrowserStep

Field	Type	Notes
`kind`	`navigate \| back \| click \| hover \| type \| select \| key \| scroll \| wait \| screenshot`	Picks the icon and the verb: Clicking while it runs, Click once done
`target`	`string`	What it acts on, as a person would say it: Sign in, Email
`url`, `text`, `key`, `direction`		Per kind: the address, typed text, key chord, scroll direction
`secret`	`boolean`	Typed text shows as dots
`x`, `y`	`number`	0 to 1 from the top left. Moves the cursor marker there
`status`	`pending \| running \| done \| error`	Defaults from the tool state
`label`	`ReactNode`	Replaces the generated label

Keyboard

Key	Action
`Enter`, `Space`	Expand or collapse the details; on the thumbnail, open it full size
`Esc`	Close the full size view
`Tab`	Move through the header, the thumbnail and approval actions

Accessibility

Steps are an ordered list; each row carries its status as text for screen readers (Done, In progress, Failed, Not run).
The thumbnail is a button named after the page it shows, and the full size view is a titled dialog.
Typed secrets never reach the DOM, only dots.
The cursor marker and loading bar are decorative and hidden from assistive tech.

Motion

The cursor glides to each point over 450 ms with the in-out curve, moving with transform only; a ring marks each click as it lands.
The running step's verb uses the one shimmer; the loading bar sweeps under the address while a page opens.
Under reduced motion the cursor jumps, the ring and the sweep are off, and the shimmer settles to plain text.