Camera voice viewPro

A voice call with the camera on. The agent marks what it sees with viewfinder brackets and labels, a pointer glides to whatever it is talking about, and its captions roll over the feed.

Markdown

On this page

ProA pour over coached through the camera. Each answer marks what it is about, the pointer glides there, and captions roll over the feed.

Installation

$ pnpm dlx shadcn@latest add @chatcn-pro/camera-voice-view

Usage

tsx

import { CameraVoiceView } from "@/components/ui/chatcn/pro/camera-voice-view"

<CameraVoiceView
  media={cameraStream}
  status="live"
  voiceState={voice.state}
  segments={voice.segments}
  marks={vision.marks}
  pointing={vision.focus}
  captureKey={vision.frameId}
  onEnd={voice.end}
/>

Marks use 0 to 1 coordinates of the camera frame, the same frame your vision model received, not of the box. The view places the frame covered or contained, works out the crop and puts every mark on its spot at any size, including a phone held upright. With a mirrored front camera the brackets flip with the picture and the labels stay readable. Change captureKey whenever the agent grabs a frame and a viewfinder flashes once.

Examples

Held upright

ProThe same call on a phone. The frame is cropped to the middle third and the marks still land on the right spots.

States

ProPointing, looking, starting, camera off, blocked, dropped, ended and a mirrored front camera.

Props

Prop	Type	Notes
`media`	`MediaStream \| ReactNode`	The camera stream, or any node that fills its box
`mediaAspect`	`number`	Frame width over height. Read from the stream. Default 16 / 9
`mirrored`	`boolean`	Front camera self view. Marks follow the mirror
`fit`	`"cover" \| "contain"`	Default `cover`
`status`	`"starting" \| "live" \| "off" \| "blocked" \| "error" \| "ended"`	The camera and the call
`voiceState`	`VoiceOrbState`	The agent's voice. Default `listening`
`segments`	`TranscriptSegment[]`	Captions
`marks`	`CameraMark[]`	`{ id, label, detail?, x, y, width?, height? }`. A point without a size
`pointing`	`string`	The mark the agent talks about. Only it shows its label
`captureKey`	`string \| number`	Change it when a frame is taken
`looking`	`boolean`	The status reads "Looking"
`agentName`, `userLabel`	`string`	Status and captions
`startedAt`, `endedAt`	`number`	Epoch ms, for the timer
`muted`, `captions`	`boolean`	Each with a default and a change handler
`onCameraChange`, `onFlipCamera`	`function`	Add the camera and switch camera buttons
`onEnd`, `onReconnect`, `onRetryCamera`, `onStart`	`() => void`	Call and camera actions
`blockedMessage`, `errorMessage`	`ReactNode`	Say how to fix it

Keyboard

Every control is a button in Tab order: switch camera, then mute, captions, camera and End in the bar. Nothing on the feed takes focus.

Accessibility

The view is a labelled region. The feed is hidden from assistive tech; what the agent sees is a list named "What Brew coach sees", and each move of the pointer is announced once: "Pointing at Water level, High".
State changes are announced; the timer never is. Blocked and dropped are alerts with the fix, and the call keeps going by voice whenever the camera cannot.
Chips, brackets and captions on the picture use the fixed on media pair with a shadow, so they read on any frame in both themes.

Motion

The pointer glides between marks over 450 ms on the in out curve and lands with one ring; marks rise in over 180 ms. The viewfinder flash on a new frame settles from 1.03 scale in 650 ms.
Under reduced motion the pointer moves instantly and only fades, there is no ring, and the flash is opacity only.