Voice session

One surface for voice and text: the orb carries the call state, a shared transcript holds spoken and typed turns, and the user can barge in or switch to text mid-conversation.

Markdown

On this page

A coding agent on a call: barge-in mid-answer, switch to text and back, one transcript.

Most voice UIs are a separate mode: you talk to an orb, then come back to a chat that never heard you. Here voice and text are the same conversation. Typed turns land in the transcript beside spoken ones, and the call stays live while you type.

Installation

$ pnpm dlx shadcn@latest add https://chatcn.dev/r/voice-session.json

Usage

The session takes plain props: a state, transcript segments, two levels and callbacks. Wire them to your realtime provider, or to useChatcnVoiceMock while you design.

tsx

import { VoiceSession } from "@/components/ui/chatcn/voice-session"
import { useChatcnVoiceMock } from "@/hooks/use-chatcn-voice-mock"

const call = useChatcnVoiceMock({ script, autoStart: true })

<VoiceSession
  title="Coding agent"
  state={call.state}
  segments={call.segments}
  inputLevel={call.getInputLevel}
  outputLevel={call.getOutputLevel}
  startedAt={call.startedAt}
  muted={call.muted}
  onMutedChange={call.setMuted}
  onInterrupt={call.interrupt}
  onSendText={call.sendText}
  onStart={call.start}
  onEnd={call.end}
  onReconnect={call.reconnect}
/>

Levels can be numbers or getters. A getter such as getLevel from useChatcnAudioLevel is read once per frame, so the orb animates at 60 fps without re-rendering the session.

With a realtime provider

Map the provider's events onto the props. Speech started on the input means listening (and a cut-off of any reply in progress), a response created means thinking, audio deltas mean speaking. Transcription deltas update the last segment's interim, and completed transcriptions settle it into text.

tsx

const mic = useChatcnMicrophone()
const input = useChatcnAudioLevel(mic.stream, { updateRate: 0 })
const output = useChatcnAudioLevel(remoteAudioElement, { updateRate: 0 })

<VoiceSession
  state={state}
  segments={segments}
  inputLevel={input.getLevel}
  outputLevel={output.getLevel}
  onInterrupt={() => session.cancelResponse()}
  onSendText={(text) => session.sendText(text)}
/>

Examples

Text mode

Some answers are better read than heard: a commit hash, a URL, a number. Text mode collapses the stage, keeps a small orb in the header so the call state stays in view, and focuses the composer.

Typed and spoken turns in one transcript, with the call still live in the header.

Connection states

Connecting shows the ring filling. A dropped connection splits the orb, keeps the transcript and offers Reconnect. A finished call shows its length and a way to start again.

Connecting, a dropped connection with the transcript kept, and a finished call.

States

State	Orb	Footer
`idle`	A still drop	Start talking
`connecting`	Thin ring with an orbiting arc	Controls, End reads Cancel
`listening`	Ripples travel inward with your voice	Controls with a live mic level
`thinking`	Smaller, breathing; the label shimmers	Controls
`speaking`	Ripples travel outward with the reply	Controls; tap the orb to interrupt
`interrupted`	Flattened, ripples frozen	Controls
`muted`	Drained to an outline	Controls, mic shown muted
`reconnecting`	The connecting ring	Controls
`error`	Split in two, with the reason	Close and Reconnect
`ended`	A still drop, with the call length	Start a new call

Barge-in

When the user talks over the agent, set state to interrupted for a moment, then listening. The orb flattens and its ripples freeze.
Mark the cut reply with interrupted: true and pass what was never said as unspoken. The transcript shows a clean cut and the word Interrupted, so the record matches what the user heard.
Sending a typed message while the agent speaks should interrupt it too. The mock engine does this.

Props

Prop	Type	Notes
`state`	`VoiceSessionState`	One of the ten states above
`segments`	`TranscriptSegment[]`	Spoken and typed turns, in order
`inputLevel`, `outputLevel`	`number \| () => number`	The user's and the agent's level
`visual`	`"orb" \| AudioVisualizerVariant`	Default `orb`. Or bars, radial, grid, line, dots
`mode`, `defaultMode`, `onModeChange`	`"voice" \| "text"`	Controlled or uncontrolled
`captions`, `defaultCaptions`, `onCaptionsChange`	`boolean`	Transcript in voice mode. Default on
`muted`, `onMutedChange`	`boolean`	Mute state from your provider
`onSendText`	`(text) => void`	Typed messages from text mode
`onInterrupt`	`() => void`	Makes the orb a button while the agent speaks
`onStart`, `onEnd`, `onReconnect`	`() => void`	Call lifecycle
`startedAt`, `endedAt`	`number`	Epoch ms, for the elapsed time
`title`, `userLabel`	`string`	Agent name in the header and transcript; your label
`errorMessage`	`ReactNode`	Shown in the error state. Say how to fix it
`controls`	`VoiceControlsProps`	Devices, mic errors and extra buttons, passed through
`placeholder`	`string`	Composer placeholder in text mode

Keyboard

Key	Action
`Tab`	Header mode switch, the orb while it can be interrupted, transcript, controls
Arrow keys	Switch between Voice and Text in the mode switch
`Enter`, `Space` on the orb	Interrupt the agent
`Enter` in text mode	Send. `Shift+Enter` adds a line; IME composition never sends

Accessibility

State changes are announced through a polite live region: Listening, Thinking, Speaking. The ticking timer is never announced.
The orb is an image named after the agent and its state, and a real button with an accessible name only while it can interrupt.
The transcript is the caption track: always available, a focusable log, with finished replies announced once.
Errors pair the split orb with words and a fix, never color alone. The collapsed stage in text mode is inert, so focus never lands in it.

Motion

The orb morphs between states on its own, easing every parameter, so any state can follow any other.
Switching to text collapses the stage over 280 ms and fades the header orb in while the title slides over with a transform. Switching back reverses it. Under reduced motion both snap.
Turning captions off scales the orb up from 144 to 224 px; it is drawn at full size, so it stays crisp.