Agents - Inspect Voice Agent turn latency and outcomes

<p><code>@cloudflare/voice</code> v0.4.0 now lets you inspect where each Voice Agent turn spends time and how it ends.</p> <figure class="nb-code-figure" data-nb-lang="ts"><pre class="astro-code astro-code-themes github-light github-dark nb-shiki-c6xiwz" tabindex="0" data-language="ts" data-nb-lang="ts"><code><span class="line"><span class="nb-shiki-140thh">client.</span><span class="nb-shiki-1t8gfj">addEventListener</span><span class="nb-shiki-140thh">(</span><span class="nb-shiki-mdbnqw">"turnmetrics"</span><span class="nb-shiki-140thh">, (</span><span class="nb-shiki-1jdh33">turn</span><span class="nb-shiki-140thh">) </span><span class="nb-shiki-1itgoe">=&gt;</span><span class="nb-shiki-140thh"> {</span></span> <span class="line"><span class="nb-shiki-140thh"> console.</span><span class="nb-shiki-1t8gfj">log</span><span class="nb-shiki-140thh">(turn.outcome, turn.turnTotalMs);</span></span> <span class="line"><span class="nb-shiki-140thh">});</span></span></code></pre></figure> <div tabindex="-1" class="heading-wrapper level-h4"><h4 id="about-the-voice-package">About the Voice package</h4><a class="anchor-link" href="#about-the-voice-package"><span aria-hidden="true" class="anchor-icon"><svg width="16" height="16" viewBox="0 0 24 24"><path fill="currentcolor" d="m12.11 15.39-3.88 3.88a2.52 2.52 0 0 1-3.5 0 2.47 2.47 0 0 1 0-3.5l3.88-3.88a1 1 0 0 0-1.42-1.42l-3.88 3.89a4.48 4.48 0 0 0 6.33 6.33l3.89-3.88a1 1 0 1 0-1.42-1.42Zm8.58-12.08a4.49 4.49 0 0 0-6.33 0l-3.89 3.88a1 1 0 0 0 1.42 1.42l3.88-3.88a2.52 2.52 0 0 1 3.5 0 2.47 2.47 0 0 1 0 3.5l-3.88 3.88a1 1 0 1 0 1.42 1.42l3.88-3.89a4.49 4.49 0 0 0 0-6.33ZM8.83 15.17a1 1 0 0 0 1.1.22 1 1 0 0 0 .32-.22l4.92-4.92a1 1 0 0 0-1.42-1.42l-4.92 4.92a1 1 0 0 0 0 1.42Z"></path></svg></span></a></div> <p>The <code>@cloudflare/voice</code> package lets you build real-time voice agents with Cloudflare Agents. It streams microphone audio to an Agent over WebSocket, transcribes speech, runs your model through <code>onTurn()</code>, converts the response to speech, and streams audio back to the caller.</p> <p>A turn moves through several stages:</p> <figure class="nb-code-figure" data-nb-lang="txt"><pre class="astro-code astro-code-themes github-light github-dark nb-shiki-c6xiwz" tabindex="0" data-language="txt" data-nb-lang="txt"><code><span class="line"><span class="nb-shiki-wvjl67">User speaks -&gt; speech-to-text -&gt; model -&gt; text-to-speech -&gt; audio</span></span></code></pre></figure> <p>Previously, the package's four aggregate metrics covered successful, non-empty speech turns. They did not show how failed, aborted, empty, or text turns ended.</p> <div tabindex="-1" class="heading-wrapper level-h4"><h4 id="turn-metrics">Turn metrics</h4><a class="anchor-link" href="#turn-metrics"><span aria-hidden="true" class="anchor-icon"><svg width="16" height="16" viewBox="0 0 24 24"><path fill="currentcolor" d="m12.11 15.39-3.88 3.88a2.52 2.52 0 0 1-3.5 0 2.47 2.47 0 0 1 0-3.5l3.88-3.88a1 1 0 0 0-1.42-1.42l-3.88 3.89a4.48 4.48 0 0 0 6.33 6.33l3.89-3.88a1 1 0 1 0-1.42-1.42Zm8.58-12.08a4.49 4.49 0 0 0-6.33 0l-3.89 3.88a1 1 0 0 0 1.42 1.42l3.88-3.88a2.52 2.52 0 0 1 3.5 0 2.47 2.47 0 0 1 0 3.5l-3.88 3.88a1 1 0 1 0 1.42 1.42l3.88-3.89a4.49 4.49 0 0 0 0-6.33ZM8.83 15.17a1 1 0 0 0 1.1.22 1 1 0 0 0 .32-.22l4.92-4.92a1 1 0 0 0-1.42-1.42l-4.92 4.92a1 1 0 0 0 0 1.42Z"></path></svg></span></a></div> <p>Each speech or text turn now produces a typed <code>VoiceTurnMetrics</code> summary with:</p> <ul> <li>A <code>turnId</code> for correlating events from the same turn.</li> <li>A terminal outcome such as <code>completed</code>, <code>no_output</code>, <code>output_limit</code>, <code>content_filtered</code>, <code>model_error</code>, <code>tts_error</code>, or <code>aborted</code>.</li> <li>Timings for important stages, including speech-to-final-transcript, model-to-first-text, TTS-to-first-audio, and total turn duration.</li> </ul> <p>These timings can overlap and are not additive. Timings for stages that a turn did not reach are omitted.</p> <p>The latest summary is available through <code>VoiceClient</code>, <code>useVoiceAgent()</code>, and <code>useVoiceInput()</code>. Voice input includes only the speech and transcription timings it can measure.</p> <p>If an agent produces no audio, you can now distinguish between the model returning no output, reaching an output limit, encountering content filtering, or failing.</p> <div tabindex="-1" class="heading-wrapper level-h4"><h4 id="additional-diagnostics">Additional diagnostics</h4><a class="anchor-link" href="#additional-diagnostics"><span aria-hidden="true" class="anchor-icon"><svg width="16" height="16" viewBox="0 0 24 24"><path fill="currentcolor" d="m12.11 15.39-3.88 3.88a2.52 2.52 0 0 1-3.5 0 2.47 2.47 0 0 1 0-3.5l3.88-3.88a1 1 0 0 0-1.42-1.42l-3.88 3.89a4.48 4.48 0 0 0 6.33 6.33l3.89-3.88a1 1 0 1 0-1.42-1.42Zm8.58-12.08a4.49 4.49 0 0 0-6.33 0l-3.89 3.88a1 1 0 0 0 1.42 1.42l3.88-3.88a2.52 2.52 0 0 1 3.5 0 2.47 2.47 0 0 1 0 3.5l-3.88 3.88a1 1 0 1 0 1.42 1.42l3.88-3.89a4.49 4.49 0 0 0 0-6.33ZM8.83 15.17a1 1 0 0 0 1.1.22 1 1 0 0 0 .32-.22l4.92-4.92a1 1 0 0 0-1.42-1.42l-4.92 4.92a1 1 0 0 0 0 1.42Z"></path></svg></span></a></div> <p>For local debugging, you can forward server lifecycle events to the browser console:</p> <div><div data-nb-tabs data-nb-sync-key="workersExamples" class><div class="relative flex border-b border-border" role="tablist" data-nb-tabs-list><span class="bg-primary pointer-events-none absolute -bottom-px h-0.5 rounded-t-sm transition-[left,width] duration-200 ease-out" data-nb-tabs-indicator aria-hidden="true"></span></div><div class="mt-3"><div role="tabpanel" data-nb-tabs-content data-nb-tab-label="JavaScript" class><figure class="nb-code-figure" data-nb-lang="js"><pre class="astro-code astro-code-themes github-light github-dark nb-shiki-c6xiwz" tabindex="0" data-language="js" data-nb-lang="js"><code><span class="line"><span class="nb-shiki-1itgoe">import</span><span class="nb-shiki-140thh"> { Agent } </span><span class="nb-shiki-1itgoe">from</span><span class="nb-shiki-mdbnqw"> "agents"</span><span class="nb-shiki-140thh">;</span></span> <span class="line"><span class="nb-shiki-1itgoe">import</span><span class="nb-shiki-140thh"> { withVoice } </span><span class="nb-shiki-1itgoe">from</span><span class="nb-shiki-mdbnqw"> "@cloudflare/voice"</span><span class="nb-shiki-140thh">;</span></span> <span class="line"></span> <span class="line"><span class="nb-shiki-1itgoe">const</span><span class="nb-shiki-dzsirb"> VoiceAgent</span><span class="nb-shiki-1itgoe"> =</span><span class="nb-shiki-1t8gfj"> withVoice</span><span class="nb-shiki-140thh">(Agent, {</span></span> <span class="line"><span class="nb-shiki-140thh"> diagnostics: {</span></span> <span class="line"><span class="nb-shiki-140thh"> browserConsole: </span><span class="nb-shiki-dzsirb">true</span><span class="nb-shiki-140thh">,</span></span> <span class="line"><span class="nb-shiki-140thh"> },</span></span> <span class="line"><span class="nb-shiki-140thh">});</span></span></code></pre></figure></div><div role="tabpanel" data-nb-tabs-content data-nb-tab-label="TypeScript" class><figure class="nb-code-figure" data-nb-lang="ts"><pre class="astro-code astro-code-themes github-light github-dark nb-shiki-c6xiwz" tabindex="0" data-language="ts" data-nb-lang="ts"><code><span class="line"><span class="nb-shiki-1itgoe">import</span><span class="nb-shiki-140thh"> { Agent } </span><span class="nb-shiki-1itgoe">from</span><span class="nb-shiki-mdbnqw"> "agents"</span><span class="nb-shiki-140thh">;</span></span> <span class="line"><span class="nb-shiki-1itgoe">import</span><span class="nb-shiki-140thh"> { withVoice } </span><span class="nb-shiki-1itgoe">from</span><span class="nb-shiki-mdbnqw"> "@cloudflare/voice"</span><span class="nb-shiki-140thh">;</span></span> <span class="line"></span> <span class="line"><span class="nb-shiki-1itgoe">const</span><span class="nb-shiki-dzsirb"> VoiceAgent</span><span class="nb-shiki-1itgoe"> =</span><span class="nb-shiki-1t8gfj"> withVoice</span><span class="nb-shiki-140thh">(Agent, {</span></span> <span class="line"><span class="nb-shiki-140thh"> diagnostics: {</span></span> <span class="line"><span class="nb-shiki-140thh"> browserConsole: </span><span class="nb-shiki-dzsirb">true</span><span class="nb-shiki-140thh">,</span></span> <span class="line"><span class="nb-shiki-140thh"> },</span></span> <span class="line"><span class="nb-shiki-140thh">});</span></span></code></pre></figure></div></div></div><script type="module" src="https://developers.cloudflare.com/home/runner/work/cloudflare-docs/cloudflare-docs/src/components/ui/tabs/Tabs.astro?astro&type=script&index=0&lang.ts"></script></div> <p>The browser console combines server lifecycle events with local microphone, connection, and playback events, including model start, first model text, first audio, and playback start. Diagnostics are off by default, and their event names and fields can change.</p> <p><code>VoiceClient</code> also exposes typed events for speech-to-text failures, connection errors, and model outcomes. The SDK removes known content fields and does not read arbitrary provider responses, but custom error messages must not contain sensitive data.</p> <p>Install the release with a compatible Agents SDK version:</p> <script> if (!customElements.get("nb-pm-restore")) { customElements.define( "nb-pm-restore", class extends HTMLElement { connectedCallback() { const card = this.closest("[data-nb-pm]"); if (!card) return; let saved; try { saved = sessionStorage.getItem("ui-pm-tab"); } catch { return; } if (!saved) return; const tabs = card.querySelectorAll("[data-nb-pm-tab]"); let idx = -1; tabs.forEach(function (t, i) { if (t.textContent.trim() === saved) idx = i; }); if (idx < 1) return; tabs.forEach(function (t, i) { t.setAttribute("aria-selected", String(i === idx)); }); card.querySelectorAll("[data-nb-pm-panel]").forEach(function (p, i) { p.hidden = i !== idx; }); } }, ); } </script><div data-nb-pm class="w-full"><div class="flex w-full flex-col overflow-hidden rounded-lg text-sm ring ring-border bg-card"><div class="flex items-center gap-2 px-3 py-2 text-[0.8125rem] font-medium leading-5 text-muted-foreground bg-[var(--nb-surface-sunken)]" role="tablist" aria-label="Package manager"><button role="tab" type="button" aria-selected="true" aria-controls="pm-panel-pm-7bccdbaa-npm" id="pm-tab-pm-7bccdbaa-npm" data-nb-pm-tab class="text-muted-foreground hover:bg-accent hover:text-foreground aria-selected:bg-selected aria-selected:text-foreground focus-visible:outline-ring m-0 cursor-pointer rounded-md border-0 bg-transparent px-2 py-0.5 text-xs leading-5 font-medium transition-colors focus-visible:outline-2 focus-visible:outline-offset-2">npm</button><button role="tab" type="button" aria-selected="false" aria-controls="pm-panel-pm-7bccdbaa-yarn" id="pm-tab-pm-7bccdbaa-yarn" data-nb-pm-tab class="text-muted-foreground hover:bg-accent hover:text-foreground aria-selected:bg-selected aria-selected:text-foreground focus-visible:outline-ring m-0 cursor-pointer rounded-md border-0 bg-transparent px-2 py-0.5 text-xs leading-5 font-medium transition-colors focus-visible:outline-2 focus-visible:outline-offset-2">yarn</button><button role="tab" type="button" aria-selected="false" aria-controls="pm-panel-pm-7bccdbaa-pnpm" id="pm-tab-pm-7bccdbaa-pnpm" data-nb-pm-tab class="text-muted-foreground hover:bg-accent hover:text-foreground aria-selected:bg-selected aria-selected:text-foreground focus-visible:outline-ring m-0 cursor-pointer rounded-md border-0 bg-transparent px-2 py-0.5 text-xs leading-5 font-medium transition-colors focus-visible:outline-2 focus-visible:outline-offset-2">pnpm</button><button role="tab" type="button" aria-selected="false" aria-controls="pm-panel-pm-7bccdbaa-bun" id="pm-tab-pm-7bccdbaa-bun" data-nb-pm-tab class="text-muted-foreground hover:bg-accent hover:text-foreground aria-selected:bg-selected aria-selected:text-foreground focus-visible:outline-ring m-0 cursor-pointer rounded-md border-0 bg-transparent px-2 py-0.5 text-xs leading-5 font-medium transition-colors focus-visible:outline-2 focus-visible:outline-offset-2">bun</button></div><div role="tabpanel" id="pm-panel-pm-7bccdbaa-npm" aria-labelledby="pm-tab-pm-7bccdbaa-npm" data-nb-pm-panel class="bg-card ring-border relative overflow-hidden rounded-lg text-inherit ring"><div class="flex items-stretch"><pre class="text-foreground my-0 min-w-0 grow overflow-x-auto border-0 bg-transparent px-4 py-3 font-mono text-sm leading-relaxed whitespace-pre"><code data-nb-pm-code><span class="text-success">npm</span><span class="text-warning"> i @cloudflare/voice@^0.4.0 agents@^0.22.0</span></code></pre><button type="button" data-nb-pm-copy data-nb-command="npm i @cloudflare/voice@^0.4.0 agents@^0.22.0" aria-label="Copy to clipboard" class="border-border text-muted-foreground hover:text-foreground focus-visible:outline-ring m-0 flex shrink-0 cursor-pointer items-center justify-center border-0 border-l border-solid bg-transparent px-3 transition-colors focus-visible:outline-2 focus-visible:-outline-offset-2"><svg width="1em" height="1em" viewBox="0 0 256 256" class="h-[18px] w-[18px]" data-icon="ph:copy"><path fill="currentColor" d="M216 32H88a8 8 0 0 0-8 8v40H40a8 8 0 0 0-8 8v128a8 8 0 0 0 8 8h128a8 8 0 0 0 8-8v-40h40a8 8 0 0 0 8-8V40a8 8 0 0 0-8-8m-56 176H48V96h112Zm48-48h-32V88a8 8 0 0 0-8-8H96V48h112Z"/></svg><svg width="1em" height="1em" viewBox="0 0 256 256" class="hidden h-[18px] w-[18px]" data-icon="ph:check"><path fill="currentColor" d="m229.66 77.66l-128 128a8 8 0 0 1-11.32 0l-56-56a8 8 0 0 1 11.32-11.32L96 188.69L218.34 66.34a8 8 0 0 1 11.32 11.32"/></svg></button></div></div><div role="tabpanel" id="pm-panel-pm-7bccdbaa-yarn" aria-labelledby="pm-tab-pm-7bccdbaa-yarn" hidden data-nb-pm-panel class="bg-card ring-border relative overflow-hidden rounded-lg text-inherit ring"><div class="flex items-stretch"><pre class="text-foreground my-0 min-w-0 grow overflow-x-auto border-0 bg-transparent px-4 py-3 font-mono text-sm leading-relaxed whitespace-pre"><code data-nb-pm-code><span class="text-success">yarn</span><span class="text-warning"> add @cloudflare/voice@^0.4.0 agents@^0.22.0</span></code></pre><button type="button" data-nb-pm-copy data-nb-command="yarn add @cloudflare/voice@^0.4.0 agents@^0.22.0" aria-label="Copy to clipboard" class="border-border text-muted-foreground hover:text-foreground focus-visible:outline-ring m-0 flex shrink-0 cursor-pointer items-center justify-center border-0 border-l border-solid bg-transparent px-3 transition-colors focus-visible:outline-2 focus-visible:-outline-offset-2"><svg width="1em" height="1em" viewBox="0 0 256 256" class="h-[18px] w-[18px]" data-icon="ph:copy"><path fill="currentColor" d="M216 32H88a8 8 0 0 0-8 8v40H40a8 8 0 0 0-8 8v128a8 8 0 0 0 8 8h128a8 8 0 0 0 8-8v-40h40a8 8 0 0 0 8-8V40a8 8 0 0 0-8-8m-56 176H48V96h112Zm48-48h-32V88a8 8 0 0 0-8-8H96V48h112Z"/></svg><svg width="1em" height="1em" viewBox="0 0 256 256" class="hidden h-[18px] w-[18px]" data-icon="ph:check"><path fill="currentColor" d="m229.66 77.66l-128 128a8 8 0 0 1-11.32 0l-56-56a8 8 0 0 1 11.32-11.32L96 188.69L218.34 66.34a8 8 0 0 1 11.32 11.32"/></svg></button></div></div><div role="tabpanel" id="pm-panel-pm-7bccdbaa-pnpm" aria-labelledby="pm-tab-pm-7bccdbaa-pnpm" hidden data-nb-pm-panel class="bg-card ring-border relative overflow-hidden rounded-lg text-inherit ring"><div class="flex items-stretch"><pre class="text-foreground my-0 min-w-0 grow overflow-x-auto border-0 bg-transparent px-4 py-3 font-mono text-sm leading-relaxed whitespace-pre"><code data-nb-pm-code><span class="text-success">pnpm</span><span class="text-warning"> add @cloudflare/voice@^0.4.0 agents@^0.22.0</span></code></pre><button type="button" data-nb-pm-copy data-nb-command="pnpm add @cloudflare/voice@^0.4.0 agents@^0.22.0" aria-label="Copy to clipboard" class="border-border text-muted-foreground hover:text-foreground focus-visible:outline-ring m-0 flex shrink-0 cursor-pointer items-center justify-center border-0 border-l border-solid bg-transparent px-3 transition-colors focus-visible:outline-2 focus-visible:-outline-offset-2"><svg width="1em" height="1em" viewBox="0 0 256 256" class="h-[18px] w-[18px]" data-icon="ph:copy"><path fill="currentColor" d="M216 32H88a8 8 0 0 0-8 8v40H40a8 8 0 0 0-8 8v128a8 8 0 0 0 8 8h128a8 8 0 0 0 8-8v-40h40a8 8 0 0 0 8-8V40a8 8 0 0 0-8-8m-56 176H48V96h112Zm48-48h-32V88a8 8 0 0 0-8-8H96V48h112Z"/></svg><svg width="1em" height="1em" viewBox="0 0 256 256" class="hidden h-[18px] w-[18px]" data-icon="ph:check"><path fill="currentColor" d="m229.66 77.66l-128 128a8 8 0 0 1-11.32 0l-56-56a8 8 0 0 1 11.32-11.32L96 188.69L218.34 66.34a8 8 0 0 1 11.32 11.32"/></svg></button></div></div><div role="tabpanel" id="pm-panel-pm-7bccdbaa-bun" aria-labelledby="pm-tab-pm-7bccdbaa-bun" hidden data-nb-pm-panel class="bg-card ring-border relative overflow-hidden rounded-lg text-inherit ring"><div class="flex items-stretch"><pre class="text-foreground my-0 min-w-0 grow overflow-x-auto border-0 bg-transparent px-4 py-3 font-mono text-sm leading-relaxed whitespace-pre"><code data-nb-pm-code><span class="text-success">bun</span><span class="text-warning"> add @cloudflare/voice@^0.4.0 agents@^0.22.0</span></code></pre><button type="button" data-nb-pm-copy data-nb-command="bun add @cloudflare/voice@^0.4.0 agents@^0.22.0" aria-label="Copy to clipboard" class="border-border text-muted-foreground hover:text-foreground focus-visible:outline-ring m-0 flex shrink-0 cursor-pointer items-center justify-center border-0 border-l border-solid bg-transparent px-3 transition-colors focus-visible:outline-2 focus-visible:-outline-offset-2"><svg width="1em" height="1em" viewBox="0 0 256 256" class="h-[18px] w-[18px]" data-icon="ph:copy"><path fill="currentColor" d="M216 32H88a8 8 0 0 0-8 8v40H40a8 8 0 0 0-8 8v128a8 8 0 0 0 8 8h128a8 8 0 0 0 8-8v-40h40a8 8 0 0 0 8-8V40a8 8 0 0 0-8-8m-56 176H48V96h112Zm48-48h-32V88a8 8 0 0 0-8-8H96V48h112Z"/></svg><svg width="1em" height="1em" viewBox="0 0 256 256" class="hidden h-[18px] w-[18px]" data-icon="ph:check"><path fill="currentColor" d="m229.66 77.66l-128 128a8 8 0 0 1-11.32 0l-56-56a8 8 0 0 1 11.32-11.32L96 188.69L218.34 66.34a8 8 0 0 1 11.32 11.32"/></svg></button></div></div><nb-pm-restore style="display:contents"></nb-pm-restore></div></div><script type="module" src="https://developers.cloudflare.com/home/runner/work/cloudflare-docs/cloudflare-docs/src/components/ui/package-managers/PackageManagerTabs.astro?astro&type=script&index=0&lang.ts"></script> <p>Refer to the <a href="https://developers.cloudflare.com/agents/communication-channels/voice/#pipeline-metrics">Voice pipeline metrics</a> and <a href="https://github.com/cloudflare/agents/tree/main/examples/voice-agent" target="_blank" rel="noopener">Voice Agent example<span class="external-link"> ↗︎</span></a> to get started.</p>