Introduction

TTP Agent SDK is a powerful JavaScript library for building AI-powered voice and chat interactions in your web applications.

Installation

NPM

npm install ttp-agent-sdk

CDN

<script src="https://unpkg.com/ttp-agent-sdk/dist/agent-widget.js"></script>

Import

// ES6 Import
import { VoiceSDK } from 'ttp-agent-sdk';

// CommonJS
const { VoiceSDK } = require('ttp-agent-sdk');

// Browser Global
const sdk = new window.TTPAgentSDK.VoiceSDK(config);

Quick Start

Get up and running in 5 minutes with this simple example.

import { VoiceSDK } from 'ttp-agent-sdk';

const voiceSDK = new VoiceSDK({
  agentId: 'agent_123',     // The AI agent to connect to
  appId: 'your_app_id',     // Your application ID
  
  // Optional: Configure audio formats (v2 protocol)
  outputContainer: 'raw',      // 'raw' or 'wav'
  outputEncoding: 'pcm',       // 'pcm', 'pcmu', 'pcma'
  outputSampleRate: 24000,     // Typical server/TTS output (default)
  protocolVersion: 2           // Use v2 protocol for format negotiation
});

// Listen to events
voiceSDK.on('connected', () => {
  console.log('✅ Connected to agent');
});

voiceSDK.on('formatNegotiated', (format) => {
  console.log('✅ Format negotiated:', format);
  // Format contains: container, encoding, sampleRate, channels, bitDepth
});

voiceSDK.on('message', (msg) => {
  if (msg.t === 'agent_response') {
    console.log('Agent:', msg.agent_response);
  }
});

// Connect
await voiceSDK.connect();

// Start recording
await voiceSDK.startRecording();

// Stop recording
await voiceSDK.stopRecording();

Authentication

The SDK connects directly using agentId and appId. No server-side authentication step is needed. Access control is managed via domain whitelist in your agent's admin panel.

How It Works

const voiceSDK = new VoiceSDK({
  agentId: 'agent_123',     // Your AI agent ID
  appId: 'your_app_id'      // Your application ID
});

Domain Whitelist

To control which websites can use your agent, configure a domain whitelist in your agent's admin panel. Only requests originating from whitelisted domains will be accepted.

Configuration Parameters

Agent Settings Override

NEW FEATURE

Dynamically customize agent behavior, voice, and personality on a per-session basis.

How It Works

1. Configure: Set up a domain whitelist for your agent in the admin panel
2. Initialize: Pass agentSettingsOverride in the SDK configuration
3. Connect: The SDK sends overrides in the hello message
4. TTP Backend: Validates the domain and applies your overrides

Example

const voiceSDK = new VoiceSDK({
  agentId: 'agent_123',     // The AI agent to connect to
  appId: 'your_app_id',     // Your application ID
  
  // Override agent settings
  agentSettingsOverride: {
    // Core settings
    prompt: "You are a friendly Spanish-speaking travel assistant",
    language: "es",
    temperature: 0.9,
    maxTokens: 200,
    
    // Voice settings
    voiceSpeed: 1.2,
    voiceId: "nova",  // Use voiceId (not selectedVoice)
    
    // Behavior
    firstMessage: "¡Hola! ¿Cómo puedo ayudarte hoy?",
    disableInterruptions: false,
    autoDetectLanguage: true,
    
    // Tools (optional)
    toolIds: [123, 456, 789],              // Custom tool IDs
    internalToolIds: ['calendar', 'email'] // Internal tool IDs
  }
});

Available Override Settings

15 out of 16 settings can be overridden. Only model selection is not supported (requires infrastructure changes).

Variables in Hello Request

Overview

Variables allow you to pass dynamic values to your agent that will be used to replace placeholders in the system prompt and first message. Variables sent in the hello request take precedence over default variables stored in the agent configuration.

Hello Message Format

SDK v2 Format (Recommended)

When using VoiceSDK v2, variables are passed in the SDK constructor:

import { VoiceSDK_v2 } from 'ttp-agent-sdk';

const voiceSDK = new VoiceSDK_v2({
  agentId: 'agent_5a2b984c1',
  appId: 'app_Bc01EqMQt2Euehl4qqZSi6l3FJP42Q9vJ0pC',
  
  // Variables (optional)
  variables: {
    USER_NAME: 'John',
    ACCOUNT_TYPE: 'premium',
    LANGUAGE: 'en-US'
  },
  
  // Audio format configuration
  sampleRate: 16000,
  channels: 1,
  bitDepth: 16,
  outputContainer: 'raw',
  outputEncoding: 'pcm',
  outputSampleRate: 24000,
  outputChannels: 1,
  outputBitDepth: 16,
  outputFrameDurationMs: 600
});

await voiceSDK.connect();

Raw WebSocket Format

If connecting via raw WebSocket (without SDK), send variables in the hello message:

{
  "t": "hello",
  "v": 2,
  "variables": {
    "USER_NAME": "John",
    "ACCOUNT_TYPE": "premium",
    "LANGUAGE": "en-US"
  },
  "inputFormat": {
    "encoding": "pcm",
    "sampleRate": 16000,
    "channels": 1,
    "bitDepth": 16
  },
  "requestedOutputFormat": {
    "encoding": "pcm",
    "sampleRate": 24000,
    "channels": 1,
    "bitDepth": 16,
    "container": "raw"
  },
  "outputFrameDurationMs": 600
}

Variable Format

Variables are sent as a JSON object where:

• Keys: Variable names (case-sensitive, e.g., USER_NAME)
• Values: String values that will replace {{VARIABLE_NAME}} in the prompt

Example

{
  "variables": {
    "USER_NAME": "John",
    "ACCOUNT_TYPE": "premium",
    "LANGUAGE": "en-US",
    "COMPANY": "Acme Corp"
  }
}

Variable Replacement Priority

Variables are replaced in the following priority order:

1. Hello Variables (highest priority) - Variables sent in the hello request
2. Default Variables - Variables stored in agent configuration (Redis)
3. Leave as-is - If no value found, {{VARIABLE_NAME}} remains unchanged

Example Priority

Agent Configuration (Redis):

{
  "USER_NAME": "David",
  "ACCOUNT_TYPE": "premium"
}

Hello Request:

{
  "variables": {
    "USER_NAME": "John"
  }
}

Result:

• {{USER_NAME}} → "John" (from hello - takes precedence)
• {{ACCOUNT_TYPE}} → "premium" (from defaults - hello doesn't override)

Usage Examples

Example 1: JavaScript/TypeScript with SDK v2

import { VoiceSDK_v2 } from 'ttp-agent-sdk';

const voiceSDK = new VoiceSDK_v2({
  agentId: 'agent_5a2b984c1',
  appId: 'app_Bc01EqMQt2Euehl4qqZSi6l3FJP42Q9vJ0pC',
  
  variables: {
    USER_NAME: 'John Doe',
    ACCOUNT_TYPE: 'premium',
    LANGUAGE: 'en-US'
  },
  
  // ... audio format config
});

await voiceSDK.connect();

Example 2: Raw WebSocket (JavaScript)

const ws = new WebSocket('wss://speech.talktopc.com/ws/conv?agentId=agent_5a2b984c1&appId=app_Bc01EqMQt2Euehl4qqZSi6l3FJP42Q9vJ0pC');

ws.onopen = () => {
  const helloMessage = {
    t: 'hello',
    v: 2,
    variables: {
      USER_NAME: 'John',
      ACCOUNT_TYPE: 'premium',
      LANGUAGE: 'en-US'
    },
    inputFormat: {
      encoding: 'pcm',
      sampleRate: 16000,
      channels: 1,
      bitDepth: 16
    },
    requestedOutputFormat: {
      encoding: 'pcm',
      sampleRate: 24000,
      channels: 1,
      bitDepth: 16,
      container: 'raw'
    },
    outputFrameDurationMs: 600
  };
  
  ws.send(JSON.stringify(helloMessage));
};

Example 3: Python WebSocket

import websocket
import json

def on_open(ws):
    hello_message = {
        "t": "hello",
        "v": 2,
        "variables": {
            "USER_NAME": "John",
            "ACCOUNT_TYPE": "premium",
            "LANGUAGE": "en-US"
        },
        "inputFormat": {
            "encoding": "pcm",
            "sampleRate": 16000,
            "channels": 1,
            "bitDepth": 16
        },
        "requestedOutputFormat": {
            "encoding": "pcm",
            "sampleRate": 24000,
            "channels": 1,
            "bitDepth": 16,
            "container": "raw"
        },
        "outputFrameDurationMs": 600
    }
    
    ws.send(json.dumps(hello_message))

ws = websocket.WebSocketApp(
    "wss://speech.talktopc.com/ws/conv?agentId=agent_5a2b984c1&appId=app_Bc01EqMQt2Euehl4qqZSi6l3FJP42Q9vJ0pC",
    on_open=on_open
)
ws.run_forever()

Example 4: cURL / wscat

# Using wscat
wscat -c "wss://speech.talktopc.com/ws/conv?agentId=agent_5a2b984c1&appId=app_Bc01EqMQt2Euehl4qqZSi6l3FJP42Q9vJ0pC"

# Then send:
{"t":"hello","v":2,"variables":{"USER_NAME":"John","ACCOUNT_TYPE":"premium"},"inputFormat":{"encoding":"pcm","sampleRate":16000,"channels":1,"bitDepth":16},"requestedOutputFormat":{"encoding":"pcm","sampleRate":24000,"channels":1,"bitDepth":16,"container":"raw"},"outputFrameDurationMs":600}

Agent Prompt Setup

To use variables in your agent, include placeholders in the system prompt or first message:

System Prompt Example

Your name is {{AGENT_NAME}}.
You are helping {{USER_NAME}} who has a {{ACCOUNT_TYPE}} account.
Speak in {{LANGUAGE}}.

First Message Example

Hello {{USER_NAME}}! Welcome to our {{ACCOUNT_TYPE}} service.

Backend Processing

When the hello request is received with variables:

1. Variables are extracted from the hello message
2. Default variables are loaded from agent configuration (Redis)
3. Variables are merged (hello variables take precedence)
4. Prompt is processed - {{VARIABLE_NAME}} placeholders are replaced
5. First message is processed - Variables are replaced here too
6. Metadata is added - Variables metadata section is appended to prompt

Server Logs

After sending hello with variables, check server logs for:

📝 Processing variables from hello message: [USER_NAME, ACCOUNT_TYPE, LANGUAGE]
✅ Variables processed and prompt updated
📝 FINAL PROCESSED PROMPT (agentId: ...):
[Prompt with variables replaced]
✅ Processed variables: 3 hello variables, 2 default variables, metadata: added

Variable Naming Conventions

• Use UPPERCASE with underscores: USER_NAME, ACCOUNT_TYPE
• Variable names are case-sensitive
• Avoid special characters except underscores
• Recommended format: {{VARIABLE_NAME}} in prompts

Common Use Cases

1. User Personalization

{
  "variables": {
    "USER_NAME": "John Doe",
    "USER_EMAIL": "john@example.com"
  }
}

2. Account Context

{
  "variables": {
    "ACCOUNT_TYPE": "premium",
    "SUBSCRIPTION_STATUS": "active"
  }
}

3. Language/Localization

{
  "variables": {
    "LANGUAGE": "en-US",
    "CURRENCY": "USD"
  }
}

4. Session Context

{
  "variables": {
    "SESSION_ID": "abc123",
    "PAGE_URL": "https://example.com/products"
  }
}

Error Handling

Missing Variables

If a variable is referenced in the prompt but not provided:

• Has default value: Uses default from agent configuration
• No default value: Placeholder remains unchanged ({{VARIABLE_NAME}})

Invalid Variable Format

• Variables must be a JSON object
• Values should be strings (will be converted to string if needed)
• Empty object {} or null is valid (will use defaults only)

Best Practices

1. Set defaults in agent configuration for all variables
2. Override with hello variables only when you have dynamic values
3. Use descriptive names that clearly indicate the variable's purpose
4. Document variables in your agent's description or notes
5. Test variables by checking server logs for "FINAL PROCESSED PROMPT"

API Reference

Hello Message Structure

interface HelloMessage {
  t: "hello";                    // Message type
  v?: number;                    // SDK version (2 for v2)
  variables?: {                   // Optional variables object
    [key: string]: string;        // Variable name -> value mapping
  };
  inputFormat?: AudioFormat;     // Input audio format
  requestedOutputFormat?: AudioFormat; // Output audio format
  outputFrameDurationMs?: number; // Frame duration for streaming
}

interface AudioFormat {
  encoding: "pcm" | "pcmu" | "pcma";
  sampleRate: number;
  channels: number;
  bitDepth: number;
  container?: "raw" | "wav";      // For output format only
}

Troubleshooting

Variables Not Being Replaced

1. Check variable names match exactly (case-sensitive)
2. Verify variables are sent in hello message (check logs)
3. Check server logs for "FINAL PROCESSED PROMPT" to see actual replacement

Variables Not in Hello Message

• SDK v2: Check if SDK supports variables in constructor
• Raw WebSocket: Ensure variables field is included in JSON
• Check WebSocket message is sent after connection opens

Default Variables Not Used

• Verify variables are stored in Redis (check agent configuration)
• Check extractVariablesFromAgentConfig is working
• Look for "Default variables not found in state" warnings in logs

Events & Callbacks

The SDK emits events for all important state changes and interactions.

Event Categories

Connection Events

voiceSDK.on('connected', () => {
  console.log('✅ Connected to agent');
});

voiceSDK.on('disconnected', (event) => {
  console.log('❌ Disconnected:', event.reason);
  console.log('Close code:', event.code);
});

voiceSDK.on('error', (error) => {
  console.error('Error:', error);
});

Recording Events

voiceSDK.on('recordingStarted', () => {
  console.log('🎤 Recording started');
});

voiceSDK.on('recordingStopped', () => {
  console.log('⏹️ Recording stopped');
});

Message Events

voiceSDK.on('message', (msg) => {
  switch(msg.type) {
    case 'agent_response':
      console.log('Agent:', msg.agent_response);
      break;
    case 'transcription':
      console.log('You said:', msg.text);
      break;
    // ... other message types
  }
});

Audio Events

voiceSDK.on('playbackStarted', () => {
  console.log('🔊 Audio playback started');
});

voiceSDK.on('playbackStopped', () => {
  console.log('🔇 Audio playback stopped');
});

voiceSDK.on('audioData', (audioData) => {
  // Raw audio data (Uint8Array)
});

Pause Events

// Call paused (server acknowledged)
voiceSDK.on('callPaused', (data) => {
  console.log('⏸️ Call paused, timeout:', data.timeoutSeconds, 'seconds');
});

// Call resumed (server acknowledged, STT ready)
voiceSDK.on('callResumed', () => {
  console.log('▶️ Call resumed');
});

// Pause timeout (call auto-ended because pause lasted too long)
voiceSDK.on('pauseTimeout', () => {
  console.log('⏱️ Pause timeout — call ended');
});

Special Events

// Barge-in (user interrupts agent)
voiceSDK.on('bargeIn', (message) => {
  console.log('User interrupted the agent');
});

// Format negotiation (v2 protocol only)
voiceSDK.on('formatNegotiated', (format) => {
  console.log('Format negotiated:', format);
  // format: { container, encoding, sampleRate, channels, bitDepth }
});

// Greeting audio
voiceSDK.on('greetingStarted', () => {
  console.log('Playing greeting message');
});

// Domain whitelist error
voiceSDK.on('domainError', (error) => {
  console.error('Domain not whitelisted:', error.reason);
});

// Server-driven disclaimer (voice v2) — see #server-driven-disclaimer
voiceSDK.on('disclaimersRequired', (payload) => {
  // Show your UI, then call voiceSDK.sendDisclaimerAck(true|false)
});
voiceSDK.on('disclaimerRejected', ({ code, message }) => {
  // DISCLAIMER_DECLINED, DISCLAIMER_TIMEOUT, DISCLAIMER_HASH_MISMATCH, etc.
});

Protocol v2 - Format Negotiation

The SDK v2 introduces format negotiation, allowing you to specify exactly what audio format you want to receive from the server.

Server-driven disclaimer (voice)

Some deployments must show exact legal or policy copy from the server before speech recognition and the agent greeting run. The conversation server can require an explicit acknowledgement step after hello_ack. This applies to VoiceSDK v2 (protocol version 2) and the Voice & Chat Widget voice path.

When the gate is active

• The agent has a non-empty disclaimers list in backend storage (Redis field disclaimers as a JSON array of plain-text strings). An empty array [] means no disclaimer gate.
• The session is not a resumed voice call (resume skips the gate).

While the gate is open, the server:

• Does not open STT or play the greeting.
• Rejects start_continuous_mode with {"ok":false,"t":"error","code":"DISCLAIMER_PENDING",...}.
• Drops uplink binary audio (microphone data) until the gate clears.
• Starts a server-side timer; if the user never acknowledges, the session is closed with DISCLAIMER_TIMEOUT (duration is configured on the server, typically on the order of minutes).

iOS Safari: microphone uplink

Voice capture uses an AudioWorklet. On iPhone and iPad, WebKit often runs the capture AudioContext at the device hardware rate (commonly 44.1 kHz or 48 kHz) even when a lower rate was requested, while the server expects PCM at the input rate negotiated in hello_ack (typically 16 kHz). The SDK resamples uplink PCM to that negotiated rate before sending binary frames. The recorder connects the worklet through a zero-gain node to the destination so WebKit reliably pulls the processor (graphs that dead-end at the worklet alone may not run on some builds). On mobile, after getUserMedia succeeds, the SDK may prime the shared recorder context while user activation is still fresh, before the WebSocket handshake and hello_ack finish. For embedded widgets, use allow="microphone" on the iframe when the host page is cross-origin.

Embed sites: CSP and “Unable to load a worklet's module”

Strict Content-Security-Policy on the parent page can block audioWorklet.addModule() when the processor URL points at another host (for example cdn.talktopc.com), which surfaces as AbortError: Unable to load a worklet's module and voice never reaches the server. The SDK tries that URL first, then automatically retries using the capture worklet source bundled inside the widget script and a blob: URL—this works on many shops without whitelisting our CDN. If both attempts fail, relax CSP (often worker-src / script-src must allow blob:, and/or your CDN origin for the processor), or host audio-processor.js on your own domain and set voice.audioProcessorPath to that same-origin URL.

hello_ack fields (gate active)

When disclaimers apply, the server includes:

Client message: disclaimer_ack

After the user accepts or declines, send:

{
  "t": "disclaimer_ack",
  "accepted": true,
  "disclaimersHash": "sha256-from-hello_ack",
  "conversationId": "optional-matches-hello_ack"
}

For decline, set "accepted": false. Duplicate acks for the same session are ignored. The SDK method sendDisclaimerAck(accepted) builds this frame using the hash and conversation id from hello_ack.

Server error frames (t: "error")

Using VoiceSDK v2 (custom apps)

1. Use protocol v2 (default in current SDK): protocolVersion: 2.
2. After connect(), wait for hello_ack. The SDK sets voiceSDK.disclaimersPending === true when the gate is active.
3. Listen for disclaimersRequired. The payload includes texts, disclaimersHash, disclaimerTimeoutMs, and conversationId for your UI.
4. Show your own modal or screen with the given texts. Do not call startRecording() until the user has accepted and you have called sendDisclaimerAck(true) (calling startRecording() while disclaimersPending is still true throws with error.code === 'DISCLAIMER_PENDING').
5. On accept: voiceSDK.sendDisclaimerAck(true). The server then opens STT, plays the greeting, and accepts start_continuous_mode.
6. On decline: voiceSDK.sendDisclaimerAck(false). The server closes the session; the SDK emits disclaimerRejected and the raw message event for the error frame.
7. Handle disclaimerRejected for terminal server outcomes (DISCLAIMER_DECLINED, DISCLAIMER_TIMEOUT, DISCLAIMER_HASH_MISMATCH). Handle error for DISCLAIMER_PENDING (ordering bug or race—fix by ack first).

const voiceSDK = new VoiceSDK({ agentId, appId, protocolVersion: 2 });

voiceSDK.on('disclaimersRequired', (payload) => {
  showMyModal({
    texts: payload.texts,
    onAccept: () => voiceSDK.sendDisclaimerAck(true),
    onDecline: () => voiceSDK.sendDisclaimerAck(false)
  });
});

voiceSDK.on('disclaimerRejected', ({ code, message }) => {
  console.warn('Disclaimer flow ended:', code, message);
});

voiceSDK.on('error', (err) => {
  if (err.code === 'DISCLAIMER_PENDING') {
    console.warn('Start recording only after sendDisclaimerAck(true)');
  }
});

await voiceSDK.connect();
// Only after ack (or if disclaimersRequired never fired):
await voiceSDK.startRecording();

SDK state you may read: disclaimersPending, disclaimersHash, lastDisclaimerPayload (set from hello_ack). After sendDisclaimerAck runs, the SDK clears disclaimersPending locally when the WebSocket send succeeds.

Voice & Chat Widget (built-in behavior)

No extra widget options are required. When the server sends the disclaimer gate:

• VoiceInterface waits after the WebSocket is up and hello_ack is processed.
• It opens a built-in modal (Notice / Accept / Decline) with disclaimerTexts from the server.
• Accept: on desktop, the widget sends sendDisclaimerAck(true) immediately, then requests the microphone and startListening. On mobile, it waits until the user has granted microphone access (and the post-grant audio delay) before sending sendDisclaimerAck(true), so the server does not stream the greeting over the system permission sheet or get cut off when capture starts.
• Decline (No thanks) calls sendDisclaimerAck(false), waits briefly so the ack reaches the server (which ends the conversation and closes the socket), then disconnects the client, invokes onConversationEnd on the wrapper SDK since recording never started, resets UI, and returns to landing (or idle voice in voice-only mode).

To match your site language, use the widget’s existing language / translation hooks for general UI; disclaimer body text always comes from the server (compliance copy).

Resume & text chat

Resume: Resumed voice sessions omit the disclaimer gate.

Text chat: Server-driven disclaimer is implemented for voice in this release. The text WebSocket hello path does not yet mirror these fields; extend the backend and TextChatSDK if you need the same gate for text-only sessions.

Supported Formats

Input Formats (What SDK Sends)

Output Formats (What SDK Receives)

Format Negotiation Flow

Example: High-Quality Audio

const voiceSDK = new VoiceSDK({
  agentId: 'agent_123',
  appId: 'your_app_id',
  
  // Request high-quality audio
  outputContainer: 'raw',        // Raw PCM for lower latency
  outputEncoding: 'pcm',         // Uncompressed PCM
  outputSampleRate: 48000,       // 48kHz for high quality
  outputBitDepth: 16,            // 16-bit depth
  outputChannels: 1,             // Mono
  outputFrameDurationMs: 600,    // 600ms frames
  
  protocolVersion: 2             // Enable format negotiation
});

voiceSDK.on('formatNegotiated', (format) => {
  console.log('Negotiated format:', format);
  // If backend sends different format, SDK will convert automatically
});

Example: Bandwidth-Optimized Audio

const voiceSDK = new VoiceSDK({
  agentId: 'agent_123',
  appId: 'your_app_id',
  
  // Request compressed, low-bandwidth audio
  outputContainer: 'raw',
  outputEncoding: 'pcmu',        // μ-law compression (8kHz equivalent)
  outputSampleRate: 8000,        // 8kHz for bandwidth savings
  outputBitDepth: 16,
  outputChannels: 1,
  
  protocolVersion: 2
});

Format Conversion

If the backend sends audio in a different format than requested, the SDK automatically converts it:

• Container: WAV ↔ Raw PCM extraction/wrapping
• Encoding: PCM ↔ PCMU/PCMA encoding/decoding
• Sample Rate: Automatic resampling using Web Audio API
• Bit Depth: 8-bit ↔ 16-bit ↔ 24-bit conversion
• Channels: Mono/stereo conversion (if needed)

Voice & Chat Widget

Pre-built, customizable widget with voice and text chat - perfect for adding AI conversation to any website.

Installation

<!-- Add the SDK script to your page -->
<script src="https://unpkg.com/ttp-agent-sdk/dist/agent-widget.js"></script>

<!-- Initialize the widget -->
<script>
  const widget = new TTPAgentSDK.TTPChatWidget({
    agentId: 'agent_123',
    appId: 'your_app_id'
  });
</script>

Basic Configuration

const widget = new TTPAgentSDK.TTPChatWidget({
  // Required
  agentId: 'agent_123',       // Your AI agent ID
  appId: 'your_app_id',       // Your application ID

  // Optional — root only (voice idle hero name & avatar initial when no image)
  agentName: 'Alex',
  
  // Optional - Agent Settings Override (available when domain whitelist is configured)
  agentSettingsOverride: {
    prompt: "You are a helpful customer service assistant.",
    temperature: 0.8,
    voiceId: "F2",
    voiceSpeed: 1.2,
    firstMessage: "Hello! How can I help you today?",
    disableInterruptions: false,
    maxCallDuration: 600
  },
  
  // Optional - Appearance
  primaryColor: '#7C3AED',    // Widget theme color
  position: {                 // Or shorthand: 'bottom-right', 'bottom-left'
    vertical: 'bottom',
    horizontal: 'right',
    offset: { x: 20, y: 20 },
    draggable: false,         // true = user can drag launcher + panel
    draggablePersist: true    // remember drag position in localStorage
  },
  language: 'en',             // 'en', 'es', 'fr', 'de', 'he', etc.
  direction: 'ltr',           // 'ltr' or 'rtl' for right-to-left languages
  
  // Optional - Variables
  variables: {
    userName: 'John Doe',
    page: 'homepage',
    customData: 'value'
  }
});

Server-driven disclaimer (voice)

If your agent has non-empty disclaimers configured on the server, the widget shows a Notice modal with the server-provided text before microphone streaming and the greeting. The user must tap Accept or Decline; you do not need extra embed code. For protocol fields, SDK hooks, and custom implementations, see Server-driven disclaimer (voice).

Access Control

The widget connects directly using agentId and appId. No backend authentication step is needed:

const widget = new TTPAgentSDK.TTPChatWidget({
  agentId: 'agent_123',
  appId: 'your_app_id',
  variables: {
    userName: 'John Doe',
    page: 'homepage'
  }
});

Advanced Customization

Icon Customization

const widget = new TTPAgentSDK.TTPChatWidget({
  agentId: 'agent_123',
  appId: 'your_app_id',
  
  icon: {
    type: 'custom',                              // 'microphone', 'emoji', or 'custom'
    // Omit customImage (or use '') for default animated waveform on the desktop pill
    customImage: 'https://your-site.com/logo.png', // Optional: pill icon image URL
    size: 60,                                    // Icon size in pixels
    backgroundColor: '#FFFFFF',                  // Background color
    borderRadius: '50%'                         // Border radius (50% for circle)
  }
});

Chat Window Customization

const widget = new TTPAgentSDK.TTPChatWidget({
  agentId: 'agent_123',
  appId: 'your_app_id',
  
  chatWindow: {
    width: 400,                    // Width in pixels
    height: 600,                   // Height in pixels
    title: 'Chat with us!',        // Custom title
    subtitle: 'We reply instantly', // Custom subtitle
    placeholder: 'Type here...',   // Input placeholder
    borderRadius: 12               // Window border radius
  }
});

Branding

const widget = new TTPAgentSDK.TTPChatWidget({
  agentId: 'agent_123',
  appId: 'your_app_id',
  
  branding: {
    companyName: 'Your Company',
    logo: 'https://your-site.com/logo.png',
    showPoweredBy: false           // Hide "Powered by TTP" footer
  }
});

Agent Settings Override

Dynamically customize agent behavior, voice, and personality on a per-session basis:

const widget = new TTPAgentSDK.TTPChatWidget({
  agentId: 'agent_123',
  appId: 'your_app_id',
  
  // Override agent settings dynamically
  agentSettingsOverride: {
    // Core settings
    prompt: "You are a friendly customer service assistant",
    temperature: 0.8,
    maxTokens: 200,
    
    // Voice settings
    voiceId: "F2",
    voiceSpeed: 1.2,
    
    // Behavior
    firstMessage: "Hello! How can I help you today?",
    disableInterruptions: false,
    maxCallDuration: 600,
    
    // Language
    language: "en",
    autoDetectLanguage: false
  }
});

See the Agent Settings Override section for complete documentation of all available override settings.

RTL (Right-to-Left) Support

// For Hebrew, Arabic, etc.
const widget = new TTPAgentSDK.TTPChatWidget({
  agentId: 'agent_123',
  appId: 'your_app_id',
  direction: 'rtl',
  language: 'he',                  // Hebrew
  position: 'bottom-left'          // Better for RTL
});

Widget Methods

// Open chat from your own button
document.getElementById('myButton').onclick = () => {
  widget.open();
};

widget.close();

widget.toggle();

widget.minimize();

widget.maximize();

widget.destroy();

widget.updateConfig({
  primaryColor: '#FF5733',
  language: 'es',
  agentName: 'Jordan'  // root only; voice.agentName in this object is ignored
});

Widget Event Callbacks

Pass these as top-level config properties when constructing TTPChatWidget:

const widget = new TTPAgentSDK.TTPChatWidget({
  agentId: 'agent_123',
  appId: 'your_app_id',

  onConversationStart: () => {
    console.log('Voice conversation started');
  },

  onConversationEnd: () => {
    console.log('Voice conversation ended');
  },

  onBargeIn: () => {
    console.log('User interrupted the agent');
  },

  onAudioStartPlaying: () => {
    console.log('Agent audio started');
  },

  onAudioStoppedPlaying: () => {
    console.log('Agent audio stopped');
  },

  onSubtitleDisplay: (subtitle) => {
    console.log('Subtitle:', subtitle);
  },

  onVoiceCallButtonClick: () => {
    // Return false to prevent starting the call
    return true;
  }
});

For lower-level VoiceSDK events (onConnected, onMessage, etc.), use widget.voiceInterface.sdk or instantiate VoiceSDK directly. See Events & Callbacks.

Complete Example

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>My Website with AI Chat</title>
</head>
<body>
  <h1>Welcome to my website!</h1>
  
  <!-- Load the SDK -->
  <script src="https://unpkg.com/ttp-agent-sdk/dist/agent-widget.js"></script>
  
  <!-- Initialize widget -->
  <script>
    const widget = new TTPAgentSDK.TTPChatWidget({
      agentId: 'agent_123',
      appId: 'your_app_id',
      
      // Customize appearance
      primaryColor: '#7C3AED',
      position: 'bottom-right',
      language: 'en',
      
      // Custom branding
      chatWindow: {
        title: 'Chat with us!',
        subtitle: 'We typically reply instantly'
      },
      
      // Pass context variables
      variables: {
        userName: 'Visitor',
        page: window.location.pathname,
        referrer: document.referrer
      },
      
      // Event handlers
      onReady: () => {
        console.log('Chat widget ready!');
      },
      
      onMessage: (message) => {
        // Track messages in analytics
        console.log('Message:', message);
      }
    });
    
    // Optional: Open chat programmatically
    // widget.open();
  </script>
</body>
</html>

Configuration Reference

Agent naming: use root agentName for the voice idle hero and avatar letter fallback. Do not use voice.agentName (removed on merge). Do not use legacy root headerTitle (ignored). See Agent display name above.

See the live customization demo to experiment with all options interactively, including quick themes (Default, Light, Sunset, Hebrew, S-Law).

// WordPress (default - Shadow DOM enabled)
const widget = new TTPChatWidget({
  agentId: 'your-agent-id',
  // useShadowDOM defaults to true, so widget is isolated from theme CSS
});

// Shopify (disable Shadow DOM)
const widget = new TTPChatWidget({
  agentId: 'your-agent-id',
  useShadowDOM: false  // Required for Shopify compatibility
});

// If widget is invisible, try disabling Shadow DOM
const widget = new TTPChatWidget({
  agentId: 'your-agent-id',
  useShadowDOM: false  // Fallback if Shadow DOM causes rendering issues
});

const widget = new TTPAgentSDK.TTPChatWidget({
  agentId: 'your-agent-id',
  appId: 'your-app-id',
  panel: {
    backgroundColor: '#ffffff',
    border: '1px solid rgba(0,0,0,0.06)'
  },
  voice: {
    pillGradient: 'linear-gradient(135deg, #7c3aed, #6d28d9)',
    pillTextColor: '#ffffff',
    heroGradient1: '#ede9fe',
    heroGradient2: '#f5f3ff',
    agentNameColor: '#1e1b4b',
    headlineColor: '#1e1b4b',
    sublineColor: '#6b7280',
    primaryBtnGradient1: '#7c3aed',
    primaryBtnGradient2: '#a78bfa',
    secondaryBtnBg: '#f5f3ff',
    secondaryBtnBorder: 'rgba(124,58,237,0.15)',
    secondaryBtnTextColor: '#6d28d9'
  }
});

const widget = new TTPAgentSDK.TTPChatWidget({
  agentId: 'your-agent-id',
  appId: 'your-app-id',
  direction: 'rtl',
  header: { title: 'עוזרת חכמה', onlineIndicatorText: 'מחוברת' },
  panel: { backgroundColor: '#0f172a' },
  voice: {
    pillGradient: 'linear-gradient(135deg, #1e3a5f, #1e40af, #0f172a)',
    avatarGradient1: '#3b82f6',
    avatarGradient2: '#1d4ed8',
    heroGradient1: '#1a2744',
    heroGradient2: '#0f172a',
    primaryBtnGradient1: '#3b82f6',
    primaryBtnGradient2: '#1d4ed8',
    startCallButtonText: 'התחל שיחה קולית',
    sendMessageText: 'שלח הודעה',
    agentRole: 'עוזרת קולית חכמה',
    headline: 'היי, מה שלומך? 👋',
    subline: 'שאל/י אותי הכל — אני עונה מיידית בקול או בטקסט.'
  }
});

visualAssistant: {
  enabled: true,           // Required to activate visual assistant
  allowHighlight: true,    // highlight_element
  allowScroll: true,       // scroll_to_element
  allowNavigate: true,     // navigate_to
  allowFillForm: true,     // fill_form
  allowClick: true         // click_element
}

📖 Tip: All configuration options support the spread operator (...), so you can pass additional custom properties that will be merged with defaults.

Use Cases

Vanilla JavaScript Guide

Use the SDK in any JavaScript application without frameworks.

Complete Example

import { VoiceSDK } from 'ttp-agent-sdk';

class VoiceAssistant {
  constructor() {
    this.sdk = null;
    this.isConnected = false;
    this.isRecording = false;
  }

  async initialize(agentId, overrides = {}) {
    this.sdk = new VoiceSDK({
      agentId: agentId,
      appId: 'your_app_id',
      agentSettingsOverride: overrides
    });

    // Setup event listeners
    this.setupEventListeners();

    // Connect
    await this.sdk.connect();
  }

  setupEventListeners() {
    this.sdk.on('connected', () => {
      this.isConnected = true;
      this.updateUI('connected');
    });

    this.sdk.on('disconnected', () => {
      this.isConnected = false;
      this.isRecording = false;
      this.updateUI('disconnected');
    });

    this.sdk.on('recordingStarted', () => {
      this.isRecording = true;
      this.updateUI('recording');
    });

    this.sdk.on('recordingStopped', () => {
      this.isRecording = false;
      this.updateUI('connected');
    });

    this.sdk.on('message', (msg) => {
      if (msg.type === 'agent_response') {
        this.displayMessage('agent', msg.agent_response);
      }
    });

    this.sdk.on('error', (error) => {
      this.handleError(error);
    });
  }

  async toggleRecording() {
    if (!this.isConnected) return;

    if (this.isRecording) {
      await this.sdk.stopRecording();
    } else {
      await this.sdk.startRecording();
    }
  }

  disconnect() {
    if (this.sdk) {
      this.sdk.disconnect();
      this.sdk = null;
    }
  }

  updateUI(state) {
    // Update your UI based on state
  }

  displayMessage(role, text) {
    // Display message in your UI
  }

  handleError(error) {
    // Handle errors
  }
}

// Usage
const assistant = new VoiceAssistant();

await assistant.initialize('agent_123', {
  language: 'es',
  temperature: 0.9
});

React Integration

Use the SDK in React applications with hooks and components.

Using Hooks

import React, { useState, useEffect, useRef } from 'react';
import { VoiceSDK } from 'ttp-agent-sdk';

function VoiceChat() {
  const [status, setStatus] = useState('disconnected');
  const [isRecording, setIsRecording] = useState(false);
  const [messages, setMessages] = useState([]);
  const sdkRef = useRef(null);

  // Initialize SDK
  useEffect(() => {
    async function initSDK() {
      const sdk = new VoiceSDK({
        agentId: 'agent_123',
        appId: 'your_app_id',
        agentSettingsOverride: {
          language: 'es',
          temperature: 0.9
        }
      });

      // Event listeners
      sdk.on('connected', () => setStatus('connected'));
      sdk.on('disconnected', () => {
        setStatus('disconnected');
        setIsRecording(false);
      });
      sdk.on('recordingStarted', () => setIsRecording(true));
      sdk.on('recordingStopped', () => setIsRecording(false));
      sdk.on('message', (msg) => {
        if (msg.type === 'agent_response') {
          setMessages(prev => [
            ...prev,
            { role: 'agent', text: msg.agent_response }
          ]);
        }
      });

      await sdk.connect();
      sdkRef.current = sdk;
    }

    initSDK();

    // Cleanup
    return () => {
      if (sdkRef.current) {
        sdkRef.current.disconnect();
      }
    };
  }, []);

  const toggleRecording = async () => {
    if (sdkRef.current) {
      await sdkRef.current.toggleRecording();
    }
  };

  return (
    <div>
      <div>Status: {status}</div>
      <button onClick={toggleRecording} disabled={status !== 'connected'}>
        {isRecording ? 'Stop' : 'Start'} Recording
      </button>
      <div>
        {messages.map((msg, i) => (
          <div key={i}>{msg.role}: {msg.text}</div>
        ))}
      </div>
    </div>
  );
}

export default VoiceChat;

VoiceButton Component

Pre-built React component for quick integration.

Installation

import { VoiceButton } from 'ttp-agent-sdk/react';

Basic Usage

import React from 'react';
import { VoiceButton } from 'ttp-agent-sdk/react';

function App() {
  return (
    <VoiceButton
      agentId="agent_123"
      appId="your_app_id"
      agentSettingsOverride={{
        language: 'es',
        temperature: 0.9
      }}
      onConnected={() => console.log('Connected')}
      onMessage={(msg) => console.log('Message:', msg)}
      onError={(error) => console.error('Error:', error)}
    />
  );
}

Props

Event Callbacks

Fly-to-Cart Animation

When users tap "Add to Cart" in the e-commerce widget, the product card flies down into the cart icon with a tornado funnel effect. Uses html2canvas to capture the card as an image, avoiding stacking/overflow clipping in iframes or containers with overflow:hidden. Built-in for TTPEcommerceWidget. For custom React product widgets, use the useFlyToCart hook.

React Hook (useFlyToCart)

import { useFlyToCart } from 'ttp-agent-sdk';

const cartIconRef = useRef(null);
const { triggerFly, isAnimating } = useFlyToCart(cartIconRef);

const handleAddToCart = (product) => {
  triggerFly(cardRef.current, product, () => {
    addToCartAPI(product);
  });
};

Vanilla JS (FlyToCart)

import { FlyToCart } from 'ttp-agent-sdk';

const flyToCart = new FlyToCart({
  getCartTarget: () => document.querySelector('.cart-icon'),
  onCartBump: () => { /* optional */ },
});

flyToCart.triggerFly(cardElement, product, () => {
  addToCartAPI(product);
});

E-commerce: Adding to cart (partner API & front-end cart)

Cart logic is owned by your integration (partner APIs, mock stores, or a Shopify storefront). The widget renders products, sends user intent over the WebSocket, and updates the cart summary when it receives the right messages. The SDK does not keep its own authoritative cart array: totals and counts should reflect what the backend (or the browser cart, then the backend) considers true.

Who does the "add"?

Flow 1 — User taps Add / Update on a product card

1. The widget stops playback (so the agent does not talk over the action) and sends t: "product_selected" on the voice WebSocket. Payload includes productId, productName, price, quantity (absolute units or weight), and sellBy (quantity or weight).
2. Your backend decides what happens next:
   • Update the partner cart via API, then push cart_updated to the session, or
   • For Shopify on-domain, send add_to_store_cart and let the widget perform /cart/add.js, then consume cart_add_result on the server, or
   • Hybrid: tool or internal step on the server plus a follow-up cart_updated.
3. EcommerceManager.handleCartUpdated updates the bottom cart bar when cartTotal and cartItemCount are present. Optional action: "added" drives the short confirmation UX.

Flow 2 — User asks in voice (or the agent adds without a card click)

For TTP's Java conversation backend, e-commerce uses internal tools such as search_products, add_to_cart, and get_cart (names match what the LLM sees; they are not the same as optional SDK client_tool_call names unless you wire them).

1. The model calls add_to_cart with productId and quantity.
2. The server runs the partner integration:
   • Partner API cart: The service calls the partner, receives success and line items, then typically sends cart_updated to the widget with totals and item count so the UI matches the server.
   • Shopify theme cart: The service sends add_to_store_cart to the widget instead of mutating cart only on the server; the widget performs the Ajax add and returns cart_add_result. Reading the cart may use get_store_cart from the server, which the widget answers with cart_state_result after GET /cart.js.
3. The model receives a text summary of the tool result and can speak to the user.

Message cheat sheet (widget ↔ backend)

Custom client tools: Register handlers with registerToolHandler on the widget or AgentSDK so the toolName in each client_tool_call matches what your backend defines.

Widget Flavors

The TTP SDK supports domain-specific "flavors" that customize the widget experience for different verticals. Each flavor provides specialized UI components, backend tools, and message handling.

Available Flavors

E-commerce WebSocket messages: The widget listens for t: "show_items" and t: "show_products" (same handler; backend tools often send show_products with products, title, layout). Use TTPEcommerceWidget (or TTPChatWidget with flavor.type: 'ecommerce') so these handlers are registered. For end-to-end add-to-cart flows (partner API vs Shopify Ajax vs client tools), see E-commerce cart flows.

Desktop voice strip (flavor.callView: 'minimized'): When flavor.callView is 'minimized', a fixed bottom voice surface is used on viewports wider than 768px. This works with or without a flavor: it requires only a flavor object carrying callView: 'minimized' (e.g. flavor: { callView: 'minimized' }, no type/partner needed) — a flavor type is optional. It is a rounded floating dock with layered glass styling (rim light, ambient shadow, soft indigo outer glow), an inset transcript “well,” squircle control tiles, and a blurred LIVE capsule—same layout and controls as before. It is centered with a capped width (up to about 720px with horizontal inset) and bottom spacing plus env(safe-area-inset-bottom). It uses the same tokens as the floating pill (voice.pillGradient, pillTextColor, pillDotColor, endCallButtonColor). Mute, pause, speaker, keyboard (text inject), and end-call align with the in-widget voice UI. The in-panel desktop “active call” UI stays hidden; the floating panel collapses if it was open, and the launcher pill is hidden for the duration of the call. The “Powered by” footer (footer.show, footer.brand) is rendered inside the strip (below the call controls) for the duration of the call—not as a separate floating pill at the widget corner. Viewports 768px and below skip the strip and use the mobile minimized bar flow instead. A single transcript line shows either the assistant (streaming) or the user (speech-to-text interim/final), never both at once; user text uses a warm highlight color and no You: prefix. When the widget language is Hebrew or Arabic (base he / ar) or direction is 'rtl', chrome uses RTL. Transcript lines use explicit dir="rtl" when the string contains Hebrew or Arabic letters (even if the widget Language is English), so neutral punctuation follows the sentence; Latin-only lines use dir="ltr". Mic/pause/speaker/end layout is unchanged.

Configuration

Set the flavor when creating the widget:

const widget = new TTPChatWidget({
  agentId: 'agent_...',
  appId: 'app_...',
  flavor: {
    type: 'pharma',        // 'ecommerce' | 'hotels' | 'pharma' | 'restaurants' | 'tours'
    partnerId: 'mock-pharm' // partner-specific data source
  }
});

Pharma Flavor

The pharmacy flavor provides a medication search and prescription management experience.

// Pharma configuration
flavor: {
  type: 'pharma',
  partnerId: 'mock-pharm'
}

// Backend tools injected automatically:
// - search_medications: Search the medication catalog
// - add_to_prescription: Add a medication to the prescription
// - get_prescription: View current prescription contents

// Frontend message types:
// - show_items: Displays medication cards
// - prescription_updated: Updates the prescription summary bar

The pharma flavor does not include gallery support (show_media).

Restaurants Flavor

The restaurants flavor provides a menu browsing, ordering, and photo gallery experience.

// Restaurant configuration
flavor: {
  type: 'restaurants',
  partnerId: 'mock-restaurant'
}

// Backend tools injected automatically:
// - search_menu: Search the restaurant menu
// - add_to_order: Add a menu item to the order
// - get_order: View current order contents
// - show_media: Display restaurant photo gallery

// Frontend message types:
// - show_items: Displays menu item cards
// - order_updated: Updates the order summary bar
// - show_media / dismiss_media: Gallery with dish photos, ambiance, etc.

Menu item cards display allergen warnings and dietary tags automatically.

Tours Flavor

The tours flavor provides a tour browsing, booking, and photo gallery experience.

// Tour configuration
flavor: {
  type: 'tours',
  partnerId: 'mock-tour'
}

// Backend tools injected automatically:
// - search_tours: Search available tours and activities by query
// - book_tour: Book a tour or activity
// - get_tour_booking: View current booking contents
// - show_media: Display tour photo gallery

// Frontend message types:
// - show_items: Displays tour item cards
// - cart_updated: Updates the booking summary bar
// - show_media / dismiss_media: Gallery with tour photos, highlights, etc.

Tour cards display activity tags and pricing. Reuses the same UI infrastructure as the restaurants flavor.

Client-Script Tools

Client-script tools let you attach backend-authored JavaScript to a client tool (tool_type: 'client') that runs in the visitor's browser when the LLM calls the tool — no host-page registerToolHandler() code required. The script is configured in the dashboard (tool form → "Agent scripts"), stored with the tool, and delivered to the widget automatically at session start. Available since SDK v2.45.3.

How it works

1. Session init — the backend pushes a partner_bundle message with the reserved partner id __client_tools__, carrying every scripted tool attached to the agent (plus auto-run library scripts). The bundle is pushed on both the voice and text-chat channels, works with or without a widget flavor, and is re-pushed on reconnect (a new bundle replaces the previous one).
2. Compile on load — each entry compiles into a strict-mode async function immediately when the bundle lands. A syntax error poisons only that entry (logged at load time); the rest of the bundle still works.
3. Invocation — when the LLM calls the tool, the SDK runs the compiled script and returns its result to the backend.
4. Auto-run — library scripts flagged auto_run execute exactly once when the bundle arrives (e.g. to set up page listeners).

Authoring styles

Two styles compile — the SDK detects the shape and picks the right wrapping:

// 1. Raw statement body (typical in the dashboard UI; `ctx` is in scope)
alert("hi");
return { ok: true, clicked: true };

// 2. Function expression
async (ctx) => {
  const res = await ctx.fetch('/api/something');
  return { ok: true, data: await res.json() };
}

Both run inside an async function, so await is always allowed. A script that returns nothing resolves to { ok: true }.

The ctx object

Client-tool scripts run with a generic context — the ecommerce-only helpers (ctx.platform, ctx.normalizeCart, ctx.refreshUI) are not available; those exist only in ecommerce partner-adapter bundles.

Pre / main / post steps

A scripted tool may carry up to three steps, executed in order: pre → main → post.

• Pre is best-effort — if it throws, the error is logged and main still runs.
• Main produces the tool result. A thrown error becomes an ok: false result.
• Post runs only when main succeeded; its ctx.args._mainResult holds main's return value.

Sync vs async results

Configured per tool in the dashboard ("Wait for result"):

Limits

Wire protocol (reference)

Script invocations arrive on two wire forms, both handled by the SDK on both channels:

// Bundle (session init / reconnect)
{ "t": "partner_bundle", "partner_id": "__client_tools__",
  "adapters": { "my_tool": { "code_js": "...", "pre_code_js": "...", "post_code_js": "..." } },
  "autoRun":  { "my_script": { "code_js": "..." } } }

// Form A — dedicated envelope (async path / chain steps)
→ { "t": "run_partner_script", "requestId": "...", "partnerId": "__client_tools__",
    "action": "my_tool", "args": { ... } }
← { "t": "run_partner_script_result", "requestId": "...", "ok": true, "result": { ... } }

// Form B — client_tool_call piggyback (backend sync-await path)
→ { "t": "client_tool_call", "toolCallId": "...", "toolName": "run_partner_script",
    "parameters": { "action": "my_tool", "args": { ... }, "partner_id": "__client_tools__" } }
← { "t": "client_tool_result", "toolCallId": "...", "result": { ... } }

Routing rule: partner_id === '__client_tools__' → the flavor-independent ClientScriptManager; any other partner id → the ecommerce flavor's partner-adapter path. The two bundles coexist.

Troubleshooting

For the full internals reference see CLIENT_SCRIPT_TOOLS_GUIDE.md in the repository root.

Chain Tools

A chain tool (tool_type: 'chain') runs a graph of steps — server tools, client tools, agent switches, and library scripts — as a single LLM tool call. Chains are edited visually in the dashboard (Chain Editor: nodes are steps, edges are data-flow dependencies) and executed by the backend; the widget participates only when a step targets the browser.

Execution model

• Steps with no dependencies run first; steps at the same level run in parallel; levels run sequentially.
• A step's input_from lists the upstream steps whose outputs feed it. Dependent steps receive the original LLM arguments shallow-merged with each upstream result (a non-object result lands under the upstream step's id).
• Fail-fast: the first failing step aborts the chain with { "success": false, "error": ..., "step": ... }.
• The LLM receives the terminal step's result (multiple terminals merge keyed by step id).
• Inside a chain, every step is awaited — a referenced client tool's own "wait for result = off" setting is ignored.

Step types and the widget

Scripts and scripted client tools referenced by a chain are resolved into the session bundle automatically — they do not need to be attached to the agent.

Constraints

• 1–30 steps per chain; no cycles; chains cannot reference other chain tools (no recursion).
• Deleting a tool or library script that a chain references is blocked in the dashboard (the delete dialog lists the referencing chains).

VoiceSDK Class

Core class for voice interaction functionality. Server-driven disclaimer gating (compliance copy before STT/greeting) is described in detail under Server-driven disclaimer (voice) and applies when using protocol v2.

Constructor

new VoiceSDK(config)

Configuration Object

Audio Format Configuration (v2 Protocol)

The SDK v2 supports format negotiation with the backend. You can specify both input and output audio formats:

Example: Custom Audio Format

const voiceSDK = new VoiceSDK({
  agentId: 'agent_123',
  appId: 'your_app_id',
  
  // Input format (what we send to server)
  sampleRate: 16000,
  channels: 1,
  bitDepth: 16,
  
  // Output format (what we want from server)
  outputContainer: 'raw',        // 'raw' or 'wav'
  outputEncoding: 'pcm',          // 'pcm', 'pcmu', 'pcma'
  outputSampleRate: 24000,       // Default; typical TTS/server output
  outputChannels: 1,
  outputBitDepth: 16,
  outputFrameDurationMs: 600,    // Frame duration for streaming
  
  // Protocol version
  protocolVersion: 2              // Use v2 protocol for format negotiation
});

// Listen for format negotiation
voiceSDK.on('formatNegotiated', (format) => {
  console.log('Format negotiated:', format);
  // format contains: { container, encoding, sampleRate, channels, bitDepth }
});

Methods

await voiceSDK.connect();

voiceSDK.disconnect();

await voiceSDK.startRecording();

voiceSDK.sendDisclaimerAck(true);

await voiceSDK.stopRecording();

await voiceSDK.toggleRecording();

voiceSDK.pauseCall();
// or via AgentSDK:
agentSDK.pauseCall();

voiceSDK.resumeCall();
// or via AgentSDK:
agentSDK.resumeCall();

if (voiceSDK.isPaused) {
  console.log('Call is paused');
}
// or via AgentSDK:
if (agentSDK.isPaused) {
  console.log('Call is paused');
}

const status = voiceSDK.getStatus();
// Returns: {
//   version: '2.0.0',
//   isConnected: boolean,
//   isRecording: boolean,
//   isPlaying: boolean,
//   outputFormat: object,      // Negotiated output format (v2)
//   audioPlayer: object,       // AudioPlayer status
//   audioRecorder: object      // AudioRecorder status
// }

const error = voiceSDK.validateInputFormat({
  encoding: 'pcm',
  sampleRate: 16000,
  bitDepth: 16,
  channels: 1
});
if (error) {
  console.error('Invalid format:', error);
}

const error = voiceSDK.validateOutputFormat({
  container: 'raw',
  encoding: 'pcm',
  sampleRate: 24000,
  bitDepth: 16,
  channels: 1
});
if (error) {
  console.error('Invalid format:', error);
}

voiceSDK.updateConfig({
  outputSampleRate: 48000,
  outputEncoding: 'pcmu'
});

await voiceSDK.reconnect();

voiceSDK.stopAudioPlayback();

voiceSDK.on('connected', () => {
  console.log('Connected!');
});

voiceSDK.destroy();

Events Reference

Configuration Options

Agent Settings Override

Complete reference for all overridable settings:

Core Settings

Voice Settings

Behavior Settings

Advanced Settings

Text-to-Speech API

PUBLIC REST API

Generate high-quality voice audio from text using our public REST API endpoint.

Endpoint

POST https://backend.talktopc.com/api/public/agents/tts/generate

Authentication

Include your API key in the Authorization header:

Authorization: Bearer YOUR_API_KEY

Request Parameters

Example Requests

curl -X POST https://backend.talktopc.com/api/public/agents/tts/generate \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "text": "Hello! Welcome to our service.",
    "voiceId": "nova",
    "voiceSpeed": 1.2,
    "language": "en"
  }' \
  --output speech.mp3

const response = await fetch('https://backend.talktopc.com/api/public/agents/tts/generate', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${process.env.TTP_API_KEY}`
  },
  body: JSON.stringify({
    text: 'Hello! Welcome to our service.',
    voiceId: 'nova',
    voiceSpeed: 1.2,
    language: 'en'
  })
});

// Response is audio file
const audioBuffer = await response.arrayBuffer();
const audioBlob = new Blob([audioBuffer], { type: 'audio/mpeg' });

// Play or save the audio
const audioUrl = URL.createObjectURL(audioBlob);
const audio = new Audio(audioUrl);
audio.play();

import requests
import os

url = "https://backend.talktopc.com/api/public/agents/tts/generate"
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {os.environ['TTP_API_KEY']}"
}
data = {
    "text": "Hello! Welcome to our service.",
    "voiceId": "nova",
    "voiceSpeed": 1.2,
    "language": "en"
}

response = requests.post(url, json=data, headers=headers)

if response.status_code == 200:
    # Save audio to file
    with open("speech.mp3", "wb") as f:
        f.write(response.content)
    print("Audio saved to speech.mp3")
else:
    print(f"Error: {response.status_code}")

<?php
$url = "https://backend.talktopc.com/api/public/agents/tts/generate";
$apiKey = getenv('TTP_API_KEY');

$data = [
    'text' => 'Hello! Welcome to our service.',
    'voiceId' => 'nova',
    'voiceSpeed' => 1.2,
    'language' => 'en'
];

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, [
    'Content-Type: application/json',
    'Authorization: Bearer ' . $apiKey
]);

$response = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);

if ($httpCode === 200) {
    file_put_contents('speech.mp3', $response);
    echo "Audio saved to speech.mp3";
} else {
    echo "Error: HTTP $httpCode";
}
?>

import java.net.http.*;
import java.net.URI;
import java.nio.file.*;

public class TTSExample {
    public static void main(String[] args) throws Exception {
        String url = "https://backend.talktopc.com/api/public/agents/tts/generate";
        String apiKey = System.getenv("TTP_API_KEY");
        
        String json = """
            {
                "text": "Hello! Welcome to our service.",
                "voiceId": "nova",
                "voiceSpeed": 1.2,
                "language": "en"
            }
            """;
        
        HttpClient client = HttpClient.newHttpClient();
        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create(url))
            .header("Content-Type", "application/json")
            .header("Authorization", "Bearer " + apiKey)
            .POST(HttpRequest.BodyPublishers.ofString(json))
            .build();
        
        HttpResponse<byte[]> response = client.send(
            request, 
            HttpResponse.BodyHandlers.ofByteArray()
        );
        
        if (response.statusCode() == 200) {
            Files.write(Paths.get("speech.mp3"), response.body());
            System.out.println("Audio saved to speech.mp3");
        } else {
            System.out.println("Error: " + response.statusCode());
        }
    }
}

Response

The endpoint returns audio data directly with the following headers:

Voice Speed Examples

Backend Implementation Example

// Your backend endpoint
app.post('/api/generate-speech', async (req, res) => {
  const { text, voiceSpeed = 1.0 } = req.body;
  
  // Call TTP TTS API
  const ttpResponse = await fetch('https://backend.talktopc.com/api/public/agents/tts/generate', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${process.env.TTP_API_KEY}`  // 🔒 Secret!
    },
    body: JSON.stringify({
      text: text,
      voiceSpeed: voiceSpeed,
      voiceId: 'nova',
      language: 'en'
    })
  });
  
  if (!ttpResponse.ok) {
    return res.status(ttpResponse.status).json({ 
      error: 'TTS generation failed' 
    });
  }
  
  // Forward audio to client
  const audioBuffer = await ttpResponse.arrayBuffer();
  res.set('Content-Type', 'audio/mpeg');
  res.send(Buffer.from(audioBuffer));
});

Error Responses

Use Cases

Java SDK

Server-side Java SDK for text-to-speech conversion. Perfect for backend applications, phone systems, and server-to-server integrations.

Installation

Maven

<repositories>
    <repository>
        <id>github</id>
        <url>https://maven.pkg.github.com/TTP-GO/java-sdk</url>
    </repository>
</repositories>

<dependencies>
    <dependency>
        <groupId>com.talktopc</groupId>
        <artifactId>ttp-agent-sdk-java</artifactId>
        <version>1.0.5</version>
    </dependency>
</dependencies>

<settings>
    <servers>
        <server>
            <id>github</id>
            <username>YOUR_GITHUB_USERNAME</username>
            <password>YOUR_GITHUB_TOKEN</password>
        </server>
    </servers>
</settings>

Gradle

repositories {
    maven {
        url = uri("https://maven.pkg.github.com/TTP-GO/java-sdk")
        credentials {
            username = project.findProperty("gpr.user") ?: System.getenv("USERNAME")
            password = project.findProperty("gpr.key") ?: System.getenv("TOKEN")
        }
    }
}

dependencies {
    implementation 'com.talktopc:ttp-agent-sdk-java:1.0.5'
}

Quick Start

1. Initialize SDK

import com.talktopc.sdk.VoiceSDK;

// Get API key from environment variable
String apiKey = System.getenv("TALKTOPC_API_KEY");

VoiceSDK sdk = VoiceSDK.builder()
    .apiKey(apiKey)
    .baseUrl("https://speech.talktopc.com")  // Optional
    .build();

2. Simple TTS (Blocking)

// Generate complete audio file
byte[] audio = sdk.textToSpeech("Hello world", "mamre");

// Save to file
Files.write(Paths.get("output.wav"), audio);

// Or send to phone system
phoneSystem.playAudio(audio);

3. Streaming TTS (Real-time)

// Stream audio chunks as they're generated
sdk.textToSpeechStream(
    "Hello world, this is a longer text that will be streamed",
    "mamre",
    audioChunk -> {
        // Receive chunks in real-time
        phoneSystem.playAudio(audioChunk);
    }
);

API Reference

VoiceSDK

Main SDK entry point for text-to-speech operations.

Builder Methods

Methods

TTSRequest Builder

Configure TTS requests with audio format options.

Basic Configuration

TTSRequest request = TTSRequest.builder()
    .text("Hello world")              // Required
    .voiceId("mamre")                // Required
    .speed(1.0)                      // Optional (0.1 - 3.0)
    .build();

Audio Format Configuration

TTSRequest request = TTSRequest.builder()
    .text("Hello world")
    .voiceId("mamre")
    .outputContainer("raw")          // "raw" or "wav"
    .outputEncoding("pcm")           // "pcm", "pcmu", "pcma"
    .outputSampleRate(24000)         // Hz (8000, 16000, 22050, 24000, 44100, 48000)
    .outputBitDepth(16)              // bits (8, 16, 24)
    .outputChannels(1)               // 1 (mono) or 2 (stereo)
    .outputFrameDurationMs(600)     // ms per frame (for streaming)
    .build();

Preset Methods

TTSResponse

Response object containing audio and metadata.

Examples

Basic Usage

VoiceSDK sdk = VoiceSDK.builder()
    .apiKey(System.getenv("TALKTOPC_API_KEY"))
    .build();

// Simple TTS
byte[] audio = sdk.textToSpeech("Welcome to TalkToPC", "mamre");
System.out.println("Generated " + audio.length + " bytes of audio");

// Save to file
Files.write(Paths.get("output.wav"), audio);

With Speed Control

// Faster speech (1.5x speed)
byte[] fastAudio = sdk.textToSpeech("Quick message", "mamre", 1.5);

// Slower speech (0.8x speed)
byte[] slowAudio = sdk.textToSpeech("Slow and clear", "mamre", 0.8);

Streaming with Metadata

sdk.textToSpeechStream(
    TTSRequest.builder()
        .text("Streaming example with full configuration")
        .voiceId("mamre")
        .speed(1.0)
        .build(),
    audioChunk -> {
        // Handle each audio chunk
        System.out.println("Received chunk: " + audioChunk.length + " bytes");
        phoneSystem.playAudio(audioChunk);
    },
    metadata -> {
        // Handle completion
        System.out.println("Stream completed:");
        System.out.println("  Total chunks: " + metadata.getTotalChunks());
        System.out.println("  Total bytes: " + metadata.getTotalBytes());
        System.out.println("  Duration: " + metadata.getDurationMs() + " ms");
        System.out.println("  Credits: " + metadata.getCreditsUsed());
    },
    error -> {
        // Handle errors
        System.err.println("Stream error: " + error.getMessage());
    }
);

Phone System Integration

Perfect for Twilio, Telnyx, or custom VoIP systems.

Standard Phone System (PCMU @ 8kHz)

// Using convenient phoneSystem() preset
TTSRequest request = TTSRequest.builder()
    .text("Hello, thank you for calling. How can I help you today?")
    .voiceId("en-US-female")
    .phoneSystem()  // ✅ PCMU @ 8kHz, 20ms frames
    .build();

sdk.textToSpeechStream(
    request,
    audioChunk -> {
        // audioChunk is PCMU @ 8kHz, 20ms frames (160 bytes)
        // Ready to send directly to phone connection
        phoneConnection.sendAudio(audioChunk);
    }
);

Twilio Integration

TTSRequest request = TTSRequest.builder()
    .text("Your appointment is confirmed for tomorrow at 3 PM")
    .voiceId("en-US-male")
    .outputContainer("raw")
    .outputEncoding("pcmu")      // μ-law for Twilio
    .outputSampleRate(8000)       // 8kHz
    .outputBitDepth(16)
    .outputChannels(1)            // Mono
    .outputFrameDurationMs(20)    // 20ms frames
    .build();

sdk.textToSpeechStream(
    request,
    audioChunk -> {
        // Send to Twilio Media Stream
        twilioStream.sendMedia(audioChunk);
    }
);

Custom Audio Format

TTSRequest request = TTSRequest.builder()
    .text("Custom format example")
    .voiceId("mamre")
    .outputContainer("raw")
    .outputEncoding("pcm")
    .outputSampleRate(16000)      // 16kHz
    .outputBitDepth(16)           // 16-bit
    .outputChannels(1)            // Mono
    .outputFrameDurationMs(100)   // 100ms frames
    .build();

byte[] audio = sdk.textToSpeech(request);
// Expected: 16kHz PCM, 16-bit, mono

High Quality Audio

TTSRequest request = TTSRequest.builder()
    .text("This is a high quality recording")
    .voiceId("mamre")
    .highQuality()  // WAV @ 44.1kHz
    .build();

byte[] audio = sdk.textToSpeech(request);
Files.write(Paths.get("high_quality.wav"), audio);

Error Handling

import com.talktopc.sdk.exception.TtsException;

try {
    byte[] audio = sdk.textToSpeech("Test", "mamre");
} catch (TtsException e) {
    System.err.println("TTS Error [" + e.getStatusCode() + "]: " + e.getErrorMessage());
    
    switch (e.getStatusCode()) {
        case 401:
            System.err.println("→ Invalid API key");
            break;
        case 402:
            System.err.println("→ Insufficient credits");
            break;
        case 400:
            System.err.println("→ Invalid parameters");
            break;
        default:
            System.err.println("→ Other error");
    }
}

Full Configuration Example

import com.talktopc.sdk.models.TTSRequest;
import com.talktopc.sdk.models.TTSResponse;

// Build request with all options
TTSRequest request = TTSRequest.builder()
    .text("Full configuration example")
    .voiceId("mamre")
    .speed(1.2)
    .outputContainer("wav")
    .outputEncoding("pcm")
    .outputSampleRate(24000)
    .outputBitDepth(16)
    .outputChannels(1)
    .build();

// Get response with metadata
TTSResponse response = sdk.synthesize(request);

System.out.println("Audio: " + response.getAudioSizeBytes() + " bytes");
System.out.println("Sample rate: " + response.getSampleRate() + " Hz");
System.out.println("Duration: " + response.getDurationMs() + " ms");
System.out.println("Credits: " + response.getCreditsUsed());

// Save audio
Files.write(Paths.get("output.wav"), response.getAudio());

Supported Audio Formats

Requirements

• Java 11 or higher
• Valid TalkToPC API key
• No external dependencies - Uses Java 11+ HttpClient

Resources

• GitHub Repository: TTP-GO/java-sdk
• Maven Package: com.talktopc:ttp-agent-sdk-java:1.0.5
• Documentation: See README.md in the repository
• Examples: Check src/main/java/com/talktopc/sdk/examples/