Interaction Events

Interaction Events let you control and customize live conversations with a PAL in real time. You can send interaction events to the Conversational Video Interface (CVI) and listen to events the PAL sends back during the call.

Interaction Types

Observable Events

Utterance Events
Utterance Streaming Events
Tool Call Events
Perception Tool Call Events
Perception Analysis Events
Canvas Interaction Events - Magic Canvas user actions (submit, skip, dismiss, etc.) on cards the PAL showed
Started/Stopped Speaking - conversation.started_speaking / conversation.stopped_speaking with properties.role of "pal" or "user" (legacy duplicate PAL events use "replica")

Which Interaction Should I Send?

Interaction	Use when
`conversation.respond`	A user typed a message and the PAL should respond as if the user had spoken that text. This is the natural text-input event for chat-style UI.
`conversation.echo`	Your app supplies text or audio for the PAL to speak directly, such as echo-mode or custom ASR flows.
`conversation.interrupt`	Your app needs to stop the PAL while it is speaking.

Daily `sendAppMessage` Payloads

Use Daily’s sendAppMessage(interaction, '*') to send interaction events over the call data channel.

Text input: `conversation.respond`

call.sendAppMessage(
  {
    message_type: 'conversation',
    event_type: 'conversation.respond',
    conversation_id: 'YOUR_CONVERSATION_ID',
    properties: {
      text: 'User message as if they had just finished speaking.',
    },
  },
  '*'
);

Face speaks supplied content: `conversation.echo`

call.sendAppMessage(
  {
    message_type: 'conversation',
    event_type: 'conversation.echo',
    conversation_id: 'YOUR_CONVERSATION_ID',
    properties: {
      modality: 'text',
      text: 'Text for the PAL to speak directly.',
      done: true,
    },
  },
  '*'
);

For audio echo, set modality: 'audio', pass base64 audio, include sample_rate, and keep done: false until the final audio chunk.

Stop the face: `conversation.interrupt`

call.sendAppMessage(
  {
    message_type: 'conversation',
    event_type: 'conversation.interrupt',
    conversation_id: 'YOUR_CONVERSATION_ID',
  },
  '*'
);

Event Ordering and Turn Tracking

All events broadcasted by Tavus include the following fields for timing, ordering, and grouping:

timestamp (number) - Unix timestamp (seconds since epoch) indicating when the event was created. Use this to build timestamped transcripts or reconstruct the full timeline of a conversation.
seq (integer) - A globally monotonic sequence number. Every event gets the next value in the sequence, so a higher seq always means the event was sent later. Use this to reconcile events that may arrive out of order over the data channel.
turn_idx (integer, optional) - The conversation turn index. This value increments each time a conversation.respond interaction is received, and groups all events that belong to the same conversational turn. Use it to correlate related events - for example, an utterance, its tool calls, and the PAL speaking state changes that all stem from the same user input. This field is present on conversation-related events (utterances, tool calls, speaking state changes, perception events, etc.) and omitted on events that are not tied to a specific turn.
inference_id (string, optional) - A stable identifier for a generated utterance or inference. Use it with conversation.utterance, conversation.utterance.streaming, and tool-call events to reconcile optimistic UI state with the final events Tavus emits.

Call Client Example

Interaction events use a WebRTC data channel for communication. In Tavus’s case, this is powered by Daily, which makes setting up the call client quick and simple.

Daily JS
Daily Python
Daily React

Here’s an example of using DailyJS to create a call client in JavaScript:

The Daily app-message event is used to send and receive events and interactions between your server and CVI.

<html>
  <script crossorigin src="https://unpkg.com/@daily-co/daily-js"></script>
  <body>
    <!-- Add input field and send button -->
    <input type="text" id="messageInput" placeholder="Enter your message">
    <button onclick="sendAppMessage()">Send Message</button>

    <script>
      call = window.Daily.createFrame();
      call.on('app-message', (event) => {
        console.log('app-message', event);
      });
      
      call.join({ url: 'YOUR_CONVERSATION_URL' });

      function sendAppMessage() {
        const messageInput = document.getElementById('messageInput');
        const message = messageInput.value;
        if (message) {
          const interaction = {
            "message_type": "conversation",
            "event_type": "conversation.respond",
            "conversation_id": "YOUR_CONVERSATION_ID",
            "properties": {
              "text": `${message}`
            }
          }
          const hi = call.sendAppMessage(interaction, '*');
          console.log('Sending message: ', hi);
          console.log('Sent message: ', interaction);
          messageInput.value = '';
        }
      }
    </script>
  </body>
</html>

Here’s an example of using Daily Python to create a call client in Python:

The Daily app-message event is used to send and receive events and interactions between your server and CVI.

call_client = None

class RoomHandler(EventHandler):
    def __init__(self):
        super().__init__()
    
    def on_app_message(self, message, sender: str) -> None:
        print(f"Incoming app message from {sender}: {message}")

def join_room(url):
    global call_client
    try:
        Daily.init()
        output_handler = RoomHandler()
        call_client = CallClient(event_handler=output_handler)
        call_client.join(url)
    except Exception as e:
        print(f"Error joining room: {e}")
        raise

def send_message(message):
    global call_client
    call_client.send_app_message(message)

Here’s an example of using Daily React to create a call client in React:

The Daily app-message event is used to send and receive events and interactions between your server and CVI.

"use client"

import React, { useEffect, useRef, useState } from 'react';


const TavusConversation = () => {
  const [message, setMessage] = useState('');
  const callRef = useRef(null);
  const containerRef = useRef(null);


  useEffect(() => {
    const loadDaily = async () => {
      const DailyIframe = (await import('@daily-co/daily-js')).default;


      callRef.current = DailyIframe.createFrame({
        iframeStyle: {
          width: '100%',
          height: '500px',
          border: '0',
        }
      });


      if (containerRef.current) {
        containerRef.current.appendChild(callRef.current.iframe());
      }


      callRef.current.on('app-message', (event) => {
        console.log('app-message received:', event);
      });


      callRef.current.join({
        url: 'YOUR_CONVERSATION_URL',
      });
    };


    loadDaily();


    return () => {
      if (callRef.current) {
        callRef.current.leave();
        callRef.current.destroy();
      }
    };
  }, []);


  const sendAppMessage = () => {
    if (!message || !callRef.current) return;


    const interaction = {
      message_type: 'conversation',
      event_type: 'conversation.respond',
      conversation_id: 'YOUR_CONVERSATION_ID',
      properties: { text: message }
    };


    callRef.current.sendAppMessage(interaction, '*');
    setMessage('');
  };


  return (
    <div className="w-full h-full flex flex-col items-center">
      <div ref={containerRef} className="w-full mb-4" />
      <div>
        <input
          type="text"
          className="border p-2 mr-2"
          value={message}
          onChange={(e) => setMessage(e.target.value)}
          placeholder="Type a message"
        />
        <button onClick={sendAppMessage} className="bg-blue-500 text-white px-4 py-2 rounded">
          Send Message
        </button>
      </div>
    </div>
  );
};


export default TavusConversation;

​Interaction Types

​Observable Events

​Which Interaction Should I Send?

​Daily sendAppMessage Payloads

​Text input: conversation.respond

​Face speaks supplied content: conversation.echo

​Stop the face: conversation.interrupt

​Event Ordering and Turn Tracking

​Call Client Example

Interaction Types

Observable Events

Which Interaction Should I Send?

Daily `sendAppMessage` Payloads

Text input: `conversation.respond`

Face speaks supplied content: `conversation.echo`

Stop the face: `conversation.interrupt`

Event Ordering and Turn Tracking

Call Client Example