{
  "message_type": "conversation",
  "event_type": "conversation.perception_tool_call",
  "conversation_id": "c123456",
  "properties": {
    "modality": "vision",
    "name": "notify_hat_detected",
    "arguments": "{\"reason\":\"The user said 'well, yeah'\"}",
    "frames": [
      {
        "data": "<string>",
        "mime_type": "image/jpeg"
      }
    ]
  }
}

Perception Tool Call Event

This is an event broadcasted by Tavus.

A perception_tool_call event is broadcast when a perception tool is triggered by Raven based on visual or audio input. The event always includes eventType conversation.perception_tool_call, a modality in data.properties ("vision" or "audio"), the tool name, and arguments.

Modality-specific payload:

modality: "audio" — Triggered by audio tools (audio_tool_prompt / audio_tools). arguments is a JSON string (e.g. "{\"reason\":\"The user said …\"}"). There is no frames array.
modality: "vision" — Triggered by visual tools (visual_tool_prompt / visual_tools). arguments is an object with tool-defined fields. Includes a frames array of objects with data (base64-encoded JPEG) and mime_type (e.g. "image/jpeg") for the images that triggered the call.

Perception tool calls can be used to trigger automated actions in response to visual or audio cues detected by the Raven perception system. For configuration details, see Tool Calling for Perception and Perception.

message_type

string

Message type indicates what product this event will be used for. In this case, the message_type will be conversation

Example:

"conversation"

event_type

string

This is the type of event that is being sent back. This field will be present on all events and can be used to distinguish between different event types.

Example:

"conversation.perception_tool_call"

conversation_id

string

The unique identifier for the conversation.

Example:

"c123456"

properties

object

Contains the tool call payload. Includes modality (vision or audio), name, arguments, and for vision calls, frames.

Show child attributes

⌘I

Introduction

Conversation

Persona

Replica

Objectives

Guardrails

Knowledge Base

Video Generation

Perception Tool Call Event