​
Video Callbacks

If a callback_url is providing in the POST /videos call, you will receive callbacks on video generation completed and on video error.

​
Video Generation Completed

{
  "created_at": "2024-08-28 15:27:40.824457",
  "data": {
    "script": "Hello this is a test to give examples of callbacks"
  },
  "download_url": "https://stream.mux.com/H5H029h02tY7XDpNj9JFDbLleTyUpsJr5npddO8gRsKqY/high.mp4?download=1e30440cf9",
  "generation_progress": "100/100",
  "hosted_url": "https://videos.tavus.io/video/1e30440cf9",
  "replica_id": "r79e1c033f",
  "status": "ready",
  "status_details": "Your request has processed successfully!",
  "stream_url": "https://stream.mux.com/H5H029h02tY7XDpNj9JFDbLleTyUpsJr5npddO8gRsKqY.m3u8",
  "updated_at": "2024-08-28 15:29:19.802670",
  "video_id": "1e30440cf9",
  "video_name": "replica_id: r79e1c033f - August 28, 2024 - video: 1e30440cf9"
}

​
Video Generation Error

On error, the status_details parameter will contain the error message. You can learn more about API Errors and Status Details here

{
  "created_at": "2024-08-28 15:32:53.058894",
  "data": {
    "script": "This is a test script to show how videos error"
  },
  "download_url": null,
  "error_details": null,
  "generation_progress": "0/100",
  "hosted_url": "https://videos.tavus.io/video/c9b85a6d36",
  "replica_id": "ra5ed77426",
  "status": "error",
  "status_details": "An error occurred while generating this request. Please check your inputs or try your request again.",
  "stream_url": null,
  "updated_at": "2024-08-28 15:35:03.762392",
  "video_id": "c9b85a6d36",
  "video_name": "replica_id: ra5ed77426 - August 28, 2024 - video: c9b85a6d36"
}

​
Replica Training

If a callback_url is provided in the POST /replicas call, you will receive a callback on replica training completion or on replica training error.

​
Replica Training Completed

{
  "replica_id": "rxxxxxxxxx",
  "status": "ready",
}

​
Replica Training Error

On error, the error_message parameter will contain the error message. You can learn more about API Errors and Status Details here

{
  "replica_id": "rxxxxxxxxx",
  "status": "error",
  "error_message": "There was an issue processing your training video. The video provided does not meet the minimum duration requirement for training"
}

​
Conversations

​
Overview

If callback_url is provided in the Create Conversation API Request, callbacks will be sent to provide inside into the state of the conversation. Our callbacks range from system-related callbacks like replica joins and room shutdowns, to application-related callbacks like final transcription parsing and recording ready webhooks, with many more webhooks coming soon!

​
Conversation Callback Types

Our callbacks are split into two main categories:

  • System Callbacks

These callbacks are to provide insight into system-related events in a conversation. They are:

  • system.replica_joined: This is fired when the replica becomes ready for a conversation.

  • system.shutdown: This is fired when the room shuts down, for any of the following reasons:

    • max_call_duration reached
    • participant_left_timeout reached
    • participant_absent_timeout reached
    • bot_could_not_join_meeting_it_was_probably_ended
    • daily_room_has_been_deleted
    • exception_encountered_during_conversation_startup
    • end_conversation_endpoint_hit
    • internal error occurred at step x

  • Application Callbacks

These callbacks are to inform developers about logical events that take place. They are:

  • application.transcription_ready: This is fired after ending a conversation, where the chat history is saved and returned.
  • application.recording_ready: This is fired if you had enabled recording on, set up a custom S3 bucket for recording and started a recording inside the room at any point. This will point to the key at which your new recording lies, useful for serving recordings through a CDN.
  • application.perception_analysis: This is fired after ending a conversation, when the replica has finished summarizing the visual artifacts that were detected throughout the call. This is a feature that is only available when the persona has raven-0 specified in the Perception Layer.

​
Conversation Callback Structure

All Conversation callbacks will share a similar structure with differences occuring in the properties object.

Here is the basic structure for all callbacks:

{
  "properties": {
    "replica_id": "<replica_id>"
  },
  "conversation_id": "<conversation_id>",
  "webhook_url": "<webhook_url>",
  "event_type": "<event_type>",
  "message_type": "<system/application>",
  "timestamp": "<timestamp>"
}

​
Conversation Callback Examples

system.replica_joined

{
  "properties": {
    "replica_id": "<replica_id>"
  },
  "conversation_id": "<conversation_id>>",
  "webhook_url": "<webhook_url>",
  "event_type": "system.replica_joined",
  "message_type": "system",
  "timestamp": "2025-02-10T21:15:09.860974Z"
}

system.shutdown

{
  "properties": {
    "replica_id": "<replica_id>",
    "shutdown_reason": "participant_left_timeout"
  },
  "conversation_id": "<conversation_id>",
  "webhook_url": "<webhook_url>",
  "event_type": "system.shutdown",
  "message_type": "system",
  "timestamp": "2025-02-10T21:15:29.565571Z"
}

application.transcription_ready

{
  "properties": {
    "replica_id": "<replica_id>",
    "transcript": [
      {
        "role": "system",
        "content": "You are in a live video conference call with a user. You will get user message with two identifiers, 'USER SPEECH:' and 'VISUAL SCENE:', where 'USER SPEECH:' is what the person actually tells you, and 'VISUAL SCENE:' is what you are seeing when you look at them. Only use the information provided in 'VISUAL SCENE:' if the user asks what you see. Don't output identifiers such as 'USER SPEECH:' or 'VISUAL SCENE:' in your response. Reply in short sentences, talk to the user in a casual way.Respond only in english.   "
      },
      {
        "role": "user",
        "content": " Hello, tell me a story. "
      },
      {
        "role": "assistant",
        "content": "I've got a great one about a guy who traveled back in time.  Want to hear it? "
      },
      {
        "role": "user",
        "content": "USER_SPEECH:  Yeah I'd love to hear it.  VISUAL_SCENE: The image shows a close-up of a person's face, focusing on their forehead, eyes, and nose. In the background, there is a television screen mounted on a wall. The setting appears to be indoors, possibly in a public or commercial space."
      },
      {
        "role": "assistant",
        "content": "Let me think for a sec.  Alright, so there was this mysterious island that appeared out of nowhere,  and people started disappearing when they went to explore it.  "
      },
    ]
  },
  "conversation_id": "<conversation_id>",
  "webhook_url": "<webhook_url>",
  "message_type": "application",
  "event_type": "application.transcription_ready",
  "timestamp": "2025-02-10T21:30:06.141454Z"
}