Customer Service Agent

Customer Service Agent Configuration

This example uses the tools registry: tools are created once at /v2/tools and attached to the PAL. Tools here use the default app-message delivery, so calls arrive as conversation.tool_call events for your frontend to handle.

The agent uses two tools - an LLM tool that logs the customer’s issue, and a vision tool that reports the user’s emotional state when Raven sees a strong cue.

resolve_customer_issue (LLM tool)

{
  "name": "resolve_customer_issue",
  "description": "Attempt to resolve the user's issue by logging the product, issue, and urgency for appropriate follow-up or resolution.",
  "parameters": {
    "type": "object",
    "required": ["product", "issue_description", "urgency"],
    "properties": {
      "product": {
        "type": "string",
        "description": "The product or service the user is having trouble with"
      },
      "issue_description": {
        "type": "string",
        "description": "The specific problem or complaint reported by the user"
      },
      "urgency": {
        "type": "string",
        "enum": ["low", "medium", "high"],
        "description": "How urgent or critical the issue is for the user"
      }
    }
  },
  "origin": "llm"
}

user_emotional_state (vision tool)

{
  "name": "user_emotional_state",
  "description": "Report the user's emotional state as inferred from body language and voice tone. Trigger when facial expressions or posture indicate a strong emotional state such as frustration, confusion, or calmness.",
  "parameters": {
    "type": "object",
    "required": ["emotional_state", "indicator"],
    "properties": {
      "emotional_state": {
        "type": "string",
        "description": "Inferred emotion from the user's body language (e.g., frustrated, calm, confused)"
      },
      "indicator": {
        "type": "string",
        "description": "The visual behavior that triggered the inference (e.g., furrowed brow, fidgeting, sighing)"
      }
    }
  },
  "origin": "vision"
}

The PAL itself no longer defines tools inline - it sets the models and ambient queries, and the tools are attached separately:

PAL configuration

{
  "pal_name": "Customer Service Agent",
  "pipeline_mode": "full",
  "system_prompt": "You are a calm, helpful customer service agent. You assist users with product or service issues, and adapt based on their emotional state. Remain professional and empathetic at all times.",
  "context": "User needs support with a product or service. Listen carefully, identify the issue, and offer a helpful resolution. Monitor body language and voice tone to adapt your responses when the user appears frustrated or confused.",
  "default_face_id": "r90bbd427f71",
  "layers": {
    "tts": {
      "tts_engine": "cartesia",
      "tts_emotion_control": true
    },
    "llm": {
      "model": "tavus-gemma-4",
      "speculative_inference": true
    },
    "perception": {
      "perception_model": "raven-1",
      "ambient_awareness_queries": [
        "Does the user appear frustrated or confused?",
        "Is the user sighing, fidgeting, or visibly anxious?",
        "Is the user's posture disengaged or tense?",
        "Is the user calm and cooperative?"
      ]
    },
    "conversational_flow": {
      "turn_detection_model": "sparrow-1",
      "turn_taking_patience": "low",
      "pal_interruptibility": "medium"
    }
  }
}

This PAL is configured to handle real product or service issues with empathy. It includes:

PAL Identity: A professional customer service agent that helps users with real product or service issues. The agent speaks clearly and responds with empathy, adjusting based on how the user sounds or looks.
Full Pipeline Mode: Enables the full Tavus conversational pipeline, including Perception, STT, LLM, and TTS.
System Prompt: Tells the agent to act professionally and respond helpfully, while being aware of the user’s emotional state.
Context: Describes a real customer support situation. The agent listens to the user’s issue, helps resolve it, and changes its tone or pace if the user seems frustrated or confused.
PAL Layer:
- LLM Layer: The attached resolve_customer_issue tool gathers:
  - product: what the issue is about
  - issue_description: a short explanation of the problem
  - urgency: how serious the issue is (low, medium, or high)
- Perception Layer: Uses the raven-1 model to watch for signs like fidgeting, slouching, or facial expressions. When the user appears upset, the attached user_emotional_state vision tool fires with:
  - emotional_state: what the user seems to feel (e.g., frustrated, calm)
  - indicator: what was observed (e.g., sighing, avoiding eye contact)
- TTS Layer: Employs the cartesia voice engine with emotion control.
- Conversational Flow Layer: Uses sparrow-1 turn detection model with low turn-taking patience for fast responses and medium face interruptibility for balanced conversation flow.

Create a Conversation with the Customer Service Agent PAL

Step 1: Create the tools

Create each tool at /v2/tools. The response returns a tool_id (e.g. t1234567890) you’ll attach to the PAL in Step 3.

curl --request POST \
  --url https://tavusapi.com/v2/tools \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: <api-key>' \
  --data '{
    "name": "resolve_customer_issue",
    "description": "Attempt to resolve the user'\''s issue by logging the product, issue, and urgency for appropriate follow-up or resolution.",
    "parameters": {
      "type": "object",
      "required": ["product", "issue_description", "urgency"],
      "properties": {
        "product": {
          "type": "string",
          "description": "The product or service the user is having trouble with"
        },
        "issue_description": {
          "type": "string",
          "description": "The specific problem or complaint reported by the user"
        },
        "urgency": {
          "type": "string",
          "enum": ["low", "medium", "high"],
          "description": "How urgent or critical the issue is for the user"
        }
      }
    },
    "origin": "llm"
  }'

curl --request POST \
  --url https://tavusapi.com/v2/tools \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: <api-key>' \
  --data '{
    "name": "user_emotional_state",
    "description": "Report the user'\''s emotional state as inferred from body language and voice tone. Trigger when facial expressions or posture indicate a strong emotional state such as frustration, confusion, or calmness.",
    "parameters": {
      "type": "object",
      "required": ["emotional_state", "indicator"],
      "properties": {
        "emotional_state": {
          "type": "string",
          "description": "Inferred emotion from the user'\''s body language (e.g., frustrated, calm, confused)"
        },
        "indicator": {
          "type": "string",
          "description": "The visual behavior that triggered the inference (e.g., furrowed brow, fidgeting, sighing)"
        }
      }
    },
    "origin": "vision"
  }'

Step 2: Create the PAL

Create the PAL using the following request. Note there are no inline tools - they’re attached in the next step.

cURL

curl --request POST \
  --url https://tavusapi.com/v2/pals \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: <api-key>' \
  --data '{
    "pal_name": "Customer Service Agent",
    "pipeline_mode": "full",
    "system_prompt": "You are a calm, helpful customer service agent. You assist users with product or service issues, and adapt based on their emotional state. Remain professional and empathetic at all times.",
    "context": "User needs support with a product or service. Listen carefully, identify the issue, and offer a helpful resolution. Monitor body language and voice tone to adapt your responses when the user appears frustrated or confused.",
    "default_face_id": "r90bbd427f71",
    "layers": {
      "tts": {
        "tts_engine": "cartesia",
        "tts_emotion_control": true
      },
      "llm": {
        "model": "tavus-gemma-4",
        "speculative_inference": true
      },
      "perception": {
        "perception_model": "raven-1",
        "ambient_awareness_queries": [
          "Does the user appear frustrated or confused?",
          "Is the user sighing, fidgeting, or visibly anxious?",
          "Is the user'\''s posture disengaged or tense?",
          "Is the user calm and cooperative?"
        ]
      },
      "conversational_flow": {
        "turn_detection_model": "sparrow-1",
        "turn_taking_patience": "low",
        "pal_interruptibility": "medium"
      }
    }
  }'

Replace <api-key> with your actual API key. You can generate one in the PAL Maker.

Step 3: Attach the tools to the PAL

Attach both tools to the PAL by their tool_ids. Vision tools require perception_model: "raven-1" on the PAL, which the configuration above already sets.

cURL

curl --request POST \
  --url https://tavusapi.com/v2/pals/<customer_service_pal_id>/tools \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: <api-key>' \
  --data '{
    "tool_ids": ["<resolve_customer_issue_tool_id>", "<user_emotional_state_tool_id>"]
  }'

Step 4: Create a Conversation

Use the following request body example:

cURL

curl --request POST \
  --url https://tavusapi.com/v2/conversations \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: <api_key>' \
  --data '{
  "pal_id": "<customer_service_pal_id>"
}'

Replace <api_key> with your actual API key.
Replace <customer_service_pal_id> with the ID of the PAL you created using the Customer Service Agent configuration.

Step 5: Join the Conversation

Click the link in the conversation_url field to join the conversation:

{
  "conversation_id": "c7f3fc6d766f",
  "conversation_name": "New Conversation 1747719531479",
  "conversation_url": "<conversation_link>",
  "status": "active",
  "callback_url": "",
  "created_at": "2025-05-20T05:38:51.501467Z"
}

Getting Started

Build

Deploy

Debug

Guides

Resources

Customer Service Agent