Speech-to-Speech

We recommend using Tavus’s Full Pipeline in its entirety for the lowest latency and most optimized multimodal experience. Third party integrations only provide rendering, while our Full Pipeline includes perception, turn-taking, and rendering for complete conversational intelligence.

Speech-to-speech models listen and speak in one step. You do not run separate STT, LLM, and TTS stages for the voice path. This page covers OpenAI GPT Live / GPTLive 1.5 support. In code, set the model to gpt-live-1. Pipecat or LiveKit runs that model; Tavus renders a synchronized face from the model’s audio. Use this page when you want a Tavus face on top of GPT Live. For cascade (STT → LLM → TTS) integrations, see Pipecat and LiveKit Agent.

How it works

Your app connects the user to Pipecat or LiveKit.
GPT Live streams spoken audio out of that stack.
Tavus receives that audio (through TavusTransport or tavus.AvatarSession) and publishes face video and audio into the room.

Tavus detects steady realtime audio streams and applies low-latency rendering optimizations automatically. Keep continuous audio flowing to Tavus (including silence between utterances) so the stream stays classified as realtime.

Prerequisites

Tavus API Key
Tavus face_id
- Stock faces or your own custom face both work.
- Use a Phoenix-4.5 Pro face when you can. Those faces are tuned for low-latency realtime rendering.
OpenAI API key with access to GPT Live
One of:
- A Pipecat Python app (pipecat-ai with the Tavus and Daily extras)
- A LiveKit Agents app (Python or Node.js)

Pipecat’s Tavus transport still takes a replica_id argument. Pass your Tavus Face ID there. LiveKit’s plugin uses face_id / faceId (with replica_id as a deprecated alias).

Pipecat + GPT Live

Pipecat’s OpenAILiveLLMService talks to OpenAI’s Live API. TavusTransport creates a Tavus conversation, joins the room, and drives the face from the live audio output. For the OpenAI Live service itself, see Pipecat’s OpenAI Live docs. For the Tavus transport options, see Pipecat.

Integration guide

Step 1: Install and configure

Install Pipecat with the OpenAI, Tavus, and Daily extras:

pip install "pipecat-ai[openai,tavus,daily]"

Add these values to your .env file:

OPENAI_API_KEY=<openai-api-key>
TAVUS_API_KEY=<tavus-api-key>
TAVUS_FACE_ID=<your_face_id>

Generate a Tavus API key in the PAL Maker. See Authentication for how Tavus APIs use x-api-key.

Step 2: Wire GPT Live into TavusTransport

Create a TavusTransport, put OpenAILiveLLMService in the pipeline, and leave the separate STT / TTS stages out. GPT Live owns the speech path.

import os

import aiohttp
from dotenv import load_dotenv
from pipecat.frames.frames import LLMRunFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineParams, PipelineWorker
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import (
    LLMContextAggregatorPair,
)
from pipecat.services.openai.live.llm import OpenAILiveLLMService
from pipecat.transports.tavus.transport import TavusParams, TavusTransport
from pipecat.workers.runner import WorkerRunner

load_dotenv()

SYSTEM_INSTRUCTION = (
    "You are a helpful assistant in a voice conversation. "
    "Keep replies short and easy to speak aloud."
)

async def main() -> None:
    async with aiohttp.ClientSession() as session:
        transport = TavusTransport(
            bot_name="Pipecat Bot",
            session=session,
            api_key=os.environ["TAVUS_API_KEY"],
            replica_id=os.environ["TAVUS_FACE_ID"],
            params=TavusParams(audio_in_enabled=True, audio_out_enabled=True),
        )

        llm = OpenAILiveLLMService(
            api_key=os.environ["OPENAI_API_KEY"],
            settings=OpenAILiveLLMService.Settings(
                model="gpt-live-1",
                voice="cedar",
                system_instruction=SYSTEM_INSTRUCTION,
            ),
        )

        context = LLMContext()
        user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context)

        pipeline = Pipeline(
            [
                transport.input(),
                user_aggregator,
                llm,
                transport.output(),
                assistant_aggregator,
            ]
        )
        worker = PipelineWorker(pipeline, params=PipelineParams(enable_metrics=True))
        runner = WorkerRunner(handle_sigint=True)
        await runner.add_workers(worker)

        @transport.event_handler("on_client_connected")
        async def on_client_connected(transport, participant):
            context.add_message(
                {
                    "role": "developer",
                    "content": "Start by concisely introducing yourself.",
                }
            )
            await worker.queue_frames([LLMRunFrame()])

        await runner.run()


if __name__ == "__main__":
    import asyncio

    asyncio.run(main())

Step 3: Run the bot

python <file-name>.py

Join the Tavus conversation URL that Pipecat creates for the session. The face speaks with GPT Live’s audio.

LiveKit + GPT Live

LiveKit Agents runs GPT Live through openai.realtime.GPTLiveModel. Tavus joins the same room as an avatar participant through tavus.AvatarSession and renders the face from the agent’s audio. For GPT Live on LiveKit, see LiveKit’s GPT-Live plugin guide. For the Tavus avatar plugin, see LiveKit Agent.

Integration guide

Step 1: Install and configure

Install the OpenAI and Tavus plugins:

pip install "livekit-agents[openai,tavus]~=1.8"

Set these values in your .env file:

OPENAI_API_KEY=<openai-api-key>
TAVUS_API_KEY=<tavus-api-key>
TAVUS_FACE_ID=<your_face_id>

Step 2: Start GPT Live with a Tavus avatar

Pass GPTLiveModel into AgentSession, then start tavus.AvatarSession before the agent session.

import os

from dotenv import load_dotenv
from livekit.agents import Agent, AgentServer, AgentSession, JobContext, cli
from livekit.plugins import openai, tavus

load_dotenv()

server = AgentServer()


@server.rtc_session()
async def entrypoint(ctx: JobContext):
    session = AgentSession(
        llm=openai.realtime.GPTLiveModel(
            model="gpt-live-1",
            voice="marin",
        ),
        resume_false_interruption=False,
    )

    avatar = tavus.AvatarSession(
        face_id=os.environ["TAVUS_FACE_ID"],
    )
    await avatar.start(session, room=ctx.room)

    await session.start(
        agent=Agent(
            instructions=(
                "You are a helpful voice assistant. "
                "Keep replies short and conversational."
            )
        ),
        room=ctx.room,
    )

    session.generate_reply(instructions="Say hello to the user.")


if __name__ == "__main__":
    cli.run_app(server)

Piece	Role
`openai.realtime.GPTLiveModel`	Full-duplex speech-to-speech model (`gpt-live-1` by default)
`tavus.AvatarSession`	Joins the LiveKit room and renders the Tavus face
`Agent.instructions`	Spoken persona for GPT Live (keep this separate from any backend/tool instructions)

LiveKit’s GPT Live plugin can also delegate reasoning and tools to a backend Responses model through responses_options. See LiveKit’s GPT-Live plugin guide when you need that path.

Tips for GPT Live with Tavus

Prefer Phoenix-4.5 Pro faces for speech-to-speech rendering.
On Pipecat, stay on a recent pipecat-ai release (1.5.0 or newer) so the Tavus transport’s realtime audio path is available.
Do not strip silence frames before they reach Tavus. GPT Live often sends continuous audio, including silence between utterances. Filtering that silence can stop Tavus from treating the stream as realtime and can cause lip-sync artifacts.
Speech-to-speech through Pipecat or LiveKit is a rendering integration. Perception, Tavus turn-taking, and the rest of the Full Pipeline are not part of this path.

Getting Started

Guides

Build

Deploy

Debug

Teams & SSO

Resources

Speech-to-Speech

How it works

Prerequisites

Pipecat + GPT Live

Integration guide

LiveKit + GPT Live

Integration guide

Tips for GPT Live with Tavus

​How it works

​Prerequisites

​Pipecat + GPT Live

​Integration guide

​LiveKit + GPT Live

​Integration guide

​Tips for GPT Live with Tavus

How it works

Prerequisites

Pipecat + GPT Live

Integration guide

LiveKit + GPT Live

Integration guide

Tips for GPT Live with Tavus