Skip to main content
If your custom LLM sits behind a firewall or compliance allowlist that requires requests to come from a known set of IPs, you can opt your PAL into the Tavus LLM proxy. When enabled, Tavus routes outbound calls to your base_url through a fixed-IP egress in us-east-1 instead of dialing your endpoint directly from variable Tavus infrastructure IPs.
This page is shared with customers under direct enablement. Reach out to your Tavus account team to confirm the egress IPs to allowlist for your account before turning the flag on.

When To Use This

Turn this on when all of the following apply:
  • You are using a custom LLM (base_url + api_key are set on the LLM layer).
  • Your LLM endpoint enforces an IP allowlist or sits behind a network policy that only permits traffic from declared sources.
  • Some additional egress latency from us-east-1 is acceptable for your use case.
If you do not have an allowlist requirement, leave this disabled and call your LLM directly - it is the lower-latency default.

Enabling The Proxy

Set enable_llm_proxy: true on the llm layer of your PAL. The flag is only honored when both base_url and api_key are set; sending it for a Tavus-hosted model is rejected.
LLM Layer

Behavior

When enable_llm_proxy: false (the default) the request continues to be sent to your base_url directly, exactly as before.

Setting It On An Existing PAL

You can flip the flag on an existing PAL with the Patch PAL API:
The change applies to new conversations created after the patch; in-flight conversations keep their existing routing until they end.

Validation

The PAL API rejects configurations that combine the proxy flag with a Tavus-hosted model:

Operational Notes

  • Egress IPs are stable but may be expanded over time. Coordinate with your Tavus account team before allowlisting and re-confirm before any compliance audit.
  • The proxy adds a small amount of latency on top of your LLM’s response time. If you observe regressions, run an A/B with enable_llm_proxy: false on a clone of the PAL to isolate the proxy hop.
  • All other LLM-layer fields (headers, extra_body, default_query, tools, speculative_inference, etc.) behave identically whether the proxy is on or off.