> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tavus.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom LLM via Fixed-IP Proxy

> Route custom LLM calls through a Tavus-hosted proxy with stable, allowlistable IPs.

If your custom LLM sits behind a firewall or compliance allowlist that requires requests to come from a known set of IPs, you can opt your PAL into the Tavus **LLM proxy**. When enabled, Tavus routes outbound calls to your `base_url` through a fixed-IP egress in `us-east-1` instead of dialing your endpoint directly from variable Tavus infrastructure IPs.

> [!NOTE]
>
> This page is shared with customers under direct enablement. Reach out to your Tavus account team to confirm the egress IPs to allowlist for your account before turning the flag on.

## When To Use This

Turn this on when **all** of the following apply:

* You are using a [custom LLM](https://docs.tavus.io/sections/conversational-video-interface/pal/llm.md#custom-llms) (`base_url` + `api_key` are set on the LLM layer).
* Your LLM endpoint enforces an IP allowlist or sits behind a network policy that only permits traffic from declared sources.
* Some additional egress latency from `us-east-1` is acceptable for your use case.

If you do not have an allowlist requirement, leave this disabled and call your LLM directly - it is the lower-latency default.

## Enabling The Proxy

Set `enable_llm_proxy: true` on the `llm` layer of your PAL. The flag is only honored when both `base_url` and `api_key` are set; sending it for a Tavus-hosted model is rejected.

```json LLM Layer theme={null}
"llm": {
  "model": "gpt-4o",
  "base_url": "https://your-llm.internal/v1",
  "api_key": "your-api-key",
  "enable_llm_proxy": true
}
```

### Behavior

| Field | Type | Required | Description |
| - | - | - | - |
| `enable_llm_proxy` | boolean | optional | When `true`, Tavus routes the LLM HTTP calls for this PAL through the fixed-IP proxy in `us-east-1`. Defaults to `false`. Requires a custom LLM (`base_url` + `api_key`). |

When `enable_llm_proxy: false` (the default) the request continues to be sent to your `base_url` directly, exactly as before.

## Setting It On An Existing PAL

You can flip the flag on an existing PAL with the [Patch PAL API](https://docs.tavus.io/api-reference/pals/patch-pal.md):

```shell theme={null}
curl --request PATCH \
  --url https://tavusapi.com/v2/pals/{pal_id} \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: <api-key>' \
  --data '[
    { "op": "replace", "path": "/layers/llm/enable_llm_proxy", "value": true }
  ]'
```

The change applies to new conversations created after the patch; in-flight conversations keep their existing routing until they end.

## Validation

The PAL API rejects configurations that combine the proxy flag with a Tavus-hosted model:

```json theme={null}
{
  "message": "enable_llm_proxy=True is only supported with a custom LLM. Please provide both base_url and api_key."
}
```

## Operational Notes

* Egress IPs are stable but may be expanded over time. Coordinate with your Tavus account team before allowlisting and re-confirm before any compliance audit.
* The proxy adds a small amount of latency on top of your LLM's response time. If you observe regressions, run an A/B with `enable_llm_proxy: false` on a clone of the PAL to isolate the proxy hop.
* All other LLM-layer fields (`headers`, `extra_body`, `default_query`, `tools`, `speculative_inference`, etc.) behave identically whether the proxy is on or off.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.