Screen Share

When someone shares their screen in a CVI call, Raven can see that shared content the same way it sees the camera feed. The PAL can read on-screen text, notice errors, follow a walkthrough, and answer questions like “what tab am I on?” without you wiring up a custom vision tool. There is nothing extra to switch on. If Raven is running, screen perception comes with it.

How it works

A participant starts screen share in the call (browser prompt for window / tab / display).
CVI receives the shared feed and Raven analyzes rolling frames of the shared surface.
Short screen summaries are injected into the LLM context (alongside webcam / audio perception), so the PAL can respond with that context on the next turn.

Two different directionsScreen share here means the user shares with the PAL. That is different from Presentation, where the PAL publishes slides for participants to watch.

Requirements

Requirement	Detail
Raven on	Set `layers.perception.perception_model` to `raven-1` (default).
Participant shares	Someone must actually start screen share in the client.
Video conversation	Screen perception needs a live video room (not chat-only / audio-only).

With Raven on, no extra perception fields are required for basic “see my screen” behavior.

{
  "layers": {
    "perception": {
      "perception_model": "raven-1"
    }
  }
}

Screen sharing is available by default. To let participants use it, include a screen-share control in your frontend. The Conversation block includes one, or you can add ScreenShareButton from @tavus/cvi-ui to a custom layout.

const canShareScreen = user.role === "presenter";

return (
  <CallControls>
    {canShareScreen && <ScreenShareButton />}
  </CallControls>
);

To disallow screen sharing, leave the control out of your UI. You can use the same pattern to show it only for certain roles or workflows.

OptionalIf you do not need visual perception at all, set perception_model to "off". This also turns off camera perception.

What the PAL receives

With Raven on, screen understanding shows up in the same perception path as camera context. For a custom LLM, that arrives as system context that can include screen content, for example:

{
  "role": "system",
  "content": "<user_appearance>...</user_appearance> <user_emotions>...</user_emotions> <user_screenshare>...</user_screenshare>"
}

Those are rolling natural-language descriptions of what is on the shared screen, available for the next reply. Raven also answers prompted visual questions against the screen when the user is sharing (for example, “what error is on my screen?”), routing those to the screenshare stream instead of the webcam when that is the right source.

Make the PAL act on what it sees

Objectives with visual modality

For structured goals that depend on the shared screen, use objectives with "modality": "visual":

{
  "objective_name": "capture_error_code",
  "objective_prompt": "Check if an error message is visible on the user's shared screen. Extract the error code and message text.",
  "modality": "visual",
  "confirmation_mode": "auto"
}

Ambient visual queries

visual_awareness_queries help Raven watch for ongoing cues during the call. For screen-heavy workflows, write queries that refer to on-screen content (error dialogs, forms, ID documents, IDE state). Pair them with a system prompt that tells the PAL how to react when those cues appear.

{
  "layers": {
    "perception": {
      "perception_model": "raven-1",
      "visual_awareness_queries": [
        "Is an error dialog or stack trace visible on the shared screen?",
        "Is the user looking at a login or settings form?"
      ]
    }
  }
}

Frontend tips

Prefer the CVI UI Conversation block if you want share controls and sensible layout out of the box.
For a custom Daily React layout, show the shared screen as the main surface and keep face video available (often as PiP) while they share.
Ask users to share a specific window or tab when possible. Full-desktop shares work, but a focused tab is usually cleaner for the PAL to read.

Perception: Raven configuration
Models → Raven: model overview
Presentation: PAL shares slides to the user

Getting Started

Guides

Build

Deploy

Debug

Resources

Screen Share

How it works

Requirements

What the PAL receives

Make the PAL act on what it sees

Objectives with visual modality

Ambient visual queries

Frontend tips

​How it works

​Requirements

​Control access to screen share

​What the PAL receives

​Make the PAL act on what it sees

​Objectives with visual modality

​Ambient visual queries

​Frontend tips

​Related

How it works

Requirements

Control access to screen share

What the PAL receives

Make the PAL act on what it sees

Objectives with visual modality

Ambient visual queries

Frontend tips

Related