Visual Overlay API

Visualize detected elements directly in the browser with color-coded borders and importance indicators. Perfect for debugging element detection, understanding importance scoring, and verifying critical UI elements.

Overview

The Visual Overlay API displays a visual overlay highlighting elements in the browser without taking a new snapshot. This is useful for:

• Debugging - See exactly which elements Predicate detects on complex pages
• Analysis - Understand which elements rank highest by importance
• Validation - Verify critical buttons/links are being detected correctly
• Learning - Understand how importance scoring and visual cues work
• Target Highlighting - Show which element the agent will interact with

show_overlay()

Display a visual overlay highlighting elements in the browser with color-coded borders.

Function Signature

from predicate

Parameters

• browser (PredicateBrowser) - Browser instance
• elements - Can be:
  • Snapshot object (will use snapshot.elements)
  • List of Element objects (from snapshot.elements)
  • List of raw element dicts (from API or snapshot result)
• target_element_id (int, optional) - ID of element to highlight in red

Visual Indicators

Color Coding

• 🔴 Red - Target element (when target_element_id is specified)
• 🔵 Blue - Primary elements (is_primary=true)
• 🟢 Green - Regular interactive elements

Additional Indicators

• Border thickness and opacity scale with importance score
• Semi-transparent fill for better visibility
• Importance badges showing numeric scores
• ⭐ Star icon for primary elements
• 🎯 Target emoji for the target element
• Auto-clear - Overlay disappears after 5 seconds

Basic Usage

from predicate import PredicateBrowser, snapshot, show_overlay

browser = PredicateBrowser()
browser.start()
browser.page.goto("https://example.com")

# Take snapshot once
snap = snapshot(browser)

Highlighting Specific Elements

Show overlay with specific elements or highlight a target element in red:

from predicate import PredicateBrowser, snapshot, show_overlay, find

browser = PredicateBrowser()
browser.start()
browser.page.goto("https://example.com")

# Take snapshot
snap = snapshot(

clear_overlay()

Manually clear the visual overlay before the 5-second auto-clear.

Function Signature

from predicate import clear_overlay

clear_overlay(browser)

Parameters

• browser (PredicateBrowser) - Browser instance

Usage

from predicate import PredicateBrowser, snapshot, show_overlay, clear_overlay
import time

browser = PredicateBrowser()
browser.start()
browser.page.goto("https://example.com")

snap = snapshot(

Using with Snapshot Options

You can also show overlays automatically when taking snapshots:

from predicate import PredicateBrowser, snapshot, SnapshotOptions
import time

browser = PredicateBrowser()
browser.start()
browser.page.goto("https://example.com")

# Automatically show overlay during snapshot
snap =

Debugging Example

Use visual overlay to debug why an element isn't being detected:

from predicate import PredicateBrowser, snapshot, show_overlay, find, clear_overlay
import time

browser = PredicateBrowser()
browser.start()
browser.page.goto("https://example.com")

# Show overlay to see what's detected

Use Cases

1. Debugging Element Detection

When an element isn't found, use overlay to see what Predicate detects:

# Check if critical buttons are detected
snap = snapshot(browser)
show_overlay(browser, snap)
time.sleep(6)

# Verify login button is present
login_btn = find(snap, "role=button text~'login'")
if not login_btn:
    print("⚠️ Login button not detected - check overlay!")

2. Understanding Importance Scoring

See which elements rank highest on a complex page:

# Show overlay with importance badges
snap = snapshot(browser)
show_overlay(browser, snap)

# Print top 10 elements by importance
for elem in snap.elements[:10]:
    print(f"{elem.importance}: {elem.role} - {elem.text[:40]}")

3. Validating Agent Actions

Highlight which element the agent will interact with:

# Find target element
snap = snapshot(browser)
target = find(snap, "role=button text~'Add to Cart'")

# Show what the agent sees (target in red)
show_overlay(browser, snap, target_element_id=target.id)
time.sleep(3)

# Proceed with action
click(browser, target.id)

4. Visual QA Testing

Verify critical UI elements are always detected:

# List of critical elements to verify
critical_elements = [
    "role=button text~'Checkout'",
    "role=link text~'Cart'",
    "role=textbox text~'Search'"
]

snap = snapshot(browser)
show_overlay(browser, snap)

for selector in critical_elements:
    elem = find(snap, selector)
    if not elem:
        print(f"❌ MISSING: {selector}")
    else:
        print(f"✅ Found: {elem.text}")

Pro Tips

1. Use show_overlay() separately when you want to re-visualize elements without re-snapshotting
2. Combine with find() or query() to highlight specific filtered elements
3. Perfect for debugging when elements aren't being detected as expected
4. Check importance scores via the overlay badges to understand element ranking
5. Use in headless mode by setting headless=False temporarily for debugging

Related APIs

• Snapshot API - Capture page state with show_overlay=True option
• Query API - Find elements to highlight with overlay
• Tracing & Debugging - Other debugging techniques

Next Steps

• Expect API → - Testing and assertions
• Snapshot API → - Capturing page state
• Examples → - More debugging examples