Docs/Core Concepts/Semantic Queries

Semantic Queries

Learn how to find elements on web pages using Predicate's semantic query language. Query by role, text, importance, and visual cues - not brittle CSS selectors.

Why Semantic Queries?

Traditional browser automation relies on CSS selectors that break whenever a website redesigns. Vision models are expensive and non-deterministic. Predicate provides a better way: query elements by their semantic meaning instead of their implementation details.

The Problem with CSS Selectors

# Breaks on redesign
page.click('.btn-primary-submit-form')
page.click('#checkout-button-2024')
page.click('div.container > button:nth-child(3)')

Issues:

  • Breaks when CSS classes change
  • Requires constant maintenance
  • No understanding of element purpose

The Predicate Way

# Resilient semantic queries
button = find(snap, "role=button text~'Submit'")
button = find(snap, "role=button text~'Checkout'")
button = find(snap, "role=button importance>500")

Benefits:

  • Survives redesigns (same meaning, different styling)
  • Self-documenting (clear intent)
  • Ranked by user importance

Query Operators

Text Matching

Match element text or labels using powerful text operators:

| Operator | Description | Example | |----------|-------------|---------| | text= | Exact match | text='Submit Form' | | text~ | Contains (case-insensitive) | text~'submit' | | text^ | Starts with | text^'Click' | | text$ | Ends with | text$'now' |

from predicate import snapshot, find

snap = snapshot(browser)

# Exact match
find(snap, "text='Submit Form'")

# Contains
find(snap, "text~'submit'")  # Matches "Submit", "SUBMIT", "submit now"

Role Matching

Find elements by their semantic role (ARIA role or inferred):

find(snap, "role=button")
find(snap, "role=textbox")
find(snap, "role=link")
find(snap, "role=heading")

Common Roles:

  • button - Clickable buttons
  • textbox - Input fields
  • link - Hyperlinks
  • heading - H1, H2, etc.
  • checkbox - Checkboxes
  • listitem - List items

Importance Ranking

Elements are ranked by importance (0-1000+) based on visual prominence, position, and user interactions:

# Find most important button
find(snap, "role=button importance>500")

# Find less prominent elements
find(snap, "role=link importance<200")

# Combine with text
find(snap, "role=button text~'submit' importance>300")

Importance Factors:

  • Visual prominence (size, color, contrast)
  • Position (center, top, above fold)
  • User interaction likelihood
  • Call-to-action indicators

Combining Operators

Combine multiple operators with spaces (AND logic):

# Find submit button with high importance
button = find(snap, "role=button text~'submit' importance>500")

# Find email input field
email = find(snap, "role=textbox text~'email'")

# Find primary CTA link
cta = find(snap, "role=link text~'get started' importance>600"

Common Patterns

Finding Form Elements

# Login form
email_input = find(snap, "role=textbox text~'email'")
password_input = find(snap, "role=textbox text~'password'")
submit_btn = find(snap, "role=button text~'log in'")

# Search form
search_box = find(snap, "role=textbox text~'search'")
search_btn = find(snap, "role=button text~'search'")

Finding Navigation Elements

# Main navigation
home_link = find(snap, "role=link text='Home'")
about_link = find(snap, "role=link text='About'")

# Breadcrumbs
docs_link = find(snap, "role=link text='Docs'")

Finding Primary CTAs

# Look for high-importance buttons
cta = find(snap, "role=button importance>700")

# Or by common CTA text
signup = find(snap, "role=button text~'sign up'")
start = find(snap, "role=button text~'get started'")

Best Practices

1. Use Semantic Roles First

# Good - starts with role
find(snap, "role=button text~'submit'")

# Avoid - text only (less precise)
find(snap, "text~'submit'")

2. Be Specific But Not Brittle

# Good - flexible text matching
find(snap, "role=button text~'checkout'")

# Too specific - might break
find(snap, "role=button text='Proceed to Checkout →'")

3. Use Importance for Disambiguation

# When multiple matches exist
primary_btn = find(snap, "role=button text~'continue' importance>500")
secondary_btn = find(snap, "role=button text~'cancel' importance<300")

4. Handle Missing Elements

# Check if element was found
button = find(snap, "role=button text~'submit'")
if button:
    click(browser, button.id)
else:
    print("Submit button not found")

Next Steps

Previous
Analysis
Next
Action Execution