metabuilder/docs/PLAYWRIGHT_INTERPRETER_GUIDE.md

Playwright Test Interpreter Guide

Overview

The Playwright Test Interpreter (e2e/tests.spec.ts) is a production-grade, feature-complete implementation that converts JSON test definitions into executable Playwright tests. It supports 25+ actions and 20+ assertions with multiple locator strategies including test IDs, ARIA roles, and complex CSS selectors.

Locator Strategies

The interpreter tries locator strategies in priority order:

1. Test ID (Recommended)

{
  "action": "click",
  "testId": "submit-button"
}

Uses page.getByTestId() - most reliable for testing.

2. ARIA Role + Name (Accessibility-First)

{
  "action": "click",
  "role": "button",
  "text": "Submit"
}

Uses page.getByRole() - finds elements by accessible role and name.

3. Label

{
  "action": "fill",
  "label": "Email"
}

Uses page.getByLabel() - finds form inputs by label text.

4. Placeholder

{
  "action": "fill",
  "placeholder": "Enter email"
}

Uses page.getByPlaceholder() - finds inputs by placeholder text.

5. Alt Text

{
  "action": "expect",
  "alt": "Profile picture",
  "assertion": { "matcher": "toBeVisible" }
}

Uses page.getByAltText() - finds images by alt text.

6. Title

{
  "action": "hover",
  "title": "Tooltip text"
}

Uses page.getByTitle() - finds elements by title attribute.

7. Text Content

{
  "action": "click",
  "text": "Save Changes"
}

Uses page.getByText() - finds elements by visible text.

8. CSS Selector (Fallback)

{
  "action": "click",
  "selector": ".submit-btn[data-status='active']"
}

Uses page.locator() - CSS selector fallback.

Actions

Navigation

Navigate

{
  "action": "navigate",
  "url": "/dashboard",
  "waitUntil": "networkidle"
}

Options: "domcontentloaded", "load", "networkidle"

Wait for Load State

{
  "action": "waitForLoadState",
  "state": "networkidle"
}

Reload

{
  "action": "reload"
}

Go Back

{
  "action": "goBack"
}

Go Forward

{
  "action": "goForward"
}

User Interactions

Click

{
  "action": "click",
  "testId": "submit-btn",
  "button": "left",
  "clickCount": 1,
  "delay": 0
}

Fill (Clear & Type)

{
  "action": "fill",
  "testId": "email-input",
  "value": "user@example.com"
}

Type (Character by Character)

{
  "action": "type",
  "testId": "search-input",
  "text": "search term",
  "delay": 50
}

Select (Dropdown)

{
  "action": "select",
  "selector": "select[name='country']",
  "value": ["option1", "option2"]
}

Hover

{
  "action": "hover",
  "role": "button",
  "text": "Options"
}

Focus

{
  "action": "focus",
  "testId": "email-input"
}

Blur

{
  "action": "blur",
  "testId": "email-input"
}

Press Key

{
  "action": "press",
  "testId": "search-input",
  "key": "Enter"
}

Keyboard Shortcut

{
  "action": "keyboard",
  "keys": ["Control", "A"]
}

Or single key: "keys": "Escape"

Waiting

Wait for Navigation

{
  "action": "waitForNavigation"
}

Wait for Selector

{
  "action": "waitForSelector",
  "selector": ".data-loaded",
  "timeout": 5000
}

Wait for URL

{
  "action": "waitForURL",
  "urlPattern": "**/dashboard",
  "timeout": 5000
}

Wait (Time)

{
  "action": "wait",
  "timeout": 2000
}

Scrolling

Scroll to Element

{
  "action": "scroll",
  "selector": "#footer"
}

Scroll by Coordinates

{
  "action": "scroll",
  "x": 0,
  "y": 500
}

Screenshots

Take Screenshot

{
  "action": "screenshot",
  "filename": "page.png",
  "fullPage": true
}

JavaScript Execution

Evaluate

{
  "action": "evaluate",
  "script": "window.localStorage.getItem('token')"
}

Assertions

Visibility

To Be Visible

{
  "action": "expect",
  "testId": "submit-btn",
  "assertion": { "matcher": "toBeVisible" }
}

To Be Hidden

{
  "action": "expect",
  "testId": "loading",
  "assertion": { "matcher": "toBeHidden" }
}

State

To Be Enabled

{
  "action": "expect",
  "testId": "submit-btn",
  "assertion": { "matcher": "toBeEnabled" }
}

To Be Disabled

{
  "action": "expect",
  "testId": "submit-btn",
  "assertion": { "matcher": "toBeDisabled" }
}

To Be Checked

{
  "action": "expect",
  "testId": "accept-terms",
  "assertion": { "matcher": "toBeChecked" }
}

To Be Empty

{
  "action": "expect",
  "testId": "message-list",
  "assertion": { "matcher": "toBeEmpty" }
}

To Be Editable

{
  "action": "expect",
  "testId": "email-input",
  "assertion": { "matcher": "toBeEditable" }
}

Content

Contain Text

{
  "action": "expect",
  "testId": "error-message",
  "assertion": {
    "matcher": "toContainText",
    "text": "required field"
  }
}

Have Text (Exact)

{
  "action": "expect",
  "role": "heading",
  "assertion": {
    "matcher": "toHaveText",
    "text": "Welcome"
  }
}

Attributes

Have Attribute

{
  "action": "expect",
  "testId": "profile-link",
  "assertion": {
    "matcher": "toHaveAttribute",
    "name": "href",
    "value": "/profile"
  }
}

Have Class

{
  "action": "expect",
  "testId": "status-badge",
  "assertion": {
    "matcher": "toHaveClass",
    "className": "active"
  }
}

Have Value

{
  "action": "expect",
  "testId": "email-input",
  "assertion": {
    "matcher": "toHaveValue",
    "value": "user@example.com"
  }
}

Have Count

{
  "action": "expect",
  "role": "listitem",
  "assertion": {
    "matcher": "toHaveCount",
    "count": 5
  }
}

Styling (Complex Checks)

Have CSS

{
  "action": "expect",
  "testId": "header",
  "assertion": {
    "matcher": "toHaveCSS",
    "property": "background-color",
    "value": "rgb(255, 0, 0)"
  }
}

Visual

Have Screenshot

{
  "action": "expect",
  "testId": "login-form",
  "assertion": {
    "matcher": "toHaveScreenshot",
    "name": "login-form.png"
  }
}

In Viewport

{
  "action": "expect",
  "testId": "footer",
  "assertion": { "matcher": "toBeInViewport" }
}

Page-Level

Have URL

{
  "action": "expect",
  "assertion": {
    "matcher": "toHaveURL",
    "url": "**/dashboard"
  }
}

Have Title

{
  "action": "expect",
  "assertion": {
    "matcher": "toHaveTitle",
    "title": "Dashboard - MyApp"
  }
}

Custom

Custom JavaScript

{
  "action": "expect",
  "assertion": {
    "matcher": "custom",
    "script": "window.localStorage.getItem('token') !== null"
  }
}

Complete Example Test

{
  "$schema": "https://metabuilder.dev/schemas/package-playwright.schema.json",
  "package": "example",
  "tests": [
    {
      "name": "Login flow with validation",
      "timeout": 10000,
      "steps": [
        {
          "action": "navigate",
          "url": "/login",
          "waitUntil": "domcontentloaded"
        },
        {
          "action": "waitForLoadState",
          "state": "networkidle"
        },
        {
          "action": "expect",
          "role": "heading",
          "assertion": { "matcher": "toBeVisible" }
        },
        {
          "action": "fill",
          "label": "Email",
          "value": "user@example.com"
        },
        {
          "action": "fill",
          "label": "Password",
          "value": "SecurePassword123!"
        },
        {
          "action": "click",
          "role": "button",
          "text": "Login"
        },
        {
          "action": "waitForURL",
          "urlPattern": "**/dashboard",
          "timeout": 5000
        },
        {
          "action": "expect",
          "testId": "user-profile",
          "assertion": { "matcher": "toBeVisible" }
        },
        {
          "action": "expect",
          "testId": "sidebar",
          "assertion": {
            "matcher": "toHaveClass",
            "className": "expanded"
          }
        },
        {
          "action": "screenshot",
          "filename": "dashboard.png",
          "fullPage": true
        }
      ]
    }
  ]
}

Best Practices

1. Use Test IDs First

// ✅ BEST - Most reliable
{"action": "click", "testId": "submit-btn"}

// ✅ GOOD - Accessible
{"action": "click", "role": "button", "text": "Submit"}

// ⚠️ FRAGILE - Can break with styling changes
{"action": "click", "selector": ".submit-btn"}

2. Accessibility-First

Use ARIA roles to encourage accessible UI:

{
  "action": "expect",
  "role": "button",
  "text": "Save",
  "assertion": { "matcher": "toBeVisible" }
}

3. Wait for State, Not Time

// ✅ GOOD - Waits for actual condition
{"action": "waitForLoadState", "state": "networkidle"}

// ⚠️ FLAKY - Time-based waits can fail
{"action": "wait", "timeout": 2000}

4. Explicit Selectors

// ✅ GOOD - Clear intent
{
  "action": "click",
  "testId": "delete-confirm-btn",
  "button": "left"
}

// ⚠️ AMBIGUOUS - Unclear behavior
{"action": "click", "selector": "button"}

5. Comprehensive Assertions

// ✅ GOOD - Multiple checks
[
  {"action": "expect", "testId": "form", "assertion": {"matcher": "toBeVisible"}},
  {"action": "expect", "testId": "submit", "assertion": {"matcher": "toBeEnabled"}},
  {"action": "expect", "testId": "error", "assertion": {"matcher": "toBeHidden"}}
]

// ⚠️ MINIMAL - Might miss issues
[
  {"action": "expect", "testId": "form", "assertion": {"matcher": "toBeVisible"}}
]

Error Handling

The interpreter provides detailed error messages:

Failed step in test "Login flow": fill - No locator strategy provided in step

This includes:

• Test name
• Action that failed
• Reason for failure
• Suggestions for fixes

Performance Tips

1. Use waitForLoadState correctly:

   • "domcontentloaded" - Page DOM ready (fastest)
   • "load" - All resources loaded
   • "networkidle" - Network activity stopped (slowest)

2. Parallel execution: Playwright runs tests in parallel by default

   • Each test runs in its own browser context
   • No shared state between tests

3. Timeouts: Set appropriate timeouts:

   {
     "name": "Long operation",
     "timeout": 30000,
     "steps": [...]
   }
   

Advanced Features

Skip Tests

{
  "name": "Feature under development",
  "skip": true,
  "steps": [...]
}

Focus on Specific Test

{
  "name": "Critical path",
  "only": true,
  "steps": [...]
}

Test Description

{
  "name": "Should validate email format",
  "description": "Verifies email field only accepts valid emails",
  "steps": [...]
}

Troubleshooting

Test Times Out

1. Check network conditions (use waitForLoadState)
2. Verify locator strategy is correct
3. Increase timeout if needed

Locator Not Found

1. Verify test ID exists in HTML
2. Check ARIA role and text match
3. Use browser DevTools to inspect elements

Flaky Tests

1. Avoid time-based waits
2. Use state-based waits (waitForLoadState)
3. Ensure proper element visibility before interaction

Schema Validation

All tests must conform to https://metabuilder.dev/schemas/package-playwright.schema.json

Test the schema with:

jq '.tests[0].steps[0]' packages/*/playwright/tests.json

---

This interpreter proves that declarative E2E tests work at production scale with full feature support and zero boilerplate.