metabuilder/docs/testing/DECLARATIVE_TESTING_ARCHITECTURE.md

Declarative Testing Architecture for MetaBuilder

Status: DESIGN DOCUMENT - Ready for Implementation Date: January 21, 2026 Objective: Complete 100% data-driven architecture by making all tests declarative (Playwright, Storybook, Unit)

---

Executive Summary

MetaBuilder already achieves 95% data-driven architecture through declarative configuration. The final 5% missing is the testing layer. This document outlines how to make ALL testing declarative (not TypeScript) by consolidating Playwright E2E flows, Storybook component stories, and unit tests into the packages system as JSON definitions.

Current State:

• ✅ Playwright tests: Declarative JSON in packages/*/playwright/tests.json (exists!)
• ✅ Storybook stories: Declarative JSON in packages/*/storybook/stories.json (exists!)
• ❌ Unit tests: Still TypeScript .spec.ts files (needs conversion!)

Proposed End State:

• ✅ Unit tests: Declarative JSON in packages/*/unit-tests/tests.json (NEW!)
• ✅ Centralized test runner that executes all three declaratively
• ✅ 100% data-driven architecture achieved

---

Current Testing Infrastructure

1. Playwright E2E Tests (ALREADY DECLARATIVE) ✅

Location: packages/*/playwright/tests.json

Example: /packages/ui_home/playwright/tests.json

{
  "$schema": "https://metabuilder.dev/schemas/package-playwright.schema.json",
  "package": "ui_home",
  "tests": [
    {
      "name": "should load home page successfully",
      "tags": ["@smoke", "@critical"],
      "steps": [
        { "action": "navigate", "url": "/" },
        { "action": "waitForLoadState", "state": "domcontentloaded" },
        { "action": "expect", "selector": "body", "assertion": { "matcher": "toContainText", "expected": "MetaBuilder" } }
      ]
    }
  ]
}

Runner: /e2e/json-runner/ - Loads and executes declarative tests

---

2. Storybook Stories (ALREADY DECLARATIVE) ✅

Location: packages/*/storybook/stories.json

Example: /packages/ui_home/storybook/stories.json

{
  "$schema": "https://metabuilder.dev/schemas/package-storybook.schema.json",
  "title": "Home Page Components",
  "stories": [
    {
      "name": "HomePage",
      "render": "home_ui",
      "description": "Complete home page with all sections"
    }
  ]
}

Runner: storybook/json-loader/ - Loads and renders declarative stories

---

3. Unit Tests (NOT YET DECLARATIVE) ❌

Current Location: e2e/**/*.spec.ts (TypeScript)

Problem:

• Tests live outside package system
• Written in TypeScript (not data)
• Cannot be managed as configuration
• Not discoverable by package system

Proposed Location: packages/*/unit-tests/tests.json (JSON)

Schema: Schema already exists at /schemas/package-schemas/tests_schema.json

---

Proposed Unit Test Declarative Format

Package Structure

packages/[packageId]/
├── unit-tests/
│   ├── tests.json              [Declarative unit tests]
│   ├── fixtures.json           [Test data/mocks]
│   └── README.md               [Test documentation]
├── playwright/
│   ├── tests.json              [E2E flows]
│   └── metadata.json           [Configuration]
└── storybook/
    ├── stories.json            [Component stories]
    └── config.json             [Configuration]

Unit Test JSON Format Example

{
  "$schema": "https://metabuilder.dev/schemas/tests.schema.json",
  "schemaVersion": "2.0.0",
  "package": "ui_home",
  "description": "Unit tests for Home Page components",
  "imports": [
    {
      "from": "@testing-library/react",
      "import": ["render", "screen", "fireEvent"]
    },
    {
      "from": "@/components/home",
      "import": ["HomePage", "HeroSection"]
    }
  ],
  "setup": {
    "beforeEach": [
      {
        "type": "fixture",
        "name": "clearMocks",
        "config": {
          "target": "localStorage"
        }
      }
    ]
  },
  "testSuites": [
    {
      "id": "hero-section",
      "name": "HeroSection Component",
      "tests": [
        {
          "id": "renders-title",
          "name": "should render hero title",
          "arrange": {
            "component": "HeroSection",
            "props": {
              "title": "Build Anything, Visually",
              "subtitle": "No code required."
            }
          },
          "act": {
            "render": true
          },
          "assert": [
            {
              "selector": ".hero-title",
              "matcher": "toHaveTextContent",
              "expected": "Build Anything"
            },
            {
              "selector": ".hero-subtitle",
              "matcher": "toBeVisible"
            }
          ]
        },
        {
          "id": "renders-cta-buttons",
          "name": "should render call-to-action buttons",
          "arrange": {
            "component": "HeroSection",
            "props": { "title": "Test", "subtitle": "Test" }
          },
          "act": { "render": true },
          "assert": [
            {
              "role": "button",
              "text": "Get Started",
              "matcher": "toBeVisible"
            },
            {
              "role": "button",
              "text": "Watch Demo",
              "matcher": "toBeVisible"
            }
          ]
        }
      ]
    },
    {
      "id": "home-page",
      "name": "HomePage Component",
      "tests": [
        {
          "id": "renders-all-sections",
          "name": "should render all page sections",
          "arrange": {
            "component": "HomePage"
          },
          "act": { "render": true },
          "assert": [
            { "selector": ".hero-section", "matcher": "toBeInTheDocument" },
            { "selector": ".features-section", "matcher": "toBeInTheDocument" },
            { "selector": ".about-section", "matcher": "toBeInTheDocument" },
            { "selector": ".contact-section", "matcher": "toBeInTheDocument" }
          ]
        }
      ]
    }
  ]
}

---

Universal Test Runner

Centralized Test Execution

npm run test           # Run all tests (all types)
npm run test:e2e       # Run Playwright E2E tests
npm run test:stories   # Validate Storybook stories
npm run test:unit      # Run unit tests

# Filter by package
npm run test -- --package ui_home
npm run test -- --tag @smoke
npm run test -- --package admin --tag @critical

Test Runner Architecture

/test-runners/
├── runner.ts                 [Main orchestrator]
├── e2e/
│   └── playwright-runner.ts  [Runs Playwright tests]
├── storybook/
│   └── story-runner.ts       [Validates Storybook stories]
└── unit/
    └── vitest-runner.ts      [Runs unit tests]

---

Implementation Phases

Phase 1: Create Unit Test Declarative Format (WEEK 1)

Tasks:

1. Finalize tests_schema.json (already ~80% complete)
2. Create unit test format documentation
3. Define AAA (Arrange-Act-Assert) pattern for JSON
4. Create example unit-tests package

Deliverables:

• Complete JSON schema for unit tests
• 3-4 example unit test JSON files
• Runner implementation draft

Estimated Effort: 2-3 days

---

Phase 2: Create Test Runner (WEEK 1-2)

Tasks:

1. Build unified test runner that:

   • Discovers tests across all packages
   • Loads JSON test definitions
   • Executes in correct order (unit → story → E2E)
   • Generates consolidated report

2. Integrate with existing runners:

   • Hook into Playwright runner
   • Hook into Vitest runner
   • Hook into Storybook generator

Deliverables:

• Unified test runner CLI
• Package discovery engine
• Test orchestration logic

Estimated Effort: 3-4 days

---

Phase 3: Migrate Existing Tests (WEEK 2-3)

Tasks:

1. Convert 50 existing .spec.ts files to JSON
2. Move tests into package structure
3. Validate all conversions
4. Run full test suite and verify passes

Deliverables:

• All existing tests migrated to JSON
• Test coverage verified
• 100% pass rate maintained

Estimated Effort: 4-5 days

---

Phase 4: Documentation & Training (WEEK 3)

Tasks:

1. Write comprehensive guide for writing unit tests as JSON
2. Create migration examples
3. Document best practices
4. Train team on new approach

Deliverables:

• DECLARATIVE_UNIT_TESTS_GUIDE.md (1000+ words)
• Video tutorial
• Code examples for common patterns

Estimated Effort: 2-3 days

---

Benefits of Declarative Testing

1. 100% Data-Driven Architecture

• Everything is configuration, not code
• Follows MetaBuilder's core principle
• Enables non-developers to write tests

2. Better Test Discovery

• All tests discoverable via package system
• Automatic test collection
• Test reports grouped by package

3. Easier Test Maintenance

• No code duplication
• Consistent patterns across all tests
• Version control friendly (JSON diffs)

4. AI-Friendly

• LLMs can easily generate test JSON
• Tests can be generated from requirements
• Automated test generation possible

5. Improved Traceability

• Tests map to components via package
• Clear requirements → tests → code lineage
• Easier coverage analysis

6. Better Test Organization

• Tests live with their components
• No scattered test files
• Natural grouping by feature

---

Example: Complete System Flows Test Package

New Package Structure

packages/system_flows/
├── unit-tests/
│   └── tests.json              [User flow unit tests]
├── playwright/
│   ├── tests.json              [E2E flows: public → login → admin]
│   └── metadata.json           [Test metadata]
├── storybook/
│   └── stories.json            [UI component stories]
└── package.json

Unit Test Example

{
  "schemaVersion": "2.0.0",
  "package": "system_flows",
  "testSuites": [
    {
      "id": "permission-levels",
      "name": "Permission Level Access Control",
      "tests": [
        {
          "id": "level-0-public",
          "name": "Level 0 (Public) should access homepage",
          "arrange": {
            "user": { "level": 0, "authenticated": false }
          },
          "act": {
            "navigate": "/"
          },
          "assert": [
            { "selector": "body", "matcher": "toBeVisible" },
            { "text": "Sign In", "matcher": "toBeVisible" }
          ]
        },
        {
          "id": "level-5-supergod",
          "name": "Level 5 (Supergod) should access schema editor",
          "arrange": {
            "user": { "level": 5, "authenticated": true }
          },
          "act": {
            "navigate": "/admin/schema-editor"
          },
          "assert": [
            { "selector": ".schema-editor", "matcher": "toBeInTheDocument" },
            { "text": "Create Entity", "matcher": "toBeVisible" }
          ]
        }
      ]
    }
  ]
}

Playwright E2E Example

{
  "package": "system_flows",
  "tests": [
    {
      "name": "Complete user flow: public → login → admin",
      "tags": ["@critical", "@flow"],
      "steps": [
        { "action": "navigate", "url": "/" },
        { "action": "expect", "text": "Build Anything", "assertion": { "matcher": "toBeVisible" } },
        { "action": "click", "role": "button", "text": "Sign In" },
        { "action": "fill", "label": "Username", "value": "admin" },
        { "action": "fill", "label": "Password", "value": "password" },
        { "action": "click", "role": "button", "text": "Login" },
        { "action": "waitForLoadState", "state": "networkidle" },
        { "action": "expect", "selector": ".admin-dashboard", "assertion": { "matcher": "toBeVisible" } },
        { "action": "click", "role": "link", "text": "Package Manager" },
        { "action": "expect", "text": "Installed Packages", "assertion": { "matcher": "toBeVisible" } }
      ]
    }
  ]
}

---

Technical Roadmap

Required Changes

1. Schema Enhancement

   • Finalize tests_schema.json with all AAA matchers
   • Add fixture definitions
   • Add mock/spy support definitions

2. Runner Implementation

   • Create test-runners/unit-runner.ts to execute JSON tests
   • Update main test orchestrator
   • Integrate with CI/CD pipeline

3. Package System Updates

   • Add unit-tests/ as recognized entity type
   • Update package discovery to include unit tests
   • Add metadata validation

4. Test Migration

   • Script to help convert .spec.ts → tests.json
   • Validation tool to ensure equivalence
   • Coverage report generator

---

Questions for Implementation

1. Test Execution Environment

   • Should JSON tests be compiled to Vitest at runtime?
   • Or create dedicated JSON test executor?
   • Recommendation: Runtime compilation for better error reporting

2. Fixture Management

   • Where do mocked data/fixtures live?
   • In unit-tests/fixtures.json?
   • Or inline in test definitions?
   • Recommendation: Both - inline for simple, separate for complex

3. Snapshot Testing

   • How to handle snapshot updates in JSON format?
   • Store snapshots separately with references?
   • Recommendation: Use hash-based references to external snapshots

4. CI/CD Integration

   • Run all tests in parallel or sequentially?
   • Per-package test isolation?
   • Recommendation: Parallel by package, sequential within package

---

Success Criteria

✅ All tests are declarative (JSON) ✅ No TypeScript test files (100% migrated) ✅ All existing tests pass (100% coverage maintained) ✅ Unified test runner works seamlessly ✅ Package discovery includes unit tests ✅ Documentation complete with examples ✅ Team trained on new approach ✅ CI/CD integrated successfully

---

Conclusion

This document outlines the final step to achieve 100% data-driven architecture in MetaBuilder by making unit tests declarative. The infrastructure is already in place (schemas exist, runners exist), requiring primarily:

1. Schema finalization (~2 days)
2. Runner implementation (~4 days)
3. Test migration (~5 days)
4. Documentation (~3 days)

Total Estimated Effort: 14 days (2 weeks) for full implementation

This positions MetaBuilder as the most data-driven full-stack system available - everything from UI components to security policies to tests is declarative configuration, not code.

---

Ready for review and implementation planning! 🚀