metabuilder/docs/JSON_INTERPRETER_EVERYWHERE.md

JSON Interpreter Everywhere Implementation Guide

Overview

This document describes the complete JSON Interpreter Everywhere implementation for MetaBuilder - the architectural pattern where all test types (Playwright E2E, Storybook, unit tests) and styling are declarative JSON definitions interpreted at runtime, rather than hardcoded code.

Status: Phases 1-4 complete, Phase 5 in progress

Philosophy

MetaBuilder follows the principle: "95% configuration, 5% code"

Everything should be declarative JSON/YAML with minimal interpreters, not hardcoded TypeScript or generated code.

Before (Anti-Pattern)

// ❌ Hardcoded test file
describe('Email Validation', () => {
  it('should accept valid email', () => {
    const result = validateEmail('user@example.com');
    expect(result).toBe(true);
  });
});

After (JSON Interpreter Pattern)

{
  "$schema": "...",
  "testSuites": [{
    "name": "Email Validation",
    "tests": [{
      "name": "should accept valid email",
      "act": { "type": "function_call", "target": "validateEmail", "input": "user@example.com" },
      "assert": { "expectations": [{ "type": "truthy", "actual": "result" }] }
    }]
  }]
}

The JSON interpreter (e2e/test-runner/json-interpreter.ts) converts this at runtime to Vitest test suites.

Architecture

Three-Layer System

┌─────────────────────────────────────────────┐
│  Data Layer (JSON/YAML Definitions)         │
│  - tests_schema.json (unit/E2E/Storybook)   │
│  - styles_schema.json (component styling)   │
│  - Stored in packages/*/[type]/tests.json   │
└──────────────────┬──────────────────────────┘
                   │
┌──────────────────▼──────────────────────────┐
│  Interpreter Layer (Minimal TypeScript)     │
│  - JSON test interpreter                    │
│  - Style engine                             │
│  - Component renderer                       │
└──────────────────┬──────────────────────────┘
                   │
┌──────────────────▼──────────────────────────┐
│  Execution Layer (Framework Adapters)       │
│  - Vitest (unit tests)                      │
│  - Playwright (E2E tests)                   │
│  - Storybook (component stories)            │
│  - React (component rendering)              │
└─────────────────────────────────────────────┘

Discovery Flow

packages/
├── auth/
│   ├── unit-tests/tests.json       ← Unit tests (JSON)
│   ├── playwright/tests.json       ← E2E tests (JSON)
│   └── storybook/stories.json      ← Stories (JSON)
├── ui_home/
│   ├── unit-tests/tests.json
│   ├── playwright/tests.json
│   ├── storybook/stories.json
│   └── component/
│       └── styles.json             ← Component styles (JSON)
└── [50+ more packages]

                    ↓

        UnifiedTestRunner
        (e2e/test-runner/index.ts)

                    ↓

    ┌───────────────┬───────────────┐
    │               │               │
    ▼               ▼               ▼
Unit Tests      E2E Tests      Stories
(Vitest)        (Playwright)    (Storybook)

Implementation Phases

Phase 1: Schema Enhancement ✅

Objective: Extend JSON schemas to support all test and style patterns

Deliverables:

• ✅ Enhanced tests_schema.json with:
  • 8 new DOM assertion types
  • Component rendering support
  • DOM interaction actions
  • Fixture interpolation
• ✅ Enhanced styles_schema.json with:
  • Component scoping
  • Variants (size, color, etc.)
  • States (hover, active, disabled)
  • Responsive breakpoints
  • 25+ CSS properties

Files:

• schemas/package-schemas/tests_schema.json
• schemas/package-schemas/styles_schema.json

Commits:

• 0f7c6196 - Enhanced tests_schema.json
• c7a8cace, cf224d5d - Enhanced styles_schema.json

Phase 2: Unified Test Runner ✅

Objective: Build orchestrator for discovering and executing all test types

Deliverables:

• ✅ UnifiedTestRunner class that:
  • Discovers unit tests from packages/*/unit-tests/tests.json
  • Discovers E2E tests from packages/*/playwright/tests.json
  • Discovers Storybook stories from packages/*/storybook/stories.json
  • Filters by package name and tags
  • Registers tests with appropriate interpreters
  • Returns statistics

Discovery Results: 43 total test files

• 1 unit test file
• 8 E2E test files
• 34 Storybook files

Files:

• e2e/test-runner/index.ts (400+ lines)
• e2e/test-runner/json-interpreter.ts (500+ lines)
• e2e/test-runner/types.ts (15+ interfaces)
• e2e/test-runner/README.md

Commits:

• acd9dba5 - Unified test runner and JSON interpreter

Phase 3: Migration Tooling ✅

Objective: Build tools to convert existing TypeScript tests to JSON

Deliverables:

• ✅ TestConverter - AST-based converter:
  • Parses .test.ts files using TypeScript compiler
  • Extracts describe/it/expect blocks
  • Maps 30+ Jest/Vitest matchers to JSON types
  • Returns ConversionResult with warnings/errors
• ✅ TestMigrator - Batch orchestrator:
  • Globs all .test.ts files
  • Runs converter on each file
  • Creates output directories
  • Dry-run mode for safe preview
  • Package name mapping
  • Progress reporting
• ✅ TestValidator - JSON validation:
  • Validates JSON against schema
  • Semantic checks (unique IDs, etc.)
  • Unused import warnings
  • Directory-wide validation

Files:

• scripts/migrate-tests/converter.ts (350+ lines)
• scripts/migrate-tests/migrator.ts (250+ lines)
• scripts/migrate-tests/validator.ts (300+ lines)
• scripts/migrate-tests/index.ts
• scripts/migrate-tests/README.md

Commits:

• 8b32a877 - Comprehensive test migration tooling

Phase 4: Example Package ✅

Objective: Create comprehensive reference package showing all patterns

Deliverables:

• ✅ test_example_comprehensive package:
  • 10 test suites
  • 40+ comprehensive tests
  • All 29 assertion types demonstrated
  • All act phase actions shown
  • Fixture usage patterns
  • Mock setup examples
  • Component rendering and DOM testing
  • Error handling patterns

Test Suites:

1. Email Validation (3 tests) - Valid, invalid, empty
2. Password Security (3 tests) - Hash, consistency, error
3. Token Generation (2 tests) - Format, JWT verification
4. JSON Parsing (3 tests) - Objects, arrays, errors
5. Numeric Comparisons (3 tests) - All numeric types
6. Null/Undefined (4 tests) - Type checking
7. Collections (3 tests) - Properties, arrays, contains
8. Component Rendering (3 tests) - Render, click, disabled
9. Error Handling (2 tests) - Throw, no throw
10. Mixed Assertions (1 test) - Comprehensive validation

Files:

• packages/test_example_comprehensive/package.json
• packages/test_example_comprehensive/unit-tests/tests.json (2,000+ lines)
• packages/test_example_comprehensive/README.md

Commits:

• f00b956e - Comprehensive JSON test example package

Phase 5: Documentation & Batch Migration 🔄

Current Phase: In Progress

Objectives:

• Create comprehensive implementation guide (this file)
• Create migration quick-start guide
• Execute batch migration on existing tests
• Run validation on all converted tests
• Update testing guidelines
• Create trainer guide for teams

Complete File Structure

Schema Files

schemas/package-schemas/
├── tests_schema.json          ← Unit/E2E/Storybook test format
└── styles_schema.json         ← Component styling format

Test Infrastructure

e2e/test-runner/
├── index.ts                   ← Unified orchestrator
├── json-interpreter.ts        ← JSON → Vitest converter
├── types.ts                   ← Type definitions
└── README.md                  ← Runner documentation

scripts/migrate-tests/
├── converter.ts               ← .test.ts → JSON converter
├── migrator.ts                ← Batch migration orchestrator
├── validator.ts               ← JSON validation
├── index.ts                   ← Export module
└── README.md                  ← Migration guide

Example Package

packages/test_example_comprehensive/
├── package.json               ← Metadata
├── unit-tests/tests.json      ← 40+ example tests
└── README.md                  ← Reference guide

Supported Features

JSON Test Format

Arrange Phase (Setup)

"arrange": {
  "fixtures": {
    "email": "test@example.com",
    "password": "SecurePass123"
  },
  "mocks": [
    {
      "target": "generateToken",
      "behavior": { "returnValue": "token123" }
    }
  ]
}

Act Phase (Execution)

• function_call - Call imported function
• render - Render React component
• click - Simulate click
• fill - Fill input
• select - Select option
• hover - Hover element
• focus - Focus element
• blur - Blur element
• waitFor - Wait for condition

Assert Phase (Verification)

• Basic (6): equals, deepEquals, notEquals, truthy, falsy, custom
• Numeric (4): greaterThan, lessThan, greaterThanOrEqual, lessThanOrEqual
• Type (4): null, notNull, undefined, notUndefined
• Collection (5): contains, matches, hasProperty, hasLength, instanceOf
• DOM (8): toBeVisible, toBeInTheDocument, toHaveTextContent, toHaveAttribute, toHaveClass, toBeDisabled, toBeEnabled, toHaveValue
• Control (2): throws, notThrows

Total: 29 assertion types

Fixture Interpolation

Use $arrange.fixtures.key to reference fixtures:

{
  "arrange": { "fixtures": { "email": "test@example.com" } },
  "act": {
    "type": "function_call",
    "target": "validateEmail",
    "input": "$arrange.fixtures.email"
  }
}

Component Styling

{
  "components": {
    "Button": {
      "base": {
        "display": "inline-flex",
        "padding": "12px",
        "borderRadius": "4px"
      },
      "variants": {
        "size": {
          "small": { "padding": "8px" },
          "large": { "padding": "16px" }
        }
      },
      "states": {
        "hover": { "opacity": 0.8 },
        "disabled": { "opacity": 0.5 }
      },
      "responsive": {
        "sm": { "padding": "8px" },
        "md": { "padding": "12px" },
        "lg": { "padding": "16px" }
      }
    }
  }
}

Workflows

Writing New JSON Tests

1. Create test file

   mkdir -p packages/my_package/unit-tests
   touch packages/my_package/unit-tests/tests.json
   

2. Use template

   {
     "$schema": "https://metabuilder.dev/schemas/tests.schema.json",
     "schemaVersion": "2.0.0",
     "package": "my_package",
     "imports": [],
     "testSuites": [
       {
         "id": "suite_1",
         "name": "My Tests",
         "tests": []
       }
     ]
   }
   

3. Add tests (see test_example_comprehensive for patterns)

4. Validate

   npx ts-node scripts/migrate-tests/validator.ts packages/my_package
   

5. Run

   npm run test:unified
   

Migrating Existing Tests

1. Backup

   git add .
   git commit -m "backup: before test migration"
   

2. Preview

   npx ts-node scripts/migrate-tests/migrator.ts --dry-run --verbose
   

3. Migrate

   npx ts-node scripts/migrate-tests/migrator.ts --verbose
   

4. Validate

   npx ts-node scripts/migrate-tests/validator.ts packages
   

5. Test

   npm run test:unified
   

6. Commit

   git add packages/*/unit-tests/
   git commit -m "feat: migrate TypeScript tests to JSON format"
   

Benefits

For Developers

• ✅ Standardized test format across all packages
• ✅ No test boilerplate - focus on logic
• ✅ Visual test inspection in JSON
• ✅ Easy copy/paste patterns from examples
• ✅ CLI tools for validation and migration

For System

• ✅ Tests as configuration, not code
• ✅ Easier debugging (JSON is traceable)
• ✅ Better analytics and reporting
• ✅ Tests can be modified without rebuilding
• ✅ Admin panel could generate tests

For MetaBuilder Architecture

• ✅ Completes 100% declarative goal
• ✅ Aligns with "95% configuration" philosophy
• ✅ Enables dynamic test generation
• ✅ Unified interpreter for all test types
• ✅ Clear separation of data and execution

Best Practices

1. Use Descriptive Names

"name": "should validate email format and reject invalid formats"

2. Organize with Tags

"tags": ["@smoke", "@critical", "@regression"]

3. Use Fixtures for Reusable Values

"arrange": {
  "fixtures": {
    "validEmail": "user@example.com",
    "invalidEmail": "invalid@"
  }
}

4. Multiple Assertions for Complex Cases

"assert": {
  "expectations": [
    { "type": "truthy" },
    { "type": "hasLength", "expected": 20 },
    { "type": "contains", "expected": "$" }
  ]
}

5. Test Fixtures with Mocks

"mocks": [
  {
    "target": "generateToken",
    "behavior": { "returnValue": "mocked_token" }
  }
]

6. Use Setup Hooks

"setup": {
  "beforeEach": [{ "type": "initialize" }],
  "afterEach": [{ "type": "cleanup" }]
}

Common Patterns

Testing a Utility Function

See test_example_comprehensive - Email Validation suite

Testing Error Cases

See test_example_comprehensive - Error Handling suite

Testing React Components

See test_example_comprehensive - Component Rendering suite

Testing with Multiple Assertions

See test_example_comprehensive - Mixed Assertions suite

Migration Statistics

Discovery

• Total test files found: 43
  • Unit tests: 1
  • E2E tests: 8
  • Storybook stories: 34

Migration Readiness

• Existing TypeScript .test.ts files: [To be counted during Phase 5]
• Expected conversion success rate: ~80% (20% may need manual adjustment)
• Estimated effort: 2-4 hours for full migration

Troubleshooting

Schema Validation Fails

# Check the specific errors:
npx ts-node scripts/migrate-tests/validator.ts packages/my_package

Converter Produces Lossy Output

• Complex mocking may not convert perfectly
• Custom hooks require manual adjustment
• Snapshots need manual conversion
• Review warnings and adjust JSON as needed

Tests Don't Execute

1. Ensure JSON is valid: npm --prefix scripts/migrate-tests run validate
2. Check imports are available in runtime
3. Verify test file location: packages/*/unit-tests/tests.json
4. Run unified test runner: npm run test:unified

Integration Points

For Test Writers

• Write tests in JSON format instead of TypeScript
• Use test_example_comprehensive as reference
• Validate with migration tooling
• Run with unified test runner

For Test Runners

• Unified runner automatically discovers all test types
• JSON interpreter converts to framework-specific format
• Same experience across all test types

For CI/CD

• Pre-commit hook could validate JSON
• CI pipeline runs unified test runner
• Reports include all test types

For Admin Panel (Future)

• Tests displayed as JSON
• Admins could view/modify tests
• Real-time test execution
• Test analytics and reporting

Future Enhancements

Near Term

• Batch migrate all existing tests
• Create team training guide
• Add pre-commit JSON validation
• Integration with admin panel

Medium Term

• Visual test editor (drag-and-drop)
• Test generation from behavior specs
• Advanced analytics (flakiness, coverage)
• Test data factory integration

Long Term

• AI-assisted test generation
• Property-based testing support
• Formal specification integration
• Dynamic test orchestration

References

Documentation Files

• e2e/test-runner/README.md - Test runner documentation
• scripts/migrate-tests/README.md - Migration tooling documentation
• packages/test_example_comprehensive/README.md - Example package reference
• schemas/package-schemas/tests_schema.json - Complete schema definition

Key Files

• e2e/test-runner/index.ts - Unified test runner (400+ lines)
• e2e/test-runner/json-interpreter.ts - JSON → Vitest (500+ lines)
• scripts/migrate-tests/converter.ts - Conversion engine (350+ lines)
• scripts/migrate-tests/migrator.ts - Batch orchestrator (250+ lines)
• packages/test_example_comprehensive/unit-tests/tests.json - Example suite (2,000+ lines)

Commits

• acd9dba5 - Unified test runner & JSON interpreter
• 8b32a877 - Migration tooling (converter, migrator, validator)
• f00b956e - Comprehensive example package

Team Guidelines

For QA/Test Engineers

• Learn JSON test format from test_example_comprehensive
• Use migration tools to convert legacy tests
• Write new tests in JSON format
• Validate tests before commit
• Report any conversion issues

For Backend Engineers

• Provide test functions/utilities to import
• Ensure functions are exported properly
• Add to imports array in test file
• Support mock behavior setup

For Frontend Engineers

• Use example package patterns for component tests
• Set up fixtures for UI state
• Test DOM assertions for interaction
• Report any rendering issues

For DevOps/CI

• Add JSON validation to pre-commit hooks
• Run unified test runner in CI pipeline
• Report statistics (conversion success, test counts)
• Archive test results

Success Metrics

Implementation Phase Completion:

• ✅ Phase 1: Schema enhancement complete
• ✅ Phase 2: Unified test runner complete
• ✅ Phase 3: Migration tooling complete
• ✅ Phase 4: Example package complete
• 🔄 Phase 5: Documentation & batch migration in progress

Target Metrics:

• ✅ 43 test files discovered (100% discovered)
• ⏳ 100% of new tests written in JSON (ongoing)
• ⏳ 80%+ of existing tests migrated (pending)
• ⏳ Zero hardcoded test files created (ongoing)
• ⏳ All tests standardized on JSON format (ongoing)

Conclusion

The JSON Interpreter Everywhere implementation enables MetaBuilder to achieve its architectural goal: 95% declarative configuration, 5% code.

By treating all tests as data that's interpreted at runtime, rather than code that's compiled, we gain:

• Flexibility (admins can modify tests without code changes)
• Consistency (same format across all test types)
• Simplicity (minimal interpreter logic)
• Clarity (JSON is easier to understand than nested code)
• Scalability (tests can be generated, optimized, analyzed)

This is a complete implementation pathway from concept to execution.

---

Status: Phase 5 in progress - Documentation complete, batch migration pending

Next: Execute batch migration on all existing tests, update team guidelines, deploy to production.