metabuilder/docs/development/typescript-reduction-guide.md

TypeScript Dependency Reduction - Complete Guide

Objective

Minimize TypeScript files by making the platform 95% data-driven, where functionality is defined through JSON configurations and Lua scripts rather than hardcoded TSX files.

Current Architecture (Post-Iteration 25)

Infrastructure Layer (TypeScript - Required)

These files MUST remain as TypeScript because they provide core platform functionality:

Core App Files (~300 lines)

• App.tsx - React application entry point, routing, authentication flow
• ErrorFallback.tsx - Error boundary component

Generic Renderers (~500 lines)

• GenericPage.tsx - Universal page renderer (supports 4 layouts)
• RenderComponent.tsx - Recursive component tree renderer

Data & Logic Engines (~800 lines)

• Database.ts - KV store wrapper, all data persistence
• LuaEngine.ts - Lua script executor with sandboxing
• WorkflowEngine.ts - Workflow execution engine

Type Definitions (~400 lines)

• level-types.ts - Core type definitions
• builder-types.ts - Builder-specific types
• schema-types.ts - Schema definitions
• package-types.ts - Package system types

Helper Libraries (~600 lines)

• auth.ts - Authentication utilities
• utils.ts - General utilities
• component-registry.ts - Component type registry
• password-utils.ts - Password & email utilities

Shadcn UI Components (~3000 lines)

• 45 components in /components/ui/
• Pre-built, reusable UI library
• Can't be replaced without losing design system

Total Infrastructure: ~5600 lines of essential TypeScript

Content Layer (Can Be Pure Data)

These files contain content that can be defined as JSON + Lua:

⚠️ To Be Removed (~1800 lines)

• Level1.tsx - Homepage content (400 lines)
• Level2.tsx - User dashboard content (500 lines)
• Level3.tsx - Admin panel content (450 lines)
• Level4.tsx - God tier builder (400 lines)
• Level5.tsx - Super god panel (350 lines)

These will be replaced by PageDefinitions in database.

⚠️ Complex Components (~2500 lines)

These are partially data-driven but still have TSX:

• Builder.tsx - Visual builder interface
• Canvas.tsx - Drag-and-drop canvas
• ComponentHierarchyEditor.tsx - Tree editor
• PropertyInspector.tsx - Props editor
• WorkflowEditor.tsx - Workflow diagram
• SchemaEditor.tsx - Database schema editor
• ThemeEditor.tsx - Theme customizer
• UserManagement.tsx - User CRUD
• PageRoutesManager.tsx - Route config
• And 30+ more specialized components

These should eventually be defined declaratively too!

✅ Already Declarative

• IRC Webchat - 100% JSON + Lua
• Forum components - Defined in packages
• All page content - Via PageDefinitionBuilder

Reduction Roadmap

Phase 1: ✅ COMPLETE (Iteration 24-25)

Goal: Create generic rendering infrastructure

✅ Created GenericPage.tsx - Universal page renderer ✅ Created PageRenderer system ✅ Created PageDefinitionBuilder ✅ Created modular seed data system (/src/seed-data/) ✅ Reduced seed-data.ts from 280 lines to 5 lines (98% reduction) ✅ Documented data-driven architecture

Impact: Foundation for procedural generation in place

Phase 2: 🎯 NEXT (Iteration 26)

Goal: Replace hardcoded level files

Tasks:

1. Update App.tsx to use GenericPage for Level 1:

   // OLD
   <Level1 onNavigate={handleNavigate} />
   
   // NEW
   <GenericPage 
     pageId="page_level1_home"
     user={currentUser}
     level={1}
     onNavigate={handleNavigate}
   />
   

2. Test Level 1 functionality:

   • Homepage renders correctly
   • Navigation works
   • God credentials display
   • Hamburger menu functions

3. Repeat for Level 2:

   // OLD
   <Level2 user={currentUser} onLogout={handleLogout} onNavigate={handleNavigate} />
   
   // NEW
   <GenericPage 
     pageId="page_level2_dashboard"
     user={currentUser}
     level={2}
     onNavigate={handleNavigate}
     onLogout={handleLogout}
   />
   

4. Repeat for Level 3:

   <GenericPage 
     pageId="page_level3_admin"
     user={currentUser}
     level={3}
     onNavigate={handleNavigate}
     onLogout={handleLogout}
   />
   

5. Delete TSX files:

   • ❌ Delete Level1.tsx (400 lines)
   • ❌ Delete Level2.tsx (500 lines)
   • ❌ Delete Level3.tsx (450 lines)

Impact: ~1350 lines of TSX removed, replaced by JSON configurations

Phase 3: 📋 FUTURE (Iteration 27-30)

Goal: Make Level 4/5 data-driven

Current state:

• Level 4/5 have complex UI builders
• These are still hardcoded TSX
• Need to be procedurally generated

Strategy:

1. Create "Meta-Builder" - A builder that builds builders
2. Define Level 4/5 UI as component trees
3. Store in database like other pages
4. Render via GenericPage

Example:

{
  "id": "page_level4_builder",
  "level": 4,
  "layout": "dashboard",
  "components": [
    {
      "type": "Sidebar",
      "children": [
        {
          "type": "SidebarItem",
          "props": {
            "label": "Pages",
            "icon": "File",
            "onClick": "script_navigate_pages"
          }
        },
        {
          "type": "SidebarItem",
          "props": {
            "label": "Components",
            "icon": "Cube",
            "onClick": "script_navigate_components"
          }
        }
      ]
    },
    {
      "type": "MainContent",
      "children": [
        {
          "type": "PageEditor",
          "props": {
            "pageId": "{{context.currentPageId}}"
          }
        }
      ]
    }
  ]
}

Challenges:

• Level 4/5 are self-referential (they edit themselves)
• Need bootstrap mechanism
• Complex interactions (drag-drop, code editors)
• Real-time preview

Solutions:

• Keep minimal "bootstrap" Level 4/5 TSX
• Move 90% of functionality to declarative definitions
• Use Lua for complex interactions
• Store editor state in database

Impact: ~2300 lines of TSX reduced to ~300 lines of bootstrap code

Phase 4: 🚀 AMBITIOUS (Future)

Goal: Declarative specialized components

Current state:

• 30+ specialized components (PropertyInspector, WorkflowEditor, etc.)
• All hardcoded TSX
• Total: ~2500 lines

Vision:

• Define component editors as JSON + Lua
• Create generic "MetaComponent" renderer
• Store all UI definitions in database

Example:

{
  "id": "component_property_inspector",
  "type": "MetaComponent",
  "definition": {
    "layout": "form",
    "fields": [
      {
        "name": "className",
        "type": "text",
        "label": "CSS Classes",
        "validator": "script_validate_css_classes"
      },
      {
        "name": "variant",
        "type": "select",
        "label": "Variant",
        "options": "{{lua:getVariantOptions(component.type)}}"
      }
    ],
    "actions": [
      {
        "label": "Save",
        "onClick": "workflow_save_component_props"
      }
    ]
  }
}

Impact: ~2000 lines of TSX converted to JSON

Target Metrics

Current (Post-Iteration 25)

Infrastructure:    5,600 lines (TypeScript) - Required
Content TSX:       4,600 lines (TypeScript) - Can be converted
Seed Data:         1,200 lines (TypeScript) - Already modular
Total TSX:        11,400 lines

Data-driven:          80%
Hardcoded:            20%

After Phase 2

Infrastructure:    5,600 lines (TypeScript) - Required
Content TSX:       3,250 lines (TypeScript) - Can be converted
Seed Data:         1,200 lines (TypeScript) - Already modular
Total TSX:        10,050 lines

Data-driven:          85%
Hardcoded:            15%

Reduction: -1,350 lines (12% reduction)

After Phase 3

Infrastructure:    5,600 lines (TypeScript) - Required
Content TSX:         950 lines (TypeScript) - Minimal bootstrap
Seed Data:         1,200 lines (TypeScript) - Already modular
Total TSX:         7,750 lines

Data-driven:          92%
Hardcoded:             8%

Reduction: -3,650 lines (32% total reduction)

After Phase 4 (Target)

Infrastructure:    5,600 lines (TypeScript) - Required
Bootstrap:           500 lines (TypeScript) - Minimal init
Seed Data:         1,200 lines (TypeScript) - Already modular
Total TSX:         7,300 lines

Data-driven:          95%
Hardcoded:             5%

Reduction: -4,100 lines (36% total reduction)

Key Insights

What Can't Be Removed

1. React Framework Code - App.tsx, main.tsx
2. Generic Renderers - GenericPage, RenderComponent
3. Core Engines - Database, LuaEngine, WorkflowEngine
4. Type Definitions - Essential for TypeScript compilation
5. UI Library - Shadcn components (or equivalent)

What Should Be Removed

1. Level-specific TSX files - Replace with PageDefinitions
2. Hardcoded page content - Move to database
3. Static component configurations - Use JSON configs
4. Hardcoded workflows - Define as node graphs
5. Inline business logic - Extract to Lua scripts

Benefits of Reduction

1. Maintainability

• Less code to maintain
• Clear separation: infrastructure vs. content
• Easy to find and fix bugs

2. Flexibility

• Change content without recompiling
• A/B test different UIs
• User-customizable interfaces

3. Distribution

• Share pages/components as data
• Package ecosystem
• Version control for content

4. Performance

• Smaller bundle size
• Lazy load content
• Dynamic optimization

5. Accessibility

• Non-developers can contribute content
• Visual editors for all content
• No TypeScript knowledge required

Implementation Guide

Converting a TSX File to Declarative

Before (Level1.tsx):

export function Level1({ onNavigate }: Level1Props) {
  const [menuOpen, setMenuOpen] = useState(false)
  
  return (
    <div className="min-h-screen bg-background">
      <header>
        <Button onClick={() => setMenuOpen(!menuOpen)}>
          <List />
        </Button>
      </header>
      
      <main className="container mx-auto py-12">
        <Card>
          <CardHeader>
            <CardTitle>Welcome to MetaBuilder</CardTitle>
          </CardHeader>
          <CardContent>
            <p>Build applications without code</p>
            <Button onClick={() => onNavigate(2)}>
              Get Started
            </Button>
          </CardContent>
        </Card>
      </main>
    </div>
  )
}

After (JSON in database):

{
  "id": "page_level1_home",
  "level": 1,
  "title": "Home",
  "layout": "default",
  "components": [
    {
      "id": "header",
      "type": "Header",
      "props": {
        "className": "border-b"
      },
      "children": [
        {
          "id": "menu_button",
          "type": "Button",
          "props": {
            "variant": "ghost",
            "size": "icon"
          },
          "children": [
            {
              "id": "menu_icon",
              "type": "Icon",
              "props": {
                "name": "List"
              },
              "children": []
            }
          ],
          "events": {
            "onClick": "script_toggle_menu"
          }
        }
      ]
    },
    {
      "id": "main_container",
      "type": "Container",
      "props": {
        "className": "mx-auto py-12"
      },
      "children": [
        {
          "id": "hero_card",
          "type": "Card",
          "children": [
            {
              "id": "card_header",
              "type": "CardHeader",
              "children": [
                {
                  "id": "card_title",
                  "type": "CardTitle",
                  "props": {
                    "children": "Welcome to MetaBuilder"
                  },
                  "children": []
                }
              ]
            },
            {
              "id": "card_content",
              "type": "CardContent",
              "children": [
                {
                  "id": "description",
                  "type": "Text",
                  "props": {
                    "children": "Build applications without code"
                  },
                  "children": []
                },
                {
                  "id": "cta_button",
                  "type": "Button",
                  "props": {
                    "children": "Get Started",
                    "variant": "default"
                  },
                  "children": [],
                  "events": {
                    "onClick": "workflow_navigate_dashboard"
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "luaScripts": {
    "onLoad": "script_page_load_analytics"
  }
}

Lua Scripts:

-- script_toggle_menu.lua
function toggleMenu()
  local state = context.state.menuOpen or false
  context.setState({ menuOpen = not state })
  return true
end

return toggleMenu

-- script_page_load_analytics.lua
function logPageView()
  local userId = context.user.id or "anonymous"
  local timestamp = os.time()
  
  print("Page view: Level 1 by " .. userId .. " at " .. timestamp)
  
  -- Could also call workflow or external API
  return true
end

return logPageView

Workflow:

{
  "id": "workflow_navigate_dashboard",
  "name": "Navigate to Dashboard",
  "nodes": [
    {
      "id": "check_auth",
      "type": "condition",
      "label": "Is Authenticated?",
      "config": {
        "condition": "context.user != null"
      }
    },
    {
      "id": "navigate_dashboard",
      "type": "action",
      "label": "Go to Dashboard",
      "config": {
        "action": "navigate",
        "target": 2
      }
    },
    {
      "id": "navigate_login",
      "type": "action",
      "label": "Go to Login",
      "config": {
        "action": "navigate",
        "target": 2
      }
    }
  ],
  "edges": [
    { "source": "check_auth", "target": "navigate_dashboard", "label": "true" },
    { "source": "check_auth", "target": "navigate_login", "label": "false" }
  ]
}

Rendered by:

<GenericPage 
  pageId="page_level1_home"
  user={currentUser}
  level={1}
  onNavigate={handleNavigate}
/>

Conclusion

The path to minimal TypeScript dependency is clear:

1. ✅ Phase 1 Complete: Infrastructure in place
2. 🎯 Phase 2 Next: Replace Level 1-3 TSX files
3. 📋 Phase 3 Future: Make Level 4-5 data-driven
4. 🚀 Phase 4 Ambitious: Declarative specialized components

End Goal: 95% data-driven platform with only 5% infrastructure code.

---

Last updated: Iteration 25 Current progress: 80% data-driven Target: 95% data-driven