git/metabuilder

Fork 0

mirror of https://github.com/johndoe6345789/metabuilder.git synced 2026-04-24 13:54:57 +00:00

Files

johndoe6345789 862cc29457 various changes

2026-03-09 22:30:41 +00:00

101 KiB

Raw Permalink Blame History

MetaBuilder Roadmap

The Single Source of Truth for MetaBuilder Development

Version: 0.8.8 Last Updated: March 4, 2026 Status: 🎯 C++ DBAL Production Live ✅ — Event-Driven Workflows, JWT Auth, Full-Stack Pastebin Structure: Universal Platform Monorepo

📁 Monorepo Structure

metabuilder/
├── cadquerywrapper/      # Parametric 3D CAD modeling (Python, CadQuery)
├── caproverforge/        # CapRover PaaS mobile client (Android, Kotlin)
├── codegen/              # Visual code generation studio (React, Vite)
├── config/               # Lint, test, misc configs
├── dbal/                 # Database Abstraction Layer
│   ├── development/      # TypeScript implementation
│   ├── production/       # C++ implementation
│   └── shared/api/schema/# JSON entities (SOURCE OF TRUTH)
├── deployment/           # Docker & infrastructure
├── dockerterminal/       # Docker Swarm management (Next.js)
├── docs/                 # Documentation (organized)
│   ├── analysis/         # Status reports, assessments
│   ├── architecture/     # System design docs
│   ├── guides/           # Quick references, how-tos
│   ├── implementation/   # Implementation details
│   ├── packages/         # Package-specific docs
│   ├── phases/           # Phase completion reports
│   ├── testing/          # E2E and test docs
│   └── workflow/         # Workflow engine docs
├── e2e/                  # End-to-end Playwright tests
├── fakemui/              # Material UI clone (React, QML)
├── frontends/            # Multiple frontends
│   ├── cli/              # Command-line interface
│   ├── nextjs/           # Primary web UI
│   └── qt6/              # Desktop application
├── gameengine/           # SDL3 C++ game engine
├── mojo/                 # Mojo examples (systems programming)
│   └── examples/         # Official Modular examples
├── packagerepo/          # Package repository service (Python, FastAPI)
├── packages/             # 62+ MetaBuilder feature packages
├── pastebin/             # Snippet pastebin (Next.js)
├── pcbgenerator/         # PCB design library (Python)
├── postgres/             # PostgreSQL admin dashboard (Next.js, Drizzle)
├── repoforge/            # GitHub Android client (Kotlin, Compose)
├── schemas/              # JSON validation schemas
├── scripts/              # Build and migration scripts
├── services/             # Background services
├── smtprelay/            # SMTP relay server (Python, Twisted)
├── sparkos/              # Minimal Linux distro + Qt6 app
├── storybook/            # Component documentation (React, Vite)
├── workflow/             # Workflow engine
│   ├── executor/         # Multi-language executors (ts, python, cpp)
│   ├── examples/         # Workflow examples
│   └── plugins/          # Workflow plugins
└── [root files]          # package.json, playwright.config.ts, etc.

Standalone Projects (15)

Project	Purpose	Tech Stack
`cadquerywrapper/`	Parametric 3D CAD modeling	Python, CadQuery
`caproverforge/`	CapRover PaaS mobile client	Android, Kotlin
`codegen/`	Visual code generation studio	React, Vite, TypeScript
`dockerterminal/`	Docker Swarm management	Next.js, TypeScript
`fakemui/`	Material UI clone (React + QML)	React, QML, TypeScript
`gameengine/`	2D/3D game engine	C++, SDL3
`mojo/`	Systems programming examples	Mojo (Python superset)
`packagerepo/`	Package repository service	Python, FastAPI
`pastebin/`	Code snippet sharing	Next.js, TypeScript
`pcbgenerator/`	PCB design automation	Python
`postgres/`	PostgreSQL admin dashboard	Next.js, Drizzle ORM
`repoforge/`	GitHub client for Android	Kotlin, Compose
`smtprelay/`	Email relay server	Python, Twisted
`sparkos/`	Minimal Linux + Qt6 app	C++, Qt6, Linux
`storybook/`	Component documentation	React, Vite, Storybook

🚀 Executive Summary

What's Done (Phase 0–2 - ✅ Complete):

✅ Core architecture (Next.js, Prisma, DBAL, Multi-tenant)
✅ Authentication & authorization (Session-based, 6-level permissions)
✅ CRUD operations (Schema-driven, all major endpoints)
✅ Package system (62 packages, auto-loading, dynamic routing)
✅ Generic component renderer (JSON-to-React)
✅ Infrastructure (Docker, PostgreSQL, Redis, Nginx)
✅ Game engine (SDL3/bgfx, 27/27 tests passing)
✅ FakeMUI (167 components — Material Design clone)
✅ WorkflowUI visual editor (n8n-style, 152 plugin nodes)

What's Done (Phase 7 — C++ DBAL Production ✅ Complete, shipped Mar 2026):

✅ C++ DBAL daemon in production (Drogon HTTP, 14 DB backends)
✅ Full YAML → JSON migration (63 files, yaml-cpp removed)
✅ JWT authentication + JSON ACL config
✅ Event-driven workflow engine (pastebin.User.created → 15-node JSON workflow)
✅ Declarative seed data (dbal/shared/seeds/database/ — 3 users, 12 namespaces, 28 snippets)
✅ Flask auth backend (register, login, JWT, Python runner)
✅ Pastebin full-stack (Next.js + Redux + IndexedDB + Flask + DBAL C++)
✅ Dark/light theme switcher, i18n (EN/ES)

What's Planned (Phase 3+ - Future):

🔮 Phase 3: Enhanced CRUD (Rich forms, bulk operations, relationships)
🔮 Phase 4: God Panel (Admin dashboard, system management)
🔮 Phase 5: Advanced Features (Webhooks, DBAL workflow triggers)
🔮 Phase 6: Advanced Auth (SSO, SAML, OAuth)
🔮 Phase 8: Multi-Source Packages (NPM, git, http sources)
🔮 Phase 9: Universal Platform (State Machine, Command Bus, Event Stream, VFS)

Status Dashboard

Phase	Name	Status	Completion	Timeline
0	Foundation	✅ Complete	100%	Pre-2026
1	MVP	✅ Complete	100%	January 2026
2	Backend Integration	✅ Complete	100%	Q1 2026
2.5	Monorepo Consolidation	✅ Complete	100%	January 2026
3	Enhanced CRUD	🔄 In Progress	25%	Q1-Q2 2026
4	God Panel	🔮 Planned	0%	Q2 2026
5	Advanced Features	🔮 Planned	0%	Q2-Q3 2026
6	Advanced Auth	🔮 Planned	0%	Q3 2026
7	C++ DBAL Production	✅ Complete	100%	March 2026
8	Multi-Source Packages	🔮 Planned	0%	Q4 2026
9	Universal Platform	🔮 Planned	0%	2027

📚 What To Read First

Quick status? → See Executive Summary above (2 min read)
What's currently working? → Go to Current Status: What's Working Today (5 min read)
Details on Phase 2? → Jump to Phase 2: Backend Integration (15 min read)
Full phase details? → See Roadmap Phases (30 min read)
Project history? → See Project History at end (10 min read)

Vision

MetaBuilder is an ultra-generic, data-driven multi-tenant platform where everything flows through the database. No hardcoded routes, no component imports, no assumptions—the entire application structure lives in the database, rendered by a generic JSON-to-React engine.

This vision evolved from our original Spark-based admin panel generator (see Project History) into a more scalable, enterprise-ready platform with enhanced multi-tenancy, security, and flexibility.

Core Philosophy

Browser URL → Database Query → JSON Component → Generic Renderer → React → User

Zero hardcoded connections. Everything is data.

Project History

Evolution from Spark-Based Version

MetaBuilder began as a Django-style admin panel generator built on GitHub's Spark framework. The original version (preserved in /old directory) demonstrated the core concept of declarative, data-driven application building but was limited in scope and scalability.

Original Version (Spark-Based) - Pre-2026

Architecture:

Framework: GitHub Spark + Vite + React 19
UI Library: Radix UI components with Tailwind CSS
Storage: KV (Key-Value) storage via Spark
Scripting: Lua integration (Fengari) for custom logic
Permission System: 4 levels (Public, User, Admin, God)
Scope: Single-tenant admin panel generator

Key Features:

Declarative schema definition via JSON
Auto-generated CRUD interfaces
Visual component hierarchy editor with drag-and-drop
Monaco code editor for Lua scripts
Page routing configuration
Database export/import

Limitations:

No multi-tenancy support
Limited scalability (KV storage)
Single deployment architecture
No package system
No production-grade database support
Limited permission granularity

Current Version (Next.js-Based) - 2026+

Architecture:

Framework: Next.js 16 + React 19 + App Router
UI Library: Material-UI (MUI) with SCSS modules
Database: PostgreSQL (production) / SQLite (development) via Prisma ORM
DBAL: Dual implementation (TypeScript for dev, C++ daemon for production)
Scripting: JSON-based scripts with controlled execution
Permission System: 6 levels (Public → User → Moderator → Admin → God → Supergod)
Scope: Multi-tenant enterprise platform

Major Enhancements:

✨ Multi-tenancy: Complete tenant isolation with per-tenant data
✨ Package System: 52 built-in packages with dynamic loading
✨ DBAL: Language-agnostic database abstraction with credential isolation
✨ Production-Ready: PostgreSQL, Redis caching, C++ services
✨ Enhanced Security: 6-level permissions, row-level security, C++ credential isolation
✨ Scalability: Docker deployment, Kubernetes support, multi-instance ready
✨ Testing: Comprehensive test suite (464 tests, 100% pass rate)
✨ Type Safety: Full TypeScript with Prisma ORM

Migration Path

What Was Preserved

✅ Core philosophy of data-driven architecture
✅ Declarative component definitions (enhanced to JSON)
✅ Generic rendering system (JSON-to-React)
✅ Visual configuration capabilities
✅ Schema-driven CRUD generation
✅ Permission-based access control (expanded)

What Was Reimplemented

🔄 Storage: KV → PostgreSQL/SQLite with Prisma
🔄 UI Library: Radix + Tailwind → Material-UI + SCSS
🔄 Scripting: Lua → JSON-based scripts
🔄 Framework: Vite/Spark → Next.js App Router
🔄 Architecture: Single-tenant → Multi-tenant
🔄 Permissions: 4 levels → 6 levels

What Was Added

➕ Multi-tenant support with complete isolation
➕ Package system with 52 built-in packages
➕ DBAL (TypeScript + C++ dual implementation)
➕ Production deployment infrastructure (Docker, Nginx, Redis)
➕ C++ media processing daemon
➕ Comprehensive test suite (unit, integration, E2E)
➕ Static site generation with ISR
➕ Session management with expiry tracking
➕ Remote package loading capability

Architecture Comparison

Aspect	Old (Spark)	New (Next.js)
Framework	Vite + Spark	Next.js 16 App Router
UI Components	Radix UI	Material-UI (MUI)
Styling	Tailwind CSS	SCSS Modules + MUI sx prop
Database	KV Storage	PostgreSQL/SQLite + Prisma
Scripting	Lua (Fengari)	JSON Scripts
Permissions	4 levels	6 levels
Tenancy	Single	Multi-tenant
Package System	None	52 packages + dynamic loading
Deployment	Single process	Docker + microservices
Testing	Limited	464 tests (100% pass)
Type Safety	TypeScript	TypeScript + Prisma
Production Ready	Dev only	Production + C++ services

Why We Migrated

Scalability Needs

The KV storage approach limited our ability to:

Handle complex queries and relationships
Scale to thousands of users
Support multi-tenancy efficiently
Provide ACID guarantees

Enterprise Requirements

Production deployments required:

Robust database with transactions
Credential isolation (C++ DBAL)
Multi-tenant data isolation
Professional deployment options
Comprehensive testing

Ecosystem Maturity

PostgreSQL provides battle-tested reliability
Prisma ORM offers excellent type safety
Next.js App Router enables optimal performance
Material-UI provides comprehensive component library

Legacy Code Location

The original Spark-based version is preserved in /old directory for reference:

Documentation: /old/*.md
Source code: /old/src/
Configuration: /old/package.json, /old/tsconfig.json

Note: The old version is maintained for historical reference only and is not actively developed.

Start Here (Essential Reading):

Executive Summary - Status overview (2 min)
Status Dashboard - Phase completion matrix (1 min)
Current Status - What's working today (5 min)

Development Reference: 4. Technology Stack - Technologies used 5. System Architecture - How it all works together 6. Roadmap Phases - Detailed phase breakdown

Information: 7. Package Ecosystem - Available packages 8. Feature Status Matrix - Feature checklist 9. Release History - What changed when 10. Known Issues & Technical Debt - What needs fixing

Deep Dives (Optional): 11. Project History - Evolution from Spark to Next.js 12. MVP Milestone - MVP completion details 13. Testing Strategy & Best Practices 14. Development Best Practices 15. Long-Term Vision - 5-year roadmap 16. Success Metrics - How we measure success 17. Contributing - How to contribute

Current Status

Metric	Value
Current Phase	Phase 7 Complete ✅ — C++ DBAL Production shipped
Version	0.8.8 (pastebin)
Build Status	✅ Production
Game Engine Tests	27/27 passing (100%)
Last Release	March 4, 2026 (C++ DBAL + workflows + JWT auth)
Last Update	March 4, 2026

Quick Stats

Entity Schemas: 39 JSON entity definitions (was YAML, migrated Mar 2026)
DB Backends: 14 (memory, sqlite, postgres, mysql, mariadb, cockroachdb, mongodb, redis, elasticsearch, cassandra, surrealdb, supabase, prisma)
Built-in Packages: 62+ packages ready to use
Standalone Projects: 15 integrated into monorepo
Technology Stack: Next.js 16, React 19, TypeScript 5.9, Flask, Drogon C++
Services: Next.js frontend, Flask auth, DBAL C++ daemon, PostgreSQL, Nginx
Workflow Steps: 7 (dbal.uuid, dbal.timestamp, dbal.entity.create/get/list, dbal.var.set, dbal.log)
Languages: TypeScript, Python, C++, Kotlin, Mojo, Go, Rust

What's Working Today

✅ Core Platform

✨ Data-driven routing system (PageConfig + InstalledPackage)
✨ 6-level permission system (Public → User → Moderator → Admin → God → Supergod)
✨ Multi-tenant architecture with complete tenant isolation
✨ DBAL C++ daemon (production) — 14 database backends, Drogon HTTP
✨ Event-driven workflow engine — JSON workflows fire async on CRUD events
✨ JWT authentication + JSON ACL rules (auth.json)
✨ Declarative seed data — JSON files in dbal/shared/seeds/database/
✨ Generic JSON-to-React component renderer
✨ Package system with auto-loading seed data (62 packages available)
✨ Dynamic package loading from filesystem
✨ SQLite (dev) and PostgreSQL (production) support

✅ Authentication & Authorization

✨ JWT authentication (Flask issues tokens, DBAL validates via auth.json ACL)
✨ Session-based authentication with secure cookies
✨ Role-to-level mapping (public=0, user=1, moderator=2, admin=3, god=4, supergod=5)
✨ Permission level checks with automatic redirects
✨ Flask auth backend (register, login, token refresh, Python runner)
✨ Session expiry tracking and IP/user-agent logging

✅ CRUD Operations

✨ Schema-driven entity list views with dynamic tables
✨ Entity detail views with field-by-field display
✨ Entity create forms (auto-generated from schema)
✨ Entity edit forms (pre-populated with existing data)
✨ Placeholder API client infrastructure (ready for backend)
✨ Error handling and user feedback
✨ Breadcrumb navigation

✅ Development Tools

✨ TypeScript/JavaScript compiler (esbuild integration)
✨ Minification support with configurable options
✨ Source map generation for debugging
✨ Hot-reload development server
✨ Vitest testing framework with 97.9% pass rate
✨ ESLint + Prettier code formatting
✨ Prisma ORM with type-safe database access

✅ Static Site Generation

✨ Database-driven generateStaticParams() for UI pages
✨ ISR (Incremental Static Regeneration) support
✨ Build-time page generation
✨ Graceful fallback to dynamic rendering

✅ Infrastructure

✨ PostgreSQL database with Prisma ORM
✨ Next.js 16 frontend (App Router + Server Components)
✨ Docker deployment with docker-compose
✨ Nginx reverse proxy with SSL support
✨ Redis caching layer
✨ Development and production environments
✨ Health checks and monitoring

What's NOT Done Yet (Phase 2 Remaining & Future Phases)

❌ Phase 2 - Final Items (see /TECH_DEBT.md for detailed instructions):

TD-1: DBAL Refactoring - Move database logic from frontend to DBAL (🔴 CRITICAL)
TD-2: Rate limiting - API endpoint protection (🟡 HIGH)
TD-3: OpenAPI/Swagger API documentation (🟡 HIGH)
TD-4: Error handling documentation (🟡 MEDIUM)

🔮 Phase 3+ (blocked until Phase 2 completes):

TD-5: Rich form editors with nested object/array support
TD-6: Bulk operations (multi-select, bulk delete, export)
TD-7: Advanced filtering UI with visual builder
TD-8: Relationship/foreign key dropdown selectors
TD-9: God Panel (system admin dashboard)
TD-10: Workflow automation UI
TD-11: Advanced authentication (SSO, SAML, OAuth)
TD-12: C++ DBAL daemon (production security mode)
TD-13: Multi-source package system (NPM, git, http)

See /TECH_DEBT.md for bot-actionable instructions on each task.

Technology Stack

Frontend

Technology	Version	Purpose
Next.js	16.1.1	React framework with App Router + RSC
React	19.2.3	UI library
TypeScript	5.9.3	Type-safe JavaScript
SASS	1.97.1	Styling with SCSS modules
esbuild	0.27.2	Fast JavaScript/TypeScript compilation
Zod	4.3.5	Schema validation
Vitest	4.0.16	Unit testing framework

Backend & Database

Technology	Version	Purpose
Prisma	7.2.0	ORM and database toolkit
PostgreSQL	Latest	Production database
SQLite	via better-sqlite3	Development database
DBAL (TypeScript)	Custom	Development database abstraction
DBAL (C++)	Custom	Production database abstraction

Infrastructure

Technology	Version	Purpose
Docker	Latest	Containerization
Docker Compose	Latest	Multi-container orchestration
Nginx	Latest	Reverse proxy and SSL termination
Redis	Latest	Caching layer

Development Tools

Tool	Purpose
ESLint	Code linting
Prettier	Code formatting
Monaco Editor	In-browser code editor
Octokit	GitHub API integration

C++ Services

Service	Purpose	Status
DBAL Daemon	Secure database access, JWT auth, event-driven workflows, 14 backends	✅ Production
Flask Auth	JWT issuance, user registration/login, Python runner	✅ Production
Media Daemon	Video/audio transcoding, radio streaming, TV channels	✅ Functional

System Architecture

High-Level Overview

┌─────────────┐
│   Browser   │
└──────┬──────┘
       │ HTTPS
┌──────▼──────┐     ┌───────────┐
│   Nginx     │────▶│   Redis   │ (Cache)
│ (SSL/Proxy) │     └───────────┘
└──────┬──────┘
       │
┌──────▼──────────┐     ┌──────────────┐
│  Next.js        │────▶│  DBAL        │
│  Frontend       │     │  (TS/C++)    │
│  (React 19 +    │     │              │
│   App Router)   │     └──────┬───────┘
└─────────────────┘            │
                               │
                        ┌──────▼────────┐
                        │  PostgreSQL   │
                        │  (Production) │
                        │  or SQLite    │
                        │  (Development)│
                        └───────────────┘

┌─────────────────┐     ┌──────────────┐
│  Media Daemon   │────▶│ Media Files  │
│  (C++)          │     │ (Storage)    │
└─────────────────┘     └──────────────┘

Database Schema (9 Models)

User - User accounts with role-based permissions
Credential - Password hashes (separate for security)
Session - Active user sessions with expiry tracking
PageConfig - Route configurations (God panel routes)
ComponentNode - Component tree hierarchy
ComponentConfig - Component properties and styling
Workflow - Automated workflow definitions
InstalledPackage - Package installation and configuration
PackageData - Package-specific data storage

Data Flow Architecture

1. User Request → Nginx → Next.js
2. Next.js → Check PageConfig (God panel routes)
3. If not found → Check InstalledPackage (default routes)
4. Load package from /packages/{packageId}/seed/
5. Parse JSON component definition
6. Generic renderer → Convert JSON to React
7. DBAL → Query data (with tenant isolation)
8. Render page → Send to browser

Permission System (6 Levels)

Level	Role	Capabilities
0	Public	Unauthenticated access to public pages
1	User	Basic authenticated access
2	Moderator	Content moderation, user management
3	Admin	Package installation, configuration
4	God	Route configuration, schema editing
5	Supergod	System-wide configuration, all access

Package Architecture

packages/{packageId}/
├── seed/
│   ├── metadata.json         # Package info, version, dependencies
│   ├── components.json       # Component definitions (JSON)
│   ├── scripts/              # JSON scripts organized by function
│   │   ├── init/             # Initialization scripts
│   │   ├── actions/          # User-triggered actions
│   │   └── cron/             # Scheduled tasks
│   └── index.ts              # Exports packageSeed object
├── src/                      # Optional React components (TSX)
└── static_content/           # Assets (images, CSS, etc.)

DBAL Architecture

Purpose: Language-agnostic database abstraction with credential isolation and event-driven workflows

Production Implementation (C++ — shipped Mar 2026):

Drogon HTTP framework, 14 database backends
JWT auth + JSON ACL (/app/schemas/auth/auth.json)
Event-driven workflow engine (fires async on CRUD events, detached thread)
Auto-seed on startup (DBAL_SEED_ON_STARTUP=true → SeedLoaderAction)
All schemas pure JSON — yaml-cpp removed

Event Flow (on user registration):

POST /pastebin/pastebin/User
  └── handleCreate() → dispatchAsync("pastebin.User.created")
        └── detached thread → on_user_created.json
              ├── 7× dbal.uuid nodes (namespace + snippet IDs)
              ├── 1× dbal.timestamp
              ├── 2× dbal.entity.create → Namespace (Default, Examples)
              └── 5× dbal.entity.create → seed Snippets

Key Features:

Credential isolation (users never see database URLs)
Row-level security enforcement
Multi-tenant query filtering (tenantId on every query)
Declarative seed data (JSON files, idempotent, loaded at startup)
JSON entity schemas as single source of truth (39 entities)

Package Ecosystem

MetaBuilder includes 52 built-in packages that provide ready-to-use functionality. All packages follow the same structure and can be installed, configured, and removed through the system.

Core UI Packages (9)

Essential UI components and pages

Package	Purpose	Status
ui_home	Homepage component	✅ Ready
ui_login	Login/authentication UI	✅ Ready
ui_header	Site header component	✅ Ready
ui_footer	Site footer component	✅ Ready
ui_auth	Authentication flows	✅ Ready
ui_dialogs	Modal dialog components	✅ Ready
ui_intro	Introduction/onboarding	✅ Ready
ui_pages	Generic page templates	✅ Ready
ui_permissions	Permission management UI	✅ Ready

Permission-Level UI (5)

UI packages for each permission level

Package	Level	Purpose	Status
ui_level2	Moderator	Moderator dashboard	✅ Ready
ui_level3	Admin	Admin dashboard	✅ Ready
ui_level4	God	God panel UI	✅ Ready
ui_level5	Supergod	Supergod controls	✅ Ready
ui_level6	Reserved	Future use	✅ Ready

Admin & Configuration (12)

System management and configuration tools

Package	Purpose	Status
admin_dialog	Admin modal dialogs	✅ Ready
config_summary	System configuration viewer	✅ Ready
database_manager	Database admin UI	✅ Ready
package_manager	Package installation/removal	✅ Ready
package_validator	Package validation tools	✅ Ready
route_manager	Route configuration UI	✅ Ready
schema_editor	Database schema editor	✅ Ready
theme_editor	UI theme customization	✅ Ready
user_manager	User management UI	✅ Ready
role_editor	Role and permission editor	✅ Ready
smtp_config	Email configuration	✅ Ready
workflow_editor	Workflow builder UI	✅ Ready

Development Tools (6)

Tools for developers and power users

Package	Purpose	Status
code_editor	In-browser code editor	✅ Ready
codegen_studio	Code generation tools	✅ Ready
component_editor	Visual component builder	✅ Ready
css_designer	CSS design tools	✅ Ready
nerd_mode_ide	Advanced IDE features	✅ Ready
screenshot_analyzer	Screenshot analysis tools	✅ Ready

Data & Content (5)

Data management and content tools

Package	Purpose	Status
data_table	Generic data table component	✅ Ready
form_builder	Dynamic form builder	✅ Ready
dashboard	Dashboard layouts	✅ Ready
audit_log	System audit logging	✅ Ready
dropdown_manager	Dropdown/select manager	✅ Ready

Communication and social features

Package	Purpose	Status
forum_forge	Forum/discussion boards	✅ Ready
irc_webchat	IRC web chat client	✅ Ready
social_hub	Social features hub	✅ Ready

Media & Streaming (3)

Media processing and streaming

Package	Purpose	Status
media_center	Media library UI	✅ Ready
stream_cast	Live streaming tools	✅ Ready
arcade_lobby	Game/arcade interface	✅ Ready

Integrations & Tools (5)

Third-party integrations and utilities

Package	Purpose	Status
github_tools	GitHub integration	✅ Ready
dbal_demo	DBAL demonstration	✅ Ready
json_script_example	JSON script examples	✅ Ready
notification_center	Notification system	✅ Ready
quick_guide	Quick start guide	✅ Ready

Utility (4)

Miscellaneous utilities

Package	Purpose	Status
nav_menu	Navigation menu component	✅ Ready
stats_grid	Statistics dashboard	✅ Ready
testing	Test utilities	✅ Ready
workflow_editor	Workflow visual editor	✅ Ready

Package Installation

All packages are installed by default in the system. They can be:

Enabled/disabled per tenant via InstalledPackage table
Configured with custom settings via the config JSON field
Set as default routes using defaultRoute in config
Overridden by God panel routes in PageConfig table

Creating New Packages

See docs/architecture/packages.md for package development guide.

Basic structure:

// packages/my_package/seed/metadata.json
{
  "name": "My Package",
  "version": "1.0.0",
  "description": "Package description",
  "author": "Your Name",
  "minLevel": 0,
  "dependencies": [],
  "exports": ["component1", "component2"]
}

MVP Milestone ✨

Achieved: January 2026

The MVP represents the minimum viable product that demonstrates MetaBuilder's core value proposition: a fully data-driven platform with zero hardcoded routes or components.

MVP Acceptance Criteria

All criteria met ✅

1. Authentication & Authorization ✅

Session-based user authentication
Permission level checks (0-5 scale)
Auth requirement enforcement (redirect to login)
Access denied UI for insufficient permissions
Server-side getCurrentUser() function
Comprehensive test coverage (11 tests)

2. Dynamic Package Loading ✅

Load packages from filesystem
Home component discovery (priority: 'home_page' > 'HomePage' > 'Home')
404 handling for missing packages
Package metadata integration
Error handling and logging

3. CRUD Operations ✅

Schema-driven entity list view
Entity detail view with field display
Entity create form (schema-generated)
Entity edit form (pre-populated)
API client utilities (placeholder implementation)
Error handling and user feedback

4. Static Page Generation ✅

Database query for active pages
generateStaticParams() implementation
Next.js static export support
ISR (Incremental Static Regeneration) compatibility
Graceful build-time error handling

5. Code Compilation ✅

esbuild integration
TypeScript/JavaScript compilation
Minification option
Source map generation
Error handling with source preservation
Comprehensive test coverage (9 tests)

MVP Deliverables

Code Changes:

✅ 5 major features implemented
✅ 20 new tests added (100% passing)
✅ Zero breaking changes
✅ All TODO items from docs/TODO_MVP_IMPLEMENTATION.md completed

Documentation:

✅ Implementation summary (docs/MVP_IMPLEMENTATION_SUMMARY.md)
✅ Architecture documentation in README.md
✅ API documentation for new utilities
✅ Test documentation

Quality Metrics:

✅ TypeScript compilation: 0 errors
✅ ESLint: All new files pass
✅ Test pass rate: 97.9% (188/192)
✅ Code review: Approved

Release History

v0.1.0-alpha (January 2026) - MVP Release ✨

Status: Current

Features:

Authentication & session management
Dynamic package loading
CRUD operations (schema-driven)
Static page generation
Code compilation (esbuild)

Files Changed:

frontends/nextjs/src/lib/auth/get-current-user.ts (new)
frontends/nextjs/src/lib/auth/get-current-user.test.ts (new)
frontends/nextjs/src/components/AccessDenied.tsx (new)
frontends/nextjs/src/lib/entities/load-entity-schema.ts (new)
frontends/nextjs/src/lib/entities/api-client.ts (new)
frontends/nextjs/src/lib/compiler/index.ts (enhanced)
frontends/nextjs/src/lib/compiler/index.test.ts (new)
frontends/nextjs/src/app/page.tsx (enhanced)
frontends/nextjs/src/app/[tenant]/[package]/page.tsx (enhanced)
frontends/nextjs/src/app/[tenant]/[package]/[...slug]/page.tsx (enhanced)
frontends/nextjs/src/app/ui/[[...slug]]/page.tsx (enhanced)

Dependencies Added:

esbuild - Fast JavaScript/TypeScript compiler

Test Results:

New tests: 20
Total tests: 192
Pass rate: 97.9%

v0.0.0 (Pre-MVP)

Status: Foundation

Core Infrastructure:

Next.js 14 frontend setup
Prisma + PostgreSQL database
DBAL (TypeScript development + C++ production)
Package system architecture
Generic JSON component renderer
Multi-tenancy support
6-level permission system foundation

Roadmap Phases

✅ Phase 0: Foundation (Completed)

Timeline: Pre-2026 Goal: Establish core architecture

Next.js 14 with App Router
Prisma ORM + PostgreSQL
DBAL architecture (TypeScript + C++)
Package system foundation
Generic component renderer
Multi-tenant database schema
Permission level system (0-5)

✅ Phase 1: MVP (Completed - January 2026)

Timeline: January 2026 Goal: Minimum viable product with core features

User authentication & authorization
Session management
Permission checks with redirects
Dynamic package loading
CRUD operations (schema-driven)
Static page generation
Code compilation (esbuild)
Comprehensive testing

Success Metrics:

✅ All TODO items resolved
✅ Test coverage >80%
✅ Zero breaking changes
✅ Documentation complete

🔄 Phase 2: Backend Integration (In Progress) Timeline: Q1 2026 (January - March) Goal: Connect frontend to real backend APIs Status: 🚀 90% Complete - Core APIs, Validation, Pagination, Filtering Implemented

Priority: HIGH ⭐

✅ Completed (January 8, 2026):

API endpoints fully implemented in /api/v1/[...slug]/route.ts
Session-based authentication middleware
Multi-tenant access validation
CRUD operations (list, read, create, update, delete)
Custom package action support
Standardized error responses
TypeScript API client (api-client.ts) with all methods
Retry utility with exponential backoff (38 tests)
Pagination utilities - offset and cursor-based (35 tests)
Filtering and sorting utilities (36 tests)
Zod validation utilities for request/response validation (39 tests)
Unit tests for API client (29 tests)
Unit tests for API route structure (10 tests)
E2E tests for CRUD operations (14 scenarios)
Total new utilities tests: 148 tests
Overall test coverage: 414/418 passing (99.0%)

Implementation Tasks

2.1 API Endpoint Implementation ✅ COMPLETE

Status: ✅ All endpoints implemented (January 2026)

GET /api/v1/{tenant}/{package}/{entity} - List entities
- Pagination support (page, limit query params)
- Filtering support (filter query param with JSON)
- Sorting support (sort query param)
- Tenant isolation checks
- Response time logging
- Unit tests (10+ test cases)
- E2E tests (4 scenarios)
GET /api/v1/{tenant}/{package}/{entity}/{id} - Get single entity
- Entity ID validation
- Return 404 for non-existent entities
- Tenant isolation checks
- Unit tests (5+ test cases)
- E2E tests (2 scenarios)
POST /api/v1/{tenant}/{package}/{entity} - Create entity
- Route handler with POST method
- JSON body parsing and validation
- Return created entity with 201 status
- Handle validation errors with 400 status
- Tenant isolation
- Unit tests (5+ test cases)
- E2E tests (3 scenarios)
PUT /api/v1/{tenant}/{package}/{entity}/{id} - Update entity
- Route handler with PUT method
- JSON body parsing
- Support partial updates
- Return 404 for non-existent entities
- Return updated entity with 200 status
- Tenant isolation
- Unit tests (5+ test cases)
- E2E tests (3 scenarios)
DELETE /api/v1/{tenant}/{package}/{entity}/{id} - Delete entity
- Route handler with DELETE method
- Return 404 for non-existent entities
- Return 200 status on success
- Tenant isolation
- Unit tests (5+ test cases)
- E2E tests (2 scenarios)

2.2 API Client Integration ✅ COMPLETE

Status: ✅ All methods implemented with retry logic (January 8, 2026)

Update api-client.ts - Fully functional implementation
- listEntities() with fetch calls and query params
- getEntity() with error handling
- createEntity() with JSON body
- updateEntity() with actual fetch calls and partial updates
- deleteEntity() with actual fetch calls and proper status codes
- Error handling (network errors, API errors)
- Request timeout handling
- Unit tests with parameterized scenarios (29 tests)
- Retry logic for transient failures (retry utility with exponential backoff)
- Comprehensive retry tests (38 tests)

2.3 Request/Response Validation ✅ COMPLETE

Status: ✅ All validation utilities implemented (January 8, 2026)

Zod Schema Generation
- Create utility to generate Zod schemas from entity definitions
- Support all field types (string, number, boolean, date, enum, array, object, relation)
- Support validation rules (required, min, max, pattern, email, url, custom)
- Support nested objects and arrays
- Write tests for schema generation (39 test cases - exceeded target)
Validation Middleware
- Create validation middleware for API routes
- Validate request body against entity schema
- Return detailed validation errors with formatted messages
- Support custom validation messages
- Common schema patterns (email, uuid, phone, password, username)
- Write comprehensive tests for validation (39 tests total)

2.4 Pagination Implementation ✅ COMPLETE

Status: ✅ All pagination components and utilities implemented (January 8, 2026)

Pagination Utilities ✅ COMPLETE
- Create pagination helper functions
- Support cursor-based pagination
- Support offset-based pagination
- Calculate total pages and items
- Return pagination metadata in responses
- Cursor encoding/decoding utilities
- Page number generation for UI
- Write tests for pagination (35 test cases - exceeded target)
Frontend Pagination Components ✅ COMPLETE
- Create pagination UI component (PaginationControls.tsx using fakemui)
- Add page navigation controls (first, last, prev, next buttons)
- Add items-per-page selector (ItemsPerPageSelector.tsx)
- Add pagination info display (PaginationInfo.tsx)
- Write unit tests for pagination components (25 tests)
- Update list views to use pagination (pending integration)
- Write E2E tests for pagination UI

2.5 Filtering and Sorting ✅ COMPLETE

Status: ✅ All filtering and sorting utilities implemented (January 8, 2026)

Filter Implementation ✅ COMPLETE
- Support equality filters (eq, ne)
- Support comparison filters (gt, gte, lt, lte)
- Support array filters (in, notIn)
- Support text search filters (contains, startsWith, endsWith)
- Support null checks (isNull, isNotNull)
- Prisma query builder integration
- SQL injection prevention with field validation
- Write tests for filtering (36 test cases - exceeded target)
Sort Implementation ✅ COMPLETE
- Support single field sorting (sort=field)
- Support multi-field sorting (sort=field1,-field2)
- Support ascending/descending (- prefix for desc)
- Prisma orderBy integration
- Field name validation for security
- Write comprehensive tests for sorting (included in 36 tests)

2.6 Authentication Middleware ✅ COMPLETE

Status: ✅ All authentication middleware implemented (January 8, 2026)

API Authentication ✅ COMPLETE
- Create auth middleware for API routes (auth-middleware.ts)
- Validate session tokens from cookies via getCurrentUser()
- Check user permission levels (0-5 scale)
- Return 401 for unauthenticated requests
- Return 403 for insufficient permissions
- Add auth bypass for public endpoints (allowPublic option)
- Support custom permission checks
- Provide requireAuth helper for simplified usage
- Write tests for auth middleware (21 test cases - exceeded target)

2.7 Rate Limiting

Target: Week 5-6 of Q1 2026

Rate Limiter Implementation
- Install @upstash/ratelimit or similar
- Configure rate limits per endpoint
- Configure rate limits per user/tenant
- Return 429 status when rate limit exceeded
- Add rate limit headers to responses
- Write tests for rate limiting (8+ test cases)

2.8 Error Handling

Target: Ongoing throughout implementation

Standardized Error Responses
- Create error response format (code, message, details)
- Handle validation errors (400)
- Handle authentication errors (401)
- Handle authorization errors (403)
- Handle not found errors (404)
- Handle rate limit errors (429)
- Handle server errors (500)
- Log all errors to error tracking service
- Write tests for error handling (20+ test cases)

Testing Requirements

Unit Tests: Target 150+ new tests - ✅ EXCEEDED (194 tests implemented)

API route handlers: 50 tests ✅ Complete
API client functions: 29 tests ✅ Complete
Retry utilities: 38 tests ✅ Complete
Validation utilities: 39 tests ✅ Complete
Pagination utilities: 35 tests ✅ Complete
Filtering/sorting utilities: 36 tests ✅ Complete
Pagination components: 25 tests ✅ Complete
Auth middleware: 21 tests ✅ Complete
Rate limiting: 8 tests 🔄 Pending
Error handling: 20 tests 🔄 Pending

Integration Tests: Target 30+ new tests - 🔄 Partially Complete

Full CRUD flows: 15 tests 🔄 Partially Complete
Multi-tenant isolation: 5 tests 🔄 Partially Complete
Permission-based access: 10 tests 🔄 Partially Complete

E2E Tests: Target 15+ new tests - 🔄 Partially Complete

Complete CRUD user flows: 14 tests ✅ Complete
Authentication flows: 3 tests 🔄 Pending
Permission-based UI changes: 4 tests 🔄 Pending
Pagination UI: 3 tests 🔄 Pending
Filtering/sorting UI: 3 tests 🔄 Pending

Performance Benchmarks

Endpoint	Target Response Time	Target Throughput
GET (list)	<100ms (p50), <200ms (p95)	>1000 req/s
GET (single)	<50ms (p50), <100ms (p95)	>2000 req/s
POST (create)	<150ms (p50), <300ms (p95)	>500 req/s
PUT (update)	<150ms (p50), <300ms (p95)	>500 req/s
DELETE	<100ms (p50), <200ms (p95)	>1000 req/s

Documentation Requirements

OpenAPI/Swagger specification for all endpoints
API authentication guide
Rate limiting documentation
Error response format documentation
Pagination documentation
Filtering and sorting guide
Example API requests for all endpoints

Success Metrics:

All CRUD operations functional (100% coverage)
API tests passing (>95% pass rate)
Performance benchmarks met (<200ms avg response p95)
Zero security vulnerabilities in API layer
API documentation complete (OpenAPI spec)

🔮 Phase 3: Enhanced CRUD (Planned)

Timeline: Q1-Q2 2026 Goal: Rich editing experience

Priority: HIGH

RenderComponent integration for forms
- Form field generation from schema
- Field type support (text, number, date, select, etc.)
- Nested object/array editing
- File upload support
Client-side form validation
- Required field validation
- Type validation
- Custom validation rules from schema
- Real-time validation feedback
Advanced list features
- Search/filter UI
- Column sorting
- Bulk operations (select, delete, export)
- Customizable views (table, grid, list)
Relationship handling
- Foreign key dropdowns
- Related entity displays
- Many-to-many relationship UI

Success Metrics:

Form validation: 100% schema coverage
List operations: <100ms client-side filtering
User satisfaction: Positive feedback from beta testers

🔮 Phase 4: God Panel (Planned)

Timeline: Q2 2026 Goal: Admin UI for system configuration

Priority: MEDIUM

Route management UI
- Add/edit/delete routes in PageConfig
- Visual route priority editor
- Route testing/preview
Package management UI
- Install/uninstall packages
- Package configuration editor
- Dependency visualization
User management
- Create/edit users
- Role assignment
- Permission testing
Schema editor
- Visual entity schema builder
- Field type selector
- Validation rule editor
Component builder
- Visual JSON component editor
- Component preview
- Component templates

Success Metrics:

God users can manage routes without code
Package installation: <2 minutes
Schema changes: No manual database migrations

🔮 Phase 5: Advanced Features (Planned)

Timeline: Q2-Q3 2026 Goal: Production-ready enhancements

Priority: MEDIUM

Search & Discovery
- Full-text search across entities
- Global search UI
- Search result ranking
- Search filters
Caching & Performance
- Redis integration
- Query result caching
- Component render caching
- CDN support for static assets
Audit & Logging
- Change history tracking
- User activity logs
- System audit trail
- Log viewer UI
Webhooks & Events
- Entity change webhooks
- Custom event triggers
- Webhook management UI
- Event replay capability
Import/Export
- CSV import/export
- JSON bulk operations
- Schema migration tools
- Data backup/restore

Success Metrics:

Cache hit rate: >80%
Search response time: <100ms
Audit coverage: 100% of mutations

🔮 Phase 6: Advanced Auth (Planned)

Timeline: Q3 2026 Goal: Enterprise-grade authentication

Priority: MEDIUM

Multi-factor authentication (MFA)
- TOTP support (Google Authenticator)
- SMS verification
- Backup codes
OAuth integration
- Google OAuth
- GitHub OAuth
- Microsoft Azure AD
Session enhancements
- "Remember Me" functionality
- Session refresh tokens
- Device management
- Concurrent session control
Security features
- Brute force protection
- IP whitelisting
- Anomaly detection
- Security audit logs

Success Metrics:

MFA adoption rate: >50% of users
OAuth login success rate: >95%
Zero security incidents

🔮 Phase 7: C++ DBAL Production (Planned)

Timeline: Q3-Q4 2026 Goal: Production-ready C++ daemon

Priority: MEDIUM

C++ implementation conformance
- All TypeScript DBAL features implemented
- Conformance test suite: 100% passing
Performance optimization
- Connection pooling
- Query optimization
- Memory management
Security hardening
- Credential isolation (env vars only)
- Row-level security enforcement
- SQL injection prevention
Monitoring & observability
- Metrics endpoint
- Health checks
- Performance profiling

Success Metrics:

Performance: 10x faster than TypeScript DBAL
Memory: <100MB resident
Security: Zero credential exposure

🔮 Phase 8: Multi-Source Packages (Planned)

Timeline: Q4 2026 Goal: Package ecosystem

Priority: LOW

Remote package repositories
- Package index API
- Remote package loading
- Version management
Package marketplace
- Package discovery UI
- Package ratings/reviews
- Package documentation
Conflict resolution
- Priority-based resolution
- Latest-version resolution
- Local-first resolution
- Custom resolution strategies

Success Metrics:

Package ecosystem: >20 public packages
Installation time: <30 seconds per package
Conflict resolution: 0 manual interventions

Feature Status Matrix

Feature	Status	Phase	Priority	Notes
Core Platform
Next.js Frontend	✅ Complete	0	-	App Router with RSC
PostgreSQL Database	✅ Complete	0	-	Prisma ORM
DBAL (TypeScript)	✅ Complete	0	-	Development mode
DBAL (C++)	🔨 In Progress	7	Medium	Production daemon
Multi-tenancy	✅ Complete	0	-	Tenant isolation
Package System	✅ Complete	0	-	JSON packages
Component Renderer	✅ Complete	0	-	JSON → React
Authentication
Session Management	✅ Complete	1	-	Cookie-based
getCurrentUser()	✅ Complete	1	-	Server function
Permission Checks	✅ Complete	1	-	6-level system
Access Denied UI	✅ Complete	1	-	User-friendly
OAuth Integration	📋 Planned	6	Medium	Google, GitHub, Azure
Multi-Factor Auth	📋 Planned	6	Medium	TOTP, SMS
Session Refresh	📋 Planned	6	Medium	Auto-refresh tokens
CRUD Operations
Entity List View	✅ Complete	1	-	Schema-driven
Entity Detail View	✅ Complete	1	-	All fields
Entity Create Form	✅ Complete	1	-	Schema-generated
Entity Edit Form	✅ Complete	1	-	Pre-populated
API Client (Placeholder)	✅ Complete	1	-	Ready for backend
API Backend	📋 Planned	2	High	Real endpoints
RenderComponent Forms	📋 Planned	3	High	Enhanced UX
Client-side Validation	📋 Planned	3	High	Real-time feedback
Pagination	📋 Planned	2	High	List views
Filtering/Sorting	📋 Planned	2	High	List views
Bulk Operations	📋 Planned	3	Medium	Multi-select
Routing
Priority Routing	✅ Complete	0	-	PageConfig + Packages
Dynamic Package Routes	✅ Complete	1	-	Filesystem loading
Static Generation	✅ Complete	1	-	generateStaticParams
ISR Support	✅ Complete	1	-	Revalidation
Route Management UI	📋 Planned	4	Medium	God panel
Development Tools
Compiler (esbuild)	✅ Complete	1	-	TS/JS compilation
Minification	✅ Complete	1	-	Optional
Source Maps	✅ Complete	1	-	Debugging support
Component Preview	📋 Planned	4	Low	Live preview
Schema Editor	📋 Planned	4	Medium	Visual builder
Performance
Redis Caching	📋 Planned	5	Medium	Query results
CDN Support	📋 Planned	5	Medium	Static assets
Component Caching	📋 Planned	5	Low	Render cache
Data Management
CSV Import/Export	📋 Planned	5	Low	Bulk operations
JSON Bulk Ops	📋 Planned	5	Low	API-driven
Backup/Restore	📋 Planned	5	Medium	System-wide
Change History	📋 Planned	5	Medium	Audit trail
Search
Full-Text Search	📋 Planned	5	Medium	PostgreSQL FTS
Global Search UI	📋 Planned	5	Medium	Unified interface
Search Filters	📋 Planned	5	Low	Advanced queries
Integration
Webhooks	📋 Planned	5	Low	Entity changes
Event System	📋 Planned	5	Low	Custom triggers
Package Marketplace	📋 Planned	8	Low	Public packages
Remote Packages	📋 Planned	8	Low	Multi-source

Legend:

✅ Complete - Implemented and tested
🔨 In Progress - Currently being developed
📋 Planned - On the roadmap, not started
⏸️ On Hold - Deprioritized
❌ Cancelled - No longer planned

Testing Strategy & Best Practices

MetaBuilder follows a comprehensive testing strategy to ensure code quality, reliability, and maintainability.

Testing Philosophy

We emphasize Test-Driven Development (TDD) as a core practice:

Write tests first - Design APIs through tests
Fast feedback - Tests provide immediate confidence
High coverage - Target >80% code coverage
Maintainable tests - Tests are as important as production code
Realistic scenarios - Tests reflect real-world usage

Testing Pyramid

              /\
             /  \
            /E2E \       Few, Slow, Expensive (15-30 tests)
           /------\      Critical user flows
          /        \
         /Integration\   More, Medium Speed (50-100 tests)
        /------------\   Components working together
       /              \
      /   Unit Tests   \  Many, Fast, Cheap (200-500 tests)
     /------------------\ Individual functions

Target Test Distribution

Test Type	Current	Target	Coverage Focus
Unit Tests	220	500+	Individual functions, utilities
Integration Tests	30	100+	API endpoints, database operations
E2E Tests	8	30+	Critical user journeys
Total	258	630+	Overall system

Critical Test Scenarios

Unit Test Scenarios (Priority Order)

API Layer (High Priority)

CRUD Operations (50 tests)
- List entities with pagination
- List entities with filters
- List entities with sorting
- Get single entity by ID
- Get non-existent entity (404)
- Create entity with valid data
- Create entity with invalid data (validation)
- Update entity with valid data
- Update entity with partial data
- Update non-existent entity (404)
- Delete entity by ID
- Delete non-existent entity (404)
Authentication & Authorization (30 tests)
- Valid session token
- Invalid session token (401)
- Expired session token (401)
- Missing session token (401)
- Insufficient permissions (403)
- Permission level checks (0-5)
- Tenant isolation validation
- Cross-tenant access prevention
Validation (40 tests)
- Required field validation
- Type validation (string, number, boolean, date)
- Min/max length validation
- Min/max value validation
- Pattern/regex validation
- Enum validation
- Nested object validation
- Array validation
- Custom validation rules
Pagination (20 tests)
- Offset-based pagination
- Cursor-based pagination
- Page size limits
- Total count calculation
- Empty result sets
- Single page results
- Multiple page results
- Invalid pagination params
Filtering (30 tests)
- Equality filters
- Comparison filters (gt, gte, lt, lte)
- Array filters (in, notIn)
- Text search filters (contains, startsWith, endsWith)
- Boolean filters
- Null/undefined filters
- Date range filters
- Nested field filters
- Multiple filter combinations
Sorting (15 tests)
- Single field ascending
- Single field descending
- Multiple field sorting
- Nested field sorting
- Invalid sort fields
- Case-insensitive sorting
Rate Limiting (15 tests)
- Within rate limit
- Exceeding rate limit (429)
- Rate limit headers
- Different rate limits per endpoint
- Different rate limits per user
- Rate limit reset

Utility Functions (Medium Priority) 8. Schema Generation (20 tests)

Generate Zod schema from entity definition
All field type support
Validation rule support
Nested object schema
Array schema
Optional/required fields

Database Helpers (30 tests)
- Query building
- Transaction management
- Error handling
- Tenant filtering
- Soft delete support
Component Rendering (20 tests)
- JSON to React conversion
- Component props mapping
- Nested component rendering
- Error component rendering

Integration Test Scenarios (Priority Order)

API Integration (High Priority)

Complete CRUD Flow (15 tests)
- Create → Read → Update → Delete flow
- List empty collection
- List with data
- Pagination through large datasets
- Filter and sort combinations
Multi-Tenant Isolation (10 tests)
- Create entity in tenant A
- List entities in tenant A (only shows A's data)
- List entities in tenant B (only shows B's data)
- Attempt cross-tenant access (blocked)
- Tenant-specific filtering
Permission-Based Access (15 tests)
- Public user (level 0) access
- Authenticated user (level 1) access
- Moderator (level 2) access
- Admin (level 3) access
- God (level 4) access
- Supergod (level 5) access
- Permission escalation prevention
Authentication Flows (10 tests)
- Login → Create session → Make authenticated request
- Logout → Session invalidated → Request fails
- Session expiry → Request fails
- Session refresh → Continued access
Error Scenarios (10 tests)
- Network timeout handling
- Database connection failure
- Invalid JSON in request
- Large payload handling
- Concurrent request handling

Database Integration (Medium Priority) 6. Transaction Management (10 tests)

Successful transaction commit
Transaction rollback on error
Nested transaction support
Concurrent transaction isolation

Data Integrity (10 tests)
- Foreign key constraints
- Unique constraints
- Not-null constraints
- Check constraints
- Cascade delete behavior

E2E Test Scenarios (Priority Order)

Critical User Flows (High Priority)

Authentication Journey (4 tests)
- Landing page → Sign in → Dashboard
- Landing page → Register → Verify email → Dashboard
- Dashboard → Logout → Landing page
- Forgot password → Reset → Login
CRUD Operations Journey (8 tests)
- Navigate to entity list → View list
- Click "Create" → Fill form → Submit → View detail page
- Detail page → Click "Edit" → Update form → Submit → View updated detail
- Detail page → Click "Delete" → Confirm → Return to list
- List page → Use pagination → Navigate pages
- List page → Apply filters → See filtered results
- List page → Apply sorting → See sorted results
- List page → Search → See search results
Permission-Based UI (6 tests)
- Login as public user → See public pages only
- Login as user → See user pages
- Login as admin → See admin pages
- Login as god → See god panel
- Attempt to access higher-level page → See access denied
- Permission level indicator in UI
Package Rendering (4 tests)
- Navigate to package home page → See rendered components
- Navigate to package sub-route → See rendered components
- Navigate to non-existent package → See 404
- Navigate to disabled package → See access denied
Form Validation (4 tests)
- Submit empty required field → See validation error
- Submit invalid format → See format error
- Submit valid data → Success message
- Server-side validation error → Display error to user
Multi-Tenant Scenarios (2 tests)
- Login to tenant A → See tenant A data
- Login to tenant B → See tenant B data (different)
Error Handling (2 tests)
- Network error during operation → User-friendly error message
- Server error (500) → Error page with retry option

Test Performance Targets

Test Type	Target Execution Time	Parallelization
Unit test (single)	<100ms	Yes (max CPU cores)
Unit test suite	<30s	Yes
Integration test (single)	<1s	Yes (database per test)
Integration test suite	<2 minutes	Yes
E2E test (single)	<30s	Yes (browser per test)
E2E test suite	<10 minutes	Yes
Full test suite (CI)	<15 minutes	Yes

Test Data Management

Unit Tests

Use mocked data (no database)
Use factories for test data generation
Keep test data minimal and focused

Integration Tests

Use test database (separate from dev/prod)
Reset database between tests
Use database transactions (rollback after test)
Use seed data for common scenarios

E2E Tests

Use separate test environment
Use isolated test accounts
Clean up test data after suite
Use realistic but anonymized data

Test Maintenance Best Practices

Keep Tests Fast
- Mock external services
- Use in-memory databases where possible
- Parallelize test execution
- Skip slow tests in watch mode
Keep Tests Isolated
- Each test is independent
- No shared state between tests
- Clean up after each test
- Don't rely on execution order
Keep Tests Readable
- Descriptive test names
- Clear arrange-act-assert sections
- Meaningful variable names
- Document complex scenarios
Keep Tests Maintainable
- Use test utilities and helpers
- Avoid test code duplication
- Update tests when requirements change
- Remove obsolete tests

Test Coverage

Component	Unit Tests	Integration Tests	E2E Tests
Utilities	>90%	-	-
Database Layer	>85%	>70%	-
API Routes	>80%	>75%	Critical paths
React Components	>75%	-	-
User Flows	-	-	All critical paths
Overall	>80%	>70%	100% critical

Test-Driven Development (TDD)

Red-Green-Refactor Cycle:

1. 🔴 RED: Write a failing test
   ↓
2. 🟢 GREEN: Write minimal code to pass
   ↓
3. 🔵 REFACTOR: Improve code while keeping tests green
   ↓
4. ♻️  REPEAT: Move to next feature

Example TDD Workflow:

// Step 1: RED - Write failing test
describe('validatePassword', () => {
  it.each([
    { password: 'short', expected: { valid: false, error: 'Too short' } },
    { password: 'validpass123', expected: { valid: true, error: null } },
  ])('should validate "$password"', ({ password, expected }) => {
    expect(validatePassword(password)).toEqual(expected)
  })
})

// Step 2: GREEN - Implement to pass tests
export function validatePassword(password: string) {
  if (password.length < 8) {
    return { valid: false, error: 'Too short' }
  }
  return { valid: true, error: null }
}

// Step 3: REFACTOR - Improve while keeping tests green

Testing Tools

Tool	Purpose	Usage
Vitest	Unit & integration testing	`npm test`
Playwright	E2E browser testing	`npx playwright test`
Testing Library	React component testing	Integrated with Vitest
Prisma	Database testing	Test database utilities

Unit Testing Best Practices

✅ Use Parameterized Tests:

it.each([
  { input: 'hello', expected: 'HELLO' },
  { input: 'world', expected: 'WORLD' },
])('should uppercase $input', ({ input, expected }) => {
  expect(uppercase(input)).toBe(expected)
})

✅ Follow AAA Pattern:

it('should calculate total', () => {
  // ARRANGE
  const items = [{ price: 10 }, { price: 20 }]
  // ACT
  const total = calculateTotal(items)
  // ASSERT
  expect(total).toBe(30)
})

✅ Mock External Dependencies:

vi.mock('@/lib/database', () => ({
  Database: {
    getUser: vi.fn().mockResolvedValue({ id: '1', name: 'Test' })
  }
}))

E2E Testing with Playwright

Configuration: e2e/playwright.config.ts

Key Features:

Cross-browser testing (Chromium, Firefox, WebKit)
Auto-waiting for elements
Screenshots and videos on failure
Trace viewer for debugging
Page Object Model support

Example E2E Test:

import { test, expect } from '@playwright/test'

test('should login successfully', async ({ page }) => {
  await page.goto('/login')
  await page.fill('input[name="email"]', 'admin@example.com')
  await page.fill('input[name="password"]', 'password123')
  await page.click('button[type="submit"]')

  await expect(page).toHaveURL(/\/dashboard/)
  await expect(page.locator('text=Welcome')).toBeVisible()
})

Running Tests

# Unit tests
npm test                    # Run all unit tests
npm run test:watch          # Watch mode
npm run test:coverage       # With coverage report

# E2E tests
npx playwright test         # Run all E2E tests
npx playwright test --ui    # Interactive UI mode
npx playwright test --debug # Debug mode
npx playwright show-report  # View HTML report

# CI
npm run test:ci             # All tests for CI

Test Maintenance

Keep Tests Fast:

Unit tests: <1s each
Integration tests: <5s each
E2E tests: <30s each

Keep Tests Isolated:

Each test is independent
Use beforeEach/afterEach for setup/cleanup
Don't rely on test execution order

Keep Tests Readable:

Descriptive test names
Clear arrange-act-assert sections
Meaningful variable names
Document complex scenarios

CI/CD Integration

Tests run automatically on:

Every push to main/develop branches
Every pull request
Pre-deployment checks

GitHub Actions:

Unit tests with coverage reporting
E2E tests across multiple browsers
Automatic artifact upload on failure
Retry failed tests (2x in CI)

Comprehensive Testing Guide

For detailed testing documentation, examples, and best practices, see: docs/TESTING_GUIDE.md

This comprehensive guide includes:

Complete TDD workflows with examples
Advanced Playwright patterns (POM, fixtures)
Integration testing strategies
Troubleshooting common issues
CI/CD configuration examples

Development Best Practices

Core Principles

One Lambda Per File - Functions in separate files, classes only as containers
Write Tests First (TDD) - Red-Green-Refactor cycle
TypeScript Strict Mode - No any types, explicit types everywhere
Follow DRY - Don't Repeat Yourself, extract common logic
Pure Functions - Make functions pure when possible

Code Quality

✅ Good Practices:

// Explicit types
function getUser(id: string): Promise<User> {
  return Database.getUser(id)
}

// Pure function
function calculateTotal(items: Item[]): number {
  return items.reduce((sum, item) => sum + item.price, 0)
}

// One function per file
// src/lib/users/get-user-by-id.ts
export async function getUserById(id: string): Promise<User> {
  return Database.getUser(id)
}

❌ Bad Practices:

// Using 'any'
function getUser(id: any): Promise<any> {
  return Database.getUser(id)
}

// Multiple unrelated functions
export function getUserById(id) { ... }
export function formatUserName(user) { ... }
export function validateEmail(email) { ... }

Development Workflow

1. Planning → Review requirements, check docs
2. Design → Data models, API contracts, interfaces
3. TDD → Write tests, implement, refactor
4. Integration → Run tests, lint, type check
5. Review → Self-review, create PR, address feedback
6. Deploy → Staging → E2E tests → Production

Git Workflow

# Feature branch
git checkout -b feature/user-auth

# Commit frequently
git commit -m "feat: add login form"
git commit -m "test: add validation tests"

# Push and create PR
git push origin feature/user-auth

Code Review Checklist

Before PR:

All tests pass
Lint and type check pass
New code has tests (>80% coverage)
Self-reviewed changes
Documentation updated

Reviewer checks:

Code follows conventions
No security vulnerabilities
Performance considered
Accessible (a11y)
Mobile responsive

Security Best Practices

✅ Input Validation:

import { z } from 'zod'

const userSchema = z.object({
  email: z.string().email(),
  password: z.string().min(8),
})

export function createUser(input: unknown) {
  const validated = userSchema.parse(input)
  return Database.createUser(validated)
}

✅ SQL Injection Prevention:

// Use Prisma's parameterized queries
const users = await prisma.user.findMany({
  where: { email: userEmail },
})

✅ XSS Prevention:

// React automatically escapes
<div>{user.name}</div>

// Sanitize HTML when needed
import DOMPurify from 'dompurify'
<div dangerouslySetInnerHTML={{
  __html: DOMPurify.sanitize(user.bio)
}} />

Performance Best Practices

✅ Database Queries:

// Query only needed fields
const users = await Database.getUsers({
  select: ['id', 'name', 'email'],
  where: { active: true },
  limit: 10,
})

✅ React Performance:

import { useMemo, useCallback } from 'react'

function UserList({ users, onSelect }) {
  const sortedUsers = useMemo(
    () => users.sort((a, b) => a.name.localeCompare(b.name)),
    [users]
  )

  const handleSelect = useCallback(
    (userId) => onSelect(userId),
    [onSelect]
  )

  return <List items={sortedUsers} onSelect={handleSelect} />
}

// ✅ Accessible components
<button
  type="button"
  aria-label="Delete user"
  onClick={handleDelete}
>
  <DeleteIcon aria-hidden="true" />
  Delete
</button>

<form onSubmit={handleSubmit}>
  <label htmlFor="email">Email</label>
  <input
    id="email"
    type="email"
    required
    aria-required="true"
    aria-describedby="email-error"
  />
  <span id="email-error" role="alert">
    {emailError}
  </span>
</form>

Development Tools

Tool	Purpose	Command
ESLint	Code linting	`npm run lint:fix`
Prettier	Code formatting	`npm run format`
TypeScript	Type checking	`npm run typecheck`
Prisma	Database ORM	`npm run db:generate`
Vitest	Unit testing	`npm test`
Playwright	E2E testing	`npx playwright test`

Known Issues & Technical Debt

Active Issues

Test Failures (4 tests)

Status: 🔍 Investigating Impact: Low - Pre-existing failures unrelated to MVP work Tests: 188/192 passing (97.9%)

These 4 failing tests existed before MVP implementation and do not affect core functionality. They are scheduled for investigation in Phase 2.

TLA+ Specification Error

File: spec/metabuilder.tla Status: 📋 Documented Impact: Low - Affects formal verification only, not runtime

Issue: Syntax error at line 323 in PackageConsistency invariant

\* Current (incorrect)
PackageConsistency ==
    \A t \in Tenants, p \in installedPackages[t]:
        packageStates[p] \in {"installed", "disabled", "installing"}

\* Suggested fix
PackageConsistency ==
    \A t \in Tenants:
        \A p \in installedPackages[t]:
            packageStates[p] \in {"installed", "disabled", "installing"}

Note: New specifications (workflow_system.tla, collaboration.tla, integrations.tla) all pass validation.

Technical Debt

Codebase Composition (Current State)

Overview: MetaBuilder is a well-architected, data-driven system with approximately 2,047 files:

TypeScript/TSX:    1,487 files (73%)
  ├── DBAL              282 files (14%)
  ├── FakeMUI           514 files (25%)
  ├── NextJS Frontend   491 files (24%)
  ├── E2E Tests         16 files (1%)
  └── Deprecated        119 files (6%)

JSON/YAML:           560 files (27%)
  ├── Packages         376 files (18%)
  ├── Schemas          76 files (4%)
  └── Seed Config      5 files (<1%)

Data-Driven Ratio: 27% (could reach 35-40% with recommended refactoring)

High Priority (Quick Wins - ~2-3 hours)

Delete /old Directory (119 files)
- Pre-Next.js SPA implementation, completely superseded
- Contains Vite, React 19, Radix UI components
- Pure cruft from Spark-based era
- Action: Delete /old/ entirely
- Impact: Reduces confusion, removes maintenance burden
Delete Backup Files (4 files)
- /dbal/development/src/core/foundation/tenant-context.ts.backup
- /dbal/development/src/core/client/client.ts.backup
- /dbal/development/src/bridges/websocket-bridge.ts.backup
- /dbal/development/src/adapters/acl-adapter.ts.backup
- Action: Delete all .backup files
- Impact: Clean code footprint, no functionality impact (Git history preserved)
Migrate Seed Data from TypeScript to YAML
- Current: /frontends/nextjs/src/lib/db/database-admin/seed-default-data/*.ts (5 files, ~150 lines)
  - seed-users.ts - Hardcoded default users
  - seed-app-config.ts - App configuration
  - seed-css-categories.ts - CSS categories
  - seed-dropdown-configs.ts - Dropdown definitions
- Better: /seed/database/*.yaml loaded via DBAL orchestration
- Status: Current approach bypasses DBAL, should use seed system
- Effort: 4-5 hours
- Impact: 100% YAML-based seed system, better bootstrapping

Medium Priority (~4-5 hours)

Eliminate Adapter Code Duplication
- Current: Frontend duplicates adapter logic at /frontends/nextjs/src/lib/dbal-client/adapter/ (355 lines)
  - PrismaAdapter class
  - DevelopmentAdapter class
- Problem: Two implementations of same adapter, inconsistent with DBAL ownership
- Better: Frontend should only use DBAL client, not re-implement adapters
- Effort: 2-3 hours
- Impact: Single source of truth for database logic
Migrate Role/Permission Constants to Seed Data
- Current: Hardcoded in /frontends/nextjs/src/lib/constants.ts
```
export const ROLE_LEVELS = {
  public: 0, user: 1, moderator: 2, admin: 3, god: 4, supergod: 5
}
```
- Better: /schemas/permissions_schema.json or /seed/database/permissions.yaml
- Effort: 2-3 hours
- Impact: All roles database-driven, dynamic permission system
Migrate Environment Configuration
- Current: Hardcoded defaults in constants.ts (ENV_DEFAULTS, TIMEOUTS)
- Better: /seed/config/app-config.yaml loaded at startup
- Effort: 2-3 hours
- Impact: All configuration declarative, environment-specific

Low Priority (~2-3 hours)

Remove Experimental Code
- Verify /frontends/nextjs/src/lib/db/database-admin/seed-default-data/css/categories/experimental.ts
- Check usage, delete if unused
- Effort: 1 hour
- Impact: Minimal
Complete TODO Items
- Review /docs/TODO_MVP_IMPLEMENTATION.md
- Compiler implementation (/lib/compiler/index.ts - marked TODO, unimplemented)
- Status: Affects 2-3 files
- Effort: Variable depending on scope

Architecture Strengths (Keep As-Is)

✅ Infrastructure (TypeScript) - 14% of codebase

DBAL (282 files) - Type-safe database abstraction, multi-adapter support
FakeMUI (514 files) - Consistent UI components across web/QML
NextJS Frontend (491 files) - Server-side rendering, API routes

✅ Data-Driven Design (JSON/YAML) - 27% of codebase

18 interconnected schemas in /schemas/ - Single source of truth
376 component definitions in /packages/*/components/ui.json - 100% declarative
Package system - Fully self-contained with seed data

Migration Path to 35-40% Data-Driven

Task	Current %	Target %	Effort	Impact
Remove cruft (old/, backups)	73% TS	70% TS	1h	High
Migrate seed data YAML	73% TS	68% TS	4-5h	Medium
Eliminate adapter duplication	73% TS	72% TS	2-3h	Medium
Migrate constants/config	73% TS	70% TS	2-3h	Low
Total (recommended)	73%	~35-40%	9-16h	High

Recommended Timeline: Address high-priority items first (Phase 1: 1-2 weeks), then medium-priority refactoring (Phase 2: 4-6 weeks)

Deprecated Features

None - This is a new project with no deprecated features

Security Audit Status

✅ Session Management: Secure cookie-based sessions with httpOnly flag ✅ Password Storage: SHA-512 hashing (see password-utils.ts) ✅ SQL Injection: Protected by Prisma ORM ✅ CSRF: Next.js built-in protection ✅ XSS: React automatic escaping ⏸️ Rate Limiting: Planned for Phase 5 ⏸️ MFA: Planned for Phase 6 ⏸️ OAuth: Planned for Phase 6

Migration Notes

From 0.0.0 (Pre-MVP) to 0.1.0 (MVP)

No breaking changes - all changes were additive or fulfilled TODO placeholders.

New Dependencies:

esbuild (^0.27.2) - Added for compiler functionality

Database Changes: None - Used existing schema

Configuration Changes: None - All configuration remains backward compatible

Technical Debt & Prioritization Roadmap

Based on comprehensive codebase audit (January 2026), MetaBuilder has well-managed technical debt with a clear refactoring path to increase data-driven architecture from 27% to 35-40%.

Executive Summary

Current State:

2,047 total files
73% TypeScript (1,487 files) - Infrastructure + UI
27% JSON/YAML (560 files) - Schemas + packages + seed data
Goal: 35-40% JSON/YAML (data-driven) through strategic cleanup

Total Estimated Effort: 9-16 hours of refactoring Recommended Timeline: Phase 1 (1-2 weeks) + Phase 2 (4-6 weeks)

Phase 1: Critical Cleanup (1-2 weeks, ~2-3 hours)

🔴 P0 - Block Production Issues (None identified - codebase is stable)

🟠 P1 - Quick Wins, High Impact

#	Task	Effort	Impact	Files	Status
1.1	Delete `/old` directory	1h	High	119	❌ TODO
	Pre-Next.js SPA implementation
	Contains Vite, React, Radix UI
	Completely superseded by NextJS
1.2	Delete `.backup` files	10m	Low	4	❌ TODO
	Not needed with Git history
	Files: `*.backup` in `/dbal/development/`
1.3	Verify experimental code	1h	Low	1	❌ TODO
	`/css/categories/experimental.ts`
	Check usage, delete if unused

P1 Deliverables: Remove 124 files of dead code, reduce confusion

Phase 2: Architecture Refactoring (4-6 weeks, ~9-16 hours)

🟡 P2 - Medium Priority, Medium Effort

#	Task	Effort	Impact	Type	Status
2.1	Migrate seed data to YAML	4-5h	⭐⭐⭐ High	Architecture	❌ TODO
	Move `/lib/db/database-admin/seed-default-data/*.ts` to `/seed/database/`
	Load via DBAL `seedDatabase()` orchestration
	Affected files: 5 TS files (~150 lines)
	Sub-tasks:
	- `seed-users.ts` → `users.yaml`	1h
	- `seed-app-config.ts` → `app-config.yaml`	1h
	- `seed-css-categories.ts` → `css-categories.yaml`	1h
	- `seed-dropdown-configs.ts` → `dropdown-configs.yaml`	1h
	- Update DBAL `seedDatabase()` to load all YAML	1-2h
2.2	Eliminate adapter code duplication	2-3h	⭐⭐ Medium	DRY	❌ TODO
	Delete `/frontends/nextjs/src/lib/dbal-client/adapter/`
	Remove `PrismaAdapter` and `DevelopmentAdapter` classes (355 lines)
	Frontend uses DBAL client exclusively
	Sub-tasks:
	- Audit current adapter usage	30m
	- Move any unique logic to DBAL	1h
	- Delete adapter classes	30m
	- Update imports to use DBAL	30m
2.3	Migrate role/permission constants to DB	2-3h	⭐⭐ Medium	Data-Driven	❌ TODO
	Move hardcoded `ROLE_LEVELS` from `constants.ts`
	Target: `/schemas/permissions_schema.json` + seed
	Sub-tasks:
	- Update `permissions_schema.json` with role definitions	1h
	- Create `seed/database/roles.yaml`	30m
	- Update frontend to load roles from DB	1h
2.4	Migrate environment configuration	2-3h	⭐ Medium	Data-Driven	❌ TODO
	Move `ENV_DEFAULTS`, `TIMEOUTS` from `constants.ts`
	Target: `/seed/config/app-config.yaml`
	Sub-tasks:
	- Extract config to YAML	1h
	- Add config loading at startup	1h
	- Update all config references	30m

P2 Deliverables: Reach 35-40% data-driven architecture, eliminate 3 sources of hardcoded data

Phase 3: Polish & Optimization (Future, 2-3 hours)

🟢 P3 - Low Priority, Low Effort

#	Task	Effort	Impact	Type	Status
3.1	Complete TODO items	Varies	Low	Code Quality	❌ TODO
	Compiler implementation (`/lib/compiler/index.ts`)
	Review `/docs/TODO_MVP_IMPLEMENTATION.md`	1h
3.2	Add error boundaries	2h	Low	UX	❌ TODO
	Comprehensive React error boundaries
3.3	Improve TypeScript coverage	2-3h	Low	Type Safety	❌ TODO
	Strict mode compliance, eliminate `any` types

Implementation Timeline

Week 1 (P1 - Quick Wins)
├── Day 1-2: Delete /old and backups (1.5h)
└── Day 3-5: Verify experimental code (1h)

Week 2-6 (P2 - Architecture Refactoring)
├── Week 2: Migrate seed data to YAML (2.1) - 4-5h
├── Week 3: Eliminate adapter duplication (2.2) - 2-3h
├── Week 4: Migrate roles to DB (2.3) - 2-3h
├── Week 5: Migrate environment config (2.4) - 2-3h
└── Week 6: Integration testing & validation

Ongoing (P3 - Polish)
└── As time permits after P1 + P2

Dependencies & Blocking Issues

No blockers identified - All work is independent and can be started immediately.

Recommended Sequence:

Start P1 immediately (delete old code) - quick win, high clarity
Start P2.1 next (seed data) - foundation for other refactoring
Parallel P2.2, P2.3, P2.4 - Can work in parallel after P2.1

Success Metrics

Metric	Current	Target	Timeline
Data-Driven Ratio	27% JSON/YAML	35-40%	After P2
Hardcoded Data	3 sources	0 sources	After P2
Deprecated Code	119 files	0 files	After P1
Adapter Duplication	355 lines	0 lines	After P2.2
Code Clarity	73% TS overhead	60-65% TS	After P2

Risk Assessment

Low Risk: All refactoring is backward compatible

Git history preserved
No production impact (development artifacts only)
All changes are additive to data-driven architecture
Can be tested incrementally with E2E tests

Testing Plan:

Verify all existing tests pass after each phase
Run E2E tests to ensure no functionality lost
Manual testing of seed data loading
Verification of role/permission system after 2.3
Environment config testing after 2.4

Post-Refactoring Priorities

Once technical debt is addressed (after Phase 2), focus on feature development:

Feature Roadmap (Q1-Q2 2026)

Q1 2026: Foundation

✅ Core platform (completed)
✅ DBAL integration (completed)
🎯 Tech Debt Cleanup (Phase 1-2, Jan-Feb 2026)

Q2 2026: Enhanced Features

🎯 Backend API Integration (real endpoints, not mocks)
🎯 Enhanced CRUD UX (forms, validation, pagination)
🎯 God Panel Foundation (route/package management)

Q3-Q4 2026: Production Hardening

🎯 C++ DBAL in production
🎯 Performance optimization
🎯 Security hardening
🎯 First production deployments

Audit Report: See "Codebase Composition" section above for detailed breakdown
Architecture: See /ARCHITECTURE.md for system design
Development: See CLAUDE.md for AI assistant guidance
Implementation: See .github/prompts/ for development workflows

Long-Term Vision

Year 1 (2026) - Foundation & Growth

Q1 2026: Platform Foundation

✅ MVP with core features (Completed)
✅ DBAL TypeScript implementation (Completed)
🎯 Tech Debt Cleanup Phase 1 (Jan-Feb, ~2-3 hours)
- Delete /old directory (119 files)
- Delete backup files (4 files)
- Remove experimental code (1 file)

Q2 2026: Architecture Hardening & Features

🎯 Tech Debt Cleanup Phase 2 (Feb-Mar, ~9-16 hours)
- Migrate seed data to YAML (4-5h)
- Eliminate adapter duplication (2-3h)
- Migrate roles to database (2-3h)
- Migrate environment config (2-3h)
🎯 Production-ready CRUD operations (integration with real API endpoints)
🎯 Enhanced CRUD UX (forms, validation, pagination)
🎯 God panel foundation (route/package management)

Q3-Q4 2026: Production Hardening & Scale

🎯 C++ DBAL in production
🎯 Performance optimization
🎯 Security hardening
🎯 First production deployments: >5 sites
🎯 Active users: >100
🎯 First 10 third-party packages

Year 2 (2027) - Ecosystem & Enterprise

🔮 Package marketplace with ratings
🔮 Advanced search capabilities (full-text, filters)
🔮 Multi-language i18n support
🔮 Enterprise authentication (OAuth, MFA, SSO)
🔮 Real-time collaboration features
🔮 Plugin ecosystem for extensions
🔮 100+ public packages
🔮 Active deployments: >50 sites
🔮 Active users: >5,000

Year 3 (2028+) - AI & Scale

🔮 Visual component builder (drag-and-drop)
🔮 AI-assisted schema generation
🔮 AI-powered package recommendations
🔮 Real-time collaborative editing
🔮 Advanced analytics and insights
🔮 Multi-cloud deployment (AWS, Azure, GCP)
🔮 Kubernetes native deployment
🔮 1000+ active deployments
🔮 Active users: >50,000
🔮 Enterprise customers: >25

Features Inspired by Original Spark Version

The following features from our original Spark-based version are being considered for future implementation in the Next.js platform:

High Priority (Q3-Q4 2026)

🎯 Visual Component Hierarchy Editor - Drag-and-drop component organization (originally in Level 4 God panel)
🎯 Monaco Code Editor Integration - In-browser code editing with syntax highlighting
🎯 Database Export/Import - Complete system backup and restoration
🎯 Live Schema Editing - Real-time schema modifications through UI

Medium Priority (2027)

🔮 Component Catalog Browser - Searchable library of available components
🔮 Visual Workflow Builder - Node-based workflow design interface
🔮 Template Library - Pre-built page and component templates
🔮 Live Preview Mode - See changes in real-time while editing

Low Priority (2028+)

🔮 Alternative Scripting Support - Consider Lua or other scripting languages as alternatives to JSON scripts
🔮 Component Version Control - Track component changes over time
🔮 Collaborative Editing - Multiple users editing simultaneously

Features Not Planned for Migration

Some features from the original version are intentionally not being migrated:

❌ KV Storage - Replaced by PostgreSQL for better scalability and reliability
❌ Radix UI - Replaced by Material-UI for comprehensive enterprise components
❌ Tailwind CSS - Replaced by SCSS modules for better component isolation
❌ 4-Level Permission System - Enhanced to 6-level system with more granular control
❌ Single-Tenant Architecture - Completely redesigned for multi-tenancy

Strategic Goals

Platform Goals

Zero Configuration Deployments - One command to deploy everything
Complete Flexibility - Everything configurable through UI
Performance First - Sub-100ms response times
Security by Default - Built-in security best practices
Developer Experience - Fast iteration, great tooling

Community Goals

Open Source Ecosystem - Vibrant package community
Comprehensive Documentation - Every feature documented
Educational Resources - Tutorials, videos, courses
Community Support - Active forums and Discord
Contributor Growth - 100+ active contributors

Business Goals

Enterprise Adoption - Fortune 500 deployments
Cloud Marketplace - Available on AWS, Azure, GCP marketplaces
Professional Services - Training, consulting, custom development
Managed Hosting - MetaBuilder Cloud service
Partner Ecosystem - Integration partners and resellers

Deployment Options

Quick Start (Development)

# Clone repository
git clone https://github.com/yourusername/metabuilder
cd metabuilder

# Install dependencies
npm install

# Setup database
npm run db:generate
npm run db:push

# Start development server
npm run dev

Visit http://localhost:3000

Docker Compose (Development)

# Start all services
docker-compose -f deployment/docker-compose.development.yml up

# Services available:
# - App: http://localhost:5173
# - DBAL API: http://localhost:8081
# - Adminer (DB UI): http://localhost:8082
# - Redis Commander: http://localhost:8083
# - Mailhog (Email): http://localhost:8025

Docker Compose (Production)

# Configure environment
cp .env.production.example .env
vim .env  # Update with production values

# Start production stack
docker-compose -f deployment/docker-compose.production.yml up -d

# Services:
# - PostgreSQL (internal)
# - DBAL Daemon (internal)
# - Next.js App (internal)
# - Nginx (public: 80, 443)
# - Redis (internal)

One-Command Deployment

# Deploy everything (PostgreSQL, DBAL, Next.js, Media daemon, Redis, Nginx)
./deployment/deploy.sh all --bootstrap

Cloud Platforms

Docker Swarm

docker swarm init
docker stack deploy -c deployment/docker-compose.production.yml metabuilder
docker service scale metabuilder_metabuilder-app=5

Kubernetes

kubectl apply -f deployment/kubernetes/
kubectl scale deployment metabuilder-app --replicas=5

Managed Services

AWS: ECS, EKS, or EC2 with docker-compose
Azure: Container Instances, AKS, or VM with docker-compose
Google Cloud: Cloud Run, GKE, or Compute Engine
DigitalOcean: App Platform or Droplet with docker-compose
Heroku: Container registry
Fly.io: Native support

Resource Requirements

Minimum (Development)

CPU: 2 cores
RAM: 4GB
Storage: 20GB
Network: Broadband internet

Recommended (Production - Small)

CPU: 4 cores
RAM: 8GB
Storage: 100GB SSD
Network: 100Mbps dedicated

Recommended (Production - Medium)

CPU: 8 cores
RAM: 16GB
Storage: 500GB SSD
Network: 1Gbps dedicated

Enterprise (Production - Large)

CPU: 16+ cores
RAM: 32GB+
Storage: 1TB+ SSD
Network: 10Gbps dedicated
Load Balancer: Required
CDN: Recommended

Development Workflow

For Contributors

Setup Development Environment

git clone <repo>
cd metabuilder
npm install
npm run db:generate
npm run db:push

Create Feature Branch
```
git checkout -b feature/my-new-feature
```
Make Changes
- Follow existing patterns
- Add tests (parameterized with it.each)
- Update documentation
- One lambda per file

Test Changes

npm run lint:fix
npm run test
npm run typecheck

Commit and Push

git add .
git commit -m "feat: add my new feature"
git push origin feature/my-new-feature

Create Pull Request
- Reference related issues
- Include screenshots for UI changes
- Ensure CI passes

Code Standards

✅ One lambda per file - Functions are separate files, classes only as containers ✅ Parameterized tests - Use it.each() for comprehensive test coverage ✅ Material-UI - Use MUI components, not Radix or Tailwind ✅ SCSS Modules - Component-specific styles in .module.scss ✅ Absolute imports - Use @/ for all imports ✅ Server-only - Mark server code with 'server-only' ✅ DBAL for data - All database access through DBAL ✅ Tenant isolation - Always filter by tenantId

Testing Standards

Unit tests: All new functions must have tests
Coverage: Target >80% for new code
Parameterized: Use it.each() for multiple test cases
Naming: Test files match source files (utils.ts → utils.test.ts)
Run tests: npm run test before committing

Release Process

Version Bump: Update version in package.json
Changelog: Document all changes
Tag Release: git tag v0.x.x
Build: Test production build
Deploy: Deploy to staging first
Verify: Run E2E tests
Production: Deploy to production
Monitor: Watch logs and metrics

Success Metrics

MVP (Achieved)

Core features implemented (5/5)
Test coverage >80% (97.9%)
Documentation complete
Zero breaking changes

Post-MVP (2026)

Production deployments: >5
Active users: >100
Public packages: >20
API response time: <200ms avg
Uptime: >99.9%
Customer satisfaction: >4.5/5

Long-Term (2027+)

Active deployments: >100
Active users: >10,000
Public packages: >100
Community contributors: >50
Enterprise customers: >10

Contributing

We welcome contributions! MetaBuil der is an open-source project that thrives on community involvement.

Ways to Contribute

Code Contributions
- Implement features from the roadmap
- Fix bugs and issues
- Improve performance
- Add tests
- Refactor code
Documentation
- Write tutorials and guides
- Improve API documentation
- Create video tutorials
- Translate documentation
Packages
- Create new packages
- Improve existing packages
- Share package templates
- Write package documentation
Testing & QA
- Report bugs
- Test new features
- Improve test coverage
- Performance testing
Community Support
- Answer questions
- Help new users
- Write blog posts
- Share use cases

Getting Started

New to MetaBuilder?

Read the README.md - Understand the core concept
Follow the Quick Start - Get it running locally
Explore /packages - See example packages
Check docs/TODO_MVP_IMPLEMENTATION.md - See what's been done
Join discussions - Ask questions, share ideas

Ready to Contribute?

Check GitHub Issues
Look for "good first issue" labels
Read Development Workflow
Follow Code Standards
Submit a pull request

Project Resources

Documentation

README.md - Project overview and quick start
ROADMAP.md (this file) - Comprehensive development roadmap
docs/TODO_MVP_IMPLEMENTATION.md - MVP implementation checklist
docs/MVP_IMPLEMENTATION_SUMMARY.md - MVP completion summary
deployment/README.md - Deployment guide
dbal/README.md - DBAL documentation
dbal/docs/AGENTS.md - AI development guidelines

Specifications

spec/ - Formal TLA+ specifications
schemas/ - JSON schemas and validation
prisma/schema.prisma - Database schema

Examples

packages/ - 52 example packages
e2e/ - End-to-end test examples
storybook/ - Component stories

Configuration

.github/prompts/ - AI agent workflows
deployment/ - Docker and deployment configs
config/ - Application configuration

Communication Channels

GitHub Issues - Bug reports and feature requests
GitHub Discussions - Questions and community discussions
Pull Requests - Code contributions and reviews
Discord (coming soon) - Real-time chat and support

Code of Conduct

Be respectful and inclusive
Help others learn and grow
Accept constructive feedback
Focus on what's best for the community
Show empathy and kindness

Quick Reference

Key Commands

# Development
npm run dev              # Start development server
npm run build            # Build for production
npm run test             # Run tests
npm run lint:fix         # Fix linting issues
npm run typecheck        # Check TypeScript types

# Database
npm run db:generate      # Generate Prisma client
npm run db:push          # Push schema to database
npm run db:migrate       # Run migrations

# Testing
npm run test             # Run all tests
npm run test:run         # Run tests once (no watch)
npm run test:watch       # Watch mode

Important Files

File	Purpose
ROADMAP.md	This comprehensive roadmap
README.md	Project overview and setup
package.json	Dependencies and scripts
prisma/schema.prisma	Database schema
frontends/nextjs/src/app/page.tsx	Root page with routing logic
frontends/nextjs/src/lib/auth/get-current-user.ts	Authentication
frontends/nextjs/src/lib/db/core/dbal-client.ts	Database access

Key Directories

Directory	Purpose
frontends/nextjs/	Next.js frontend application
packages/	52 built-in packages
dbal/	Database abstraction layer (TS + C++)
services/media_daemon/	C++ media processing service
deployment/	Docker and deployment configs
prisma/	Database schema and migrations
docs/	Project documentation
e2e/	End-to-end tests
.github/prompts/	AI development workflows

Architecture Principles

Data-Driven Everything - No hardcoded routes or components
Database as Source of Truth - All configuration in database
Generic Rendering - JSON → React via generic renderer
Multi-Tenant by Default - All queries filter by tenantId
Permission-Based Access - 6-level permission system
Package-Based Architecture - Self-contained, installable packages
DBAL Abstraction - All database access through DBAL
Zero Coupling - Frontend knows nothing about specific packages

Version History

Version	Date	Status	Highlights
0.1.0-alpha	Jan 2026	✅ Current	MVP achieved with all 5 TODO items
0.0.0	Pre-2026	✅ Complete	Foundation and architecture
0.2.0-alpha	Q1 2026	📋 Planned	Backend API integration
0.3.0-beta	Q2 2026	📋 Planned	God panel, enhanced CRUD
1.0.0	Q4 2026	🔮 Target	Production-ready release

Changelog

v0.1.0-alpha (January 2026) ✨ MVP Release

New Features:

✨ Session-based authentication with getCurrentUser()
✨ Permission level checks and access control
✨ Dynamic package loading from filesystem
✨ Schema-driven CRUD operations (list, detail, create, edit)
✨ Static page generation with ISR support
✨ Code compilation with esbuild (minification, source maps)

New Files:

frontends/nextjs/src/lib/auth/get-current-user.ts
frontends/nextjs/src/lib/auth/get-current-user.test.ts
frontends/nextjs/src/components/AccessDenied.tsx
frontends/nextjs/src/lib/entities/load-entity-schema.ts
frontends/nextjs/src/lib/entities/api-client.ts
frontends/nextjs/src/lib/compiler/index.test.ts

Enhanced Files:

frontends/nextjs/src/app/page.tsx - Added auth and permission checks
frontends/nextjs/src/app/[tenant]/[package]/page.tsx - Dynamic package loading
frontends/nextjs/src/app/[tenant]/[package]/[...slug]/page.tsx - Full CRUD views
frontends/nextjs/src/app/ui/[[...slug]]/page.tsx - Static generation
frontends/nextjs/src/lib/compiler/index.ts - Full esbuild integration

Testing:

20 new tests added (100% passing)
Total: 188/192 tests passing (97.9%)
Parameterized test coverage for all new functions

Dependencies:

Added: esbuild (^0.27.2)

Documentation:

Created comprehensive ROADMAP.md (this file)
Updated docs/TODO_MVP_IMPLEMENTATION.md - All items completed
Created docs/MVP_IMPLEMENTATION_SUMMARY.md

Breaking Changes:

None - All changes additive or fulfilling TODO placeholders

Questions or Feedback?

Documentation: See /docs directory and README.md
Issues: GitHub Issues
Discussions: GitHub Discussions
Email: support@metabuilder.dev (coming soon)

License

MIT License - See LICENSE file

Document Status: 📊 Comprehensive Single Source of Truth Last Updated: January 8, 2026 Current Version: 0.1.0-alpha Current Phase: 🎯 MVP Achieved → Post-MVP Development Next Milestone: Backend API Integration (Q1 2026)

This roadmap is a living document. It will be updated as the project evolves. All major decisions, features, and milestones are documented here to serve as the single source of truth for MetaBuilder development.

101 KiB Raw Permalink Blame History Unescape Escape

MetaBuilder Roadmap

📁 Monorepo Structure

Standalone Projects (15)

🚀 Executive Summary

Status Dashboard

📚 What To Read First

Vision

Core Philosophy

Project History

Evolution from Spark-Based Version

Original Version (Spark-Based) - Pre-2026

Current Version (Next.js-Based) - 2026+

Migration Path

What Was Preserved

What Was Reimplemented

What Was Added

Architecture Comparison

Why We Migrated

Scalability Needs

Enterprise Requirements

Ecosystem Maturity

Legacy Code Location

Table of Contents

Current Status

Quick Stats

What's Working Today

What's NOT Done Yet (Phase 2 Remaining & Future Phases)

Technology Stack

Frontend

Backend & Database

Infrastructure

Development Tools

C++ Services

System Architecture

High-Level Overview

Database Schema (9 Models)

Data Flow Architecture

Permission System (6 Levels)

Package Architecture

DBAL Architecture

Package Ecosystem

Core UI Packages (9)

Permission-Level UI (5)

Admin & Configuration (12)

Development Tools (6)

Data & Content (5)

Communication & Social (3)

Media & Streaming (3)

Integrations & Tools (5)

Utility (4)

Package Installation

Creating New Packages

MVP Milestone ✨

MVP Acceptance Criteria

1. Authentication & Authorization ✅

2. Dynamic Package Loading ✅

3. CRUD Operations ✅

4. Static Page Generation ✅

5. Code Compilation ✅

MVP Deliverables

Release History

v0.1.0-alpha (January 2026) - MVP Release ✨

v0.0.0 (Pre-MVP)

Roadmap Phases

✅ Phase 0: Foundation (Completed)

✅ Phase 1: MVP (Completed - January 2026)

Implementation Tasks

2.1 API Endpoint Implementation ✅ COMPLETE

2.2 API Client Integration ✅ COMPLETE

2.3 Request/Response Validation ✅ COMPLETE

2.4 Pagination Implementation ✅ COMPLETE

2.5 Filtering and Sorting ✅ COMPLETE

2.6 Authentication Middleware ✅ COMPLETE

2.7 Rate Limiting

2.8 Error Handling

Testing Requirements

Performance Benchmarks

Documentation Requirements

🔮 Phase 3: Enhanced CRUD (Planned)

101 KiB

Raw Permalink Blame History