From c04431a5e2a6604f403d2ae669fa0e56bf16894f Mon Sep 17 00:00:00 2001 From: johndoe6345789 Date: Sat, 3 Jan 2026 00:11:12 +0000 Subject: [PATCH] Update TLA+ specifications and validation scripts - Revised architecture alignment in README.md to reference JSON package format. - Enhanced SUMMARY.md with detailed specifications for workflow, collaboration, and integration systems. - Updated package_system.tla to include JSON package format and manifest path. - Modified validate-specs.sh to include package_system.tla in validation checks. --- README.md | 2400 ++++++++++++++++++++------------------- spec/README.md | 4 +- spec/SUMMARY.md | 516 ++++----- spec/package_system.tla | 13 +- spec/validate-specs.sh | 189 +-- 5 files changed, 1565 insertions(+), 1557 deletions(-) diff --git a/README.md b/README.md index 87c098dfd..f8f357e30 100644 --- a/README.md +++ b/README.md @@ -1,1199 +1,1201 @@ -# MetaBuilder - -A **data-driven, multi-tenant platform** where 95% of functionality lives in JSON/Lua, not TypeScript. Build enterprise applications declaratively with a 6-level permission system. - -> πŸ“‹ **Ready to populate the kanban?** See [KANBAN_READY.md](KANBAN_READY.md) for a quick guide to populate the GitHub project board with 775 TODO items. - ---- - -## Table of Contents - -1. [Quick Start](#quick-start) -2. [Quick Overview](#quick-overview) -3. [Refactor Plan](#refactor-plan) -4. [Architecture](#architecture) -5. [6-Level Permission System](#6-level-permission-system) -6. [Package System](#package-system) -7. [Database](#database) -8. [Multi-Tenant System](#multi-tenant-system) -9. [Lua Scripting](#lua-scripting) -10. [API Reference](#api-reference) -11. [Testing](#testing) -12. [Development](#development) -13. [Security](#security) -14. [Deployment](#deployment) -15. [Contributing](#contributing) -16. [Troubleshooting](#troubleshooting) -17. [Key Files Reference](#key-files-reference) -18. [Roadmap](#roadmap) - ---- - -## Quick Start - -```bash -# Clone and install -git clone -cd metabuilder/frontends/nextjs -npm install - -# Set up database -npm run db:generate -npm run db:push - -# Start development -npm run dev -``` - -Visit `http://localhost:3000` - -### Prerequisites - -- Node.js 18+ -- npm or yarn -- Docker (optional) - -### 15-Minute Sanity Check - -```bash -cd frontends/nextjs -npm ci -npm run typecheck -npm run lint -npm run test:unit -- --run -npm run build -``` - ---- - -## Quick Overview - -MetaBuilder is a **data-driven, multi-tenant platform** with these core features: - -- **6-Level Permission System**: Public β†’ User β†’ Moderator β†’ Admin β†’ God β†’ Supergod hierarchy -- **Multi-Tenant Data**: All queries filter by `tenantId`; each tenant has isolated config -- **Declarative Packages**: Ship UI/features via JSON/Lua in `packages/*/seed/` -- **Lua Sandbox**: Safe runtime execution with blocked system access -- **Type-Safe Workflow**: TypeScript, Act local CI, comprehensive testing - ---- - -## Refactor Plan - -### πŸš€ Auto Code Extractor 3000β„’ - Automated File Splitting - -**NEW: One-command solution to split large files (>150 LOC) into modular structure!** - -We have 62 files exceeding 150 lines. The Auto Code Extractor 3000β„’ automatically extracts functions into individual files following the lambda-per-file pattern. - -#### Quick Commands - -```bash -# Preview what will be extracted -npm run extract:preview - -# Extract 5 files -npm run extract:quick - -# Extract all high-priority files (automated) -npm run extract:auto -``` - -**πŸ“– [Quick Start Guide](./tools/refactoring/QUICK_START.md)** | **πŸ“š [Full Documentation](./tools/refactoring/AUTO_CODE_EXTRACTOR_3000.md)** - ---- - -### Next.js to Lua Conversion TODO - -#### Table of Contents - -- [TODOs by Phase](#todos-by-phase) -- [Prep Checklist](#prep-checklist) -- [Audit Results](#audit-results) -- [Quick File References](#quick-file-references) -- [Next Steps](#next-steps) -- [Phase Deliverables](#phase-deliverables) -- [Support Notes](#support-notes) -- [Tracking & Templates](#tracking--templates) -- [Package Design Notes](#package-design-notes) -- [Metadata Design Notes](#metadata-design-notes) - -#### TODOs by Phase - -##### Phase 0 – Foundation - -- [ ] Audit all `frontends/nextjs/src/` entry points (pages, components, hooks, lib). Determine what goes to Lua. -- [ ] Define the Lua boundary. List metadata, static, and seed assets plus their API surface. - -##### Phase 1 – Mapping - -- [ ] Map reusable TypeScript helpers to single-function Lua targets (one file, one function) and describe their inputs/outputs for future porting. -- [ ] Categorize each file as β€œport to Lua”, β€œadapter only”, or β€œkeep in TypeScript”. - -##### Phase 2 – Framework Bridge - -- [ ] Extend the core framework only where gaps remain (data loading, metadata resolution, routing) with tiny TypeScript adapters to Lua if needed. -- [ ] Align metadata/seed folder structure with the requirements (consider `lua/` or `packages/static_content`) and add placeholder files. - -##### Phase 3 – Pilot Conversion - -- [ ] Pilot a TypeScript β†’ Lua helper conversion (choose one small helper, wrap it with an adapter, and keep the rest of the build intact). -- [ ] Ensure the pilot is wired into the Next.js app without breaking the current build or tests. - -##### Phase 4 – Stabilize - -- [ ] Update documentation/tests to reflect the new Lua metadata focus and ensure existing CI steps continue to pass with the new structure. -- [ ] Iterate with feedbackβ€”confirm the team agrees on the folder layout, metadata ownership, and which TypeScript files stay as adapters. - -#### Prep Checklist - -- [ ] Confirm tooling (Lua runtime, `luacheck`, etc.) is installed if needed before creating helpers. -- [ ] Backup critical TypeScript files via comments or notes before starting ports. -- [ ] Share the plan with the team and gather quick approvals for folder names (`lua/`, `packages/static_content`). -- [ ] Identify one β€œsafe” helper to convert first, ideally small and well-tested. - -#### Audit Results - -##### App surface - -- `frontends/nextjs/src/app` has 40+ routes. -- Each route needs metadata. -- Key files: - - `page.tsx` - - `layout.tsx` - - `levels` tree - - auth/dashboards - - `providers.tsx` and theme hooks - - `packages/[...path]/route.ts` -- API handlers touch auth, users, packages, levels, power-transfers, GitHub actions, screenshot, and health. -- They depend on helpers under `frontends/nextjs/src/lib/api`, `lib/packages/loader`, `db`, and `dbal`. - -##### UI components - -- `frontends/nextjs/src/components` holds ~60 React files. -- Builders: - - `Builder` - - `Level1`, `Level2`, `Level3`, `Level4`, `Level5` - - `NerdModeIDE` -- Editors: - - `LuaEditor` - - `LuaBlocksEditor` - - `CodeEditor` - - `JsonEditor` - - `SchemaEditor`s -- Management views: - - `PackageManager` - - `DatabaseManager` - - `ComponentCatalog` - - `WorkflowEditor` -- Integrations and tools: - - `GitHubActionsFetcher` - - `IRCWebchat` variants - - `DBALDemo` - - `UnifiedLogin` -- Atoms, molecules, and organisms live in nested folders. We will tag each file in Phase 1. - -##### Hooks & libraries - -- Hooks: - - `useAuth` - - `useAutoRefresh` - - `useKV` - - `useLevelRouting` - - `useDBAL` - - `useFileTree` - - `useCodeEditor` - - `useResolvedUser` - - `useGitHubFetcher` - - `use-mobile` - - `hooks/auth` store - - `hooks/use-dbal` helpers (blob storage, kv store, cached data) -- Libraries cover: - - auth and security (`lib/auth`, `lib/security`) - - database and DBAL helpers - - package, seed, and metadata tooling (`lib/packages`, `lib/seed`) - - Lua engine and functions - - workflow and rendering helpers - - API helpers and schema utilities - - GitHub and general utilities (`lib/utils`, `lib/prisma`) -- Phase 1 will tag each file as β€œadapter” or β€œport”. - -##### Supporting assets - -- `frontends/nextjs/src/seed-data` has JSON seeds for packages and baseline data. These can move to metadata. -- Tests live in `app/api/*/*.test.tsx`, `lib/**/*test.ts`, `components/get-component-icon.test.tsx`, and `hooks/*.test.ts`. We will decide which tests follow Lua stubbed logic or stay with TypeScript. - -#### Quick File References - -- `frontends/nextjs/src/app`: each route, page, provider, and API handler needs metadata or a Lua adapter. -- `frontends/nextjs/src/components`: move builders, editors, and managers to metadata-driven definitions. -- `frontends/nextjs/src/hooks`: turn each hook into a Lua micro-function. -- `frontends/nextjs/src/lib`: catalogue auth, security, db/DBAL, packages, Lua, workflow, API, schema, GitHub, and utility modules. -- Supporting assets: seed JSONs, README docs, and tests must align with the Lua transition. - -#### God Panel / Package Mapping Notes - -- Levelβ€―4’s `Level4Tabs` exposes a β€œPackages” tab that renders `PackageManager`, so God-level users open that tab when they need to install, enable, or disable packages (`frontends/nextjs/src/components/level4/Level4Tabs.tsx`). -- `PackageManager` loads the catalog from `PACKAGE_CATALOG`, shows installed/available packages, and calls APIs (`installPackage`, `togglePackageEnabled`, `uninstallPackage`, `listInstalledPackages`) that persist package state (`frontends/nextjs/src/components/PackageManager.tsx`). -- `PACKAGE_CATALOG` lives in `frontends/nextjs/src/lib/packages/package-catalog.ts` and contains the manifest content (schemas, pages, workflows, Lua scripts, seed data) that the God panel presents when a package is selected. -- The reusable packages under `packages/` follow the structure described in `packages/README.md`: each package has `seed/metadata.json`, `components.json`, and optional `static_content/examples.json`, so new package metadata can align with what `PackageManager` expects. -- Default packages are enumerated in `frontends/nextjs/src/lib/packages/package-glue/default-packages.ts` (admin_dialog, dashboard, forum_forge, etc.), and God panel UI relies on those definitions to match catalog entries with metadata stored in `packages/`. - -#### Next Steps - -1. Create the companion metadata folder (e.g., `lua/` or `packages/static_content`) with placeholder single-function files that mirror the above references. -2. Run the audit across the identified folders. Mark each file as β€œport to Lua”, β€œadapter only”, or β€œleave in TypeScript” before editing. - -#### Phase Deliverables - -- **Phase 0**: Audit spreadsheet/list with tags for each entry. Lua boundary doc with endpoints and metadata goals. -- **Phase 1**: Mapping sheet (spreadsheet or Markdown) documenting TypeScript helper β†’ Lua micro-function mapping and file categorization. -- **Phase 2**: Adapter interface stubs plus placeholder Lua files under the new metadata folder. -- **Phase 3**: One working Lua helper wired through an adapter, with updated unit test coverage or manual verification notes. -- **Phase 4**: Documentation updates, test run results, and a summary of which files moved versus stayed. - -#### Support Notes - -- **Risks**: The Next.js frontend is large; do not remove TypeScript pages until their Lua replacements exist. Keep unit/e2e tests running in TypeScript during transition. -- **Owner**: Track who updates each phase and keep a note in this doc or comments for accountability. -- **Validation tips**: After a Lua helper is added, run `npm run lint` and `npm run test:unit` from `frontends/nextjs/`. Keep logs of failing tests if you defer them. -- **Documentation**: Update READMEs in `components/`, `hooks/`, and `lib/` with pointers to the new Lua package once core content moves. -- **Checkpoint**: Before Phase 3, ensure adapter interfaces exist for data access and that seed data has been migrated to the metadata folder. - -- **Toolbelt** - - `npm run lint` (in `frontends/nextjs/`) - - `npm run test:unit` - - `npm run test:e2e` (only after Lua helpers are wired) - - `npm run build` to verify core app still compiles - -- **Folder sketch** - - `lua/metadata/` – metadata + seed descriptors per package - - `lua/functions/` – single-function files (one helper per file) - - `lua/adapters/` – TypeScript adapters that relay calls into Lua micro-functions - - `packages/static_content/` – optional storage for JSON seeds already living in the frontend - -#### Tracking & Templates - -- **Audit table**: - 1. File path - 2. Current role (page, API, hook, lib) - 3. Port target (Lua, adapter, stay TS) - 4. Notes/risks - 5. Owner/ETA -- **Mapping sheet**: - 1. Helper name - 2. Inputs/outputs - 3. Proposed Lua file path - 4. Adapter needed (yes/no) - 5. Test expectations - -### Lua Conversion & God Panel Mapping - -- God-level builders (Levelβ€―4 tabs) expose the `PackageManager` UI where admins install, enable, and disable packages (`Level4Tabs` β†’ `PackageManager` in `frontends/nextjs/src/components`). -- `PackageManager` loads `PACKAGE_CATALOG`, drives the install/toggle APIs, and feeds catalog entries back into the God panel (`frontends/nextjs/src/lib/packages/package-catalog.ts` and `frontends/nextjs/src/components/PackageManager.tsx`). -- Catalog metadata mirrors the `packages/*/seed/metadata.json` format described in `packages/README.md`; new packages must follow that structure (metadata, components, optional static content) so the God panel can surface them. -- Default package metadata is pre-defined under `frontends/nextjs/src/lib/packages/package-glue/default-packages.ts` (admin_dialog, dashboard, forum_forge, etc.) to keep UI, Lua loaders, and catalog definitions aligned. - -#### Package Design Notes - -- Keep packages lean: each folder under `packages/` should contain `seed/`, optional `static_content/`, and light JSON/metadata entries. -- Store metadata (name, version, packageId) in `seed/metadata.json`; keep `packageId` snake_case and version semver. -- Avoid coupling packages to UI; they should expose data scripts for Lua/metadata consumers. -- Use `package/glue` logic in `frontends/nextjs/src/lib/packages` as reference when designing new packages so metadata matches runtime expectations. -- Document every package’s dependencies and optional scripts so Lua adapters can load/distribute them predictably. - -#### Metadata Design Notes - -- Define metadata per route/component via JSON tables describing inputs, outputs, Lua helper references, and UI states. -- Keep metadata separate from implementation; store static descriptors in `lua/metadata/*` and reference them via adapters. -- Version metadata files to make rollbacks easier and ensure seeds stay synchronized with database migrations. -- Use small, focused metadata files so the single-function Lua helpers can import only the data they need. -- Track metadata ownership (which team, file) inside each descriptor to simplify future audits. - -#### Reference Resources - -- **AGENTS.md**: follow repo guidelines and nested agent notes before touching `dbal/` or other special folders. -- **UI_STANDARDS.md**: reference for UI design rules while documenting Lua metadata that may affect styling. -- **packages/**: inspect existing JSON-driven package structure for inspiration (seed/metadata.json, static_content/). -- **tools/**: reuse scripts from this folder when building new metadata loaders or conversion helpers. -- **frontends/nextjs/src/lib/packages/package-glue/**: examine default packages and loaders to keep Lua metadata compatible. - -#### Decision Log - -- Document every key decision (phase changes, Lua structure choices, package additions) below with date/owner for easy onboarding. - 1. [date] - [owner] - [decision summary] - 2. ... - -#### Collaboration Notes - -- Post updates to this doc before merging any changes, especially new folder names or metadata formats. -- Tag @owner in relevant PR comments when handing off a phase or requesting review. - -## Architecture - -MetaBuilder combines: - -- **Declarative Components**: Render UIs from JSON using `RenderComponent` -- **DBAL**: TypeScript SDK + C++ daemon, language-agnostic via YAML contracts -- **Package System**: Self-contained modules in `/packages/{name}/seed/` -- **Multi-Tenancy**: All queries filter by `tenantId` - -### Project Structure - -``` -metabuilder/ -β”œβ”€β”€ frontends/nextjs/ # Next.js application -β”‚ β”œβ”€β”€ src/ -β”‚ β”‚ β”œβ”€β”€ app/ # App routes and pages -β”‚ β”‚ β”œβ”€β”€ components/ # React components (Atomic Design) -β”‚ β”‚ β”‚ β”œβ”€β”€ atoms/ # Basic UI elements -β”‚ β”‚ β”‚ β”œβ”€β”€ molecules/ # Simple composites (2-5 atoms) -β”‚ β”‚ β”‚ β”œβ”€β”€ organisms/ # Complex features with business logic -β”‚ β”‚ β”‚ └── level1-5/ # Page-level components -β”‚ β”‚ β”œβ”€β”€ lib/ # Core libraries -β”‚ β”‚ β”œβ”€β”€ hooks/ # Custom React hooks -β”‚ β”‚ β”œβ”€β”€ seed-data/ # Database seeds -β”‚ β”‚ β”œβ”€β”€ theme/ # MUI theme configuration -β”‚ β”‚ └── types/ # TypeScript type definitions -β”‚ └── e2e/ # Playwright E2E tests -β”œβ”€β”€ packages/ # Feature packages (JSON/Lua-driven) -β”œβ”€β”€ dbal/ # Database Abstraction Layer -β”‚ β”œβ”€β”€ ts/ # TypeScript implementation -β”‚ β”œβ”€β”€ cpp/ # C++ daemon (production) -β”‚ β”‚ β”œβ”€β”€ src/ # Source code -β”‚ β”‚ β”œβ”€β”€ include/ # Header files -β”‚ β”‚ β”œβ”€β”€ build-config/ # CMake, Docker configs -β”‚ β”‚ β”œβ”€β”€ lint-config/ # Clang format/tidy configs -β”‚ β”‚ └── docs/ # Implementation docs -β”‚ β”œβ”€β”€ api/ # YAML contracts -β”‚ └── docs/ # DBAL documentation -β”œβ”€β”€ prisma/ # Database schema & migrations -β”œβ”€β”€ config/ # Shared configuration -β”‚ β”œβ”€β”€ build/ # Next.js, TypeScript configs -β”‚ β”œβ”€β”€ lint/ # ESLint config -β”‚ β”œβ”€β”€ test/ # Vitest, Playwright configs -β”‚ └── misc/ # Other configs (JSON, env) -β”œβ”€β”€ tools/ # Repository utilities -β”‚ β”œβ”€β”€ analysis/ # Code analysis scripts -β”‚ β”œβ”€β”€ detection/ # Stub/pattern detection -β”‚ β”œβ”€β”€ generation/ # Code generation -β”‚ β”œβ”€β”€ quality/ # Quality metrics -β”‚ β”œβ”€β”€ security/ # Security scanning -β”‚ └── validation/ # Validation scripts -β”œβ”€β”€ docs/ # Documentation -β”‚ β”œβ”€β”€ architecture/ # System design docs -β”‚ β”œβ”€β”€ guides/ # How-to guides -β”‚ β”œβ”€β”€ security/ # Security documentation -β”‚ β”œβ”€β”€ todo/ # Task tracking -β”‚ └── navigation/ # Doc navigation files -β”œβ”€β”€ deployment/ # Docker configs -└── .github/ # GitHub workflows & prompts - β”œβ”€β”€ workflows/ # CI/CD pipelines - └── prompts/ # AI prompt templates -``` - -### Component Architecture (Atomic Design) - -| Layer | Examples | Rules | -|-------|----------|-------| -| **Atoms** | Button, Input, Badge | No custom dependencies | -| **Molecules** | AppHeader, ProfileCard | Can use Atoms only | -| **Organisms** | SchemaEditor, PackageManager | Can use Atoms, Molecules, other Organisms | -| **Pages** | Level1-5 | Can use all | - ---- - -## 6-Level Permission System - -Hierarchical access control where each level inherits all permissions from lower levels: - -| Level | Role | Access | Route | -|-------|------|--------|-------| -| 1 | Public | Read-only, unauthenticated | `/` | -| 2 | User | Personal dashboard, content creation | `/dashboard` | -| 3 | Moderator | Moderation desk, flag review, report handling | `/moderator` | -| 4 | Admin | User management, system settings | `/admin` | -| 5 | God | Workflows, advanced scripting, packages | `/builder` | -| 6 | Supergod | Full system control, tenant management | `/supergod` | - -The new Level 3 moderator desk (`/moderator`) is protected by `AuthGate` with `requiredRole="moderator"` and surfaces flagged discussions, incident reports, and quick actions before escalation to admin. - -### Permission Matrix - -| Feature | L1 | L2 | L3 | L4 | L5 | L6 | -|---------|----|----|----|----|----|----| -| View Public Data | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | -| Authenticate | | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | -| Create Content | | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | -| Moderate Content | | | βœ“ | βœ“ | βœ“ | βœ“ | -| Manage Users | | | | βœ“ | βœ“ | βœ“ | -| Run Workflows | | | | | βœ“ | βœ“ | -| System Configuration | | | | | | βœ“ | - -### Usage - -```typescript -import { canAccessLevel } from '@/lib/auth' -import { useAuth } from '@/hooks' - -export const AdminPanel = () => { - const { user } = useAuth() - - if (!canAccessLevel(user.level, 4)) { - return - } - - return -} - -// Multiple conditional features -const features = { - basicDashboard: true, - contentEditor: user.level >= 2, - moderationDesk: user.level >= 3, - userManagement: user.level >= 4, - workflowEngine: user.level >= 5, - systemConfig: user.level === 6, -} -``` - -### Backend Route Protection - -```typescript -import { validateRequest } from '@/lib/auth' - -export async function POST(req: Request) { - const user = await validateRequest(req) - - if (!canAccessLevel(user.level, 4)) { - return Response.json({ error: 'Forbidden' }, { status: 403 }) - } - - return handleAdminOperation(req) -} -``` - ---- - -## Package System - -Self-contained feature modules with seed data, components, and Lua scripts. - -### Structure - -``` -packages/{name}/ -β”œβ”€β”€ seed/ -β”‚ β”œβ”€β”€ metadata.json # Package info -β”‚ β”œβ”€β”€ components.json # Component definitions -β”‚ β”œβ”€β”€ workflows.ts # Workflow definitions -β”‚ β”œβ”€β”€ schemas.ts # Data schemas -β”‚ └── scripts/ # Lua scripts -β”‚ β”œβ”€β”€ initialize.lua -β”‚ └── helpers.lua -β”œβ”€β”€ src/ # React components (optional) -β”‚ β”œβ”€β”€ components/ -β”‚ β”œβ”€β”€ lib/ -β”‚ └── hooks/ -β”œβ”€β”€ styles/ # SCSS styles -└── static_content/ # Assets (images, templates) -``` - -### Package Metadata - -```json -{ - "name": "@metabuilder/my-feature", - "version": "1.0.0", - "metabuilder": { - "type": "feature", - "minVersion": "1.0.0", - "permissions": ["level3"], - "components": [ - { "name": "MyComponent", "path": "src/components/MyComponent.tsx", "type": "widget" } - ] - } -} -``` - -### Available Packages - -| Package | Description | -|---------|-------------| -| `admin_dialog` | Admin utilities | -| `arcade_lobby` | Games arcade | -| `codegen_studio` | Code generation tools | -| `dashboard` | Dashboard widgets | -| `data_table` | Data table component | -| `form_builder` | Form creation tools | -| `forum_forge` | Forum system | -| `nav_menu` | Navigation menus | -| `notification_center` | Notifications | -| `social_hub` | Social features | -| `spark-tools` | Development tools | -| `stream_cast` | Streaming features | - -### Creating a Package - -```bash -mkdir -p packages/my-feature/seed/scripts -mkdir -p packages/my-feature/src/{components,lib,hooks} -``` - -```typescript -// packages/my-feature/seed/index.ts -export const packageSeed = { - name: '@metabuilder/my-feature', - version: '1.0.0', - components: [], - workflows: [], -} -``` - ---- - -## Database - -### Prisma ORM - -Schema: `prisma/schema.prisma` - -```bash -npm run db:generate # Generate Prisma client -npm run db:push # Apply schema changes -npm run db:migrate # Create migration -npm run db:studio # Open database UI -npm run db:reset # Reset database (destructive!) -``` - -### Schema Example - -```prisma -model User { - id String @id @default(cuid()) - email String @unique - password String - level Int // 1-5 - tenant Tenant @relation(fields: [tenantId], references: [id]) - tenantId String - workflows Workflow[] - - @@unique([email, tenantId]) -} - -model Workflow { - id String @id @default(cuid()) - name String - config String @db.Text - user User @relation(fields: [userId], references: [id]) - userId String - tenantId String -} -``` - -### CRUD Operations - -```typescript -// Create -const newUser = await prisma.user.create({ - data: { - email: 'user@example.com', - password: hashPassword('password'), - level: 2, - tenant: { connect: { id: tenantId } }, - }, -}) - -// Read -const workflows = await prisma.workflow.findMany({ - where: { tenantId: user.tenantId, status: 'active' }, - orderBy: { createdAt: 'desc' }, - take: 10, -}) - -// Update -const updated = await prisma.user.update({ - where: { id: userId }, - data: { level: 3 }, -}) - -// Delete -await prisma.user.delete({ where: { id: userId } }) - -// Transaction -const result = await prisma.$transaction(async (tx) => { - const user = await tx.user.create({ data: userData }) - const workflow = await tx.workflow.create({ - data: { ...workflowData, userId: user.id } - }) - return { user, workflow } -}) -``` - -### DBAL (Database Abstraction Layer) - -For complex operations: - -- **TypeScript** (`dbal/development/`): Fast iteration, development -- **C++ Daemon** (`dbal/production/`): Production security, credential protection - -```typescript -import { dbalQuery } from '@/lib/database-dbal.server' - -const result = await dbalQuery({ - operation: 'transfer_workflow', - from: user1Id, - to: user2Id, -}) -``` - ---- - -## Multi-Tenant System - -Complete isolation with access control, quotas, and namespace separation. - -### Initialize Tenant - -```typescript -import { InMemoryTenantManager, TenantAwareBlobStorage } from './dbal/development/src' - -const tenantManager = new InMemoryTenantManager() - -await tenantManager.createTenant('tenant-123', { - maxBlobStorageBytes: 1024 * 1024 * 1024, // 1GB - maxBlobCount: 10000, - maxRecords: 100000, -}) -``` - -### Tenant-Aware Storage - -```typescript -const tenantStorage = new TenantAwareBlobStorage( - baseStorage, - tenantManager, - 'tenant-123', - 'user-456' -) - -// Automatically scoped to tenant namespace -await tenantStorage.upload('documents/report.pdf', pdfBuffer) -// Actual key: tenants/tenant-123/documents/report.pdf -``` - -### Key-Value Store - -```typescript -const kvStore = new InMemoryKVStore() -const context = await tenantManager.getTenantContext('tenant-123', 'user-456') - -await kvStore.set('config:theme', 'dark', context) -await kvStore.set('user:profile', { name: 'John' }, context) - -// Lists -await kvStore.listAdd('tasks', ['Task 1', 'Task 2'], context) -const tasks = await kvStore.listGet('tasks', context) -``` - -### Access Control - -```typescript -tenantManager.setUserRole('tenant-123', 'user-admin', 'admin') -tenantManager.setUserRole('tenant-123', 'user-viewer', 'viewer') - -// Viewer can only read -const viewerContext = await tenantManager.getTenantContext('tenant-123', 'user-viewer') -await kvStore.get('data', viewerContext) // βœ… Success -await kvStore.set('data', 'new', viewerContext) // ❌ Permission denied -``` - -### Quota Management - -```typescript -const usage = await tenantManager.getUsage('tenant-123') -console.log({ - blobStorage: `${usage.currentBlobStorageBytes} / ${usage.maxBlobStorageBytes}`, - records: `${usage.currentRecords} / ${usage.maxRecords}`, -}) -``` - ---- - -## Lua Scripting - -Business logic without code redeployment. Scripts run in isolated sandbox. - -### Sandbox Restrictions - -- ❌ No `os`, `io`, `require`, `loadfile`, `dofile` -- ❌ No file system access -- ❌ 5-second execution timeout -- βœ… String manipulation, math, tables - -### Example Scripts - -```lua --- Validation -function validateEmail(email) - return string.match(email, "^[^@]+@[^@]+$") ~= nil -end - --- Data transformation -function formatCurrency(amount) - return string.format("$%.2f", amount) -end - --- Business rules -function calculateDiscount(total, memberLevel) - if memberLevel >= 3 then - return total * 0.15 - elseif memberLevel >= 2 then - return total * 0.10 - end - return 0 -end -``` - -### Executing Lua - -```typescript -import { executeLua } from '@/lib/lua-sandbox' - -const result = await executeLua(luaCode, { - db: database, - user: currentUser, - config: packageConfig, -}) -``` - ---- - -## API Reference - -### Authentication - -```typescript -// POST /api/auth/login -{ email: string, password: string } -// Returns: { user: User, token: string } - -// POST /api/auth/register -{ email: string, username: string } -// Returns: { user: User } (password emailed) - -// POST /api/auth/reset-password -{ email: string } -// Returns: { success: boolean } -``` - -### Users - -```typescript -// GET /api/users -// Returns: User[] - -// GET /api/users/:id -// Returns: User - -// PATCH /api/users/:id (Level 3+) -{ level?: number, email?: string } -// Returns: User -``` - -### Packages - -```typescript -// GET /api/packages -// Returns: Package[] - -// POST /api/packages/install (Level 4+) -{ packageId: string } -// Returns: { success: boolean } - -// DELETE /api/packages/:id (Level 4+) -// Returns: { success: boolean } -``` - -### Workflows - -```typescript -// GET /api/workflows -// Returns: Workflow[] - -// POST /api/workflows (Level 4+) -{ name: string, config: object } -// Returns: Workflow - -// POST /api/workflows/:id/execute (Level 4+) -{ input?: object } -// Returns: { result: any } -``` - ---- - -## Testing - -### Commands - -```bash -# From frontends/nextjs/ -npm run test:unit # Vitest unit tests -npm run test:unit -- --run # Run once (no watch) -npm run test:unit -- --coverage # With coverage -npm run test:e2e # Playwright E2E tests -npm run lint # ESLint -npm run typecheck # TypeScript validation -``` - -### Test File Structure - -``` -src/lib/utils.ts β†’ src/lib/utils.test.ts -src/hooks/useKV.ts β†’ src/hooks/useKV.test.ts -src/components/Button.tsx β†’ src/components/Button.test.tsx -``` - -### Parameterized Tests - -```typescript -import { describe, it, expect } from 'vitest' - -describe('validateField', () => { - it.each([ - { field: { type: 'email' }, value: 'test@example.com', shouldError: false }, - { field: { type: 'email' }, value: 'invalid', shouldError: true }, - { field: { type: 'number', min: 0 }, value: -1, shouldError: true }, - ])('validates $field.type with $value', ({ field, value, shouldError }) => { - const result = validateField(field, value) - expect(!!result).toBe(shouldError) - }) -}) -``` - -### React Hook Tests - -```typescript -import { renderHook, act } from '@testing-library/react' - -describe('useIsMobile', () => { - it.each([ - { width: 400, expected: true }, - { width: 768, expected: false }, - ])('returns $expected for width $width', ({ width, expected }) => { - window.innerWidth = width - const { result } = renderHook(() => useIsMobile()) - expect(result.current).toBe(expected) - }) -}) -``` - -### Local CI Testing - -```bash -npm run act # Full CI pipeline locally -npm run act:lint # Test linting only -npm run act:build # Test build only -npm run act:diagnose # Check setup (no Docker) -``` - -#### Act Modes - -| Mode | Command | Description | -|------|---------|-------------| -| Full CI | `npm run act` | Complete workflow | -| Lint only | `npm run act:lint` | Just linting | -| Build only | `npm run act:build` | Just build | -| Diagnose | `npm run act:diagnose` | Check setup | - ---- - -## Development - -### Scripts - -```bash -# Development -npm run dev # Start dev server -npm run build # Production build -npm run preview # Preview production build - -# Quality -npm run lint # Check code -npm run lint:fix # Auto-fix issues -npm run typecheck # TypeScript check -``` - -### Code Conventions - -| Rule | Details | -|------|---------| -| **UI Library** | Material-UI (`@mui/material`) with `sx` prop | -| **No Tailwind** | Use MUI or `.module.scss` | -| **No Radix UI** | MUI equivalents only | -| **One lambda per file** | Classes only as containers | -| **Imports** | Absolute `@/` paths | -| **Components** | Functional with hooks | -| **Tests** | 1:1 source-to-test naming | - -### Component Example - -```tsx -import { Box, Button } from '@mui/material' - -interface MyComponentProps { - title: string - onAction: () => void -} - -export const MyComponent = ({ title, onAction }: MyComponentProps) => { - return ( - -

{title}

- -
- ) -} -``` - -### Generic Component Rendering - -```tsx -// βœ… Declarative - - -// ❌ Hardcoded - -``` - ---- - -## Security - -### Password Hashing - -SHA-512 for all credentials: - -```typescript -import { hashPassword, verifyPassword } from '@/lib/password-utils' - -const hash = hashPassword('mypassword') -const isValid = verifyPassword('mypassword', hash) -``` - -### Lua Sandbox - -Scripts cannot access system resources: - -```lua --- ❌ These are blocked -os.execute('rm -rf /') -io.open('/etc/passwd') -require('socket') - --- βœ… These work -local result = string.match(input, pattern) -local sum = a + b -``` - -### Security Scanning - -Code is automatically scanned for: - -| Category | Patterns | -|----------|----------| -| JavaScript | `eval()`, `innerHTML`, XSS patterns, prototype pollution | -| Lua | `os`/`io` module usage, infinite loops, global manipulation | -| JSON | `__proto__` injection, script tags | - -Critical severity blocks execution. High severity requires acknowledgment. - -### Best Practices - -- βœ… Always filter queries by `tenantId` -- βœ… Validate all user input -- βœ… Use transactions for related operations -- βœ… Hash passwords with SHA-512 -- ❌ Never trust client-side permission checks -- ❌ Don't allow users to modify their own level -- ❌ Don't store secrets in database - ---- - -## Deployment - -### Docker - -```bash -# Development -docker-compose -f deployment/docker-compose.development.yml up - -# Production -docker-compose -f deployment/docker-compose.production.yml up -``` - -### Environment Variables - -```env -# Development (SQLite) -DATABASE_URL="file:./dev.db" - -# Production (PostgreSQL) -DATABASE_URL="postgresql://user:password@host:5432/metabuilder" - -# Optional -NODE_ENV=production -NEXT_PUBLIC_API_URL=https://api.example.com -``` - -### Production Build - -```bash -npm run build -npm run start -``` - ---- - -## Contributing - -### Reporting Issues - -MetaBuilder uses structured issue templates to ensure quality bug reports and feature requests: - -- **πŸ› Bug Report**: Report bugs with environment details and reproduction steps -- **✨ Feature Request**: Propose features aligned with data-driven architecture -- **πŸ“š Documentation**: Request documentation improvements -- **πŸ“¦ Package Request**: Propose new packages for the package system -- **πŸ”§ DBAL Issue**: Report Database Abstraction Layer issues - -See [`.github/TEMPLATES.md`](.github/TEMPLATES.md) for detailed guidance on using templates. - -### Workflow - -1. Follow `.github/copilot-instructions.md` and `.github/prompts/0-kickstart.md` -2. Run `npm run lint:fix` before committing -3. Add parameterized tests for new functions -4. Commit with descriptive messages on `main` - -### Before Committing - -```bash -npm run lint -npm run typecheck -npm run test:unit -- --run -``` - -### Code Quality Targets - -- 80%+ test coverage on critical paths -- All exported functions have tests -- JSDoc comments on public APIs - -### Pull Request Process - -PRs are optional (trunk-based on `main` is default). When required: - -1. Use the comprehensive PR template (auto-populated) -2. Complete all sections and checklists -3. Include screenshots for UI changes -4. Run linter and tests -5. Update documentation -6. Add test cases -7. Use descriptive PR title - -See [`.github/TEMPLATES.md`](.github/TEMPLATES.md) for the complete PR checklist and MetaBuilder-specific guidelines. - ---- - -## Troubleshooting - -### Common Issues - -| Problem | Solution | -|---------|----------| -| Port in use | `npm run dev -- --port 5174` | -| Database connection failed | Check `DATABASE_URL`, run `npm run db:push` | -| TypeScript errors | Run `npm run db:generate` | -| Prisma client outdated | Run `npm run db:generate` | -| Build fails | Check `npm run typecheck` output | - -### Debug Commands - -```bash -# View database -npm run db:studio - -# Check migrations -ls prisma/migrations/ - -# Verbose logging -DEBUG=metabuilder:* npm run dev -``` - -### If Something Fails - -| Issue Type | Check | -|------------|-------| -| Build/config | Run sanity check above | -| Next.js app | Check terminal for errors | -| Prisma/DB | Run `npm run db:generate && npm run db:push` | -| Tests | Run `npm run test:unit -- --run` | - ---- - -## Key Files Reference - -| Purpose | Location | -|---------|----------| -| App source | `frontends/nextjs/src/` | -| Database schema | `prisma/schema.prisma` | -| Package seeds | `packages/*/seed/` | -| DBAL TypeScript | `dbal/development/src/` | -| DBAL C++ | `dbal/production/src/` | -| E2E tests | `frontends/nextjs/e2e/` | -| Shared config | `config/` | -| Analysis tools | `tools/analysis/` | -| Security tools | `tools/security/` | -| Development prompts | `.github/prompts/` | -| CI workflows | `.github/workflows/` | - ---- - -## Roadmap - -### Current Phase: Foundation & Migration - -| Status | Milestone | Target | -|--------|-----------|--------| -| βœ… | Multi-tenant architecture | Complete | -| βœ… | 6-level permission system | Complete | -| βœ… | Package system with JSON/Lua | Complete | -| βœ… | DBAL TypeScript SDK | Complete | -| πŸ”„ | Next.js to Lua conversion | In Progress | -| πŸ”„ | C++ DBAL daemon (production) | In Progress | -| ⏳ | Full Lua business logic | Q1 2025 | -| ⏳ | Package marketplace | Q2 2025 | - -### Future Goals - -- **Q1 2025**: Complete Lua conversion for 95% of frontend logic -- **Q2 2025**: Launch package marketplace for community packages -- **Q3 2025**: Self-hosted deployment templates (Docker, K8s) -- **Q4 2025**: Enterprise features (SSO, audit logs, advanced analytics) - -### Contributing to Roadmap - -See the [Refactor Plan](#refactor-plan) section for detailed phase breakdowns and how to contribute. - ---- - -## License - -MIT License - See LICENSE file +# MetaBuilder + +A **data-driven, multi-tenant platform** where 95% of functionality lives in JSON/Lua, not TypeScript. Build enterprise applications declaratively with a 6-level permission system. + +> πŸ“‹ Project kanban population guide moved; search the repository for `populate-kanban` tools. + +--- + +## Table of Contents + +1. [Quick Start](#quick-start) +2. [Quick Overview](#quick-overview) +3. [Refactor Plan](#refactor-plan) +4. [Architecture](#architecture) +5. [6-Level Permission System](#6-level-permission-system) +6. [Package System](#package-system) +7. [Database](#database) +8. [Multi-Tenant System](#multi-tenant-system) +9. [Lua Scripting](#lua-scripting) +10. [API Reference](#api-reference) +11. [Testing](#testing) +12. [Development](#development) +13. [Security](#security) +14. [Deployment](#deployment) +15. [Contributing](#contributing) +16. [Troubleshooting](#troubleshooting) +17. [Key Files Reference](#key-files-reference) +18. [Roadmap](#roadmap) + +--- + +## Quick Start + +```bash +# Clone and install +git clone +cd metabuilder/frontends/nextjs +npm install + +# Set up database +npm run db:generate +npm run db:push + +# Start development +npm run dev +``` + +Visit `http://localhost:3000` + +### Prerequisites + +- Node.js 18+ +- npm or yarn +- Docker (optional) + +### 15-Minute Sanity Check + +```bash +cd frontends/nextjs +npm ci +npm run typecheck +npm run lint +npm run test:unit -- --run +npm run build +``` + +--- + +## Quick Overview + +MetaBuilder is a **data-driven, multi-tenant platform** with these core features: + +- **6-Level Permission System**: Public β†’ User β†’ Moderator β†’ Admin β†’ God β†’ Supergod hierarchy +- **Multi-Tenant Data**: All queries filter by `tenantId`; each tenant has isolated config +- **Declarative Packages**: Ship UI/features via JSON/Lua in `packages/*/seed/` +- **Lua Sandbox**: Safe runtime execution with blocked system access +- **Type-Safe Workflow**: TypeScript, Act local CI, comprehensive testing + +--- + +## Refactor Plan + +### πŸš€ Auto Code Extractor 3000β„’ - Automated File Splitting + +**NEW: One-command solution to split large files (>150 LOC) into modular structure!** + +We have 62 files exceeding 150 lines. The Auto Code Extractor 3000β„’ automatically extracts functions into individual files following the lambda-per-file pattern. + +#### Quick Commands + +```bash +# Preview what will be extracted +npm run extract:preview + +# Extract 5 files +npm run extract:quick + +# Extract all high-priority files (automated) +npm run extract:auto +``` + +**πŸ“– Quick start**: see repository `tools/` or search for `extract:*` scripts in `config/package.json` for refactoring utilities. + +--- + +### Next.js to Lua Conversion TODO + +#### Table of Contents + +- [TODOs by Phase](#todos-by-phase) +- [Prep Checklist](#prep-checklist) +- [Audit Results](#audit-results) +- [Quick File References](#quick-file-references) +- [Next Steps](#next-steps) +- [Phase Deliverables](#phase-deliverables) +- [Support Notes](#support-notes) +- [Tracking & Templates](#tracking--templates) +- [Package Design Notes](#package-design-notes) +- [Metadata Design Notes](#metadata-design-notes) + +#### TODOs by Phase + +##### Phase 0 – Foundation + +- [ ] Audit all `frontends/nextjs/src/` entry points (pages, components, hooks, lib). Determine what goes to Lua. +- [ ] Define the Lua boundary. List metadata, static, and seed assets plus their API surface. + +##### Phase 1 – Mapping + +- [ ] Map reusable TypeScript helpers to single-function Lua targets (one file, one function) and describe their inputs/outputs for future porting. +- [ ] Categorize each file as β€œport to Lua”, β€œadapter only”, or β€œkeep in TypeScript”. + +##### Phase 2 – Framework Bridge + +- [ ] Extend the core framework only where gaps remain (data loading, metadata resolution, routing) with tiny TypeScript adapters to Lua if needed. +- [ ] Align metadata/seed folder structure with the requirements (consider `lua/` or `packages/static_content`) and add placeholder files. + +##### Phase 3 – Pilot Conversion + +- [ ] Pilot a TypeScript β†’ Lua helper conversion (choose one small helper, wrap it with an adapter, and keep the rest of the build intact). +- [ ] Ensure the pilot is wired into the Next.js app without breaking the current build or tests. + +##### Phase 4 – Stabilize + +- [ ] Update documentation/tests to reflect the new Lua metadata focus and ensure existing CI steps continue to pass with the new structure. +- [ ] Iterate with feedbackβ€”confirm the team agrees on the folder layout, metadata ownership, and which TypeScript files stay as adapters. + +#### Prep Checklist + +- [ ] Confirm tooling (Lua runtime, `luacheck`, etc.) is installed if needed before creating helpers. +- [ ] Backup critical TypeScript files via comments or notes before starting ports. +- [ ] Share the plan with the team and gather quick approvals for folder names (`lua/`, `packages/static_content`). +- [ ] Identify one β€œsafe” helper to convert first, ideally small and well-tested. + +#### Audit Results + +##### App surface + +- `frontends/nextjs/src/app` has 40+ routes. +- Each route needs metadata. +- Key files: + - `page.tsx` + - `layout.tsx` + - `levels` tree + - auth/dashboards + - `providers.tsx` and theme hooks + - `packages/[...path]/route.ts` +- API handlers touch auth, users, packages, levels, power-transfers, GitHub actions, screenshot, and health. +- They depend on helpers under `frontends/nextjs/src/lib/api`, `lib/packages/loader`, `db`, and `dbal`. + +##### UI components + +- `frontends/nextjs/src/components` holds ~60 React files. +- Builders: + - `Builder` + - `Level1`, `Level2`, `Level3`, `Level4`, `Level5` + - `NerdModeIDE` +- Editors: + - `LuaEditor` + - `LuaBlocksEditor` + - `CodeEditor` + - `JsonEditor` + - `SchemaEditor`s +- Management views: + - `PackageManager` + - `DatabaseManager` + - `ComponentCatalog` + - `WorkflowEditor` +- Integrations and tools: + - `GitHubActionsFetcher` + - `IRCWebchat` variants + - `DBALDemo` + - `UnifiedLogin` +- Atoms, molecules, and organisms live in nested folders. We will tag each file in Phase 1. + +##### Hooks & libraries + +- Hooks: + - `useAuth` + - `useAutoRefresh` + - `useKV` + - `useLevelRouting` + - `useDBAL` + - `useFileTree` + - `useCodeEditor` + - `useResolvedUser` + - `useGitHubFetcher` + - `use-mobile` + - `hooks/auth` store + - `hooks/use-dbal` helpers (blob storage, kv store, cached data) +- Libraries cover: + - auth and security (`lib/auth`, `lib/security`) + - database and DBAL helpers + - package, seed, and metadata tooling (`lib/packages`, `lib/seed`) + - Lua engine and functions + - workflow and rendering helpers + - API helpers and schema utilities + - GitHub and general utilities (`lib/utils`, `lib/prisma`) +- Phase 1 will tag each file as β€œadapter” or β€œport”. + +##### Supporting assets + +- `frontends/nextjs/src/seed-data` has JSON seeds for packages and baseline data. These can move to metadata. +- Tests live in `app/api/*/*.test.tsx`, `lib/**/*test.ts`, `components/get-component-icon.test.tsx`, and `hooks/*.test.ts`. We will decide which tests follow Lua stubbed logic or stay with TypeScript. + +#### Quick File References + +- `frontends/nextjs/src/app`: each route, page, provider, and API handler needs metadata or a Lua adapter. +- `frontends/nextjs/src/components`: move builders, editors, and managers to metadata-driven definitions. +- `frontends/nextjs/src/hooks`: turn each hook into a Lua micro-function. +- `frontends/nextjs/src/lib`: catalogue auth, security, db/DBAL, packages, Lua, workflow, API, schema, GitHub, and utility modules. +- Supporting assets: seed JSONs, README docs, and tests must align with the Lua transition. + +#### God Panel / Package Mapping Notes + +- Levelβ€―4’s `Level4Tabs` exposes a β€œPackages” tab that renders `PackageManager`, so God-level users open that tab when they need to install, enable, or disable packages (`frontends/nextjs/src/components/level4/Level4Tabs.tsx`). +- `PackageManager` loads the catalog from `PACKAGE_CATALOG`, shows installed/available packages, and calls APIs (`installPackage`, `togglePackageEnabled`, `uninstallPackage`, `listInstalledPackages`) that persist package state (`frontends/nextjs/src/components/PackageManager.tsx`). +- `PACKAGE_CATALOG` lives in `frontends/nextjs/src/lib/packages/package-catalog.ts` and contains the manifest content (schemas, pages, workflows, Lua scripts, seed data) that the God panel presents when a package is selected. +- The reusable packages under `packages/` follow the structure described in `packages/README.md`: each package has `seed/metadata.json`, `components.json`, and optional `static_content/examples.json`, so new package metadata can align with what `PackageManager` expects. +- Default packages are enumerated in `frontends/nextjs/src/lib/packages/package-glue/default-packages.ts` (admin_dialog, dashboard, forum_forge, etc.), and God panel UI relies on those definitions to match catalog entries with metadata stored in `packages/`. + +#### Next Steps + +1. Create the companion metadata folder (e.g., `lua/` or `packages/static_content`) with placeholder single-function files that mirror the above references. +2. Run the audit across the identified folders. Mark each file as β€œport to Lua”, β€œadapter only”, or β€œleave in TypeScript” before editing. + +#### Phase Deliverables + +- **Phase 0**: Audit spreadsheet/list with tags for each entry. Lua boundary doc with endpoints and metadata goals. +- **Phase 1**: Mapping sheet (spreadsheet or Markdown) documenting TypeScript helper β†’ Lua micro-function mapping and file categorization. +- **Phase 2**: Adapter interface stubs plus placeholder Lua files under the new metadata folder. +- **Phase 3**: One working Lua helper wired through an adapter, with updated unit test coverage or manual verification notes. +- **Phase 4**: Documentation updates, test run results, and a summary of which files moved versus stayed. + +#### Support Notes + +- **Risks**: The Next.js frontend is large; do not remove TypeScript pages until their Lua replacements exist. Keep unit/e2e tests running in TypeScript during transition. +- **Owner**: Track who updates each phase and keep a note in this doc or comments for accountability. +- **Validation tips**: After a Lua helper is added, run `npm run lint` and `npm run test:unit` from `frontends/nextjs/`. Keep logs of failing tests if you defer them. +- **Documentation**: Update READMEs in `components/`, `hooks/`, and `lib/` with pointers to the new Lua package once core content moves. +- **Checkpoint**: Before Phase 3, ensure adapter interfaces exist for data access and that seed data has been migrated to the metadata folder. + +- **Toolbelt** + - `npm run lint` (in `frontends/nextjs/`) + - `npm run test:unit` + - `npm run test:e2e` (only after Lua helpers are wired) + - `npm run build` to verify core app still compiles + +- **Folder sketch** + - `lua/metadata/` – metadata + seed descriptors per package + - `lua/functions/` – single-function files (one helper per file) + - `lua/adapters/` – TypeScript adapters that relay calls into Lua micro-functions + - `packages/static_content/` – optional storage for JSON seeds already living in the frontend + +#### Tracking & Templates + +- **Audit table**: + 1. File path + 2. Current role (page, API, hook, lib) + 3. Port target (Lua, adapter, stay TS) + 4. Notes/risks + 5. Owner/ETA +- **Mapping sheet**: + 1. Helper name + 2. Inputs/outputs + 3. Proposed Lua file path + 4. Adapter needed (yes/no) + 5. Test expectations + +### Lua Conversion & God Panel Mapping + +- God-level builders (Levelβ€―4 tabs) expose the `PackageManager` UI where admins install, enable, and disable packages (`Level4Tabs` β†’ `PackageManager` in `frontends/nextjs/src/components`). +- `PackageManager` loads `PACKAGE_CATALOG`, drives the install/toggle APIs, and feeds catalog entries back into the God panel (`frontends/nextjs/src/lib/packages/package-catalog.ts` and `frontends/nextjs/src/components/PackageManager.tsx`). +- Catalog metadata mirrors the `packages/*/seed/metadata.json` format described in `packages/README.md`; new packages must follow that structure (metadata, components, optional static content) so the God panel can surface them. +- Default package metadata is pre-defined under `frontends/nextjs/src/lib/packages/package-glue/default-packages.ts` (admin_dialog, dashboard, forum_forge, etc.) to keep UI, Lua loaders, and catalog definitions aligned. + +#### Package Design Notes + +- Keep packages lean: each folder under `packages/` should contain `seed/`, optional `static_content/`, and light JSON/metadata entries. +- Store metadata (name, version, packageId) in `seed/metadata.json`; keep `packageId` snake_case and version semver. +- Avoid coupling packages to UI; they should expose data scripts for Lua/metadata consumers. +- Use `package/glue` logic in `frontends/nextjs/src/lib/packages` as reference when designing new packages so metadata matches runtime expectations. +- Document every package’s dependencies and optional scripts so Lua adapters can load/distribute them predictably. + +#### Metadata Design Notes + +- Define metadata per route/component via JSON tables describing inputs, outputs, Lua helper references, and UI states. +- Keep metadata separate from implementation; store static descriptors in `lua/metadata/*` and reference them via adapters. +- Version metadata files to make rollbacks easier and ensure seeds stay synchronized with database migrations. +- Use small, focused metadata files so the single-function Lua helpers can import only the data they need. +- Track metadata ownership (which team, file) inside each descriptor to simplify future audits. + +#### Reference Resources + +- **AGENTS.md**: follow repo guidelines and nested agent notes before touching `dbal/` or other special folders. +- **UI_STANDARDS.md**: reference for UI design rules while documenting Lua metadata that may affect styling. +- **packages/**: inspect existing JSON-driven package structure for inspiration (seed/metadata.json, static_content/). +- **tools/**: reuse scripts from this folder when building new metadata loaders or conversion helpers. +- **frontends/nextjs/src/lib/packages/package-glue/**: examine default packages and loaders to keep Lua metadata compatible. + +#### Decision Log + +- Document every key decision (phase changes, Lua structure choices, package additions) below with date/owner for easy onboarding. + 1. [date] - [owner] - [decision summary] + 2. ... + +#### Collaboration Notes + +- Post updates to this doc before merging any changes, especially new folder names or metadata formats. +- Tag @owner in relevant PR comments when handing off a phase or requesting review. + +## Architecture + +MetaBuilder combines: + +- **Declarative Components**: Render UIs from JSON using `RenderComponent` +- **DBAL**: TypeScript SDK + C++ daemon, language-agnostic via YAML contracts +- **Package System**: Self-contained modules in `/packages/{name}/seed/` +- **Multi-Tenancy**: All queries filter by `tenantId` + +### Project Structure + +``` +metabuilder/ +β”œβ”€β”€ frontends/nextjs/ # Next.js application +β”‚ β”œβ”€β”€ src/ +β”‚ β”‚ β”œβ”€β”€ app/ # App routes and pages +β”‚ β”‚ β”œβ”€β”€ components/ # React components (Atomic Design) +β”‚ β”‚ β”‚ β”œβ”€β”€ atoms/ # Basic UI elements +β”‚ β”‚ β”‚ β”œβ”€β”€ molecules/ # Simple composites (2-5 atoms) +β”‚ β”‚ β”‚ β”œβ”€β”€ organisms/ # Complex features with business logic +β”‚ β”‚ β”‚ └── level1-5/ # Page-level components +β”‚ β”‚ β”œβ”€β”€ lib/ # Core libraries +β”‚ β”‚ β”œβ”€β”€ hooks/ # Custom React hooks +β”‚ β”‚ β”œβ”€β”€ seed-data/ # Database seeds +β”‚ β”‚ β”œβ”€β”€ theme/ # MUI theme configuration +β”‚ β”‚ └── types/ # TypeScript type definitions +β”‚ └── e2e/ # Playwright E2E tests +β”œβ”€β”€ packages/ # Feature packages (JSON/Lua-driven) +β”œβ”€β”€ dbal/ # Database Abstraction Layer +β”‚ β”œβ”€β”€ ts/ # TypeScript implementation +β”‚ β”œβ”€β”€ cpp/ # C++ daemon (production) +β”‚ β”‚ β”œβ”€β”€ src/ # Source code +β”‚ β”‚ β”œβ”€β”€ include/ # Header files +β”‚ β”‚ β”œβ”€β”€ build-config/ # CMake, Docker configs +β”‚ β”‚ β”œβ”€β”€ lint-config/ # Clang format/tidy configs +β”‚ β”‚ └── docs/ # Implementation docs +β”‚ β”œβ”€β”€ api/ # YAML contracts +β”‚ └── docs/ # DBAL documentation +β”œβ”€β”€ prisma/ # Database schema & migrations +β”œβ”€β”€ config/ # Shared configuration +β”‚ β”œβ”€β”€ build/ # Next.js, TypeScript configs +β”‚ β”œβ”€β”€ lint/ # ESLint config +β”‚ β”œβ”€β”€ test/ # Vitest, Playwright configs +β”‚ └── misc/ # Other configs (JSON, env) +β”œβ”€β”€ tools/ # Repository utilities +β”‚ β”œβ”€β”€ analysis/ # Code analysis scripts +β”‚ β”œβ”€β”€ detection/ # Stub/pattern detection +β”‚ β”œβ”€β”€ generation/ # Code generation +β”‚ β”œβ”€β”€ quality/ # Quality metrics +β”‚ β”œβ”€β”€ security/ # Security scanning +β”‚ └── validation/ # Validation scripts +β”œβ”€β”€ docs/ # Documentation +β”‚ β”œβ”€β”€ architecture/ # System design docs +β”‚ β”œβ”€β”€ guides/ # How-to guides +β”‚ β”œβ”€β”€ security/ # Security documentation +β”‚ β”œβ”€β”€ todo/ # Task tracking +β”‚ └── navigation/ # Doc navigation files +β”œβ”€β”€ deployment/ # Docker configs +└── .github/ # GitHub workflows & prompts + β”œβ”€β”€ workflows/ # CI/CD pipelines + └── prompts/ # AI prompt templates +``` + +### Component Architecture (Atomic Design) + +| Layer | Examples | Rules | +|-------|----------|-------| +| **Atoms** | Button, Input, Badge | No custom dependencies | +| **Molecules** | AppHeader, ProfileCard | Can use Atoms only | +| **Organisms** | SchemaEditor, PackageManager | Can use Atoms, Molecules, other Organisms | +| **Pages** | Level1-5 | Can use all | + +--- + +## 6-Level Permission System + +Hierarchical access control where each level inherits all permissions from lower levels: + +| Level | Role | Access | Route | +|-------|------|--------|-------| +| 1 | Public | Read-only, unauthenticated | `/` | +| 2 | User | Personal dashboard, content creation | `/dashboard` | +| 3 | Moderator | Moderation desk, flag review, report handling | `/moderator` | +| 4 | Admin | User management, system settings | `/admin` | +| 5 | God | Workflows, advanced scripting, packages | `/builder` | +| 6 | Supergod | Full system control, tenant management | `/supergod` | + +The new Level 3 moderator desk (`/moderator`) is protected by `AuthGate` with `requiredRole="moderator"` and surfaces flagged discussions, incident reports, and quick actions before escalation to admin. + +### Permission Matrix + +| Feature | L1 | L2 | L3 | L4 | L5 | L6 | +|---------|----|----|----|----|----|----| +| View Public Data | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | +| Authenticate | | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | +| Create Content | | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | +| Moderate Content | | | βœ“ | βœ“ | βœ“ | βœ“ | +| Manage Users | | | | βœ“ | βœ“ | βœ“ | +| Run Workflows | | | | | βœ“ | βœ“ | +| System Configuration | | | | | | βœ“ | + +### Usage + +```typescript +import { canAccessLevel } from '@/lib/auth' +import { useAuth } from '@/hooks' + +export const AdminPanel = () => { + const { user } = useAuth() + + if (!canAccessLevel(user.level, 4)) { + return + } + + return +} + +// Multiple conditional features +const features = { + basicDashboard: true, + contentEditor: user.level >= 2, + moderationDesk: user.level >= 3, + userManagement: user.level >= 4, + workflowEngine: user.level >= 5, + systemConfig: user.level === 6, +} +``` + +### Backend Route Protection + +```typescript +import { validateRequest } from '@/lib/auth' + +export async function POST(req: Request) { + const user = await validateRequest(req) + + if (!canAccessLevel(user.level, 4)) { + return Response.json({ error: 'Forbidden' }, { status: 403 }) + } + + return handleAdminOperation(req) +} +``` + +--- + +## Package System + +Self-contained feature modules with seed data, components, and Lua scripts. + +### Structure + +``` +packages/{name}/ +β”œβ”€β”€ seed/ +β”‚ β”œβ”€β”€ metadata.json # Package metadata (JSON) +β”‚ β”œβ”€β”€ components.json # Component definitions (JSON) +β”‚ β”œβ”€β”€ workflows.json # Workflow definitions (JSON) +β”‚ β”œβ”€β”€ schemas.json # Data schemas (JSON) +β”‚ └── static_content/ # Assets (images, templates) +β”œβ”€β”€ src/ # React components (optional) +β”‚ β”œβ”€β”€ components/ +β”‚ β”œβ”€β”€ lib/ +β”‚ └── hooks/ +β”œβ”€β”€ styles/ # SCSS styles +└── static_content/ # Assets (images, templates) +``` + +### Package Metadata + +Package metadata and seeds are stored as JSON under `seed/` and should include `name`, `version`, `metabuilder` metadata and component/workflow listings. + +Example (seed/metadata.json): + +```json +{ + "name": "@metabuilder/my-feature", + "version": "1.0.0", + "metabuilder": { + "type": "feature", + "minVersion": "1.0.0", + "permissions": ["level3"], + "components": [ + { "name": "MyComponent", "path": "src/components/MyComponent.tsx", "type": "widget" } + ] + } +} +``` + +### Available Packages + +| Package | Description | +|---------|-------------| +| `admin_dialog` | Admin utilities | +| `arcade_lobby` | Games arcade | +| `codegen_studio` | Code generation tools | +| `dashboard` | Dashboard widgets | +| `data_table` | Data table component | +| `form_builder` | Form creation tools | +| `forum_forge` | Forum system | +| `nav_menu` | Navigation menus | +| `notification_center` | Notifications | +| `social_hub` | Social features | + +| `stream_cast` | Streaming features | + +### Creating a Package + +```bash +mkdir -p packages/my-feature/seed/scripts +mkdir -p packages/my-feature/src/{components,lib,hooks} +``` + +```typescript +// packages/my-feature/seed/index.ts +export const packageSeed = { + name: '@metabuilder/my-feature', + version: '1.0.0', + components: [], + workflows: [], +} +``` + +--- + +## Database + +### Prisma ORM + +Schema: `prisma/schema.prisma` + +```bash +npm run db:generate # Generate Prisma client +npm run db:push # Apply schema changes +npm run db:migrate # Create migration +npm run db:studio # Open database UI +npm run db:reset # Reset database (destructive!) +``` + +### Schema Example + +```prisma +model User { + id String @id @default(cuid()) + email String @unique + password String + level Int // 1-5 + tenant Tenant @relation(fields: [tenantId], references: [id]) + tenantId String + workflows Workflow[] + + @@unique([email, tenantId]) +} + +model Workflow { + id String @id @default(cuid()) + name String + config String @db.Text + user User @relation(fields: [userId], references: [id]) + userId String + tenantId String +} +``` + +### CRUD Operations + +```typescript +// Create +const newUser = await prisma.user.create({ + data: { + email: 'user@example.com', + password: hashPassword('password'), + level: 2, + tenant: { connect: { id: tenantId } }, + }, +}) + +// Read +const workflows = await prisma.workflow.findMany({ + where: { tenantId: user.tenantId, status: 'active' }, + orderBy: { createdAt: 'desc' }, + take: 10, +}) + +// Update +const updated = await prisma.user.update({ + where: { id: userId }, + data: { level: 3 }, +}) + +// Delete +await prisma.user.delete({ where: { id: userId } }) + +// Transaction +const result = await prisma.$transaction(async (tx) => { + const user = await tx.user.create({ data: userData }) + const workflow = await tx.workflow.create({ + data: { ...workflowData, userId: user.id } + }) + return { user, workflow } +}) +``` + +### DBAL (Database Abstraction Layer) + +For complex operations: + +- **TypeScript** (`dbal/development/`): Fast iteration, development +- **C++ Daemon** (`dbal/production/`): Production security, credential protection + +```typescript +import { dbalQuery } from '@/lib/database-dbal.server' + +const result = await dbalQuery({ + operation: 'transfer_workflow', + from: user1Id, + to: user2Id, +}) +``` + +--- + +## Multi-Tenant System + +Complete isolation with access control, quotas, and namespace separation. + +### Initialize Tenant + +```typescript +import { InMemoryTenantManager, TenantAwareBlobStorage } from './dbal/development/src' + +const tenantManager = new InMemoryTenantManager() + +await tenantManager.createTenant('tenant-123', { + maxBlobStorageBytes: 1024 * 1024 * 1024, // 1GB + maxBlobCount: 10000, + maxRecords: 100000, +}) +``` + +### Tenant-Aware Storage + +```typescript +const tenantStorage = new TenantAwareBlobStorage( + baseStorage, + tenantManager, + 'tenant-123', + 'user-456' +) + +// Automatically scoped to tenant namespace +await tenantStorage.upload('documents/report.pdf', pdfBuffer) +// Actual key: tenants/tenant-123/documents/report.pdf +``` + +### Key-Value Store + +```typescript +const kvStore = new InMemoryKVStore() +const context = await tenantManager.getTenantContext('tenant-123', 'user-456') + +await kvStore.set('config:theme', 'dark', context) +await kvStore.set('user:profile', { name: 'John' }, context) + +// Lists +await kvStore.listAdd('tasks', ['Task 1', 'Task 2'], context) +const tasks = await kvStore.listGet('tasks', context) +``` + +### Access Control + +```typescript +tenantManager.setUserRole('tenant-123', 'user-admin', 'admin') +tenantManager.setUserRole('tenant-123', 'user-viewer', 'viewer') + +// Viewer can only read +const viewerContext = await tenantManager.getTenantContext('tenant-123', 'user-viewer') +await kvStore.get('data', viewerContext) // βœ… Success +await kvStore.set('data', 'new', viewerContext) // ❌ Permission denied +``` + +### Quota Management + +```typescript +const usage = await tenantManager.getUsage('tenant-123') +console.log({ + blobStorage: `${usage.currentBlobStorageBytes} / ${usage.maxBlobStorageBytes}`, + records: `${usage.currentRecords} / ${usage.maxRecords}`, +}) +``` + +--- + +## Lua Scripting + +Business logic without code redeployment. Scripts run in isolated sandbox. + +### Sandbox Restrictions + +- ❌ No `os`, `io`, `require`, `loadfile`, `dofile` +- ❌ No file system access +- ❌ 5-second execution timeout +- βœ… String manipulation, math, tables + +### Example Scripts + +```lua +-- Validation +function validateEmail(email) + return string.match(email, "^[^@]+@[^@]+$") ~= nil +end + +-- Data transformation +function formatCurrency(amount) + return string.format("$%.2f", amount) +end + +-- Business rules +function calculateDiscount(total, memberLevel) + if memberLevel >= 3 then + return total * 0.15 + elseif memberLevel >= 2 then + return total * 0.10 + end + return 0 +end +``` + +### Executing Lua + +```typescript +import { executeLua } from '@/lib/lua-sandbox' + +const result = await executeLua(luaCode, { + db: database, + user: currentUser, + config: packageConfig, +}) +``` + +--- + +## API Reference + +### Authentication + +```typescript +// POST /api/auth/login +{ email: string, password: string } +// Returns: { user: User, token: string } + +// POST /api/auth/register +{ email: string, username: string } +// Returns: { user: User } (password emailed) + +// POST /api/auth/reset-password +{ email: string } +// Returns: { success: boolean } +``` + +### Users + +```typescript +// GET /api/users +// Returns: User[] + +// GET /api/users/:id +// Returns: User + +// PATCH /api/users/:id (Level 3+) +{ level?: number, email?: string } +// Returns: User +``` + +### Packages + +```typescript +// GET /api/packages +// Returns: Package[] + +// POST /api/packages/install (Level 4+) +{ packageId: string } +// Returns: { success: boolean } + +// DELETE /api/packages/:id (Level 4+) +// Returns: { success: boolean } +``` + +### Workflows + +```typescript +// GET /api/workflows +// Returns: Workflow[] + +// POST /api/workflows (Level 4+) +{ name: string, config: object } +// Returns: Workflow + +// POST /api/workflows/:id/execute (Level 4+) +{ input?: object } +// Returns: { result: any } +``` + +--- + +## Testing + +### Commands + +```bash +# From frontends/nextjs/ +npm run test:unit # Vitest unit tests +npm run test:unit -- --run # Run once (no watch) +npm run test:unit -- --coverage # With coverage +npm run test:e2e # Playwright E2E tests +npm run lint # ESLint +npm run typecheck # TypeScript validation +``` + +### Test File Structure + +``` +src/lib/utils.ts β†’ src/lib/utils.test.ts +src/hooks/useKV.ts β†’ src/hooks/useKV.test.ts +src/components/Button.tsx β†’ src/components/Button.test.tsx +``` + +### Parameterized Tests + +```typescript +import { describe, it, expect } from 'vitest' + +describe('validateField', () => { + it.each([ + { field: { type: 'email' }, value: 'test@example.com', shouldError: false }, + { field: { type: 'email' }, value: 'invalid', shouldError: true }, + { field: { type: 'number', min: 0 }, value: -1, shouldError: true }, + ])('validates $field.type with $value', ({ field, value, shouldError }) => { + const result = validateField(field, value) + expect(!!result).toBe(shouldError) + }) +}) +``` + +### React Hook Tests + +```typescript +import { renderHook, act } from '@testing-library/react' + +describe('useIsMobile', () => { + it.each([ + { width: 400, expected: true }, + { width: 768, expected: false }, + ])('returns $expected for width $width', ({ width, expected }) => { + window.innerWidth = width + const { result } = renderHook(() => useIsMobile()) + expect(result.current).toBe(expected) + }) +}) +``` + +### Local CI Testing + +```bash +npm run act # Full CI pipeline locally +npm run act:lint # Test linting only +npm run act:build # Test build only +npm run act:diagnose # Check setup (no Docker) +``` + +#### Act Modes + +| Mode | Command | Description | +|------|---------|-------------| +| Full CI | `npm run act` | Complete workflow | +| Lint only | `npm run act:lint` | Just linting | +| Build only | `npm run act:build` | Just build | +| Diagnose | `npm run act:diagnose` | Check setup | + +--- + +## Development + +### Scripts + +```bash +# Development +npm run dev # Start dev server +npm run build # Production build +npm run preview # Preview production build + +# Quality +npm run lint # Check code +npm run lint:fix # Auto-fix issues +npm run typecheck # TypeScript check +``` + +### Code Conventions + +| Rule | Details | +|------|---------| +| **UI Library** | Material-UI (`@mui/material`) with `sx` prop | +| **No Tailwind** | Use MUI or `.module.scss` | +| **No Radix UI** | MUI equivalents only | +| **One lambda per file** | Classes only as containers | +| **Imports** | Absolute `@/` paths | +| **Components** | Functional with hooks | +| **Tests** | 1:1 source-to-test naming | + +### Component Example + +```tsx +import { Box, Button } from '@mui/material' + +interface MyComponentProps { + title: string + onAction: () => void +} + +export const MyComponent = ({ title, onAction }: MyComponentProps) => { + return ( + +

{title}

+ +
+ ) +} +``` + +### Generic Component Rendering + +```tsx +// βœ… Declarative + + +// ❌ Hardcoded + +``` + +--- + +## Security + +### Password Hashing + +SHA-512 for all credentials: + +```typescript +import { hashPassword, verifyPassword } from '@/lib/password-utils' + +const hash = hashPassword('mypassword') +const isValid = verifyPassword('mypassword', hash) +``` + +### Lua Sandbox + +Scripts cannot access system resources: + +```lua +-- ❌ These are blocked +os.execute('rm -rf /') +io.open('/etc/passwd') +require('socket') + +-- βœ… These work +local result = string.match(input, pattern) +local sum = a + b +``` + +### Security Scanning + +Code is automatically scanned for: + +| Category | Patterns | +|----------|----------| +| JavaScript | `eval()`, `innerHTML`, XSS patterns, prototype pollution | +| Lua | `os`/`io` module usage, infinite loops, global manipulation | +| JSON | `__proto__` injection, script tags | + +Critical severity blocks execution. High severity requires acknowledgment. + +### Best Practices + +- βœ… Always filter queries by `tenantId` +- βœ… Validate all user input +- βœ… Use transactions for related operations +- βœ… Hash passwords with SHA-512 +- ❌ Never trust client-side permission checks +- ❌ Don't allow users to modify their own level +- ❌ Don't store secrets in database + +--- + +## Deployment + +### Docker + +```bash +# Development +docker-compose -f deployment/docker-compose.development.yml up + +# Production +docker-compose -f deployment/docker-compose.production.yml up +``` + +### Environment Variables + +```env +# Development (SQLite) +DATABASE_URL="file:./dev.db" + +# Production (PostgreSQL) +DATABASE_URL="postgresql://user:password@host:5432/metabuilder" + +# Optional +NODE_ENV=production +NEXT_PUBLIC_API_URL=https://api.example.com +``` + +### Production Build + +```bash +npm run build +npm run start +``` + +--- + +## Contributing + +### Reporting Issues + +MetaBuilder uses structured issue templates to ensure quality bug reports and feature requests: + +- **πŸ› Bug Report**: Report bugs with environment details and reproduction steps +- **✨ Feature Request**: Propose features aligned with data-driven architecture +- **πŸ“š Documentation**: Request documentation improvements +- **πŸ“¦ Package Request**: Propose new packages for the package system +- **πŸ”§ DBAL Issue**: Report Database Abstraction Layer issues + +See [`.github/TEMPLATES.md`](.github/TEMPLATES.md) for detailed guidance on using templates. + +### Workflow + +1. Follow `.github/copilot-instructions.md` and `.github/prompts/0-kickstart.md` +2. Run `npm run lint:fix` before committing +3. Add parameterized tests for new functions +4. Commit with descriptive messages on `main` + +### Before Committing + +```bash +npm run lint +npm run typecheck +npm run test:unit -- --run +``` + +### Code Quality Targets + +- 80%+ test coverage on critical paths +- All exported functions have tests +- JSDoc comments on public APIs + +### Pull Request Process + +PRs are optional (trunk-based on `main` is default). When required: + +1. Use the comprehensive PR template (auto-populated) +2. Complete all sections and checklists +3. Include screenshots for UI changes +4. Run linter and tests +5. Update documentation +6. Add test cases +7. Use descriptive PR title + +See [`.github/TEMPLATES.md`](.github/TEMPLATES.md) for the complete PR checklist and MetaBuilder-specific guidelines. + +--- + +## Troubleshooting + +### Common Issues + +| Problem | Solution | +|---------|----------| +| Port in use | `npm run dev -- --port 5174` | +| Database connection failed | Check `DATABASE_URL`, run `npm run db:push` | +| TypeScript errors | Run `npm run db:generate` | +| Prisma client outdated | Run `npm run db:generate` | +| Build fails | Check `npm run typecheck` output | + +### Debug Commands + +```bash +# View database +npm run db:studio + +# Check migrations +ls prisma/migrations/ + +# Verbose logging +DEBUG=metabuilder:* npm run dev +``` + +### If Something Fails + +| Issue Type | Check | +|------------|-------| +| Build/config | Run sanity check above | +| Next.js app | Check terminal for errors | +| Prisma/DB | Run `npm run db:generate && npm run db:push` | +| Tests | Run `npm run test:unit -- --run` | + +--- + +## Key Files Reference + +| Purpose | Location | +|---------|----------| +| App source | `frontends/nextjs/src/` | +| Database schema | `prisma/schema.prisma` | +| Package seeds | `packages/*/seed/` | +| DBAL TypeScript | `dbal/development/src/` | +| DBAL C++ | `dbal/production/src/` | +| E2E tests | `e2e/` | +| Shared config | `config/` | +| Analysis tools | `tools/analysis/` | +| Security tools | `tools/security/` | +| Development prompts | `.github/prompts/` | +| CI workflows | `.github/workflows/` | + +--- + +## Roadmap + +### Current Phase: Foundation & Migration + +| Status | Milestone | Target | +|--------|-----------|--------| +| βœ… | Multi-tenant architecture | Complete | +| βœ… | 6-level permission system | Complete | +| βœ… | Package system with JSON/Lua | Complete | +| βœ… | DBAL TypeScript SDK | Complete | +| πŸ”„ | Next.js to Lua conversion | In Progress | +| πŸ”„ | C++ DBAL daemon (production) | In Progress | +| ⏳ | Full Lua business logic | Q1 2025 | +| ⏳ | Package marketplace | Q2 2025 | + +### Future Goals + +- **Q1 2025**: Complete Lua conversion for 95% of frontend logic +- **Q2 2025**: Launch package marketplace for community packages +- **Q3 2025**: Self-hosted deployment templates (Docker, K8s) +- **Q4 2025**: Enterprise features (SSO, audit logs, advanced analytics) + +### Contributing to Roadmap + +See the [Refactor Plan](#refactor-plan) section for detailed phase breakdowns and how to contribute. + +--- + +## License + +MIT License - See LICENSE file diff --git a/spec/README.md b/spec/README.md index c3c53695d..86419f093 100644 --- a/spec/README.md +++ b/spec/README.md @@ -365,9 +365,9 @@ ExecuteWorkflow(user, workflow) == ``` ## Architecture Alignment - + `storybook/JSON_PACKAGES.md` β†’ JSON package format and discovery This specification aligns with the MetaBuilder architecture documentation: - + `storybook/JSON_PACKAGES.md` β†’ JSON package format and discovery - `docs/architecture/security-docs/5-level-system.md` β†’ Permission model (extended to 6 levels) - `schemas/package-schemas/` β†’ **16+ JSON schemas defining package structure** - `docs/packages/package-sources.md` β†’ **Multi-source package loading architecture** diff --git a/spec/SUMMARY.md b/spec/SUMMARY.md index e3fb87c06..b7a3eecf6 100644 --- a/spec/SUMMARY.md +++ b/spec/SUMMARY.md @@ -1,258 +1,258 @@ -# TLA+ Specifications Summary - -## Overview - -This directory contains formal TLA+ specifications for MetaBuilder, including both the current core system and three major future functionality areas. All specifications have been validated with TLA+ tools. - -## What Was Created - -### New Specifications (December 2025) - -Three comprehensive TLA+ modules for future functionality: - -| Specification | Lines | Operations | Properties | Status | -|---------------|-------|------------|------------|--------| -| `workflow_system.tla` | 595 | 22 | 12 (9 safety + 3 liveness) | βœ… Validated | -| `collaboration.tla` | 546 | 26 | 13 (10 safety + 3 liveness) | βœ… Validated | -| `integrations.tla` | 644 | 28 | 13 (10 safety + 3 liveness) | βœ… Validated | -| **Total** | **1,785** | **76** | **38** | | - -### Supporting Files - -- **Configuration Files**: 3 `.cfg` files for model checking -- **Documentation**: Updated `README.md` with comprehensive module documentation -- **Quick Reference**: `QUICK_REFERENCE.md` for developer workflows -- **Validation Script**: `validate-specs.sh` for automated syntax checking -- **Known Issues**: `KNOWN_ISSUES.md` documenting pre-existing core spec issue - -## Specifications Detail - -### 1. Workflow System (`workflow_system.tla`) - -**Purpose**: Formal specification for advanced workflow execution features (from TODO 20). - -**Key Features**: -- Template lifecycle (create, activate, archive) -- Manual and scheduled execution -- Step dependencies with parallel execution -- Configurable retry logic (MaxRetries) -- Per-tenant concurrency limits (MaxConcurrentRuns) - -**Safety Properties**: -1. GodOnlyTemplateCreation - Only Level 5+ can author templates -2. AdminOnlyExecution - Only Level 4+ can execute workflows -3. TenantIsolation - Workflows isolated by tenant -4. ConcurrencyLimit - Per-tenant execution limits enforced -5. RetryLimit - Failed steps don't exceed retry threshold -6. NoOverlap - Running and completed workflows don't overlap -7. DependencyEnforcement - Step dependencies respected -8. AuditCompleteness - All operations logged - -**Liveness Properties**: -1. EventualStepExecution - Pending steps eventually execute -2. EventualCompletion - Running workflows eventually complete/fail -3. EventualScheduleTrigger - Scheduled workflows eventually trigger - -**Model Checking Configuration**: -- 3 users, 2 tenants, 3 workflow templates, 3 steps -- Max 3 retries, max 2 concurrent runs per tenant -- Bounded to 15 audit log entries, 5 instances - -### 2. Collaboration System (`collaboration.tla`) - -**Purpose**: Formal specification for real-time collaboration features (from TODO 20). - -**Key Features**: -- User session states (active, idle, disconnected) -- Concurrent document editing -- Comments and @mentions -- Real-time presence tracking -- Notification system -- Document version history - -**Safety Properties**: -1. DocumentTenantIsolation - Editors only access own tenant docs -2. ConcurrentEditorLimit - Max concurrent editors enforced -3. CommentTenantConsistency - Comments belong to doc tenant -4. MentionTenantIsolation - Mentions within tenant boundaries -5. NotificationLimit - Pending notifications don't exceed limit -6. ActiveEditorsOnly - Only active users can edit -7. DisconnectedNotEditing - Disconnected users auto-stop editing -8. OperationAuthorship - All ops from authorized editors -9. VersionConsistency - Version snapshots preserve content - -**Liveness Properties**: -1. EventualNotificationHandling - Notifications eventually read/cleared -2. EventualMentionRead - Mentions eventually seen -3. EventualStopEditing - Active editors eventually stop/disconnect - -**Model Checking Configuration**: -- 3 users, 2 tenants, 2 documents -- Max 3 concurrent editors, max 10 notifications -- Bounded to 5 comments, 5 mentions, 12 activity feed entries - -### 3. Integration Ecosystem (`integrations.tla`) - -**Purpose**: Formal specification for integration features (from TODO 20). - -**Key Features**: -- Webhook management and delivery -- OAuth application framework -- API key lifecycle management -- Event subscriptions and filtering -- Rate limiting per identity -- Retry logic for webhook delivery - -**Safety Properties**: -1. AdminOnlyIntegrationManagement - Only Level 4+ manage integrations -2. WebhookTenantLimit - Webhooks per tenant limited -3. APIKeyUserLimit - API keys per user limited -4. RateLimitEnforcement - Rate limits enforced -5. WebhookTenantIsolation - Webhooks isolated by tenant -6. OAuthAppTenantIsolation - OAuth apps isolated by tenant -7. APIKeyTenantIsolation - API keys isolated by tenant -8. TokenTenantConsistency - OAuth tokens match app tenants -9. ActiveDeliveriesQueued - Active deliveries properly queued - -**Liveness Properties**: -1. EventualDeliveryCompletion - Deliveries eventually complete/fail -2. EventualRetryOrFail - Failed deliveries retry or permanently fail -3. EventualExpiration - Expired keys eventually marked expired - -**Model Checking Configuration**: -- 3 users, 2 tenants, 3 webhooks, 2 OAuth apps, 3 API keys, 3 event types -- Max 2 webhooks/tenant, 2 keys/user, 100 calls/hour -- Bounded to 15 audit entries, 5 queue items, 6 deliveries - -## Common Properties Across All Specs - -### Multi-Tenant Isolation -Every specification enforces strict tenant isolation: -- Users can only access resources in their tenant -- All operations filter by tenantId -- Cross-tenant access is impossible by design - -### Permission Hierarchy -All specs respect the 6-level hierarchy: -1. Public (Level 1) - Unauthenticated, read-only -2. User (Level 2) - Basic authenticated operations -3. Moderator (Level 3) - Content moderation -4. Admin (Level 4) - Integration management, workflow execution -5. God (Level 5) - Workflow authoring, advanced features -6. Supergod (Level 6) - Complete system control - -### Audit Logging -Every privileged operation is logged with: -- User performing the action -- Action type -- Target resource -- Tenant context -- Timestamp - -## Validation Status - -Validated using TLA+ tools v1.8.0 (tla2tools.jar): - -```bash -./validate-specs.sh -``` - -**Results**: -- βœ… workflow_system.tla: PASSED -- βœ… collaboration.tla: PASSED -- βœ… integrations.tla: PASSED -- ⚠️ metabuilder.tla: Pre-existing syntax error (line 323) - -The pre-existing error in metabuilder.tla is documented in `KNOWN_ISSUES.md` and does not affect the new specifications. - -## Using These Specifications - -### For Design Review -1. Review state transitions before implementation -2. Identify race conditions and deadlocks -3. Verify permission enforcement is complete -4. Ensure multi-tenant isolation is maintained - -### For Implementation -1. Map specification states to code structures -2. Use operations as API contract definitions -3. Implement invariants as runtime assertions -4. Follow state transition logic - -### For Testing -1. Use safety properties as test requirements -2. Create property-based tests from invariants -3. Test edge cases identified in specifications -4. Validate liveness properties with integration tests - -### Running Model Checker - -```bash -# Download TLA+ tools -wget https://github.com/tlaplus/tlaplus/releases/download/v1.8.0/tla2tools.jar - -# Check syntax -java -cp tla2tools.jar tla2sany.SANY spec/workflow_system.tla - -# Run model checker (may take several minutes) -java -cp tla2tools.jar tlc2.TLC spec/workflow_system.tla -config spec/workflow_system.cfg -``` - -## Architecture Alignment - -These specifications align with MetaBuilder's architecture: - -| Specification | Architecture Docs | -|---------------|-------------------| -| `workflow_system.tla` | `docs/todo/improvements/20-FUTURE-FEATURES-TODO.md` (Workflow Features) | -| `collaboration.tla` | `docs/todo/improvements/20-FUTURE-FEATURES-TODO.md` (Collaboration Features) | -| `integrations.tla` | `docs/todo/improvements/20-FUTURE-FEATURES-TODO.md` (Integration Ecosystem) | -| All specs | `docs/architecture/security-docs/5-level-system.md` (Permissions) | -| All specs | `docs/architecture/data/data-driven-architecture.md` (Multi-tenancy) | - -## Benefits - -These formal specifications provide: - -1. **Early Bug Detection**: Find race conditions before implementation -2. **Clear Contracts**: Precise API behavior definitions -3. **Security Validation**: Verify permission and isolation properties -4. **Implementation Guide**: Clear state machines to implement -5. **Test Blueprint**: Property-based testing requirements -6. **Documentation**: Precise system behavior description -7. **Design Tool**: Explore alternatives before coding -8. **Regression Prevention**: Detect when changes violate invariants - -## Resources - -- **Main Documentation**: `spec/README.md` -- **Quick Reference**: `spec/QUICK_REFERENCE.md` -- **Known Issues**: `spec/KNOWN_ISSUES.md` -- **Validation Script**: `spec/validate-specs.sh` -- **TLA+ Homepage**: https://lamport.azurewebsites.net/tla/tla.html -- **Learn TLA+**: https://learntla.com/ - -## Maintenance - -When implementing these features: - -1. βœ… Review the relevant TLA+ specification first -2. βœ… Use state transitions as requirements -3. βœ… Implement property-based tests -4. βœ… Validate multi-tenant isolation -5. βœ… Update spec if design changes - -When modifying specifications: - -1. Update the `.tla` file with changes -2. Run `./validate-specs.sh` to check syntax -3. Run model checker to verify properties still hold -4. Update documentation if needed -5. Document any new invariants or properties - ---- - -*Created*: 2025-12-27 -*Status*: Complete and Validated -*TLA+ Version*: 1.8.0 -*MetaBuilder Version*: Iteration 25+ +# TLA+ Specifications Summary + +## Overview + +This directory contains formal TLA+ specifications for MetaBuilder, including both the current core system and three major future functionality areas. All specifications have been validated with TLA+ tools. + +## What Was Created + +### New Specifications (December 2025) + +Three comprehensive TLA+ modules for future functionality: + +| Specification | Lines | Operations | Properties | Status | +|---------------|-------|------------|------------|--------| +| `workflow_system.tla` | 595 | 22 | 12 (9 safety + 3 liveness) | βœ… Validated | +| `collaboration.tla` | 546 | 26 | 13 (10 safety + 3 liveness) | βœ… Validated | +| `integrations.tla` | 644 | 28 | 13 (10 safety + 3 liveness) | βœ… Validated | +| **Total** | **1,785** | **76** | **38** | | + +### Supporting Files + +- **Configuration Files**: 3 `.cfg` files for model checking +- **Documentation**: Updated `README.md` with comprehensive module documentation +- **Quick Reference**: `QUICK_REFERENCE.md` for developer workflows +- **Validation Script**: `validate-specs.sh` for automated syntax checking +- **Known Issues**: `KNOWN_ISSUES.md` documenting pre-existing core spec issue + +## Specifications Detail + +### 1. Workflow System (`workflow_system.tla`) + +**Purpose**: Formal specification for advanced workflow execution features (from TODO 20). + +**Key Features**: +- Template lifecycle (create, activate, archive) +- Manual and scheduled execution +- Step dependencies with parallel execution +- Configurable retry logic (MaxRetries) +- Per-tenant concurrency limits (MaxConcurrentRuns) + +**Safety Properties**: +1. GodOnlyTemplateCreation - Only Level 5+ can author templates +2. AdminOnlyExecution - Only Level 4+ can execute workflows +3. TenantIsolation - Workflows isolated by tenant +4. ConcurrencyLimit - Per-tenant execution limits enforced +5. RetryLimit - Failed steps don't exceed retry threshold +6. NoOverlap - Running and completed workflows don't overlap +7. DependencyEnforcement - Step dependencies respected +8. AuditCompleteness - All operations logged + +**Liveness Properties**: +1. EventualStepExecution - Pending steps eventually execute +2. EventualCompletion - Running workflows eventually complete/fail +3. EventualScheduleTrigger - Scheduled workflows eventually trigger + +**Model Checking Configuration**: +- 3 users, 2 tenants, 3 workflow templates, 3 steps +- Max 3 retries, max 2 concurrent runs per tenant +- Bounded to 15 audit log entries, 5 instances + +### 2. Collaboration System (`collaboration.tla`) + +**Purpose**: Formal specification for real-time collaboration features (from TODO 20). + +**Key Features**: +- User session states (active, idle, disconnected) +- Concurrent document editing +- Comments and @mentions +- Real-time presence tracking +- Notification system +- Document version history + +**Safety Properties**: +1. DocumentTenantIsolation - Editors only access own tenant docs +2. ConcurrentEditorLimit - Max concurrent editors enforced +3. CommentTenantConsistency - Comments belong to doc tenant +4. MentionTenantIsolation - Mentions within tenant boundaries +5. NotificationLimit - Pending notifications don't exceed limit +6. ActiveEditorsOnly - Only active users can edit +7. DisconnectedNotEditing - Disconnected users auto-stop editing +8. OperationAuthorship - All ops from authorized editors +9. VersionConsistency - Version snapshots preserve content + +**Liveness Properties**: +1. EventualNotificationHandling - Notifications eventually read/cleared +2. EventualMentionRead - Mentions eventually seen +3. EventualStopEditing - Active editors eventually stop/disconnect + +**Model Checking Configuration**: +- 3 users, 2 tenants, 2 documents +- Max 3 concurrent editors, max 10 notifications +- Bounded to 5 comments, 5 mentions, 12 activity feed entries + +### 3. Integration Ecosystem (`integrations.tla`) + +**Purpose**: Formal specification for integration features (from TODO 20). + +**Key Features**: +- Webhook management and delivery +- OAuth application framework +- API key lifecycle management +- Event subscriptions and filtering +- Rate limiting per identity +- Retry logic for webhook delivery + +**Safety Properties**: +1. AdminOnlyIntegrationManagement - Only Level 4+ manage integrations +2. WebhookTenantLimit - Webhooks per tenant limited +3. APIKeyUserLimit - API keys per user limited +4. RateLimitEnforcement - Rate limits enforced +5. WebhookTenantIsolation - Webhooks isolated by tenant +6. OAuthAppTenantIsolation - OAuth apps isolated by tenant +7. APIKeyTenantIsolation - API keys isolated by tenant +8. TokenTenantConsistency - OAuth tokens match app tenants +9. ActiveDeliveriesQueued - Active deliveries properly queued + +**Liveness Properties**: +1. EventualDeliveryCompletion - Deliveries eventually complete/fail +2. EventualRetryOrFail - Failed deliveries retry or permanently fail +3. EventualExpiration - Expired keys eventually marked expired + +**Model Checking Configuration**: +- 3 users, 2 tenants, 3 webhooks, 2 OAuth apps, 3 API keys, 3 event types +- Max 2 webhooks/tenant, 2 keys/user, 100 calls/hour +- Bounded to 15 audit entries, 5 queue items, 6 deliveries + +## Common Properties Across All Specs + +### Multi-Tenant Isolation +Every specification enforces strict tenant isolation: +- Users can only access resources in their tenant +- All operations filter by tenantId +- Cross-tenant access is impossible by design + +### Permission Hierarchy +All specs respect the 6-level hierarchy: +1. Public (Level 1) - Unauthenticated, read-only +2. User (Level 2) - Basic authenticated operations +3. Moderator (Level 3) - Content moderation +4. Admin (Level 4) - Integration management, workflow execution +5. God (Level 5) - Workflow authoring, advanced features +6. Supergod (Level 6) - Complete system control + +### Audit Logging +Every privileged operation is logged with: +- User performing the action +- Action type +- Target resource +- Tenant context +- Timestamp + +## Validation Status + +Validated using TLA+ tools v1.8.0 (tla2tools.jar): + +```bash +./validate-specs.sh +``` + +**Results**: +- βœ… workflow_system.tla: PASSED +- βœ… collaboration.tla: PASSED +- βœ… integrations.tla: PASSED +- ⚠️ metabuilder.tla: Pre-existing syntax error (line 323) + +The pre-existing error in metabuilder.tla is documented in `KNOWN_ISSUES.md` and does not affect the new specifications. + +## Using These Specifications + +### For Design Review +1. Review state transitions before implementation +2. Identify race conditions and deadlocks +3. Verify permission enforcement is complete +4. Ensure multi-tenant isolation is maintained + +### For Implementation +1. Map specification states to code structures +2. Use operations as API contract definitions +3. Implement invariants as runtime assertions +4. Follow state transition logic + +### For Testing +1. Use safety properties as test requirements +2. Create property-based tests from invariants +3. Test edge cases identified in specifications +4. Validate liveness properties with integration tests + +### Running Model Checker + +```bash +# Download TLA+ tools +wget https://github.com/tlaplus/tlaplus/releases/download/v1.8.0/tla2tools.jar + +# Check syntax +java -cp tla2tools.jar tla2sany.SANY spec/workflow_system.tla + +# Run model checker (may take several minutes) +java -cp tla2tools.jar tlc2.TLC spec/workflow_system.tla -config spec/workflow_system.cfg +``` + +## Architecture Alignment + +These specifications align with MetaBuilder's architecture: + +| Specification | Architecture Docs | +|---------------|-------------------| +| `workflow_system.tla` | `storybook/JSON_PACKAGES.md` (Workflow Features & JSON packages) | +| `collaboration.tla` | `storybook/JSON_PACKAGES.md` (Collaboration Features & JSON packages) | +| `integrations.tla` | `storybook/JSON_PACKAGES.md` (Integration Ecosystem & JSON packages) | +| All specs | `docs/architecture/security-docs/5-level-system.md` (Permissions) | +| All specs | `docs/architecture/data/data-driven-architecture.md` (Multi-tenancy) | + +## Benefits + +These formal specifications provide: + +1. **Early Bug Detection**: Find race conditions before implementation +2. **Clear Contracts**: Precise API behavior definitions +3. **Security Validation**: Verify permission and isolation properties +4. **Implementation Guide**: Clear state machines to implement +5. **Test Blueprint**: Property-based testing requirements +6. **Documentation**: Precise system behavior description +7. **Design Tool**: Explore alternatives before coding +8. **Regression Prevention**: Detect when changes violate invariants + +## Resources + +- **Main Documentation**: `spec/README.md` +- **Quick Reference**: `spec/QUICK_REFERENCE.md` +- **Known Issues**: `spec/KNOWN_ISSUES.md` +- **Validation Script**: `spec/validate-specs.sh` +- **TLA+ Homepage**: https://lamport.azurewebsites.net/tla/tla.html +- **Learn TLA+**: https://learntla.com/ + +## Maintenance + +When implementing these features: + +1. βœ… Review the relevant TLA+ specification first +2. βœ… Use state transitions as requirements +3. βœ… Implement property-based tests +4. βœ… Validate multi-tenant isolation +5. βœ… Update spec if design changes + +When modifying specifications: + +1. Update the `.tla` file with changes +2. Run `./validate-specs.sh` to check syntax +3. Run model checker to verify properties still hold +4. Update documentation if needed +5. Document any new invariants or properties + +--- + +*Created*: 2025-12-27 +*Status*: Complete and Validated +*TLA+ Version*: 1.8.0 +*MetaBuilder Version*: Iteration 25+ diff --git a/spec/package_system.tla b/spec/package_system.tla index 4fe8b8b63..4a6ceab4d 100644 --- a/spec/package_system.tla +++ b/spec/package_system.tla @@ -12,8 +12,8 @@ * * * Aligned with: * * - schemas/package-schemas/*.json * - * - docs/packages/package-sources.md * - * - packages/*/package.json structure * + * - storybook/JSON_PACKAGES.md (JSON package format and discovery) * + * - packages/*/package.json or packages/*/seed/metadata.json * ***************************************************************************) EXTENDS Naturals, Sequences, FiniteSets, TLC @@ -112,7 +112,9 @@ PackageIndexEntry == [ minLevel: PermissionLevel, category: PackageCategory, source: PackageSources, - dependencies: SUBSET Packages + dependencies: SUBSET Packages, + format: {"json"}, \* Package format (JSON-based packages) + manifestPath: STRING \* Path to package manifest (package.json or seed metadata) ] \* Full package entry in registry @@ -125,13 +127,16 @@ PackageEntry == [ source: PackageSources, state: PackageState, dependencies: SUBSET Packages, + \* For JSON-based packages the manifest contains exports and metadata + manifestPath: STRING, exports: [ components: SUBSET STRING, scripts: SUBSET STRING, types: SUBSET STRING ], validated: BOOLEAN, - installedAt: Nat \cup {0} + installedAt: Nat \cup {0}, + format: {"json"} ] \* Pending operation diff --git a/spec/validate-specs.sh b/spec/validate-specs.sh index 096671dce..cce084d88 100755 --- a/spec/validate-specs.sh +++ b/spec/validate-specs.sh @@ -1,94 +1,95 @@ -#!/bin/bash -# Validate TLA+ Specifications -# This script checks the syntax of all TLA+ specifications in the spec directory - -set -e - -SPEC_DIR="$(dirname "$0")" -TLA_TOOLS="${TLA_TOOLS:-/tmp/tla2tools.jar}" - -# Color output -RED='\033[0;31m' -GREEN='\033[0;32m' -YELLOW='\033[1;33m' -NC='\033[0m' # No Color - -echo "TLA+ Specification Validator" -echo "=============================" -echo "" - -# Check if TLA+ tools are available -if [ ! -f "$TLA_TOOLS" ]; then - echo -e "${YELLOW}TLA+ tools not found at $TLA_TOOLS${NC}" - echo "Downloading TLA+ tools..." - wget -q -O "$TLA_TOOLS" https://github.com/tlaplus/tlaplus/releases/download/v1.8.0/tla2tools.jar - if [ $? -eq 0 ]; then - echo -e "${GREEN}βœ“ Downloaded TLA+ tools${NC}" - else - echo -e "${RED}βœ— Failed to download TLA+ tools${NC}" - exit 1 - fi -fi - -# Check Java -if ! command -v java &> /dev/null; then - echo -e "${RED}βœ— Java not found. Please install Java 11 or later.${NC}" - exit 1 -fi - -echo "Using TLA+ tools: $TLA_TOOLS" -echo "" - -# List of specifications to validate -SPECS=( - "metabuilder.tla" - "workflow_system.tla" - "collaboration.tla" - "integrations.tla" -) - -FAILED=0 -PASSED=0 - -# Validate each specification -for spec in "${SPECS[@]}"; do - spec_path="$SPEC_DIR/$spec" - - if [ ! -f "$spec_path" ]; then - echo -e "${YELLOW}⚠ Skipping $spec (file not found)${NC}" - continue - fi - - echo "Validating $spec..." - - # Run SANY (syntax and semantic analyzer) - output=$(java -cp "$TLA_TOOLS" tla2sany.SANY "$spec_path" 2>&1) - - # Check for errors - if echo "$output" | grep -q "Semantic errors:" || echo "$output" | grep -q "Parsing or semantic analysis failed."; then - echo -e "${RED}βœ— $spec: FAILED${NC}" - echo "$output" | grep -A 10 "Semantic errors:" - FAILED=$((FAILED + 1)) - else - echo -e "${GREEN}βœ“ $spec: PASSED${NC}" - PASSED=$((PASSED + 1)) - fi - - echo "" -done - -# Summary -echo "=============================" -echo "Validation Summary" -echo "=============================" -echo -e "${GREEN}Passed: $PASSED${NC}" -echo -e "${RED}Failed: $FAILED${NC}" -echo "" - -if [ $FAILED -eq 0 ]; then - echo -e "${GREEN}All specifications are valid!${NC}" - exit 0 -else - echo -e "${RED}Some specifications have errors. Please review the output above.${NC}" - exit 1 -fi +#!/bin/bash +# Validate TLA+ Specifications +# This script checks the syntax of all TLA+ specifications in the spec directory + +set -e + +SPEC_DIR="$(dirname "$0")" +TLA_TOOLS="${TLA_TOOLS:-/tmp/tla2tools.jar}" + +# Color output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +NC='\033[0m' # No Color + +echo "TLA+ Specification Validator" +echo "=============================" +echo "" + +# Check if TLA+ tools are available +if [ ! -f "$TLA_TOOLS" ]; then + echo -e "${YELLOW}TLA+ tools not found at $TLA_TOOLS${NC}" + echo "Downloading TLA+ tools..." + wget -q -O "$TLA_TOOLS" https://github.com/tlaplus/tlaplus/releases/download/v1.8.0/tla2tools.jar + if [ $? -eq 0 ]; then + echo -e "${GREEN}βœ“ Downloaded TLA+ tools${NC}" + else + echo -e "${RED}βœ— Failed to download TLA+ tools${NC}" + exit 1 + fi +fi + +# Check Java +if ! command -v java &> /dev/null; then + echo -e "${RED}βœ— Java not found. Please install Java 11 or later.${NC}" + exit 1 +fi + +echo "Using TLA+ tools: $TLA_TOOLS" +echo "" + +# List of specifications to validate +SPECS=( + "metabuilder.tla" + "workflow_system.tla" + "collaboration.tla" + "integrations.tla" + "package_system.tla" +) + +FAILED=0 +PASSED=0 + +# Validate each specification +for spec in "${SPECS[@]}"; do + spec_path="$SPEC_DIR/$spec" + + if [ ! -f "$spec_path" ]; then + echo -e "${YELLOW}⚠ Skipping $spec (file not found)${NC}" + continue + fi + + echo "Validating $spec..." + + # Run SANY (syntax and semantic analyzer) + output=$(java -cp "$TLA_TOOLS" tla2sany.SANY "$spec_path" 2>&1) + + # Check for errors + if echo "$output" | grep -q "Semantic errors:" || echo "$output" | grep -q "Parsing or semantic analysis failed."; then + echo -e "${RED}βœ— $spec: FAILED${NC}" + echo "$output" | grep -A 10 "Semantic errors:" + FAILED=$((FAILED + 1)) + else + echo -e "${GREEN}βœ“ $spec: PASSED${NC}" + PASSED=$((PASSED + 1)) + fi + + echo "" +done + +# Summary +echo "=============================" +echo "Validation Summary" +echo "=============================" +echo -e "${GREEN}Passed: $PASSED${NC}" +echo -e "${RED}Failed: $FAILED${NC}" +echo "" + +if [ $FAILED -eq 0 ]; then + echo -e "${GREEN}All specifications are valid!${NC}" + exit 0 +else + echo -e "${RED}Some specifications have errors. Please review the output above.${NC}" + exit 1 +fi