diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index 9f7a8cb1c..000000000 --- a/AGENTS.md +++ /dev/null @@ -1,937 +0,0 @@ -# AI Agent Development Guide for MetaBuilder - -**START HERE:** This document guides AI agents working on any part of MetaBuilder. Read the sections relevant to your task. - -## Quick Navigation - -- **Finding Tech Debt** → See `/TECH_DEBT.md` (bot-actionable task list with explicit instructions) -- **Starting Development** → [Core Principles](#core-principles) + [Scoped Rules](#scoped-rules) -- **Working on DBAL** → [DBAL Development](#dbal-development-phase-2--phase-3) -- **Working on Frontend** → [Frontend Development](#frontend-development) -- **Working on Packages** → [Package Development](#package-development) -- **Running Tests** → [Testing Strategy](#testing-strategy) -- **Deploying Code** → [Deployment](#deployment-guidelines) -- **Troubleshooting** → [Troubleshooting](#troubleshooting) - ---- - -## Core Principles - -These principles apply to all development work on MetaBuilder: - -### 1. Data-Driven Architecture (95% Rule) - -MetaBuilder is **95% JSON/YAML configuration**, not code: - -- **5% TypeScript** = Infrastructure, adapters, frameworks, bootstrapping -- **95% JSON/YAML** = UI definitions, business logic, configurations, database schemas - -**When choosing between code and config: always prefer config.** - -```typescript -// ❌ WRONG: Hardcoded UI -function HomePage() { - return
Welcome to MetaBuilder
-} - -// ✅ CORRECT: JSON definition -// File: packages/ui_home/seed/metadata.json -{ - "pages": [{ - "id": "home", - "path": "/", - "component": "DeclaredComponent", - "config": { "title": "Welcome to MetaBuilder" } - }] -} -``` - -### 2. One Lambda Per File - -Every file should be **one focused unit**: - -```typescript -// ✅ CORRECT -// File: src/lib/users/createUser.ts -export async function createUser(username: string): Promise { - // Single responsibility -} - -// ✅ CORRECT (class as container) -// File: src/lib/users/UserManager.ts -export class UserManager { - static async create = createUser - static async list = listUsers -} - -// ❌ WRONG: Multiple functions in one file -// File: src/lib/users.ts -export function createUser() { ... } -export function listUsers() { ... } -export function deleteUser() { ... } -``` - -See `.github/prompts/LAMBDA_PROMPT.md` for detailed guidance. - -### 3. Schema-First Development - -**Always define schema before implementation:** - -``` -1. Define in YAML/JSON schema (source of truth) -2. Generate types from schema -3. Implement code that conforms to types -4. Add tests -``` - -Never generate schema from code or hardcode types. - -### 4. Multi-Tenant Safety - -Every query must include tenant filtering: - -```typescript -// ❌ WRONG: No tenant filter -const users = await db.users.list() - -// ✅ CORRECT: Tenant filtered -const users = await db.users.list({ - filter: { tenantId: req.user.tenantId } -}) -``` - -### 5. ACL (Access Control) First - -Every database operation must check permissions: - -```typescript -// Always use DBAL which includes ACL checks -const db = getDBALClient() -const users = await db.users.list() // ACL applied automatically -``` - ---- - -## Scoped Rules - -**CRITICAL:** Before starting work, check for scoped `AGENTS.md` or similar files in your working directory: - -| Location | Purpose | -|----------|---------| -| `AGENTS.md` (root) | **Start here** - General MetaBuilder guidance for AI agents | -| `CLAUDE.md` (root) | Overall project instructions (more detailed than AGENTS.md) | -| `ARCHITECTURE.md` | System architecture and data flow | -| `dbal/docs/` | DBAL-specific documentation and guides | -| `dbal/shared/docs/AGENTS.md` | YAML schema development rules (if exists) | -| `dbal/development/docs/AGENTS.md` | TypeScript DBAL rules (if exists) | -| `frontends/nextjs/AGENTS.md` | Next.js frontend rules (if exists) | -| `packages/[name]/AGENTS.md` | Package-specific rules (if exists) | - -**Pattern:** -1. Find the directory/subsystem you're editing -2. Search for `AGENTS.md` or similar guidance files in that directory or parents -3. Scoped rules **override** general guidance -4. Read completely before starting work - ---- - -## DBAL Development (Phase 2 & Phase 3) - -The DBAL (Database Abstraction Layer) is a **language-agnostic contract system**: - -- **Phase 2 (Current):** TypeScript in `/dbal/development/` with Prisma adapter -- **Phase 3 (Future):** C++ daemon in `/dbal/production/` with security isolation - -### DBAL Architecture Philosophy - -``` -1. API Definition (YAML) ← Source of truth - ↓ -2. Development Implementation (TypeScript) ← Fast iteration - ↓ -3. Production Implementation (C++) ← Security & performance - ↓ -4. Conformance Tests ← Guarantee parity -``` - -### Key Rules for DBAL Work - -#### 1. Schema First (API Contract is Source of Truth) - -```bash -# CORRECT workflow: -1. Edit: /dbal/shared/api/schema/entities/user.yaml -2. Generate: npm --prefix dbal/development run codegen:prisma -3. Push: npm --prefix dbal/development run db:push -4. Implement TypeScript code -5. Add tests - -# WRONG: -❌ Edit Prisma schema directly -❌ Hardcode fields in TypeScript -❌ Generate schema from code -``` - -#### 2. File Organization - -**YAML Schemas (source of truth):** -``` -/dbal/shared/api/schema/ -├── entities/ -│ ├── core/ (User, Session, Workflow, etc.) -│ ├── access/ (Credential, PageConfig, etc.) -│ └── packages/ (Package-specific entities) -├── operations/ (Operation definitions) -└── capabilities.yaml (Adapter capabilities) -``` - -**TypeScript Implementation (Phase 2):** -``` -/dbal/development/src/ -├── core/ (Client, types, errors) -├── adapters/ (Database adapters: Prisma, SQLite, etc.) -├── runtime/ (Config, secrets, telemetry) -└── seeds/ (Seed orchestration) -``` - -**C++ Implementation (Phase 3):** -``` -/dbal/production/ -├── include/dbal/ (Public headers) -├── src/ (Implementation) -└── tests/ (Tests) -``` - -#### 3. Code Generation - -**Phase 2 (TypeScript - Current):** -```bash -# Generate Prisma schema from YAML -npm --prefix dbal/development run codegen:prisma - -# Push schema to database -npm --prefix dbal/development run db:push - -# Rebuild types -npm --prefix dbal/development run build -``` - -**Never edit generated files manually:** -- `/prisma/schema.prisma` - Auto-generated -- `/dbal/development/src/core/types.ts` - Auto-generated - -#### 4. Using DBAL Client - -```typescript -// CORRECT: Use factory from DBAL -import { getDBALClient } from '@/dbal' - -const db = getDBALClient() -const users = await db.users.list() - -// WRONG: -❌ import { prisma } from '@/lib/prisma' // Direct Prisma access -❌ import { getAdapter } from '@/lib/dbal-client/adapter' // Old adapter -``` - -#### 5. Adding Entities/Fields - -```bash -# 1. Update YAML schema -vim /dbal/shared/api/schema/entities/core/user.yaml -# Add: avatar_url: { type: string, optional: true } - -# 2. Regenerate Prisma -npm --prefix dbal/development run codegen:prisma - -# 3. Push to database -npm --prefix dbal/development run db:push - -# 4. Update TypeScript code to use new field -# (Types auto-generated, so just use the field) - -# 5. Add tests -# Update test files to cover new functionality -``` - -#### 6. Security Considerations - -❌ **Never:** -- Expose database credentials to user code -- Allow raw SQL queries from user input -- Skip ACL checks -- Trust user input without validation -- Log sensitive data (passwords, tokens, PII) - -✅ **Always:** -- Validate input against schema -- Enforce row-level security (multi-tenant) -- Use parameterized queries (Prisma handles this) -- Log security-relevant operations -- Test with malicious input - ---- - -## Frontend Development - -### Next.js Frontend (`/frontends/nextjs/`) - -#### Key Rules - -1. **Use DBAL for all database access:** -```typescript -// CORRECT -import { getDBALClient } from '@/dbal' -const db = getDBALClient() -const users = await db.users.list() - -// WRONG -❌ Direct Prisma access -❌ SQL queries -❌ Old adapter patterns -``` - -2. **Follow one lambda per file:** -``` -src/lib/users/createUser.ts (single function) -src/lib/users/UserManager.ts (class as container) -src/api/users/route.ts (single handler) -``` - -3. **Keep API routes thin:** -```typescript -// ✅ CORRECT: API route delegates to DBAL -export async function GET(request: Request) { - const db = getDBALClient() - const users = await db.users.list() - return Response.json(users) -} - -// ❌ WRONG: Complex logic in route -export async function POST(request: Request) { - const data = await request.json() - // Complex validation, transformation, etc. - // Should be in a separate function -} -``` - -4. **E2E Tests with Playwright:** -```bash -# Generate schema -npm --prefix dbal/development run codegen:prisma - -# Push to database -npm --prefix dbal/development run db:push - -# Run tests -npm run test:e2e -``` - -See `TESTING.md` for comprehensive E2E testing guide. - -#### Component Documentation (Storybook) - -**🚨 CRITICAL GUARDRAIL: Stories Are Data, Not Code** - -Like tests, Storybook stories must be JSON, not TypeScript! - -**❌ WRONG: Writing new .stories.tsx files** -```typescript -// DON'T DO THIS - No new .stories.tsx files! -export const HomePage: Story = { - render: () => -} -``` - -**✅ CORRECT: JSON story definitions in packages** -```json -// packages/ui_home/storybook/stories.json -{ - "$schema": "https://metabuilder.dev/schemas/package-storybook.schema.json", - "title": "Home Page Components", - "stories": [{ - "name": "HomePage", - "render": "home_page", - "description": "Complete home page", - "args": { - "title": "Welcome to MetaBuilder" - } - }] -} -``` - -**Story Location Rules:** -- ✅ `packages/{package_name}/storybook/stories.json` - Package stories (PREFERRED) -- ✅ `storybook/src/stories/*.stories.tsx` - Existing stories (legacy, don't add more) -- ❌ NEVER create new `.stories.tsx` files - use JSON instead! - -**JSON Story Loader:** -- Stories defined in `packages/*/storybook/stories.json` are auto-discovered -- `storybook/json-loader/` loads and renders JSON stories directly -- No code generation - JSON is rendered at runtime -- Uses `DynamicStory` component to render from JSON definitions - -See `storybook/json-loader/README.md` for complete guide. - -#### Project Structure - -``` -frontends/nextjs/src/ -├── app/ (Next.js App Router) -│ ├── api/ (API routes - keep thin) -│ ├── (pages)/ (Pages) -│ └── layout.tsx (Root layout) -├── lib/ -│ ├── db-client.ts (DBAL integration) -│ └── [feature]/ (Feature-specific utilities) -├── components/ (React components) -└── styles/ (CSS/styling) -``` - ---- - -## Package Development - -### Creating New Packages (`/packages/[name]/`) - -A package is a self-contained feature with UI, configuration, and optional database entities. - -#### Package Structure - -``` -packages/my_feature/ -├── seed/ -│ └── metadata.json (Package definition & seed data) -├── components/ (Optional: package-specific components) -├── public/ (Static assets) -└── README.md (Package documentation) -``` - -#### Package Metadata (`seed/metadata.json`) - -```json -{ - "packageId": "my_feature", - "name": "My Feature", - "version": "1.0.0", - "permissions": { - "required": ["feature:read", "feature:write"] - }, - "pages": [{ - "id": "main", - "path": "/my-feature", - "title": "My Feature", - "component": "DeclaredComponent", - "config": { } - }], - "entities": [], - "scripts": [], - "migrations": [] -} -``` - -#### Development Workflow - -1. **Create package structure:** -```bash -mkdir -p packages/my_feature/seed -``` - -2. **Define metadata (seed/metadata.json):** -```json -{ - "packageId": "my_feature", - "name": "My Feature", - "version": "1.0.0" -} -``` - -3. **Add to installed packages:** -```bash -# Edit: seed/database/installed_packages.yaml -- packageId: my_feature - source: packages/my_feature -``` - -4. **Run bootstrap:** -```bash -./deployment/scripts/bootstrap-system.sh -``` - -5. **Test:** -```bash -npm run test:e2e -``` - -#### Key Rules - -- 95% of package definition is JSON, not code -- Use `DeclaredComponent` for all UI -- Define configuration in JSON, not hardcoded -- One responsibility per component -- Test packages with E2E tests - ---- - -## Testing Strategy - -### Test Pyramid - -``` - ↑ - │ E2E Tests (Playwright) - │ - Full app flow - │ - Real database - │ - Real users - │ - │ Integration Tests - │ - Component + DBAL - │ - SQLite/test DB - │ - │ Unit Tests - │ - Individual functions - │ - Mocked dependencies - │ - └───────────────────── -``` - -### Running Tests - -**Unit tests (DBAL):** -```bash -npm --prefix dbal/development run test:unit -``` - -**Integration tests (DBAL):** -```bash -npm --prefix dbal/development run test:integration -``` - -**E2E tests (Full app):** -```bash -# 1. Generate schema -npm --prefix dbal/development run codegen:prisma - -# 2. Push to database -npm --prefix dbal/development run db:push - -# 3. Run E2E tests -npm run test:e2e -``` - -### Writing Tests - -**🚨 CRITICAL GUARDRAIL: Tests Are Data, Not Code** - -MetaBuilder follows the **95% JSON rule** for tests too. Tests must be defined as JSON, not TypeScript! - -**❌ WRONG: Writing new .spec.ts files** -```typescript -// DON'T DO THIS - No new .spec.ts files! -test('should login', async ({ page }) => { - await page.goto('/login') - await page.fill('[name="username"]', 'user') -}) -``` - -**✅ CORRECT: JSON test definitions in packages** -```json -// packages/auth/playwright/tests.json -{ - "$schema": "https://metabuilder.dev/schemas/package-playwright.schema.json", - "package": "auth", - "tests": [{ - "name": "should login", - "tags": ["@auth", "@smoke"], - "steps": [ - {"action": "navigate", "url": "/login"}, - {"action": "fill", "label": "Username", "value": "user"}, - {"action": "click", "role": "button", "text": "Login"} - ] - }] -} -``` - -**Test Location Rules:** -- ✅ `packages/{package_name}/playwright/tests.json` - Package-scoped tests (NEW) -- ✅ `e2e/*.spec.ts` - Existing manual tests (legacy, don't add more) -- ✅ `e2e/json-packages.spec.ts` - Auto-loads all package JSON tests -- ❌ NEVER create new `.spec.ts` files - use JSON instead! - -**JSON Test Runner:** -- Tests defined in `packages/*/playwright/tests.json` are auto-discovered -- `e2e/json-runner/playwright-json-runner.ts` interprets and executes JSON directly -- No code generation - JSON is executed at runtime -- Changes to JSON tests take effect immediately - -**Running JSON Tests:** -```bash -npm run test:e2e:json # All package JSON tests -npm run test:e2e -- e2e/json-packages.spec.ts # Same thing, explicit -``` - -**Unit test example (still TypeScript):** -```typescript -// tests/lib/users/createUser.test.ts -import { createUser } from '@/lib/users/createUser' - -describe('createUser', () => { - it('should create a user', async () => { - const user = await createUser('john', 'john@example.com') - expect(user.username).toBe('john') - }) -}) -``` - -See `TESTING.md` and `e2e/json-runner/README.md` for comprehensive testing guide. - ---- - -## Deployment Guidelines - -### Development Deployment - -```bash -# Terminal 1: Frontend -cd frontends/nextjs -npm run dev # http://localhost:3000 - -# Terminal 2: Generate and push schema (once) -npm --prefix dbal/development run codegen:prisma -npm --prefix dbal/development run db:push - -# Terminal 3: Run tests -npm run test:e2e -``` - -### Production Deployment - -```bash -# Using deployment script -./deployment/deploy.sh all --bootstrap - -# Or docker-compose with GHCR images -docker compose -f docker-compose.ghcr.yml up -d -``` - -**What gets deployed:** -- PostgreSQL 16 (database) -- C++ DBAL daemon (Phase 3 when ready) -- Next.js frontend -- C++ Media daemon (FFmpeg, ImageMagick) -- Redis 7 (caching) -- Nginx (reverse proxy, SSL/TLS) -- Optional: Prometheus, Grafana, Loki (monitoring) - -### Schema Management - -**Local development:** -```bash -# Edit YAML schema -vim /dbal/shared/api/schema/entities/core/user.yaml - -# Regenerate Prisma -npm --prefix dbal/development run codegen:prisma - -# Push to local database -npm --prefix dbal/development run db:push -``` - -**Before committing:** -- Always regenerate schema from YAML -- Push to ensure tables exist -- Run E2E tests -- Commit generated schema changes - ---- - -## Troubleshooting - -### "The table X does not exist" - -```bash -# Solution: Generate and push schema -npm --prefix dbal/development run codegen:prisma -npm --prefix dbal/development run db:push -``` - -### "Cannot find module '@/dbal'" - -```bash -# Solution: Verify DBAL exports -cat dbal/development/src/index.ts | grep getDBALClient -``` - -### E2E tests fail with database errors - -```bash -# Solution: Regenerate schema, push, then test -npm --prefix dbal/development run codegen:prisma -npm --prefix dbal/development run db:push -npm run test:e2e -``` - -### TypeScript type errors in DBAL - -```bash -# Solution: Rebuild DBAL -npm --prefix dbal/development run build -npm --prefix dbal/development run codegen:prisma -``` - -### Package not loading - -```bash -# Check: Is package in installed_packages.yaml? -cat seed/database/installed_packages.yaml | grep packageId - -# Check: Is metadata.json valid JSON? -cat packages/[name]/seed/metadata.json | jq . - -# Solution: Run bootstrap -./deployment/scripts/bootstrap-system.sh -``` - -### Tests fail with "seed not loaded" - -```bash -# Solution: Ensure global.setup.ts runs -# Check playwright.config.ts has: globalSetup: './e2e/global.setup.ts' - -# Manual fix: Call seed endpoint -curl http://localhost:3000/api/setup -``` - ---- - -## Best Practices Summary - -1. ✅ **Schema-first** - Define in YAML, generate code -2. ✅ **95% config, 5% code** - Prefer JSON/YAML over TypeScript -3. ✅ **One lambda per file** - Small, focused units -4. ✅ **Multi-tenant safe** - Always filter by tenantId -5. ✅ **ACL-first** - Every operation checks permissions -6. ✅ **Use DBAL** - Never bypass database abstraction -7. ✅ **Test thoroughly** - Unit → Integration → E2E -8. ✅ **Document scoped rules** - Create AGENTS.md for subsystems -9. ✅ **Principle of least privilege** - Minimal permissions -10. ✅ **Security by default** - Validate early, fail fast - ---- - -## Key Documents - -**Read These First:** -- **CLAUDE.md** - Comprehensive project instructions -- **ARCHITECTURE.md** - System architecture and data flow -- **.github/copilot-instructions.md** - AI-specific development patterns - -**Subsystem Guides:** -- **dbal/shared/docs/** - DBAL implementation guides -- **dbal/docs/** - DBAL-specific rules -- **TESTING.md** - E2E testing guide -- **DBAL_REFACTOR_PLAN.md** - Phase 2 cleanup roadmap - -**Schemas:** -- **/dbal/shared/api/schema/** - DBAL entity definitions (YAML) -- **/schemas/package-schemas/** - Package system definitions (JSON) -- **/.github/prompts/LAMBDA_PROMPT.md** - Code organization pattern -- **/.github/prompts/WORKFLOW_PROMPTS/** - Development workflow guides - -**Quick Reference:** -- **README.md** - System overview -- **ROADMAP.md** - Project vision and evolution - ---- - -## Project Structure - -``` -metabuilder/ -├── 📋 Core Documentation -│ ├── AGENTS.md ⭐ START HERE - AI agent guide for all subsystems -│ ├── CLAUDE.md Comprehensive project instructions for AI assistants -│ ├── ARCHITECTURE.md System architecture, data flow, deployment stack -│ ├── ROADMAP.md Project vision, evolution (Spark → Next.js) -│ ├── README.md System overview, quick start -│ └── TESTING.md E2E testing guide with Playwright -│ -├── 🗄️ Database (DBAL) -│ ├── dbal/ -│ │ ├── development/ ⭐ Phase 2: TypeScript DBAL implementation -│ │ │ ├── src/ Core client, adapters, seeds -│ │ │ ├── tests/ Unit & integration tests -│ │ │ └── prisma/ Prisma schema (auto-generated) -│ │ ├── production/ Phase 3: C++ DBAL daemon (future) -│ │ │ ├── include/dbal/ C++ public headers -│ │ │ ├── src/ C++ implementation -│ │ │ └── tests/ C++ tests -│ │ ├── shared/ ⭐ Shared between Phase 2 & 3 -│ │ │ ├── api/schema/ YAML entity & operation definitions (source of truth) -│ │ │ ├── backends/ Database connection utilities -│ │ │ ├── docs/ DBAL implementation guides -│ │ │ └── tools/ Code generation scripts -│ │ └── docs/ DBAL-specific documentation -│ -├── 🎨 Frontend -│ ├── frontends/ -│ │ ├── nextjs/ ⭐ Main Next.js application (Phase 2) -│ │ │ ├── src/app/ Next.js App Router (pages, API routes) -│ │ │ ├── src/lib/ Utilities (DBAL client integration, etc.) -│ │ │ ├── src/components/ React components -│ │ │ └── e2e/ Playwright E2E tests -│ │ ├── cli/ CLI frontend (optional) -│ │ ├── qt6/ Qt6 frontend (optional) -│ │ └── dbal/ DBAL exports for frontend use -│ -├── 📦 Packages -│ ├── packages/ ⭐ Modular feature packages -│ │ ├── ui_home/ Landing page -│ │ ├── ui_header/ App header/navbar -│ │ ├── ui_footer/ App footer -│ │ ├── ui_login/ Login UI -│ │ ├── dashboard/ User dashboard -│ │ ├── user_manager/ User management (Admin) -│ │ ├── package_manager/ Package installation (God level) -│ │ ├── database_manager/ Database admin (Supergod level) -│ │ ├── schema_editor/ Schema editor (Supergod level) -│ │ └── [40+ other packages]/ Feature-specific packages with JSON metadata -│ -├── 🌱 Seed Data -│ ├── seed/ -│ │ ├── database/ Base bootstrap YAML data -│ │ │ ├── installed_packages.yaml Package installation records -│ │ │ └── package_permissions.yaml Permission seed data -│ │ ├── packages/ Package-specific seed order -│ │ └── config/ Configuration files -│ -├── 📋 Schemas -│ ├── schemas/ -│ │ ├── package-schemas/ ⭐ Package system JSON schemas (18+ files) -│ │ │ ├── metadata_schema.json Package structure -│ │ │ ├── entities_schema.json Database models from packages -│ │ │ ├── components_schema.json UI component definitions -│ │ │ ├── script_schema.json JSON Script language spec (v2.2.0) -│ │ │ └── [13+ more schemas] API, validation, permissions, jobs, etc. -│ │ └── yaml-schema.yaml DBAL YAML schema validator -│ -├── 🚀 Deployment -│ ├── deployment/ -│ │ ├── deploy.sh ⭐ Main deployment orchestration script -│ │ ├── docker/ Docker images & compose files -│ │ │ ├── Dockerfile.app Next.js production image -│ │ │ ├── Dockerfile.app.dev Dev environment image -│ │ │ └── docker-compose.*.yml Development, production, monitoring -│ │ ├── scripts/ Bootstrap, schema, backup scripts -│ │ │ ├── bootstrap-system.sh Core package installation -│ │ │ ├── init-db.sh Database initialization -│ │ │ └── backup-database.sh Backup utilities -│ │ └── config/ Configuration templates -│ │ ├── dbal/ DBAL daemon config -│ │ ├── nginx/ Reverse proxy config -│ │ ├── prometheus/ Monitoring config -│ │ └── grafana/ Dashboards config -│ -├── 🧪 Testing -│ ├── e2e/ ⭐ End-to-end tests (Playwright) -│ │ ├── global.setup.ts Database initialization -│ │ ├── auth/ Authentication flows -│ │ ├── crud/ CRUD operations -│ │ ├── api/ API endpoint tests -│ │ ├── dbal-daemon/ DBAL daemon tests -│ │ └── *.spec.ts Individual test suites -│ ├── playwright.config.ts E2E test configuration -│ ├── config/test/ Test configuration directory -│ └── spec/ TLA+ formal specifications -│ -├── 🔧 Configuration -│ ├── config/ -│ │ ├── package.json ⭐ Root package definitions -│ │ ├── lint/ ESLint configuration -│ │ └── test/ Test tool configurations -│ ├── .github/ -│ │ ├── prompts/ AI agent development workflows -│ │ │ ├── workflow/ Feature planning, review, docs -│ │ │ ├── implement/ Implementation guides (DBAL, components, etc.) -│ │ │ ├── test/ Testing guides -│ │ │ ├── deploy/ Deployment procedures -│ │ │ └── maintain/ Maintenance & debugging guides -│ │ ├── copilot-instructions.md AI development patterns -│ │ └── TEMPLATES.md PR/issue templates -│ -├── 📚 Documentation -│ ├── docs/ Additional documentation -│ │ ├── TESTING_GUIDE.md TDD methodology, test pyramid -│ │ ├── CONTAINER_IMAGES.md Docker image registry -│ │ ├── PIPELINE_CONSOLIDATION.md CI/CD pipeline -│ │ └── archive/ Historical documentation -│ ├── DBAL_REFACTOR_PLAN.md Phase 2 cleanup roadmap -│ ├── SEED_SYSTEM_OVERHAUL.md Seed system improvements -│ └── README files in each subsystem -│ -├── 🎭 Other Frontends (Reference) -│ ├── fakemui/ QML/UI component library (reference) -│ ├── storybook/ Component documentation (archived) -│ └── old/ Legacy Spark implementation (archived) -│ -├── 📡 Services -│ ├── services/ -│ │ └── media_daemon/ C++ FFmpeg/ImageMagick daemon (future) -│ -├── 🔬 Specifications -│ ├── spec/ TLA+ formal specifications -│ │ ├── metabuilder.tla Core system behavior -│ │ ├── package_system.tla Package loading & management -│ │ ├── workflow_system.tla Workflow execution -│ │ └── integrations.tla Integration patterns -│ -├── 🛠️ Scripts & Tools -│ ├── scripts/ Utility scripts -│ │ ├── generate-package.ts Package scaffolding -│ │ └── check-dbal-prisma-sync.py Schema validation -│ -├── 💾 Database Files -│ ├── prisma/ -│ │ ├── schema.prisma ⭐ AUTO-GENERATED from YAML schemas -│ │ └── schema-registry.json Schema version tracking -│ ├── test-prisma-adapter.js Adapter tests -│ -└── 📦 Root Configuration - ├── package.json ⭐ Root dependencies - ├── package-lock.json Dependency lock file - ├── playwright.config.ts E2E test configuration - ├── docker-compose.ghcr.yml GHCR image deployment - └── Jenkinsfile CI/CD pipeline definition -``` - -### Key Directory Roles - -**Core Development (Always Active):** -- `dbal/development/` - TypeScript DBAL implementation -- `frontends/nextjs/` - Next.js frontend application -- `packages/` - Feature packages -- `dbal/shared/api/schema/` - Entity & operation definitions (YAML) - -**Supporting Systems:** -- `deployment/` - Deployment automation and Docker -- `e2e/` - Playwright E2E tests -- `seed/` - Bootstrap data and seeding -- `.github/prompts/` - AI agent workflow guidance - -**Configuration (Do Not Edit):** -- `prisma/schema.prisma` - AUTO-GENERATED from YAML -- `dbal/development/src/core/types.ts` - AUTO-GENERATED types - -**Reference/Archive:** -- `old/` - Legacy Spark implementation -- `fakemui/` - Legacy QML components -- `storybook/` - Archived component documentation -- `spec/` - TLA+ formal specifications - ---- - -## Questions? - -- Stuck? See `.github/prompts/misc/EEK-STUCK.md` recovery guide -- Need help? Use `/help` command in Claude Code -- Report issues: https://github.com/anthropics/claude-code/issues diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 9473e51ae..000000000 --- a/CLAUDE.md +++ /dev/null @@ -1,856 +0,0 @@ -# MetaBuilder Development Guide for AI Assistants - -**Last Updated**: 2026-01-21 -**Status**: Phase 2 Complete, Universal Platform Architecture Defined -**Overall Health**: 82/100 -**Vision**: Universal Platform - One cohesive system for code, 3D, graphics, media, and system administration - ---- - -## Table of Contents - -1. [Universal Platform Vision](#universal-platform-vision) -2. [Before Starting Any Task](#before-starting-any-task) -3. [Core Principles](#core-principles) -4. [Architecture Overview](#architecture-overview) -5. [JSON-First Philosophy](#json-first-philosophy) -6. [Multi-Tenant & Security](#multi-tenant--security) -7. [Code Organization](#code-organization) -8. [What NOT to Do](#what-not-to-do) -9. [Quick Reference](#quick-reference) - ---- - -## Universal Platform Vision - -MetaBuilder is evolving into a **Universal Platform** - a userland operating environment that provides everything through a unified data model and consistent interface. - -### The Premise - -Modern computing is fragmented. Users need dozens of apps, each with its own paradigms, file formats, and learning curves. MetaBuilder provides **one cohesive system** for code editing, 3D modeling, game development, graphics work, system administration, and media production. - -### System Architecture - -``` -┌─────────────────────────────────────────────────────────────────────────────┐ -│ FRONTENDS │ -├─────────────────┬─────────────────────┬─────────────────────────────────────┤ -│ CLI Frontend │ Qt6 Frontend │ Web Frontend (Next.js) │ -│ (Commander) │ (Native Desktop) │ (Browser/Electron/Tauri) │ -└────────┬────────┴──────────┬──────────┴──────────────────┬──────────────────┘ - │ │ │ - └───────────────────┼──────────────────────────────┘ - │ - ┌────────▼────────┐ - │ FRONTEND BUS │ - │ (WebSocket/IPC)│ - └────────┬────────┘ - │ -┌────────────────────────────▼────────────────────────────────────────────────┐ -│ METABUILDER CORE │ -│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ -│ │ State Machine│ │ Command Bus │ │ Event Stream │ │ Entity Graph │ │ -│ │ (XState-like)│ │ (CQRS) │ │ (Pub/Sub) │ │ (DBAL) │ │ -│ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘ │ -│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ -│ │ Undo/Redo │ │ Job Scheduler│ │ Plugin Host │ │ VFS Layer │ │ -│ │ (Event Src) │ │ (DAG Engine) │ │ (Registry) │ │ (Abstraction)│ │ -│ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘ │ -└─────────────────────────────────┬───────────────────────────────────────────┘ - │ -┌─────────────────────────────────▼───────────────────────────────────────────┐ -│ CAPABILITY MODULES │ -├─────────────┬─────────────┬─────────────┬─────────────┬─────────────────────┤ -│ CODE │ GRAPHICS │ 3D │ MEDIA │ SYSTEM │ -│ ──── │ ──────── │ ── │ ───── │ ────── │ -│ • Editor │ • Raster │ • Modeling │ • Audio │ • Files │ -│ • LSP Host │ • Vector │ • Sculpting │ • Video │ • Processes │ -│ • Debugger │ • Compositor│ • Animation │ • Streaming │ • Network │ -│ • Builder │ • Filters │ • Physics │ • Recording │ • Hardware │ -│ • VCS │ • AI Gen │ • Rendering │ • Encoding │ • Containers │ -├─────────────┼─────────────┼─────────────┼─────────────┼─────────────────────┤ -│ GAME │ DATA │ DOCS │ COMMS │ AI │ -│ ──── │ ──── │ ──── │ ───── │ ── │ -│ • Engine │ • Database │ • Writer │ • Chat │ • Local LLM │ -│ • Physics │ • Sheets │ • Slides │ • Email │ • Image Gen │ -│ • Audio │ • Graphs │ • Diagrams │ • Calendar │ • Code Assist │ -│ • Assets │ • ETL │ • PDF │ • Tasks │ • Agents │ -│ • Scripting │ • Analytics │ • Publishing│ • Contacts │ • Embeddings │ -└─────────────┴─────────────┴─────────────┴─────────────┴─────────────────────┘ - │ -┌─────────────────────────────────▼───────────────────────────────────────────┐ -│ RUNTIME LAYER │ -│ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ │ -│ │ Native │ │ WASM │ │ Workflow │ │ GPU │ │ -│ │ (C++/TS) │ │ (Portable) │ │ (JSON DAG) │ │ (Compute) │ │ -│ └────────────┘ └────────────┘ └────────────┘ └────────────┘ │ -└─────────────────────────────────────────────────────────────────────────────┘ -``` - -### Mapping to Current Implementation - -| Architecture Layer | Current Component | Location | -|-------------------|-------------------|----------| -| Entity Graph | DBAL | `dbal/` | -| Job Scheduler | DAG Executor | `workflow/executor/ts/executor/` | -| Plugin Host | Node Executor Registry | `workflow/executor/ts/registry/` | -| Workflow Runtime | Python/TS Plugins | `workflow/plugins/` | -| Web Frontend | Next.js App | `frontends/nextjs/` | - -### Core Subsystems (To Build) - -| Subsystem | Purpose | Status | -|-----------|---------|--------| -| State Machine | Central state management (XState-like) | Planned | -| Command Bus | CQRS command/query separation | Planned | -| Event Stream | Pub/sub for cross-module communication | Planned | -| VFS Layer | Virtual filesystem abstraction | Planned | -| Frontend Bus | WebSocket/IPC for frontend sync | Planned | - -### Capability Categories - -- **Code**: Editor, LSP, debugger, builder, VCS -- **Graphics**: Raster, vector, compositor, filters, AI generation -- **3D**: Modeling, sculpting, animation, physics, rendering -- **Media**: Audio, video, streaming, recording, encoding -- **System**: Files, processes, network, hardware, containers -- **Game**: Engine, physics, audio, assets, scripting -- **Data**: Database, sheets, graphs, ETL, analytics -- **Docs**: Writer, slides, diagrams, PDF, publishing -- **Comms**: Chat, email, calendar, tasks, contacts -- **AI**: Local LLM, image generation, code assist, agents, embeddings - -**Full architecture details**: See [docs/UNIVERSAL_PLATFORM_ARCHITECTURE.md](./docs/UNIVERSAL_PLATFORM_ARCHITECTURE.md) - ---- - -## Before Starting Any Task - -**MANDATORY: Perform exploration BEFORE any implementation work.** Do NOT skip this step or go shallow. - -### Exploration is NOT Optional -- Many task failures occur because exploration was skipped or done superficially -- Shallow exploration (reading only 1-2 files) misses critical context and conventions -- Proper exploration prevents rework, security issues, and architectural mistakes -- Use the **Explore agent** for codebase questions - it goes deep systematically - -### For EVERY Task - Follow This Sequence - -#### 1. Understand the Task Domain -- Read `/CLAUDE.md` (this file) - core principles and patterns -- Read `/AGENTS.md` - domain-specific rules for your task type -- Read `/README.md` - project overview -- Check recent git commits: What was done recently in this area? - -#### 2. Map the Relevant Codebase (Use Explore Agent) -The exploration depth depends on task type: - -**For Feature/Package Work:** -- [ ] Examine `/packages/{packageId}/` structure - all directories and files -- [ ] Review similar existing packages for patterns -- [ ] Check `/schemas/package-schemas/` for validation rules -- [ ] Look at `/dbal/shared/api/schema/entities/` if new entities needed - -**For API/Backend Work:** -- [ ] Review `/frontends/nextjs/src/app/api/v1/[...slug]/route.ts` - main router -- [ ] Check `/frontends/nextjs/src/lib/middleware/` - rate limiting, auth, multi-tenant -- [ ] Examine `/dbal/development/src/adapters/` - database patterns -- [ ] Check `/dbal/shared/api/schema/operations/` - operation specifications -- [ ] Review rate limiting setup in `/docs/RATE_LIMITING_GUIDE.md` - -**For Frontend/UI Work:** -- [ ] Review target `/packages/{packageId}/component/` structure -- [ ] Check `/frontends/nextjs/src/lib/` for rendering and utilities -- [ ] Examine similar UI packages for component patterns -- [ ] Understand the JSON→React rendering flow - -**For Database/Schema Work:** -- [ ] Review all 27 YAML entity definitions in `/dbal/shared/api/schema/entities/` -- [ ] Check `/schemas/package-schemas/` for JSON validation schemas -- [ ] Understand the two-layer schema system (YAML source → Prisma → JSON validation) -- [ ] Review `/docs/MULTI_TENANT_AUDIT.md` for tenant filtering patterns - -**For Testing Work:** -- [ ] Check `/playwright.config.ts` and `/e2e/` structure -- [ ] Review test patterns in `/packages/{packageId}/playwright/` and `tests/` -- [ ] Understand schema validation in `/schemas/package-schemas/tests/` -- [ ] Check how tests handle multi-tenancy - -#### 3. Identify Patterns & Conventions -- Look at 2-3 similar completed implementations -- Identify: naming conventions, directory structure, file organization, code patterns -- Check what's in `/schemas/` and `/dbal/` for your task area -- Note any TODOs or FIXMEs related to your task - -#### 4. Check for Existing Work/Documentation -- Search for related documentation files -- Check git log for similar work: `git log --grep="keyword"` -- Look for ADRs (Architecture Decision Records) or implementation guides -- Note if there are Phase reports or migration docs relevant to your task - -#### 5. Identify Blockers & Dependencies -- Check `/TECH_DEBT.md` and `/SYSTEM_HEALTH_ASSESSMENT.md` for known issues -- Verify dependencies are installed and buildable -- Check if related packages/modules already exist -- Understand what's needed before you can start - -### What "Going 1 Level Deep" Really Means - -**Insufficient Exploration (❌ Don't Do This):** -``` -- Only reading CLAUDE.md -- Checking 1 file and starting to code -- Skipping package structure inspection -- Missing schema validation files -- Not checking existing patterns -``` - -**Proper Exploration (✅ Do This):** -``` -# For package work: -✓ Read package.json file structure -✓ Examine component/ directory and sample files -✓ Check page-config/ for routing patterns -✓ Review workflow/ for existing JSON Script patterns -✓ Look at tests/ and playwright/ for test patterns -✓ Compare with 2-3 similar packages -✓ Check /schemas/package-schemas/ for validation rules - -# For API work: -✓ Review main router ([...slug]/route.ts) -✓ Check middleware implementations -✓ Examine 2-3 similar API endpoints -✓ Review rate limiting patterns -✓ Understand multi-tenant filtering -✓ Check DBAL client usage patterns - -# For schema work: -✓ Review all 27 YAML entities (/dbal/shared/api/schema/entities/) -✓ Check JSON validation schemas (/schemas/package-schemas/) -✓ Understand entity operations specifications -✓ Review existing entity implementations -✓ Check how tenantId is used everywhere -``` - -### When to Use the Explore Agent - -Use the **Explore agent** for: -- Understanding codebase structure and patterns -- Finding where code lives (searching across files) -- Mapping dependencies and integrations -- Answering "How does X work?" questions -- Identifying similar existing implementations -- Understanding architecture layers - -**Do NOT** use Explore for: -- Simple file reads (use Read tool directly) -- Known file paths (use Read/Glob/Grep) -- Trivial searches (use Glob for patterns) - -### Common Exploration Mistakes (And How to Fix Them) - -| Mistake | Impact | Fix | -|---------|--------|-----| -| Not reading `/CLAUDE.md` principles first | Violates 95/5 rule, multi-tenant issues | Always start with Core Principles section | -| Skipping schema review | Missing validation requirements, data structure issues | Review `/schemas/package-schemas/` before coding | -| Not checking existing patterns | Inconsistent code, duplicate work | Examine 2-3 similar implementations | -| Ignoring multi-tenant requirements | Data leaks, security issues | Read Multi-Tenant section, check all DB queries have tenantId filter | -| Missing rate limiting | API abuse risk | Review `/docs/RATE_LIMITING_GUIDE.md` for new endpoints | -| Not understanding JSON-first philosophy | TypeScript bloat, wrong patterns | Review JSON-First Philosophy section | -| Assuming you know the structure | Wrong file locations, wasted effort | Use Explore agent to map the actual structure | - ---- - -## Core Principles - -### 1. 95% Data, 5% Code - -MetaBuilder is **95% configuration/data in JSON, 5% infrastructure code in TypeScript**. - -- UI components defined as JSON -- Workflows defined as JSON Script -- Pages/routes defined as JSON -- Business logic in JSON, not TypeScript -- Code only handles rendering and persistence - -### 2. Schema-First Development - -**Two-layer schema system - YAML entities + JSON validation schemas**. - -**Layer 1: Core Database Schemas (YAML)** -- Source of truth for database structure -- Location: `/dbal/shared/api/schema/entities/` (27 files) -- 5 core system entities (User, Session, Workflow, Package, UIPage) -- 3 access control entities (Credential, ComponentNode, PageConfig) -- 6 package-specific entities (Forum, Notification, AuditLog, Media, IRC, Streaming) -- 4 domain-specific entities (Product, Game, Artist, Video) - -**Layer 2: JSON Validation Schemas (JSON Schema)** -- Location: `/schemas/package-schemas/` (27 files) -- Validates package files: metadata, entities, types, scripts, components, API, events, etc. -- Reference: [SCHEMAS_COMPREHENSIVE.md](./SCHEMAS_COMPREHENSIVE.md) - -**Development Workflow**: -```bash -# 1. Define entity schema in YAML -# /dbal/shared/api/schema/entities/core/my-entity.yaml - -# 2. Generate Prisma from YAML -npm --prefix dbal/development run codegen:prisma - -# 3. Push to database -npm --prefix dbal/development run db:push - -# 4. Create package files with proper schemas -# Use schemas/package-schemas/*.json for validation - -# 5. Code follows the schema -``` - -### 3. Multi-Tenant by Default - -**Every entity has tenantId - no exceptions.** - -```typescript -// ✅ CORRECT: Filter by tenant -const db = getDBALClient() -const records = await db.users.list({ - filter: { tenantId: user.tenantId } -}) - -// ❌ WRONG: Missing tenant filter (data leak!) -const records = await db.users.list() -``` - -### 4. Use JSON Script, Not TypeScript for Logic - -**Business logic lives in JSON Script v2.2.0, not TypeScript.** - -```json -{ - "version": "2.2.0", - "nodes": [ - { - "id": "filter", - "type": "operation", - "op": "filter", - "condition": "{{ $json.status === 'active' }}" - } - ] -} -``` - -Why: Non-technical users understand JSON, GUIs can generate it, no compilation needed. - -### 5. One Lambda Per File - -**Each file = one focused function, period.** - -```typescript -// ✅ CORRECT -// src/lib/users/createUser.ts -export async function createUser(data: UserData): Promise { ... } - -// src/lib/users/listUsers.ts -export async function listUsers(): Promise { ... } - -// ❌ WRONG: Multiple functions in one file -// src/lib/users.ts -export function createUser() { ... } -export function listUsers() { ... } -export function deleteUser() { ... } -``` - -### 6. DBAL > Prisma > Raw SQL - -**Use highest abstraction level available.** - -```typescript -// ✅ HIGHEST (handles multi-tenant, ACL, caching) -const db = getDBALClient() -const users = await db.users.list() - -// ⚠️ MIDDLE (only if DBAL doesn't support) -const prisma = getPrismaClient() -const users = await prisma.user.findMany() - -// ❌ LOWEST (never do this) -const query = "SELECT * FROM user" -``` - ---- - -## Repository Structure (Quick Reference) - -**Top-Level Directories:** -``` -/dbal/ # Database Abstraction Layer (TypeScript) - /development/src/ # DBAL implementation - /shared/api/schema/ # YAML entity definitions (27 files - SOURCE OF TRUTH) - /shared/api/schema/operations/ # Operation specifications - -/frontends/ # Multiple frontend implementations - /nextjs/src/ # Primary web UI (Next.js) - /cli/ # Command-line interface - /qt6/ # Desktop application - -/packages/ # 62 feature packages - /admin_dialog/, /audit_log/, /dashboard/, ... (each has own structure) - -/schemas/ # JSON validation schemas - /package-schemas/ # 27 JSON schema files + examples - -/docs/ # Documentation (43+ guides, implementation specs) -/deployment/ # Docker & infrastructure as code -/e2e/ # End-to-end Playwright tests -/scripts/ # Build and migration scripts -/spec/ # TLA+ formal specifications -/prisma/ # Prisma configuration -``` - -**Critical Root Files:** -- `CLAUDE.md` - THIS FILE (core principles) -- `AGENTS.md` - Domain-specific rules and AI instructions -- `README.md` - Project overview -- `package.json` - Workspace configuration -- `playwright.config.ts` - E2E test config -- `152+ .md files` - Phase reports, analysis, implementation guides - -**Understanding the Structure:** -- **Schemas drive development**: YAML entities → Prisma schema → JSON validation → Code -- **Packages are modular**: Each package is self-contained with its own structure -- **Multi-layer architecture**: Database layer (DBAL) → API (Next.js) → Frontend (React/CLI/Qt6) -- **Everything multi-tenant**: Every entity has `tenantId` - mandatory filtering - ---- - -## Architecture Overview - -### Three-Tier System - -``` -Frontend (Next.js/CLI/Qt6) - ↓ -DBAL (TypeScript Phase 2, C++ Phase 3) - ↓ -Database (SQLite dev, PostgreSQL prod) -``` - -### Routing Pattern - -``` -/api/v1/{tenant}/{package}/{entity}[/{id}[/{action}]] - -GET /api/v1/acme/forum_forge/posts → List -POST /api/v1/acme/forum_forge/posts → Create -GET /api/v1/acme/forum_forge/posts/123 → Get -PUT /api/v1/acme/forum_forge/posts/123 → Update -DELETE /api/v1/acme/forum_forge/posts/123 → Delete -POST /api/v1/acme/forum_forge/posts/123/like → Custom action -``` - ---- - -## JSON-First Philosophy - -### UI Components as JSON - -```json -{ - "id": "comp_hero", - "name": "Hero Section", - "render": { - "type": "Container", - "props": { "variant": "primary" }, - "children": [ - { - "type": "Heading", - "props": { "text": "Welcome" } - } - ] - } -} -``` - -### Pages as JSON - -```json -{ - "id": "page_home", - "path": "/", - "title": "Home", - "tenantId": "acme", - "component": "home_page", - "isPublished": true -} -``` - -### Workflows as JSON Script - -```json -{ - "version": "2.2.0", - "nodes": [ - { - "id": "step1", - "type": "operation", - "op": "transform_data", - "input": "{{ $json }}", - "output": "{{ $utils.flatten($json) }}" - } - ] -} -``` - -**No hardcoded TypeScript UI logic!** Everything is JSON that GUIs can generate. - ---- - -## Multi-Tenant & Security - -### Rate Limiting (Always Apply) - -```typescript -import { applyRateLimit } from '@/lib/middleware' - -export async function POST(request: NextRequest) { - // Apply rate limit - const limitResponse = applyRateLimit(request, 'mutation') - if (limitResponse) return limitResponse - - // Rest of handler -} -``` - -**Rate limits:** -- Login: 5 attempts/minute -- Register: 3 attempts/minute -- List: 100 requests/minute -- Mutations: 50 requests/minute -- Bootstrap: 1 attempt/hour - -### Multi-Tenant Filtering (Always Apply) - -```typescript -// ✅ CORRECT -const filter = tenantId !== undefined ? { tenantId } : {} -const records = await adapter.list(entity, { filter }) - -// ❌ WRONG: Data leak! -const records = await adapter.list(entity) -``` - -**Rule**: Every database query filters by tenantId. No exceptions. - ---- - -## Code Organization - -### Directory Structure - -``` -/dbal/ - /development/src/ - /core/client/ # DBAL implementation - /adapters/ # Prisma adapter - /seeds/ # Seed logic - /shared/api/schema/ # YAML schemas (source of truth) - /entities/ # 27 entity definitions - /operations/ # 7 operation specs - -/frontends/nextjs/ - /src/lib/ - /middleware/ # Express-style middleware - /routing/ # API routing helpers - /db-client.ts # DBAL singleton - /app/api/ - /v1/[...slug]/ # RESTful API - -/packages/ # 62 packages total - /{packageId}/ - /page-config/ # Routes - /workflow/ # Workflows (JSON Script v2.2.0) - /component/ # Components - /permissions/ # Roles & permissions - /styles/ # Design tokens - /tests/ # Unit tests - /playwright/ # E2E tests - package.json # Package metadata + file inventory - -/schemas/ - /package-schemas/ # 27 JSON schemas for validation - /{schema}.json # Metadata, entities, types, scripts, components, etc. - /examples/ # Reference templates (minimal, complete, advanced) - README.md # Schema overview - SEED_SCHEMAS.md # Seed data validation guide - yaml-schema.yaml # YAML meta-schema -``` - -### What Goes Where - -| Item | Location | Format | -|------|----------|--------| -| Entity definitions | `/dbal/shared/api/schema/entities/` | YAML (27 files) | -| Entity operations | `/dbal/shared/api/schema/operations/` | YAML (7 files) | -| API routes | `/frontends/nextjs/src/app/api/` | TypeScript | -| UI definitions | `/packages/{pkg}/component/` | JSON | -| Workflows | `/packages/{pkg}/workflow/` | JSON Script v2.2.0 | -| Pages/routes | `/packages/{pkg}/page-config/` | JSON | -| Package metadata | `/packages/{pkg}/package.json` | JSON (with file inventory) | -| Schema validation | `/schemas/package-schemas/` | JSON Schema (27 files) | - -### Package File Inventory - -Each package.json now includes a `files` section documenting all contained files: - -```json -{ - "packageId": "my_package", - "files": { - "directories": ["components", "page-config", "tests", ...], - "byType": { - "components": ["components/ui.json"], - "pages": ["page-config/page-config.json"], - "workflows": ["workflow/init.jsonscript"], - "tests": ["tests/test.json", "playwright/tests.json"], - "config": ["package.json"], - "permissions": ["permissions/roles.json"], - "styles": ["styles/tokens.json"], - "schemas": ["entities/schema.json"] - } - } -} -``` - -See [PACKAGE_INVENTORY_GUIDE.md](./PACKAGE_INVENTORY_GUIDE.md) for usage examples. - ---- - -## What NOT to Do - -### ❌ Don't Use Prisma Directly - -```typescript -// ❌ WRONG -import { prisma } from '@/lib/config/prisma' -const users = await prisma.user.findMany() - -// ✅ CORRECT -const db = getDBALClient() -const users = await db.users.list() -``` - -### ❌ Don't Hardcode Configuration - -```typescript -// ❌ WRONG: Hardcoded routes -const routes = [ - { path: '/', component: 'home' }, - { path: '/about', component: 'about' } -] - -// ✅ CORRECT: In database/JSON -// /packages/my_package/page-config/routes.json -``` - -### ❌ Don't Skip tenantId Filtering - -```typescript -// ❌ WRONG: Data leak! -const users = await db.users.list() - -// ✅ CORRECT -const users = await db.users.list({ - filter: { tenantId } -}) -``` - -### ❌ Don't Put Multiple Functions in One File - -```typescript -// ❌ WRONG -// src/lib/users.ts -export function create() { ... } -export function list() { ... } -export function delete() { ... } - -// ✅ CORRECT -// src/lib/users/create.ts -export function createUser() { ... } - -// src/lib/users/list.ts -export function listUsers() { ... } - -// src/lib/users/delete.ts -export function deleteUser() { ... } -``` - -### ❌ Don't Use TypeScript for Business Logic - -```typescript -// ❌ WRONG: TypeScript workflow logic -export function processOrder(order) { - if (order.status === 'pending') { - // Complex TypeScript logic - } -} - -// ✅ CORRECT: JSON Script -{ - "nodes": [ - { - "op": "filter", - "condition": "{{ $json.status === 'pending' }}" - } - ] -} -``` - -### ❌ Don't Forget Rate Limiting on APIs - -```typescript -// ❌ WRONG: Missing rate limit -export async function POST(request: NextRequest) { - // No protection! -} - -// ✅ CORRECT -export async function POST(request: NextRequest) { - const limitResponse = applyRateLimit(request, 'mutation') - if (limitResponse) return limitResponse -} -``` - -### ❌ Don't Ignore YAML Schemas - -```typescript -// ❌ WRONG: Schema in code -interface User { - id: string - name: string -} - -// ✅ CORRECT: YAML source of truth -// /dbal/shared/api/schema/entities/core/user.yaml -``` - ---- - -## Quick Reference - -### Common Commands - -```bash -# Development -npm run dev # Start dev server -npm run typecheck # Check TypeScript -npm run build # Build production -npm run test:e2e # Run tests - -# Database -npm --prefix dbal/development run codegen:prisma # YAML → Prisma -npm --prefix dbal/development run db:push # Apply schema - -# Documentation -http://localhost:3000/api/docs # API docs -``` - -### Key Files - -- **Rate Limiting**: `frontends/nextjs/src/lib/middleware/rate-limit.ts` -- **DBAL Client**: `frontends/nextjs/src/lib/db-client.ts` -- **API Routes**: `frontends/nextjs/src/app/api/v1/[...slug]/route.ts` -- **YAML Schemas**: `dbal/shared/api/schema/entities/` (source of truth) -- **JSON Schemas**: `schemas/package-schemas/` (validation & documentation) - -### Documentation - -**Core Principles & Development**: -- **This file**: CLAUDE.md (core principles) -- **Rate Limiting**: `/docs/RATE_LIMITING_GUIDE.md` -- **Multi-Tenant**: `/docs/MULTI_TENANT_AUDIT.md` -- **API Reference**: `/docs/API_DOCUMENTATION_GUIDE.md` -- **Strategic Guide**: `/STRATEGIC_POLISH_GUIDE.md` - -**Schemas & Package Documentation**: -- **All Schemas**: [SCHEMAS_COMPREHENSIVE.md](./SCHEMAS_COMPREHENSIVE.md) - 27 JSON schemas + 27 YAML entities -- **Quick Start**: `schemas/package-schemas/QUICKSTART.md` - 30-second patterns -- **Schema Reference**: `schemas/package-schemas/SCHEMAS_README.md` - Complete 16 schema overview -- **Package Inventory**: [PACKAGES_INVENTORY.md](./PACKAGES_INVENTORY.md) - All 62 packages with files -- **Package Guide**: [PACKAGE_INVENTORY_GUIDE.md](./PACKAGE_INVENTORY_GUIDE.md) - How to use package.json files section -- **Seed Schemas**: `schemas/SEED_SCHEMAS.md` - Seed data validation & entity types -- **Root Schemas**: `schemas/README.md` - Schema overview & core database structure - ---- - -## Development Workflow - -### Phase 1: Explore (MANDATORY - Never Skip) - -1. **Read Foundation Docs** (15 min) - - [ ] `/CLAUDE.md` core principles - - [ ] `/AGENTS.md` for your domain - - [ ] `/README.md` project overview - -2. **Map the Codebase** (30-45 min) - - [ ] Use Explore agent to understand structure - - [ ] Identify where similar work exists - - [ ] Check relevant schemas and types - - [ ] Review 2-3 similar implementations - -3. **Verify Dependencies** (10 min) - - [ ] Build passes: `npm run build` - - [ ] Tests pass: `npm run test` - - [ ] No blocking issues in `/TECH_DEBT.md` - -**Checklist Before Moving to Phase 2:** -- [ ] You can explain the existing pattern for your task type -- [ ] You've found 2-3 examples of similar work -- [ ] You understand the directory structure -- [ ] You know what schemas/types are involved -- [ ] You've identified any blockers or dependencies - -### Phase 2: Design (Before Coding) - -1. **Schema Design** (if needed) - - [ ] Update YAML in `/dbal/shared/api/schema/entities/` - - [ ] Generate Prisma: `npm --prefix dbal/development run codegen:prisma` - - [ ] Create validation schemas in `/schemas/package-schemas/` - -2. **Architecture Design** - - [ ] Sketch file structure and naming - - [ ] Identify multi-tenant filtering points - - [ ] Plan rate limiting strategy - - [ ] Document any deviations from existing patterns - -3. **Get Alignment** - - [ ] If patterns unclear, ask for clarification - - [ ] If task scope seems large, break it down - - [ ] Verify assumptions about dependencies - -### Phase 3: Implementation - -1. Start Implementation with Code Review Agent proactively -2. Apply multi-tenant filtering to ALL database queries -3. Add rate limiting to sensitive endpoints -4. Follow one-function-per-file pattern -5. Use JSON Script for business logic, not TypeScript - -### Before Committing - -- [ ] TypeScript compiles: `npm run typecheck` -- [ ] Tests pass (99%+): `npm run test:e2e` -- [ ] Build succeeds: `npm run build` -- [ ] **One function per file** rule followed -- [ ] **Multi-tenant filtering** applied everywhere (tenantId check) -- [ ] **Rate limiting** on sensitive endpoints -- [ ] **No Prisma direct usage** (use DBAL) -- [ ] **Business logic in JSON Script**, not TypeScript -- [ ] Documentation updated -- [ ] Code review completed - ---- - -**Status**: Production Ready (Phase 2 Complete) -**Next**: Universal Platform - Core Infrastructure (State Machine, Command Bus, Event Stream, VFS, Frontend Bus) - diff --git a/PROMPT.md b/PROMPT.md deleted file mode 100644 index 6c2c5b194..000000000 --- a/PROMPT.md +++ /dev/null @@ -1,710 +0,0 @@ -# 🚀 MEGA PROMPT: Hybrid TypeScript/C++ Package System - -**Version**: 1.0 -**Last Updated**: January 2026 -**For**: AI Assistants working on 3-frontend package bootstrap system - ---- - -## 🎯 PROJECT ESSENCE - -This is a **hybrid TypeScript/C++ bootloader system** that powers a modular package architecture. Three frontends (CLI, Qt6, Next.js) act as **entry points** that bootstrap a NestJS-based package system. The system is pragmatic: schemas are improvable, not sacred; patterns come from functional programming; and delightfulness matters. - -### The Mental Model - -``` -┌─────────────────────────────────────────────────────────────┐ -│ User Input │ -│ (CLI / Qt6 GUI / Next.js Web) │ -└──────────────┬──────────────────────────────────────────────┘ - │ - ▼ - ┌──────────────┐ - │ Bootloader │ ← Frontend-specific logic - │ (3x) │ - Routes/state management - └──────────────┘ - │ - ▼ - ┌──────────────┐ - │ Package │ ← Central coordinator - │ System │ - Loads packages from /packages/ - │ (NestJS) │ - Manages seed data & schemas - └──────────────┘ - │ - ┌──────┴──────┐ - ▼ ▼ - ┌────────┐ ┌────────┐ - │ DBAL │ │ C++ │ - │ Layer │ │ Daemons│ - │ │ │(Media) │ - └────────┘ └────────┘ - │ │ - └──────┬──────┘ - ▼ - ┌──────────────┐ - │ Database │ - │(PostgreSQL) │ - └──────────────┘ -``` - -**Key Principle**: Seed data lives in `/packages/*/seed/`, not scattered elsewhere. Schemas are single source of truth but improvable. - ---- - -## 📋 PRIORITY READING (In Order) - -1. **IMMEDIATE_FIXES.md** ⭐ - What's actually broken right now -2. **claude.md** - System overview and patterns -3. **agents.md** - Agent guidelines for this codebase -4. **README.md** - Project structure and setup -5. **ROADMAP.md** (if exists) - Vision and evolution -6. **docs/** - Architecture, testing, deployment guides -7. **schemas/** - Data models (understand but improve if needed) - ---- - -## 🏗️ ARCHITECTURE AT A GLANCE - -### Three Frontends (Each a Bootloader) - -| Frontend | Purpose | Tech Stack | Startup Flow | -|----------|---------|-----------|--------------| -| **CLI** | Command-line interface | TypeScript/Node | Read config → Load packages → Execute | -| **Qt6** | Desktop GUI application | C++/Qt6 | Parse Qt resources → Load packages → Render | -| **Next.js** | Web application | TypeScript/React | HTTP request → Load packages → Render page | - -**Critical**: Each frontend is a **bootloader**. It doesn't contain business logic—it loads and orchestrates packages. - -### Package System (The Heart) - -``` -/packages/ -├── [package-name]/ -│ ├── seed/ ← REAL seed data (not /dbal/shared/seeds/) -│ │ ├── metadata.json ← Package manifest -│ │ ├── page-config/ ← Routes/pages -│ │ ├── workflow/ ← Automation workflows -│ │ ├── credential/ ← API credentials -│ │ ├── component/ ← UI components -│ │ └── [entity-type]/ ← Other entity types -│ ├── src/ ← Business logic (if TypeScript package) -│ ├── package.json -│ └── README.md -``` - -**Pattern**: Each package is self-contained. Seed data describes what the package does. Business logic implements it. - -### DBAL Layer (Database Access) - -- **Phase 2** (Current): TypeScript DBAL in `/dbal/development/` -- **Phase 3** (Future): C++ daemon in `/dbal/production/` -- **Shared**: YAML schemas at `/dbal/shared/api/schema/entities/` (source of truth) - -**Rule**: Always use `getDBALClient()` from `@/dbal`, never raw Prisma. - -### Schemas: Two Layers - -| Layer | Location | Purpose | Editable? | -|-------|----------|---------|-----------| -| **Entity Schemas** | `/dbal/shared/api/schema/entities/*.yaml` | Database structure (source of truth) | Yes, improve as needed | -| **Validation Schemas** | `/schemas/seed-data/*.schema.json` | Seed data validation | Yes, extend for new features | -| **Package Schemas** | `/schemas/package-schemas/*.schema.json` | Package system architecture | Yes, treat as blueprint | - -**Philosophy**: Schemas are **improvable**. If they lack features, extend them. They're not dogma. - ---- - -## 🎨 DESIGN PHILOSOPHY (How to Think) - -### 1. Functional Programming First - -```typescript -// ✅ GOOD: Functional, pure, composable -const loadPackage = (path: string) => readJSON(path); -const filterActive = (packages: Package[]) => packages.filter(p => p.active); -const orchestrate = pipe(loadPackages, filterActive, initializeAll); - -// ❌ BAD: Imperative, class-heavy, side effects -class PackageManager { - private packages: Package[] = []; - async load(path) { this.packages = await readJSON(path); } - async filter() { this.packages = this.packages.filter(p => p.active); } -} -``` - -### 2. Make It Delightful - -- Code should be obvious. If it requires 20 comments, refactor it. -- Styling matters. Use Tailwind (web), Qt stylesheets (desktop), ANSI colors (CLI). -- Error messages should hint at solutions, not just fail. -- Performance should feel snappy. - -### 3. Schemas Are Improvable - -```json -// If schema lacks what you need: -// 1. Document the gap -// 2. Extend the schema -// 3. Update examples -// 4. Move on - -// Example: Script schema is JSON AST, but n8n-style is simpler for visual GUIs -{ - "trigger": "http.post", - "input": { "body": "{ name, email }" }, - "steps": [ - { "action": "validate.email", "target": "$input.email" }, - { "action": "db.create", "table": "users", "data": "$input" } - ], - "output": { "userId": "$steps[1].id" } -} -// This coexists with JSON AST for complex logic. Both are valid. -``` - -### 4. No Magic, No Lua - -- **No Lua**: JSON is sufficient for visual scripting (n8n, Scratch patterns). -- **No magic imports**: Be explicit. Use dependency injection. -- **No hidden state**: Logs should explain what's happening. - -### 5. GUIs Over Raw Input - -```typescript -// ❌ RAW: User types CSS directly -