docs: Create TECH_DEBT.md with bot-actionable task instructions

Add explicit, step-by-step instructions for each tech debt item so bots know exactly what to do: - TD-1: DBAL Refactoring (🔴 CRITICAL) - Move database logic from frontend to DBAL (13 steps) - TD-2: Rate limiting (🟡 HIGH) - Protect API endpoints from abuse - TD-3: OpenAPI/Swagger (🟡 HIGH) - API contract visibility and client generation - TD-4: Error handling docs (🟡 MEDIUM) - Consistent error response patterns Also included 9 Phase 3+ tasks (TD-5 through TD-13) for future reference. Each task includes: - Current status and priority - Clear problem statement - Impact analysis - Bot-actionable instructions (step-by-step) - Expected outcomes Updated ROADMAP.md, CLAUDE.md, and AGENTS.md to reference TECH_DEBT.md. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-04-24 13:54:57 +00:00 · 2026-01-15 00:49:20 +00:00
parent 9f3f8b14b6
commit ccf383a642
4 changed files with 280 additions and 22 deletions
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -4,6 +4,7 @@

 ## Quick Navigation

+- **Finding Tech Debt** → See `/TECH_DEBT.md` (bot-actionable task list with explicit instructions)
 - **Starting Development** → [Core Principles](#core-principles) + [Scoped Rules](#scoped-rules)
 - **Working on DBAL** → [DBAL Development](#dbal-development-phase-2--phase-3)
 - **Working on Frontend** → [Frontend Development](#frontend-development)
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -950,16 +950,18 @@ cmake --build build

 ### Project Guidance
 12. **AGENTS.md** - ⭐ **READ FIRST** - All agents. Core principles, DBAL development, frontend, packages, testing, deployment, troubleshooting, best practices
+13. **TECH_DEBT.md** - ⭐ **For AI Assistants** - Bot-actionable tech debt tasks (TD-1 through TD-13) with explicit step-by-step instructions

-### DBAL Documentation
-13. **dbal/shared/docs/IMPLEMENTATION_SUMMARY.md** - Phase 2 overview
-14. **dbal/shared/docs/PHASE2_IMPLEMENTATION.md** - Phase 2 detailed guide
-15. **dbal/production/docs/PHASE3_DAEMON.md** - Phase 3 design (future)
+### DBAL Documentation & Refactoring
+14. **DBAL_REFACTOR_PLAN.md** - Detailed implementation plan (reference for TD-1)
+15. **dbal/shared/docs/IMPLEMENTATION_SUMMARY.md** - Phase 2 overview
+16. **dbal/shared/docs/PHASE2_IMPLEMENTATION.md** - Phase 2 detailed guide
+17. **dbal/production/docs/PHASE3_DAEMON.md** - Phase 3 design (future)

 ### Schema & Package System
-16. **schemas/SCHEMAS_README.md** - Package system definitions
-17. **schemas/QUICKSTART.md** - Package system quick start
-18. **schemas/package-schemas/** - Complete schema definitions:
+18. **schemas/SCHEMAS_README.md** - Package system definitions
+19. **schemas/QUICKSTART.md** - Package system quick start
+20. **schemas/package-schemas/** - Complete schema definitions:
    - `script_schema.json` - JSON Script language specification (v2.2.0, planned n8n migration)
    - `metadata_schema.json` - Package structure
    - `entities_schema.json` - Database models
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -320,23 +320,24 @@ The original Spark-based version is preserved in `/old` directory for reference:

 ### What's NOT Done Yet (Phase 2 Remaining & Future Phases)

-❌ **Phase 2 - Final Items:**
- Rate limiting (API endpoint protection)
- OpenAPI/Swagger API documentation
- Detailed error handling documentation
+❌ **Phase 2 - Final Items** (see `/TECH_DEBT.md` for detailed instructions):
+- TD-1: DBAL Refactoring - Move database logic from frontend to DBAL (🔴 CRITICAL)
+- TD-2: Rate limiting - API endpoint protection (🟡 HIGH)
+- TD-3: OpenAPI/Swagger API documentation (🟡 HIGH)
+- TD-4: Error handling documentation (🟡 MEDIUM)

-🔮 **Phase 3+:**
- [ ] Rich form editors with nested object/array support
- [ ] Bulk operations (multi-select, bulk delete, export)
- [ ] Advanced filtering UI with visual builder
- [ ] Relationship/foreign key dropdown selectors
- [ ] God Panel (system admin dashboard)
- [ ] Workflow automation UI
- [ ] Advanced authentication (SSO, SAML, OAuth)
- [ ] C++ DBAL daemon (production security mode)
- [ ] Multi-source package system (NPM, git, http)
+🔮 **Phase 3+** (blocked until Phase 2 completes):
+- TD-5: Rich form editors with nested object/array support
+- TD-6: Bulk operations (multi-select, bulk delete, export)
+- TD-7: Advanced filtering UI with visual builder
+- TD-8: Relationship/foreign key dropdown selectors
+- TD-9: God Panel (system admin dashboard)
+- TD-10: Workflow automation UI
+- TD-11: Advanced authentication (SSO, SAML, OAuth)
+- TD-12: C++ DBAL daemon (production security mode)
+- TD-13: Multi-source package system (NPM, git, http)

-See [Roadmap Phases](#roadmap-phases) for detailed roadmap.
+**See `/TECH_DEBT.md` for bot-actionable instructions on each task.**

 ---

--- a/TECH_DEBT.md
+++ b/TECH_DEBT.md
@@ -0,0 +1,254 @@
+# Tech Debt Tracker
+
+> **For AI Assistants**: This file explicitly defines what needs to be done for each tech debt item. Read the "Bot Instructions" section for each task to understand exactly what's required.
+
+**Current Status**: Phase 2 - 90% Complete. 3 critical items blocking Phase 3.
+
+---
+
+## 📋 Phase 2 Tech Debt (Priority Order)
+
+### TD-1: DBAL Refactoring - Move Database Logic from Frontend to DBAL
+
+**Status**: 🔴 **CRITICAL** - Blocks Phase 3 start
+
+**Problem**: Database logic is scattered:
+- Prisma schema at `/prisma/schema.prisma` (should be in DBAL)
+- Adapter factory at `/frontends/nextjs/src/lib/dbal-client/adapter/get-adapter.ts` (duplicate code)
+- Prisma context at `/frontends/nextjs/src/lib/config/prisma.ts` (should be in DBAL)
+- Seeding split between `/seed/`, `/frontends/nextjs/src/lib/seed/`, and code (should be in DBAL)
+- Frontend uses raw adapter instead of entity operations
+
+**Impact**: DBAL can't be versioned/deployed independently, code duplication, poor separation of concerns
+
+**Reference Document**: `/DBAL_REFACTOR_PLAN.md` - Contains full implementation details
+
+**Bot Instructions**:
+1. Read `/DBAL_REFACTOR_PLAN.md` completely to understand all 3 phases
+2. Follow the "Implementation Order" section (page 356-375) exactly
+3. After each step, verify against the "Verification Checklist" (page 379-392)
+4. Commit after each phase completes successfully
+5. Total work: 13 atomic steps across 3 phases
+
+**Expected Outcome**:
+- ✅ Prisma schema moved to `/dbal/development/prisma/schema.prisma`
+- ✅ DBAL owns all Prisma initialization
+- ✅ DBAL exports `getDBALClient()`, `seedDatabase()`, `getPrismaClient()`
+- ✅ Frontend has single `/frontends/nextjs/src/lib/db-client.ts` integration point
+- ✅ All frontend code uses `db.users.list()` pattern, not `adapter.list('User')`
+- ✅ Tests pass with new structure
+- ✅ Old code deleted (`dbal-client/`, `database-dbal/`, `config/prisma.ts`)
+
+---
+
+### TD-2: Rate Limiting for API Endpoints
+
+**Status**: 🟡 **HIGH** - Needed for production
+
+**Problem**: No rate limiting on `/api/*` endpoints. Bots can enumerate users, brute force, or DoS.
+
+**Impact**: Security vulnerability. Production deployment blocked until implemented.
+
+**Bot Instructions**:
+1. Research rate limiting options for Next.js (e.g., `next-rate-limit`, `redis-based`, `in-memory`)
+2. Evaluate against requirements:
+   - Per-IP limiting (not per-user, since public endpoints)
+   - Configurable limits by endpoint
+   - Works in development (SQLite) and production (Redis)
+   - Graceful failure if Redis unavailable
+3. Implement in `/api/middleware.ts` or route handler
+4. Add tests in `e2e/rate-limiting.spec.ts`
+5. Document in `/docs/API_RATE_LIMITING.md`
+6. Update `/ROADMAP.md` to mark as complete
+
+**Expected Outcome**:
+- ✅ `GET /api/users` limited to 100 req/min per IP
+- ✅ `POST /api/users/login` limited to 5 req/min per IP
+- ✅ Exceeding limit returns 429 status
+- ✅ Tests verify both pass and rate-limited scenarios
+
+---
+
+### TD-3: OpenAPI/Swagger API Documentation
+
+**Status**: 🟡 **HIGH** - Needed for frontend/API clarity
+
+**Problem**: No API specification. Frontend developers must read code to understand endpoints.
+
+**Impact**: Slow development, inconsistency, hard to maintain.
+
+**Bot Instructions**:
+1. Choose OpenAPI generator:
+   - Option A: `@ts-rest/open-api` (lightweight, TypeScript-first)
+   - Option B: `swagger-jsdoc` (JSDoc-based, intrusive)
+   - Option C: Manual YAML at `/openapi.yaml`
+   Recommendation: Option A for tight TypeScript integration
+
+2. Generate OpenAPI 3.0 spec from route handlers:
+   - Document all GET, POST, PUT, DELETE endpoints in `/api/*`
+   - Include request/response schemas
+   - Document authentication requirements
+   - Mark deprecated endpoints
+
+3. Serve Swagger UI at `/api/docs`
+
+4. Verify spec in `/openapi.yaml` (at root, version-controlled)
+
+5. Add tests to verify:
+   - All routes documented
+   - Schemas match actual request/response types
+   - No breaking changes without major version bump
+
+6. Update AGENTS.md "API Documentation" section with usage
+
+**Expected Outcome**:
+- ✅ `/api/docs` shows interactive Swagger UI
+- ✅ `/openapi.yaml` exists with complete spec
+- ✅ All endpoints documented with examples
+- ✅ Request/response types auto-generated from TypeScript
+- ✅ Frontend can generate client SDK from spec
+
+---
+
+### TD-4: Error Handling Documentation
+
+**Status**: 🟡 **MEDIUM** - Improves developer experience
+
+**Problem**: No centralized docs on error handling patterns. Developers guess.
+
+**Impact**: Inconsistent error responses, hard to debug, poor API contracts.
+
+**Bot Instructions**:
+1. Create `/docs/ERROR_HANDLING.md`:
+   - Document error response format (should be consistent)
+   - List all possible HTTP status codes used
+   - Explain retry logic for each error type
+   - Show examples of client-side handling
+
+2. Audit all `/api/*` endpoints:
+   - Ensure consistent error response format
+   - Use correct HTTP status codes (400, 401, 403, 404, 409, 500, etc.)
+   - All errors should have `code`, `message`, `details` fields (if applicable)
+
+3. Add error handling test suite (`e2e/error-handling.spec.ts`):
+   - Test 404 on missing resource
+   - Test 401 on unauthorized access
+   - Test 400 on invalid input
+   - Test 500 on server error
+
+4. Document in AGENTS.md under "Error Handling" section
+
+**Expected Outcome**:
+- ✅ `/docs/ERROR_HANDLING.md` exists with patterns
+- ✅ All errors follow consistent JSON format
+- ✅ Tests verify error scenarios
+- ✅ Developers know how to handle errors in frontend
+
+---
+
+## 📊 Phase 3+ Tech Debt (Future)
+
+These are planned but blocked until Phase 2 completes:
+
+### TD-5: Rich Form Editors with Nested Object/Array Support
+- Implement dynamic form builder
+- Support nested objects, arrays, conditional fields
+- Use JSON Schema for validation
+- Add e2e tests
+
+### TD-6: Bulk Operations (Multi-Select, Bulk Delete, Export)
+- Add bulk delete endpoint `/api/[entity]/bulk-delete`
+- Add bulk export endpoint `/api/[entity]/export`
+- Implement multi-select UI in generic table renderer
+- Add tests for bulk operations
+
+### TD-7: Advanced Filtering UI with Visual Builder
+- Create visual filter builder component
+- Support multiple conditions (AND/OR)
+- Persist filters in URL/localStorage
+- Add tests
+
+### TD-8: Relationship/Foreign Key Dropdown Selectors
+- Auto-generate dropdowns for foreign keys
+- Support search/autocomplete for large datasets
+- Lazy-load options on demand
+- Add tests
+
+### TD-9: God Panel (System Admin Dashboard)
+- Create `/admin` page (requires God role)
+- Dashboard showing system stats
+- User management UI
+- Tenant management UI
+- Job monitoring
+- Error tracking
+
+### TD-10: Workflow Automation UI
+- Visual workflow builder
+- Trigger selection (API, schedule, manual)
+- Action selection (email, webhook, database)
+- Condition logic
+- Tests
+
+### TD-11: Advanced Authentication (SSO, SAML, OAuth)
+- Implement OAuth2 flow
+- Add SAML support
+- Add SSO configuration
+- Tests
+
+### TD-12: C++ DBAL Daemon (Production Security Mode)
+- See `/dbal/production/docs/PHASE3_DAEMON.md`
+- Requires Phase 2 complete
+
+### TD-13: Multi-Source Package System (NPM, Git, HTTP)
+- Load packages from NPM registry
+- Load packages from Git repositories
+- Load packages from HTTP endpoints
+- Verify package signatures
+- Tests
+
+---
+
+## ✅ Completed Tech Debt (Phase 0 & 1)
+
+- ✅ Core architecture (Next.js, Prisma, DBAL, Multi-tenant)
+- ✅ Authentication & authorization
+- ✅ CRUD operations
+- ✅ Package system (52 packages)
+- ✅ Generic component renderer
+- ✅ Infrastructure (Docker, PostgreSQL, Redis, Nginx)
+- ✅ Test suite (464 tests)
+- ✅ AGENTS.md expanded to cover whole project
+- ✅ CLAUDE.md synced with AGENTS.md
+- ✅ ROADMAP.md restructured for clarity
+
+---
+
+## 🎯 How Bots Should Use This File
+
+**Start here when assigned tech debt work:**
+1. Find the task ID (TD-1, TD-2, etc.)
+2. Read the "Bot Instructions" section
+3. Follow each step exactly as written
+4. Check off each step as completed
+5. Verify against any listed "Expected Outcome"
+6. Commit with message: `chore: Complete TD-X [task name]`
+
+**For sequential work:**
+- Phase 2 tasks (TD-1, TD-2, TD-3, TD-4) must complete in order
+- Phase 3+ tasks (TD-5+) are independent but require Phase 2 complete
+
+**Questions?**
+- If instructions are ambiguous, ask user before starting
+- If you get stuck, check `/CLAUDE.md` for architectural context
+- If you need implementation details, check the reference documents
+
+---
+
+## 📝 Adding New Tech Debt
+
+When adding new tech debt items:
+1. Use next available ID (TD-14, TD-15, etc.)
+2. Include: Status, Problem, Impact, Bot Instructions
+3. Link to reference documents where applicable
+4. Add to "Phase X Tech Debt" section with correct priority
+5. Update this file in git with clear commit message