From ccf383a64214b85bef4da044e65773d68297b800 Mon Sep 17 00:00:00 2001 From: johndoe6345789 Date: Thu, 15 Jan 2026 00:49:20 +0000 Subject: [PATCH] docs: Create TECH_DEBT.md with bot-actionable task instructions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add explicit, step-by-step instructions for each tech debt item so bots know exactly what to do: - TD-1: DBAL Refactoring (🔴 CRITICAL) - Move database logic from frontend to DBAL (13 steps) - TD-2: Rate limiting (🟡 HIGH) - Protect API endpoints from abuse - TD-3: OpenAPI/Swagger (🟡 HIGH) - API contract visibility and client generation - TD-4: Error handling docs (🟡 MEDIUM) - Consistent error response patterns Also included 9 Phase 3+ tasks (TD-5 through TD-13) for future reference. Each task includes: - Current status and priority - Clear problem statement - Impact analysis - Bot-actionable instructions (step-by-step) - Expected outcomes Updated ROADMAP.md, CLAUDE.md, and AGENTS.md to reference TECH_DEBT.md. Co-Authored-By: Claude Haiku 4.5 --- AGENTS.md | 1 + CLAUDE.md | 16 ++-- ROADMAP.md | 31 ++++--- TECH_DEBT.md | 254 +++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 280 insertions(+), 22 deletions(-) create mode 100644 TECH_DEBT.md diff --git a/AGENTS.md b/AGENTS.md index 72da0cb08..31f4bc302 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,6 +4,7 @@ ## Quick Navigation +- **Finding Tech Debt** → See `/TECH_DEBT.md` (bot-actionable task list with explicit instructions) - **Starting Development** → [Core Principles](#core-principles) + [Scoped Rules](#scoped-rules) - **Working on DBAL** → [DBAL Development](#dbal-development-phase-2--phase-3) - **Working on Frontend** → [Frontend Development](#frontend-development) diff --git a/CLAUDE.md b/CLAUDE.md index fe3a26928..ebaba8374 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -950,16 +950,18 @@ cmake --build build ### Project Guidance 12. **AGENTS.md** - ⭐ **READ FIRST** - All agents. Core principles, DBAL development, frontend, packages, testing, deployment, troubleshooting, best practices +13. **TECH_DEBT.md** - ⭐ **For AI Assistants** - Bot-actionable tech debt tasks (TD-1 through TD-13) with explicit step-by-step instructions -### DBAL Documentation -13. **dbal/shared/docs/IMPLEMENTATION_SUMMARY.md** - Phase 2 overview -14. **dbal/shared/docs/PHASE2_IMPLEMENTATION.md** - Phase 2 detailed guide -15. **dbal/production/docs/PHASE3_DAEMON.md** - Phase 3 design (future) +### DBAL Documentation & Refactoring +14. **DBAL_REFACTOR_PLAN.md** - Detailed implementation plan (reference for TD-1) +15. **dbal/shared/docs/IMPLEMENTATION_SUMMARY.md** - Phase 2 overview +16. **dbal/shared/docs/PHASE2_IMPLEMENTATION.md** - Phase 2 detailed guide +17. **dbal/production/docs/PHASE3_DAEMON.md** - Phase 3 design (future) ### Schema & Package System -16. **schemas/SCHEMAS_README.md** - Package system definitions -17. **schemas/QUICKSTART.md** - Package system quick start -18. **schemas/package-schemas/** - Complete schema definitions: +18. **schemas/SCHEMAS_README.md** - Package system definitions +19. **schemas/QUICKSTART.md** - Package system quick start +20. **schemas/package-schemas/** - Complete schema definitions: - `script_schema.json` - JSON Script language specification (v2.2.0, planned n8n migration) - `metadata_schema.json` - Package structure - `entities_schema.json` - Database models diff --git a/ROADMAP.md b/ROADMAP.md index 6776bb960..2acfb95a5 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -320,23 +320,24 @@ The original Spark-based version is preserved in `/old` directory for reference: ### What's NOT Done Yet (Phase 2 Remaining & Future Phases) -❌ **Phase 2 - Final Items:** -- Rate limiting (API endpoint protection) -- OpenAPI/Swagger API documentation -- Detailed error handling documentation +❌ **Phase 2 - Final Items** (see `/TECH_DEBT.md` for detailed instructions): +- TD-1: DBAL Refactoring - Move database logic from frontend to DBAL (🔴 CRITICAL) +- TD-2: Rate limiting - API endpoint protection (🟡 HIGH) +- TD-3: OpenAPI/Swagger API documentation (🟡 HIGH) +- TD-4: Error handling documentation (🟡 MEDIUM) -🔮 **Phase 3+:** -- [ ] Rich form editors with nested object/array support -- [ ] Bulk operations (multi-select, bulk delete, export) -- [ ] Advanced filtering UI with visual builder -- [ ] Relationship/foreign key dropdown selectors -- [ ] God Panel (system admin dashboard) -- [ ] Workflow automation UI -- [ ] Advanced authentication (SSO, SAML, OAuth) -- [ ] C++ DBAL daemon (production security mode) -- [ ] Multi-source package system (NPM, git, http) +🔮 **Phase 3+** (blocked until Phase 2 completes): +- TD-5: Rich form editors with nested object/array support +- TD-6: Bulk operations (multi-select, bulk delete, export) +- TD-7: Advanced filtering UI with visual builder +- TD-8: Relationship/foreign key dropdown selectors +- TD-9: God Panel (system admin dashboard) +- TD-10: Workflow automation UI +- TD-11: Advanced authentication (SSO, SAML, OAuth) +- TD-12: C++ DBAL daemon (production security mode) +- TD-13: Multi-source package system (NPM, git, http) -See [Roadmap Phases](#roadmap-phases) for detailed roadmap. +**See `/TECH_DEBT.md` for bot-actionable instructions on each task.** --- diff --git a/TECH_DEBT.md b/TECH_DEBT.md new file mode 100644 index 000000000..a35789da1 --- /dev/null +++ b/TECH_DEBT.md @@ -0,0 +1,254 @@ +# Tech Debt Tracker + +> **For AI Assistants**: This file explicitly defines what needs to be done for each tech debt item. Read the "Bot Instructions" section for each task to understand exactly what's required. + +**Current Status**: Phase 2 - 90% Complete. 3 critical items blocking Phase 3. + +--- + +## 📋 Phase 2 Tech Debt (Priority Order) + +### TD-1: DBAL Refactoring - Move Database Logic from Frontend to DBAL + +**Status**: 🔴 **CRITICAL** - Blocks Phase 3 start + +**Problem**: Database logic is scattered: +- Prisma schema at `/prisma/schema.prisma` (should be in DBAL) +- Adapter factory at `/frontends/nextjs/src/lib/dbal-client/adapter/get-adapter.ts` (duplicate code) +- Prisma context at `/frontends/nextjs/src/lib/config/prisma.ts` (should be in DBAL) +- Seeding split between `/seed/`, `/frontends/nextjs/src/lib/seed/`, and code (should be in DBAL) +- Frontend uses raw adapter instead of entity operations + +**Impact**: DBAL can't be versioned/deployed independently, code duplication, poor separation of concerns + +**Reference Document**: `/DBAL_REFACTOR_PLAN.md` - Contains full implementation details + +**Bot Instructions**: +1. Read `/DBAL_REFACTOR_PLAN.md` completely to understand all 3 phases +2. Follow the "Implementation Order" section (page 356-375) exactly +3. After each step, verify against the "Verification Checklist" (page 379-392) +4. Commit after each phase completes successfully +5. Total work: 13 atomic steps across 3 phases + +**Expected Outcome**: +- ✅ Prisma schema moved to `/dbal/development/prisma/schema.prisma` +- ✅ DBAL owns all Prisma initialization +- ✅ DBAL exports `getDBALClient()`, `seedDatabase()`, `getPrismaClient()` +- ✅ Frontend has single `/frontends/nextjs/src/lib/db-client.ts` integration point +- ✅ All frontend code uses `db.users.list()` pattern, not `adapter.list('User')` +- ✅ Tests pass with new structure +- ✅ Old code deleted (`dbal-client/`, `database-dbal/`, `config/prisma.ts`) + +--- + +### TD-2: Rate Limiting for API Endpoints + +**Status**: 🟡 **HIGH** - Needed for production + +**Problem**: No rate limiting on `/api/*` endpoints. Bots can enumerate users, brute force, or DoS. + +**Impact**: Security vulnerability. Production deployment blocked until implemented. + +**Bot Instructions**: +1. Research rate limiting options for Next.js (e.g., `next-rate-limit`, `redis-based`, `in-memory`) +2. Evaluate against requirements: + - Per-IP limiting (not per-user, since public endpoints) + - Configurable limits by endpoint + - Works in development (SQLite) and production (Redis) + - Graceful failure if Redis unavailable +3. Implement in `/api/middleware.ts` or route handler +4. Add tests in `e2e/rate-limiting.spec.ts` +5. Document in `/docs/API_RATE_LIMITING.md` +6. Update `/ROADMAP.md` to mark as complete + +**Expected Outcome**: +- ✅ `GET /api/users` limited to 100 req/min per IP +- ✅ `POST /api/users/login` limited to 5 req/min per IP +- ✅ Exceeding limit returns 429 status +- ✅ Tests verify both pass and rate-limited scenarios + +--- + +### TD-3: OpenAPI/Swagger API Documentation + +**Status**: 🟡 **HIGH** - Needed for frontend/API clarity + +**Problem**: No API specification. Frontend developers must read code to understand endpoints. + +**Impact**: Slow development, inconsistency, hard to maintain. + +**Bot Instructions**: +1. Choose OpenAPI generator: + - Option A: `@ts-rest/open-api` (lightweight, TypeScript-first) + - Option B: `swagger-jsdoc` (JSDoc-based, intrusive) + - Option C: Manual YAML at `/openapi.yaml` + Recommendation: Option A for tight TypeScript integration + +2. Generate OpenAPI 3.0 spec from route handlers: + - Document all GET, POST, PUT, DELETE endpoints in `/api/*` + - Include request/response schemas + - Document authentication requirements + - Mark deprecated endpoints + +3. Serve Swagger UI at `/api/docs` + +4. Verify spec in `/openapi.yaml` (at root, version-controlled) + +5. Add tests to verify: + - All routes documented + - Schemas match actual request/response types + - No breaking changes without major version bump + +6. Update AGENTS.md "API Documentation" section with usage + +**Expected Outcome**: +- ✅ `/api/docs` shows interactive Swagger UI +- ✅ `/openapi.yaml` exists with complete spec +- ✅ All endpoints documented with examples +- ✅ Request/response types auto-generated from TypeScript +- ✅ Frontend can generate client SDK from spec + +--- + +### TD-4: Error Handling Documentation + +**Status**: 🟡 **MEDIUM** - Improves developer experience + +**Problem**: No centralized docs on error handling patterns. Developers guess. + +**Impact**: Inconsistent error responses, hard to debug, poor API contracts. + +**Bot Instructions**: +1. Create `/docs/ERROR_HANDLING.md`: + - Document error response format (should be consistent) + - List all possible HTTP status codes used + - Explain retry logic for each error type + - Show examples of client-side handling + +2. Audit all `/api/*` endpoints: + - Ensure consistent error response format + - Use correct HTTP status codes (400, 401, 403, 404, 409, 500, etc.) + - All errors should have `code`, `message`, `details` fields (if applicable) + +3. Add error handling test suite (`e2e/error-handling.spec.ts`): + - Test 404 on missing resource + - Test 401 on unauthorized access + - Test 400 on invalid input + - Test 500 on server error + +4. Document in AGENTS.md under "Error Handling" section + +**Expected Outcome**: +- ✅ `/docs/ERROR_HANDLING.md` exists with patterns +- ✅ All errors follow consistent JSON format +- ✅ Tests verify error scenarios +- ✅ Developers know how to handle errors in frontend + +--- + +## 📊 Phase 3+ Tech Debt (Future) + +These are planned but blocked until Phase 2 completes: + +### TD-5: Rich Form Editors with Nested Object/Array Support +- Implement dynamic form builder +- Support nested objects, arrays, conditional fields +- Use JSON Schema for validation +- Add e2e tests + +### TD-6: Bulk Operations (Multi-Select, Bulk Delete, Export) +- Add bulk delete endpoint `/api/[entity]/bulk-delete` +- Add bulk export endpoint `/api/[entity]/export` +- Implement multi-select UI in generic table renderer +- Add tests for bulk operations + +### TD-7: Advanced Filtering UI with Visual Builder +- Create visual filter builder component +- Support multiple conditions (AND/OR) +- Persist filters in URL/localStorage +- Add tests + +### TD-8: Relationship/Foreign Key Dropdown Selectors +- Auto-generate dropdowns for foreign keys +- Support search/autocomplete for large datasets +- Lazy-load options on demand +- Add tests + +### TD-9: God Panel (System Admin Dashboard) +- Create `/admin` page (requires God role) +- Dashboard showing system stats +- User management UI +- Tenant management UI +- Job monitoring +- Error tracking + +### TD-10: Workflow Automation UI +- Visual workflow builder +- Trigger selection (API, schedule, manual) +- Action selection (email, webhook, database) +- Condition logic +- Tests + +### TD-11: Advanced Authentication (SSO, SAML, OAuth) +- Implement OAuth2 flow +- Add SAML support +- Add SSO configuration +- Tests + +### TD-12: C++ DBAL Daemon (Production Security Mode) +- See `/dbal/production/docs/PHASE3_DAEMON.md` +- Requires Phase 2 complete + +### TD-13: Multi-Source Package System (NPM, Git, HTTP) +- Load packages from NPM registry +- Load packages from Git repositories +- Load packages from HTTP endpoints +- Verify package signatures +- Tests + +--- + +## ✅ Completed Tech Debt (Phase 0 & 1) + +- ✅ Core architecture (Next.js, Prisma, DBAL, Multi-tenant) +- ✅ Authentication & authorization +- ✅ CRUD operations +- ✅ Package system (52 packages) +- ✅ Generic component renderer +- ✅ Infrastructure (Docker, PostgreSQL, Redis, Nginx) +- ✅ Test suite (464 tests) +- ✅ AGENTS.md expanded to cover whole project +- ✅ CLAUDE.md synced with AGENTS.md +- ✅ ROADMAP.md restructured for clarity + +--- + +## 🎯 How Bots Should Use This File + +**Start here when assigned tech debt work:** +1. Find the task ID (TD-1, TD-2, etc.) +2. Read the "Bot Instructions" section +3. Follow each step exactly as written +4. Check off each step as completed +5. Verify against any listed "Expected Outcome" +6. Commit with message: `chore: Complete TD-X [task name]` + +**For sequential work:** +- Phase 2 tasks (TD-1, TD-2, TD-3, TD-4) must complete in order +- Phase 3+ tasks (TD-5+) are independent but require Phase 2 complete + +**Questions?** +- If instructions are ambiguous, ask user before starting +- If you get stuck, check `/CLAUDE.md` for architectural context +- If you need implementation details, check the reference documents + +--- + +## 📝 Adding New Tech Debt + +When adding new tech debt items: +1. Use next available ID (TD-14, TD-15, etc.) +2. Include: Status, Problem, Impact, Bot Instructions +3. Link to reference documents where applicable +4. Add to "Phase X Tech Debt" section with correct priority +5. Update this file in git with clear commit message