Merge branch 'main' into copilot/add-missing-ui-atoms

This commit is contained in:
2025-12-27 13:58:10 +00:00
committed by GitHub
539 changed files with 8019 additions and 1795 deletions

View File

@@ -8,7 +8,7 @@
### Analysis Approach
1. **Examined existing instructions**
- `dbal/AGENTS.md` (605 lines) - DBAL-specific agent development guide
- `dbal/docs/AGENTS.md` (605 lines) - DBAL-specific agent development guide
- `.github/copilot-instructions.md` (existing) - Original generic guidance
2. **Analyzed codebase patterns** through:
@@ -116,7 +116,7 @@ Instructions now reference:
| File | Purpose | Why Referenced |
|------|---------|-----------------|
| `dbal/AGENTS.md` | DBAL development guide | Critical for DBAL changes |
| `dbal/docs/AGENTS.md` | DBAL development guide | Critical for DBAL changes |
| `src/lib/database.ts` | Database operations | 1200+ LOC utility wrapper, required for all DB access |
| `src/components/RenderComponent.tsx` | Generic renderer | 221 LOC example of declarative UI pattern |
| `src/lib/schema-utils.test.ts` | Test examples | 63 tests showing parameterized pattern |
@@ -159,7 +159,7 @@ Instructions now reference:
### Adding a new database entity
1. Read: API-First DBAL Development pattern
2. Check: DBAL-Specific Guidance (YAML → Types → Adapters)
3. Reference: `dbal/AGENTS.md` for detailed workflow
3. Reference: `dbal/docs/AGENTS.md` for detailed workflow
### Creating a new component feature
1. Read: Generic Component Rendering pattern
@@ -192,7 +192,7 @@ Agents should prioritize these when onboarding:
1. **Start**: `docs/architecture/5-level-system.md` (understand permissions)
2. **Then**: `docs/architecture/packages.md` (understand modularity)
3. **Then**: `src/lib/database.ts` (understand DB pattern)
4. **Then**: `dbal/AGENTS.md` (if working on DBAL)
4. **Then**: `dbal/docs/AGENTS.md` (if working on DBAL)
5. **Always**: `FUNCTION_TEST_COVERAGE.md` (for test requirements)
---

View File

@@ -16,8 +16,8 @@ body:
label: DBAL Implementation
description: Which DBAL implementation is affected?
options:
- TypeScript SDK (dbal/ts/)
- C++ Daemon (dbal/cpp/)
- TypeScript SDK (dbal/development/)
- C++ Daemon (dbal/production/)
- Both implementations
- YAML Contracts (api/schema/)
- Conformance Tests

View File

@@ -94,7 +94,7 @@ Report issues with the Database Abstraction Layer.
**Best For:**
- DBAL TypeScript SDK issues (`dbal/ts/`)
- DBAL C++ daemon issues (`dbal/cpp/`)
- DBAL C++ daemon issues (`dbal/production/`)
- YAML contract problems (`api/schema/`)
- Conformance test failures
- Implementation inconsistencies
@@ -285,7 +285,7 @@ Packages follow strict conventions:
### DBAL (Database Abstraction Layer)
- TypeScript implementation: `dbal/ts/` (development)
- C++ implementation: `dbal/cpp/` (production)
- C++ implementation: `dbal/production/` (production)
- YAML contracts: `api/schema/` (source of truth)
- Always update YAML first
- Run conformance tests: `python tools/conformance/run_all.py`
@@ -338,6 +338,6 @@ Please submit an issue with the "documentation" template to suggest improvements
- **Workflow Guide**: `.github/prompts/0-kickstart.md`
- **Contributing**: `README.md` → Contributing section
- **Architecture**: `docs/architecture/`
- **DBAL Guide**: `dbal/AGENTS.md`
- **DBAL Guide**: `dbal/docs/AGENTS.md`
- **UI Standards**: `UI_STANDARDS.md`
- **Copilot Instructions**: `.github/copilot-instructions.md`

View File

@@ -190,7 +190,7 @@ if (user.level >= 3) { // Admin and above
## DBAL-Specific Guidance
**TypeScript DBAL**: Fast iteration, development use. Located in `dbal/ts/src/`.
**C++ DBAL Daemon**: Production security, credential protection. Located in `dbal/cpp/src/`.
**C++ DBAL Daemon**: Production security, credential protection. Located in `dbal/production/src/`.
**Conformance Tests**: Guarantee both implementations behave identically. Update `common/contracts/` when changing YAML schemas.
If fixing a DBAL bug:
@@ -217,7 +217,7 @@ If fixing a DBAL bug:
- **Database**: `src/lib/database.ts` (all DB operations), `prisma/schema.prisma` (schema)
- **Packages**: `src/lib/package-loader.ts` (initialization), `packages/*/seed/` (definitions)
- **Tests**: `src/lib/schema-utils.test.ts` (parameterized pattern), `FUNCTION_TEST_COVERAGE.md` (auto-generated report)
- **DBAL**: `dbal/AGENTS.md` (detailed DBAL agent guide), `api/schema/` (YAML contracts)
- **DBAL**: `dbal/docs/AGENTS.md` (detailed DBAL agent guide), `api/schema/` (YAML contracts)
## Questions to Ask

View File

@@ -4,7 +4,7 @@ Run DBAL commands from `dbal/`.
Add a new entity to the DBAL following the API-first approach:
1. **Define entity** in `dbal/api/schema/entities/{name}.yaml`:
1. **Define entity** in `dbal/shared/api/schema/entities/{name}.yaml`:
```yaml
entity: EntityName
version: "1.0"
@@ -13,14 +13,14 @@ fields:
# Add fields...
```
2. **Define operations** in `dbal/api/schema/operations/{name}.ops.yaml`
2. **Define operations** in `dbal/shared/api/schema/operations/{name}.ops.yaml`
3. **Generate types**: `python tools/codegen/gen_types.py`
4. **Implement adapters** in both:
- `dbal/ts/src/adapters/`
- `dbal/cpp/src/adapters/`
- `dbal/development/src/adapters/`
- `dbal/production/src/adapters/`
5. **Add conformance tests** in `dbal/common/contracts/{name}_tests.yaml`
5. **Add conformance tests** in `dbal/shared/common/contracts/{name}_tests.yaml`
6. **Verify**: `python tools/conformance/run_all.py`

View File

@@ -36,4 +36,4 @@ static async getNewEntities(filter: { tenantId: string }) {
```
## 4. Update DBAL (if applicable)
Add entity to `dbal/api/schema/entities/`
Add entity to `dbal/shared/api/schema/entities/`

View File

@@ -10,7 +10,7 @@ Run app commands from `frontends/nextjs/` unless a step says otherwise.
npm run db:generate && npm run db:push
```
2. **DBAL contracts**: If new entity/operation, update YAML in `dbal/api/schema/`
2. **DBAL contracts**: If new entity/operation, update YAML in `dbal/shared/api/schema/`
3. **Database layer**: Add methods to `Database` class in `src/lib/database.ts`

View File

@@ -4,7 +4,7 @@ Use this as the default workflow when starting work in this repo.
## Workflow
1. Skim `docs/START_HERE.md` (if new), `docs/INDEX.md`, and relevant items in `docs/todo/`.
2. Check for scoped rules in nested `AGENTS.md` files (e.g. `dbal/AGENTS.md`) before editing those areas.
2. Check for scoped rules in nested `AGENTS.md` files (e.g. `dbal/docs/AGENTS.md`) before editing those areas.
3. Use the prompts in `.github/prompts/` as needed:
- Plan: `1-plan-feature.prompt.md`
- Design: `2-design-component.prompt.md`
@@ -19,7 +19,7 @@ Use this as the default workflow when starting work in this repo.
## Where Work Lives
- Next.js app: `frontends/nextjs/` (source in `src/`, E2E in `e2e/`, local scripts in `scripts/`).
- Component packages: `packages/` (seed JSON under `packages/*/seed/`, optional `static_content/`, schema checks in `packages/*/tests/`).
- DBAL: `dbal/` (TypeScript library in `dbal/ts/`).
- DBAL: `dbal/` (TypeScript library in `dbal/development/`).
- Prisma schema/migrations: `prisma/` (`schema.prisma`, `migrations/`).
- Shared config: `config/` (symlinked into `frontends/nextjs/`).
- Repo utilities: `tools/` (quality checks, workflow helpers, code analysis).
@@ -41,7 +41,7 @@ Run app workflows from `frontends/nextjs/`:
- Validate: `npx prisma validate`
- Coverage output: `frontends/nextjs/coverage/`
DBAL workflows live in `dbal/ts/` (`npm run build`, `npm run test:unit`).
DBAL workflows live in `dbal/development/` (`npm run build`, `npm run test:unit`).
## Source + Tests
- TypeScript + ESM. Prefer `@/…` imports inside `frontends/nextjs/src/`.

View File

@@ -5,7 +5,7 @@ Before implementing, analyze the feature requirements:
1. **Check existing docs**: `docs/architecture/` for design patterns
2. **Identify affected areas**:
- Database schema changes? → `prisma/schema.prisma`
- New API/DBAL operations? → `dbal/api/schema/`
- New API/DBAL operations? → `dbal/shared/api/schema/`
- UI components? → Use declarative `RenderComponent`
- Business logic? → Consider Lua script in `packages/*/seed/scripts/`

View File

@@ -2,6 +2,40 @@
This directory contains automated workflows for CI/CD, code quality, and comprehensive AI-assisted development throughout the entire SDLC.
## 🚦 Enterprise Gated Tree Workflow
MetaBuilder uses an **Enterprise Gated Tree Workflow** that ensures all code changes pass through multiple validation gates before being merged and deployed.
**📖 Complete Guide:** [Enterprise Gated Workflow Documentation](../../docs/ENTERPRISE_GATED_WORKFLOW.md)
### Quick Overview
All PRs must pass through 5 sequential gates:
1. **Gate 1: Code Quality** - Prisma, TypeScript, Lint, Security
2. **Gate 2: Testing** - Unit, E2E, DBAL Daemon tests
3. **Gate 3: Build & Package** - Application build, quality metrics
4. **Gate 4: Review & Approval** - Human code review (1 approval required)
5. **Gate 5: Deployment** - Staging (auto) → Production (manual approval)
**Key Benefits:**
- ✅ Sequential gates prevent wasted resources
- ✅ Automatic merge after approval
- ✅ Manual approval required for production
- ✅ Clear visibility of gate status on PRs
- ✅ Audit trail for all deployments
### Legacy Workflow Cleanup
**Deprecated and Removed (Dec 2025):**
-`ci/ci.yml` - Replaced by `gated-ci.yml` (100% redundant)
-`quality/deployment.yml` - Replaced by `gated-deployment.yml` (100% redundant)
**Modified:**
-`development.yml` - Refactored to remove redundant quality checks, kept unique Copilot features
See [Legacy Pipeline Cruft Report](../../docs/LEGACY_PIPELINE_CRUFT_REPORT.md) for analysis.
## 🤖 GitHub Copilot Integration
All workflows are designed to work seamlessly with **GitHub Copilot** to assist throughout the Software Development Lifecycle:
@@ -16,7 +50,85 @@ All workflows are designed to work seamlessly with **GitHub Copilot** to assist
## Workflows Overview
### 1. CI/CD Workflow (`ci.yml`)
### 🚦 Enterprise Gated Workflows (New)
#### 1. Enterprise Gated CI/CD Pipeline (`gated-ci.yml`)
**Triggered on:** Push to main/master/develop branches, Pull requests
**Structure:**
- **Gate 1:** Code Quality (Prisma, TypeScript, Lint, Security)
- **Gate 2:** Testing (Unit, E2E, DBAL Daemon)
- **Gate 3:** Build & Package (Build, Quality Metrics)
- **Gate 4:** Review & Approval (Human review required)
**Features:**
- Sequential gate execution for efficiency
- Clear gate status reporting on PRs
- Automatic progression through gates
- Summary report with all gate results
**Best for:** Small to medium teams, straightforward workflows
#### 1a. Enterprise Gated CI/CD Pipeline - Atomic (`gated-ci-atomic.yml`) 🆕
**Triggered on:** Push to main/master/develop branches, Pull requests
**Structure:**
- **Gate 1:** Code Quality - 7 atomic steps
- 1.1 Prisma Validation
- 1.2 TypeScript Check (+ strict mode analysis)
- 1.3 ESLint (+ any-type detection + ts-ignore detection)
- 1.4 Security Scan (+ dependency audit)
- 1.5 File Size Check
- 1.6 Code Complexity Analysis
- 1.7 Stub Implementation Detection
- **Gate 2:** Testing - 3 atomic steps
- 2.1 Unit Tests (+ coverage analysis)
- 2.2 E2E Tests
- 2.3 DBAL Daemon Tests
- **Gate 3:** Build & Package - 2 atomic steps
- 3.1 Application Build (+ bundle analysis)
- 3.2 Quality Metrics
- **Gate 4:** Review & Approval (Human review required)
**Features:**
- **Atomic validation steps** for superior visualization
- Each tool from `/tools` runs as separate job
- **Gate artifacts** persisted between steps (30-day retention)
- Granular failure detection
- Parallel execution within gates
- Complete audit trail with JSON artifacts
- Individual step timing and status
**Best for:** Large teams, enterprise compliance, audit requirements
**Documentation:** See [Atomic Gated Workflow Architecture](../../docs/ATOMIC_GATED_WORKFLOW.md)
#### 2. Enterprise Gated Deployment (`gated-deployment.yml`)
**Triggered on:** Push to main/master, Releases, Manual workflow dispatch
**Environments:**
- **Staging:** Automatic deployment after merge to main
- **Production:** Manual approval required
**Features:**
- Pre-deployment validation (schema, security, size)
- Breaking change detection and warnings
- Environment-specific deployment paths
- Post-deployment health checks
- Automatic deployment tracking issues
- Rollback preparation and procedures
**Gate 5:** Deployment gate ensures only reviewed code reaches production
### 🔄 Legacy Workflows (Still Active)
#### 3. CI/CD Workflow (`ci/ci.yml`) - ❌ REMOVED
**Status:** Deprecated and removed (Dec 2025)
**Reason:** 100% functionality superseded by `gated-ci.yml`
**Jobs:** ~~Prisma Check, Lint, Build, E2E Tests, Quality Check~~
**Replacement:** Use `gated-ci.yml` for all CI/CD operations
**Triggered on:** Push to main/master/develop branches, Pull requests
**Jobs:**
@@ -26,7 +138,7 @@ All workflows are designed to work seamlessly with **GitHub Copilot** to assist
- **E2E Tests**: Runs Playwright end-to-end tests
- **Quality Check**: Checks for console.log statements and TODO comments
### 2. Automated Code Review (`code-review.yml`)
### 4. Automated Code Review (`code-review.yml`)
**Triggered on:** Pull request opened, synchronized, or reopened
**Features:**
@@ -43,20 +155,21 @@ All workflows are designed to work seamlessly with **GitHub Copilot** to assist
- ✅ React best practices
- ✅ File size warnings
### 3. Auto Merge (`auto-merge.yml`)
### 5. Auto Merge (`auto-merge.yml`) - Updated for Gated Workflow
**Triggered on:** PR approval, CI workflow completion
**Features:**
- Automatically merges PRs when:
- PR is approved by reviewers
- All CI checks pass (lint, build, e2e tests)
- All gates pass (supports both gated and legacy CI checks)
- No merge conflicts
- PR is not in draft
- **Automatically deletes the branch** after successful merge
- Uses squash merge strategy
- Posts comments about merge status
- **Updated:** Now supports Enterprise Gated CI/CD Pipeline checks
### 4. Issue Triage (`issue-triage.yml`)
### 6. Issue Triage (`issue-triage.yml`)
**Triggered on:** New issues opened, issues labeled
**Features:**
@@ -68,7 +181,7 @@ All workflows are designed to work seamlessly with **GitHub Copilot** to assist
- Suggests automated fix attempts for simple issues
- Can create fix branches automatically with `create-pr` label
### 5. PR Management (`pr-management.yml`)
### 7. PR Management (`pr-management.yml`)
**Triggered on:** PR opened, synchronized, labeled
**Features:**
@@ -80,7 +193,7 @@ All workflows are designed to work seamlessly with **GitHub Copilot** to assist
- Links related issues automatically
- Posts comments on related issues
### 6. Merge Conflict Check (`merge-conflict-check.yml`)
### 8. Merge Conflict Check (`merge-conflict-check.yml`)
**Triggered on:** PR opened/synchronized, push to main/master
**Features:**
@@ -89,7 +202,7 @@ All workflows are designed to work seamlessly with **GitHub Copilot** to assist
- Adds/removes `merge-conflict` label
- Fails CI if conflicts exist
### 7. Planning & Design (`planning.yml`) 🆕
### 9. Planning & Design (`planning.yml`) 🆕
**Triggered on:** Issues opened or labeled with enhancement/feature-request
**Features:**
@@ -103,35 +216,28 @@ All workflows are designed to work seamlessly with **GitHub Copilot** to assist
**SDLC Phase:** Planning & Design
### 8. Development Assistance (`development.yml`) 🆕
**Triggered on:** Push to feature branches, PR updates, @copilot mentions
### 10. Development Assistance (`development.yml`) 🆕 - Refactored
**Triggered on:** Pull request updates, @copilot mentions
**Features:**
- **Continuous Quality Feedback**: Real-time code metrics and architectural compliance
- **Declarative Ratio Tracking**: Monitors JSON/Lua vs TypeScript balance
- **Component Size Monitoring**: Flags components exceeding 150 LOC
- **Refactoring Suggestions**: Identifies opportunities for improvement
- **Architectural Compliance Feedback**: Monitors declarative ratio and component sizes
- **@copilot Interaction Handler**: Responds to @copilot mentions with context-aware guidance
- **Refactoring Suggestions**: Identifies opportunities for improvement
- Provides architectural reminders and best practices
- Suggests generic renderers over hardcoded components
**Note:** Refactored to remove redundant quality checks (lint/build now in gated-ci.yml)
**SDLC Phase:** Development
### 9. Deployment & Monitoring (`deployment.yml`) 🆕
**Triggered on:** Push to main, releases, manual workflow dispatch
### 11. Deployment & Monitoring (`deployment.yml`) - ❌ REMOVED
**Status:** Deprecated and removed (Dec 2025)
**Reason:** 100% functionality superseded by `gated-deployment.yml` with improvements
**Features:**
- **Pre-Deployment Validation**: Schema validation, security audit, package size check
- **Breaking Change Detection**: Identifies breaking commits
- **Deployment Summary**: Generates release notes with categorized changes
- **Post-Deployment Health Checks**: Verifies build integrity and critical files
- **Deployment Tracking Issues**: Creates monitoring issues for releases
- **Security Dependency Audit**: Detects and reports vulnerabilities
- Auto-creates security issues for critical vulnerabilities
**Jobs:** ~~Pre-Deployment Validation, Deployment Summary, Post-Deployment Health Checks~~
**SDLC Phase:** Deployment & Operations
**Replacement:** Use `gated-deployment.yml` for all deployment operations
### 10. Code Size Limits (`size-limits.yml`)
### 12. Code Size Limits (`size-limits.yml`)
**Triggered on:** Pull requests, pushes to main (when source files change)
**Features:**

View File

@@ -1,327 +0,0 @@
name: CI/CD
on:
push:
branches: [ main, master, develop ]
pull_request:
branches: [ main, master, develop ]
jobs:
prisma-check:
name: Validate Prisma setup
runs-on: ubuntu-latest
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Validate Prisma Schema
run: bunx prisma validate
env:
DATABASE_URL: file:./dev.db
typecheck:
name: TypeScript Type Check
runs-on: ubuntu-latest
needs: prisma-check
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Run TypeScript type check
run: bun run typecheck
lint:
name: Lint Code
runs-on: ubuntu-latest
needs: prisma-check
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Run ESLint
run: bun run lint
test-unit:
name: Unit Tests
runs-on: ubuntu-latest
needs: [typecheck, lint]
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Run unit tests
run: bun run test:unit
env:
DATABASE_URL: file:./dev.db
- name: Upload coverage report
if: always()
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: coverage-report
path: frontends/nextjs/coverage/
retention-days: 7
build:
name: Build Application
runs-on: ubuntu-latest
needs: test-unit
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Build
run: bun run build
env:
DATABASE_URL: file:./dev.db
- name: Upload build artifacts
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: dist
path: frontends/nextjs/.next/
retention-days: 7
test-e2e:
name: E2E Tests
runs-on: ubuntu-latest
needs: [typecheck, lint]
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Install Playwright Browsers
run: bunx playwright install --with-deps chromium
- name: Run Playwright tests
run: bun run test:e2e
env:
DATABASE_URL: file:./dev.db
- name: Upload test results
if: always()
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: playwright-report
path: frontends/nextjs/playwright-report/
retention-days: 7
test-dbal-daemon:
name: DBAL Daemon E2E
runs-on: ubuntu-latest
needs: test-e2e
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Install Playwright Browsers
run: bunx playwright install --with-deps chromium
- name: Run DBAL daemon suite
run: bun run test:e2e:dbal-daemon
env:
DATABASE_URL: file:./dev.db
- name: Upload daemon test report
if: always()
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: playwright-report-dbal-daemon
path: frontends/nextjs/playwright-report/
retention-days: 7
quality-check:
name: Code Quality Check
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 0
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Check for console.log statements
run: |
if git diff origin/${{ github.base_ref }}...HEAD -- '*.ts' '*.tsx' '*.js' '*.jsx' | grep -E '^\+.*console\.(log|debug|info)'; then
echo "⚠️ Found console.log statements in the changes"
echo "Please remove console.log statements before merging"
exit 1
fi
continue-on-error: true
- name: Check for TODO comments
run: |
TODO_COUNT=$(git diff origin/${{ github.base_ref }}...HEAD -- '*.ts' '*.tsx' '*.js' '*.jsx' | grep -E '^\+.*TODO|FIXME' | wc -l)
if [ $TODO_COUNT -gt 0 ]; then
echo "⚠️ Found $TODO_COUNT TODO/FIXME comments in the changes"
echo "Please address TODO comments before merging or create issues for them"
fi
continue-on-error: true

View File

@@ -4,14 +4,14 @@ on:
push:
branches: [ main, develop ]
paths:
- 'dbal/cpp/**'
- 'dbal/tools/cpp-build-assistant.cjs'
- 'dbal/production/**'
- 'dbal/shared/tools/cpp-build-assistant.cjs'
- '.github/workflows/cpp-build.yml'
pull_request:
branches: [ main, develop ]
paths:
- 'dbal/cpp/**'
- 'dbal/tools/cpp-build-assistant.cjs'
- 'dbal/production/**'
- 'dbal/shared/tools/cpp-build-assistant.cjs'
- '.github/workflows/cpp-build.yml'
workflow_dispatch:
@@ -33,7 +33,7 @@ jobs:
- name: Check if C++ sources exist
id: check
run: |
if [ -d "dbal/cpp/src" ] && [ "$(find dbal/cpp/src -name '*.cpp' | wc -l)" -gt 0 ]; then
if [ -d "dbal/production/src" ] && [ "$(find dbal/production/src -name '*.cpp' | wc -l)" -gt 0 ]; then
echo "has_sources=true" >> $GITHUB_OUTPUT
echo "✓ C++ source files found"
else
@@ -112,8 +112,8 @@ jobs:
with:
name: dbal-daemon-linux
path: |
dbal/cpp/build/dbal_daemon
dbal/cpp/build/*.so
dbal/production/build/dbal_daemon
dbal/production/build/*.so
retention-days: 7
build-macos:
@@ -151,7 +151,7 @@ jobs:
CMAKE_BUILD_TYPE: ${{ matrix.build_type }}
run: |
if [ "${{ matrix.build_type }}" = "Debug" ]; then
node dbal/tools/cpp-build-assistant.cjs full --debug
node dbal/shared/tools/cpp-build-assistant.cjs full --debug
else
bun run cpp:full
fi
@@ -165,8 +165,8 @@ jobs:
with:
name: dbal-daemon-macos
path: |
dbal/cpp/build/dbal_daemon
dbal/cpp/build/*.dylib
dbal/production/build/dbal_daemon
dbal/production/build/*.dylib
retention-days: 7
build-windows:
@@ -206,7 +206,7 @@ jobs:
shell: bash
run: |
if [ "${{ matrix.build_type }}" = "Debug" ]; then
node dbal/tools/cpp-build-assistant.cjs full --debug
node dbal/shared/tools/cpp-build-assistant.cjs full --debug
else
bun run cpp:full
fi
@@ -220,8 +220,8 @@ jobs:
with:
name: dbal-daemon-windows
path: |
dbal/cpp/build/dbal_daemon.exe
dbal/cpp/build/*.dll
dbal/production/build/dbal_daemon.exe
dbal/production/build/*.dll
retention-days: 7
code-quality:
@@ -255,13 +255,13 @@ jobs:
run: |
cppcheck --enable=all --inconclusive --error-exitcode=1 \
--suppress=missingIncludeSystem \
-I dbal/cpp/include \
dbal/cpp/src/
-I dbal/production/include \
dbal/production/src/
continue-on-error: true
- name: Check formatting
run: |
find dbal/cpp/src dbal/cpp/include -name '*.cpp' -o -name '*.hpp' | \
find dbal/production/src dbal/production/include -name '*.cpp' -o -name '*.hpp' | \
xargs clang-format --dry-run --Werror
continue-on-error: true
@@ -288,15 +288,15 @@ jobs:
uses: actions/download-artifact@v4
with:
name: dbal-daemon-linux
path: dbal/cpp/build/
path: dbal/production/build/
- name: Make daemon executable
run: chmod +x dbal/cpp/build/dbal_daemon
run: chmod +x dbal/production/build/dbal_daemon
- name: Run integration tests
run: |
# Start C++ daemon
./dbal/cpp/build/dbal_daemon &
./dbal/production/build/dbal_daemon &
DAEMON_PID=$!
sleep 2

View File

@@ -16,8 +16,7 @@ jobs:
name: Continuous Quality Feedback
runs-on: ubuntu-latest
if: |
github.event_name == 'push' ||
(github.event_name == 'pull_request' && !github.event.pull_request.draft)
github.event_name == 'pull_request' && !github.event.pull_request.draft
defaults:
run:
working-directory: frontends/nextjs
@@ -27,37 +26,15 @@ jobs:
with:
fetch-depth: 0
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: '1.3.4'
- name: Cache Bun dependencies
uses: actions/cache@v4
with:
key: bun-deps-${{ runner.os }}-${{ hashFiles('bun.lock') }}
path: |
frontends/nextjs/node_modules
~/.bun
restore-keys: bun-deps-${{ runner.os }}-
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Analyze code quality
- name: Analyze code metrics (no redundant checks)
id: quality
run: |
# Run lint and capture output
bun run lint > lint-output.txt 2>&1 || echo "LINT_FAILED=true" >> $GITHUB_OUTPUT
# Note: Lint/build/tests are handled by gated-ci.yml
# This job only collects metrics for architectural feedback
# Count TypeScript files and their sizes
TOTAL_TS_FILES=$(find src -name "*.ts" -o -name "*.tsx" | wc -l)
LARGE_FILES=$(find src -name "*.ts" -o -name "*.tsx" -exec wc -l {} \; | awk '$1 > 150 {print $2}' | wc -l)
TOTAL_TS_FILES=$(find src -name "*.ts" -o -name "*.tsx" 2>/dev/null | wc -l)
LARGE_FILES=$(find src -name "*.ts" -o -name "*.tsx" -exec wc -l {} \; 2>/dev/null | awk '$1 > 150 {print $2}' | wc -l)
echo "total_ts_files=$TOTAL_TS_FILES" >> $GITHUB_OUTPUT
echo "large_files=$LARGE_FILES" >> $GITHUB_OUTPUT
@@ -68,8 +45,6 @@ jobs:
echo "json_files=$JSON_FILES" >> $GITHUB_OUTPUT
echo "lua_scripts=$LUA_SCRIPTS" >> $GITHUB_OUTPUT
cat lint-output.txt
- name: Check architectural compliance
id: architecture

1033
.github/workflows/gated-ci-atomic.yml vendored Normal file

File diff suppressed because it is too large Load Diff

610
.github/workflows/gated-ci.yml vendored Normal file
View File

@@ -0,0 +1,610 @@
name: Enterprise Gated CI/CD Pipeline
on:
push:
branches: [ main, master, develop ]
pull_request:
branches: [ main, master, develop ]
permissions:
contents: read
pull-requests: write
checks: write
statuses: write
# Enterprise Gated Tree Workflow
# Changes must pass through 5 gates before merge:
# Gate 1: Code Quality (lint, typecheck, security)
# Gate 2: Testing (unit, E2E)
# Gate 3: Build & Package
# Gate 4: Review & Approval
# Gate 5: Deployment (staging → production with manual approval)
jobs:
# ============================================================================
# GATE 1: Code Quality Gates
# ============================================================================
gate-1-start:
name: "Gate 1: Code Quality - Starting"
runs-on: ubuntu-latest
steps:
- name: Gate 1 checkpoint
run: |
echo "🚦 GATE 1: CODE QUALITY VALIDATION"
echo "================================================"
echo "Running: Prisma validation, TypeScript check, Linting, Security scan"
echo "Status: IN PROGRESS"
prisma-check:
name: "Gate 1.1: Validate Prisma Schema"
runs-on: ubuntu-latest
needs: gate-1-start
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Validate Prisma Schema
run: bunx prisma validate
env:
DATABASE_URL: file:./dev.db
typecheck:
name: "Gate 1.2: TypeScript Type Check"
runs-on: ubuntu-latest
needs: prisma-check
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Run TypeScript type check
run: bun run typecheck
lint:
name: "Gate 1.3: Lint Code"
runs-on: ubuntu-latest
needs: prisma-check
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Run ESLint
run: bun run lint
security-scan:
name: "Gate 1.4: Security Scan"
runs-on: ubuntu-latest
needs: prisma-check
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Run security audit
run: bun audit --audit-level=moderate
continue-on-error: true
- name: Check for vulnerable dependencies
run: |
echo "Checking for known vulnerabilities..."
bun audit --json > audit-results.json 2>&1 || true
if [ -f audit-results.json ]; then
echo "Security audit completed"
fi
gate-1-complete:
name: "Gate 1: Code Quality - Passed ✅"
runs-on: ubuntu-latest
needs: [prisma-check, typecheck, lint, security-scan]
steps:
- name: Gate 1 passed
run: |
echo "✅ GATE 1 PASSED: CODE QUALITY"
echo "================================================"
echo "✓ Prisma schema validated"
echo "✓ TypeScript types checked"
echo "✓ Code linted"
echo "✓ Security scan completed"
echo ""
echo "Proceeding to Gate 2: Testing..."
# ============================================================================
# GATE 2: Testing Gates
# ============================================================================
gate-2-start:
name: "Gate 2: Testing - Starting"
runs-on: ubuntu-latest
needs: gate-1-complete
steps:
- name: Gate 2 checkpoint
run: |
echo "🚦 GATE 2: TESTING VALIDATION"
echo "================================================"
echo "Running: Unit tests, E2E tests, DBAL daemon tests"
echo "Status: IN PROGRESS"
test-unit:
name: "Gate 2.1: Unit Tests"
runs-on: ubuntu-latest
needs: gate-2-start
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Run unit tests
run: bun run test:unit
env:
DATABASE_URL: file:./dev.db
- name: Upload coverage report
if: always()
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: coverage-report
path: frontends/nextjs/coverage/
retention-days: 7
test-e2e:
name: "Gate 2.2: E2E Tests"
runs-on: ubuntu-latest
needs: gate-2-start
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Install Playwright Browsers
run: bunx playwright install --with-deps chromium
- name: Run Playwright tests
run: bun run test:e2e
env:
DATABASE_URL: file:./dev.db
- name: Upload test results
if: always()
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: playwright-report
path: frontends/nextjs/playwright-report/
retention-days: 7
test-dbal-daemon:
name: "Gate 2.3: DBAL Daemon E2E"
runs-on: ubuntu-latest
needs: gate-2-start
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Install Playwright Browsers
run: bunx playwright install --with-deps chromium
- name: Run DBAL daemon suite
run: bun run test:e2e:dbal-daemon
env:
DATABASE_URL: file:./dev.db
- name: Upload daemon test report
if: always()
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: playwright-report-dbal-daemon
path: frontends/nextjs/playwright-report/
retention-days: 7
gate-2-complete:
name: "Gate 2: Testing - Passed ✅"
runs-on: ubuntu-latest
needs: [test-unit, test-e2e, test-dbal-daemon]
steps:
- name: Gate 2 passed
run: |
echo "✅ GATE 2 PASSED: TESTING"
echo "================================================"
echo "✓ Unit tests passed"
echo "✓ E2E tests passed"
echo "✓ DBAL daemon tests passed"
echo ""
echo "Proceeding to Gate 3: Build & Package..."
# ============================================================================
# GATE 3: Build & Package Gates
# ============================================================================
gate-3-start:
name: "Gate 3: Build & Package - Starting"
runs-on: ubuntu-latest
needs: gate-2-complete
steps:
- name: Gate 3 checkpoint
run: |
echo "🚦 GATE 3: BUILD & PACKAGE VALIDATION"
echo "================================================"
echo "Running: Application build, artifact packaging"
echo "Status: IN PROGRESS"
build:
name: "Gate 3.1: Build Application"
runs-on: ubuntu-latest
needs: gate-3-start
defaults:
run:
working-directory: frontends/nextjs
outputs:
build-success: ${{ steps.build-step.outcome }}
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Build
id: build-step
run: bun run build
env:
DATABASE_URL: file:./dev.db
- name: Upload build artifacts
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: dist
path: frontends/nextjs/.next/
retention-days: 7
quality-check:
name: "Gate 3.2: Code Quality Metrics"
runs-on: ubuntu-latest
needs: gate-3-start
if: github.event_name == 'pull_request'
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 0
- name: Setup Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Check for console.log statements
run: |
if git diff origin/${{ github.base_ref }}...HEAD -- '*.ts' '*.tsx' '*.js' '*.jsx' | grep -E '^\+.*console\.(log|debug|info)'; then
echo "⚠️ Found console.log statements in the changes"
echo "Please remove console.log statements before merging"
exit 1
fi
continue-on-error: true
- name: Check for TODO comments
run: |
TODO_COUNT=$(git diff origin/${{ github.base_ref }}...HEAD -- '*.ts' '*.tsx' '*.js' '*.jsx' | grep -E '^\+.*TODO|FIXME' | wc -l)
if [ $TODO_COUNT -gt 0 ]; then
echo "⚠️ Found $TODO_COUNT TODO/FIXME comments in the changes"
echo "Please address TODO comments before merging or create issues for them"
fi
continue-on-error: true
gate-3-complete:
name: "Gate 3: Build & Package - Passed ✅"
runs-on: ubuntu-latest
needs: [build, quality-check]
if: always() && needs.build.result == 'success' && (needs.quality-check.result == 'success' || needs.quality-check.result == 'skipped')
steps:
- name: Gate 3 passed
run: |
echo "✅ GATE 3 PASSED: BUILD & PACKAGE"
echo "================================================"
echo "✓ Application built successfully"
echo "✓ Build artifacts packaged"
echo "✓ Quality metrics validated"
echo ""
echo "Proceeding to Gate 4: Review & Approval..."
# ============================================================================
# GATE 4: Review & Approval Gate (PR only)
# ============================================================================
gate-4-review-required:
name: "Gate 4: Review & Approval Required"
runs-on: ubuntu-latest
needs: gate-3-complete
if: github.event_name == 'pull_request'
steps:
- name: Check PR approval status
uses: actions/github-script@v7
with:
script: |
const { data: reviews } = await github.rest.pulls.listReviews({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number
});
const latestReviews = {};
for (const review of reviews) {
latestReviews[review.user.login] = review.state;
}
const hasApproval = Object.values(latestReviews).includes('APPROVED');
const hasRequestChanges = Object.values(latestReviews).includes('CHANGES_REQUESTED');
console.log('Review Status:');
console.log('==============');
console.log('Approvals:', Object.values(latestReviews).filter(s => s === 'APPROVED').length);
console.log('Change Requests:', Object.values(latestReviews).filter(s => s === 'CHANGES_REQUESTED').length);
if (hasRequestChanges) {
core.setFailed('❌ Changes requested - PR cannot proceed to deployment');
} else if (!hasApproval) {
core.notice('⏳ PR approval required before merge - this gate will pass when approved');
} else {
console.log('✅ PR approved - gate passed');
}
gate-4-complete:
name: "Gate 4: Review & Approval - Status"
runs-on: ubuntu-latest
needs: gate-4-review-required
if: always() && github.event_name == 'pull_request'
steps:
- name: Gate 4 status
run: |
echo "🚦 GATE 4: REVIEW & APPROVAL"
echo "================================================"
echo "Note: This gate requires human approval"
echo "PR must be approved by reviewers before auto-merge"
echo ""
if [ "${{ needs.gate-4-review-required.result }}" == "success" ]; then
echo "✅ Review approval received"
echo "Proceeding to Gate 5: Deployment (post-merge)..."
else
echo "⏳ Awaiting review approval"
echo "Gate will complete when PR is approved"
fi
# ============================================================================
# GATE 5: Deployment Gate (post-merge, main branch only)
# ============================================================================
gate-5-deployment-ready:
name: "Gate 5: Deployment Ready"
runs-on: ubuntu-latest
needs: gate-3-complete
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master')
steps:
- name: Deployment gate checkpoint
run: |
echo "🚦 GATE 5: DEPLOYMENT VALIDATION"
echo "================================================"
echo "Code merged to main branch"
echo "Ready for staging deployment"
echo ""
echo "✅ ALL GATES PASSED"
echo "================================================"
echo "✓ Gate 1: Code Quality"
echo "✓ Gate 2: Testing"
echo "✓ Gate 3: Build & Package"
echo "✓ Gate 4: Review & Approval"
echo "✓ Gate 5: Ready for Deployment"
echo ""
echo "Note: Production deployment requires manual approval"
echo "Use workflow_dispatch with environment='production'"
# ============================================================================
# Summary Report
# ============================================================================
gates-summary:
name: "🎯 Gates Summary"
runs-on: ubuntu-latest
needs: [gate-1-complete, gate-2-complete, gate-3-complete]
if: always()
steps:
- name: Generate gates report
uses: actions/github-script@v7
with:
script: |
const gates = [
{ name: 'Gate 1: Code Quality', status: '${{ needs.gate-1-complete.result }}' },
{ name: 'Gate 2: Testing', status: '${{ needs.gate-2-complete.result }}' },
{ name: 'Gate 3: Build & Package', status: '${{ needs.gate-3-complete.result }}' }
];
let summary = '## 🚦 Enterprise Gated CI/CD Pipeline Summary\n\n';
for (const gate of gates) {
const icon = gate.status === 'success' ? '✅' :
gate.status === 'failure' ? '❌' :
gate.status === 'skipped' ? '⏭️' : '⏳';
summary += `${icon} **${gate.name}**: ${gate.status}\n`;
}
if (context.eventName === 'pull_request') {
summary += '\n### Next Steps\n';
summary += '- ✅ All CI gates passed\n';
summary += '- ⏳ Awaiting PR approval (Gate 4)\n';
summary += '- 📋 Once approved, PR will auto-merge\n';
summary += '- 🚀 Deployment gates (Gate 5) run after merge to main\n';
}
console.log(summary);
// Post comment on PR if applicable
if (context.eventName === 'pull_request') {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: summary
});
}

517
.github/workflows/gated-deployment.yml vendored Normal file
View File

@@ -0,0 +1,517 @@
name: Enterprise Gated Deployment
on:
push:
branches:
- main
- master
release:
types: [published]
workflow_dispatch:
inputs:
environment:
description: 'Target deployment environment'
required: true
type: choice
options:
- staging
- production
skip_tests:
description: 'Skip pre-deployment tests (emergency only)'
required: false
type: boolean
default: false
permissions:
contents: read
issues: write
pull-requests: write
deployments: write
# Enterprise Deployment with Environment Gates
# Staging: Automatic deployment after main branch push
# Production: Requires manual approval
jobs:
# ============================================================================
# Pre-Deployment Validation
# ============================================================================
pre-deployment-validation:
name: Pre-Deployment Checks
runs-on: ubuntu-latest
defaults:
run:
working-directory: frontends/nextjs
outputs:
has-breaking-changes: ${{ steps.breaking.outputs.has_breaking }}
deployment-environment: ${{ steps.determine-env.outputs.environment }}
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Determine target environment
id: determine-env
run: |
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
echo "environment=${{ inputs.environment }}" >> $GITHUB_OUTPUT
elif [ "${{ github.event_name }}" == "release" ]; then
echo "environment=production" >> $GITHUB_OUTPUT
else
echo "environment=staging" >> $GITHUB_OUTPUT
fi
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Validate database schema
run: bunx prisma validate
env:
DATABASE_URL: file:./dev.db
- name: Check for breaking changes
id: breaking
uses: actions/github-script@v7
with:
script: |
const commits = await github.rest.repos.listCommits({
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 10
});
let hasBreaking = false;
let breakingChanges = [];
for (const commit of commits.data) {
const message = commit.commit.message.toLowerCase();
if (message.includes('breaking') || message.includes('breaking:') || message.startsWith('!')) {
hasBreaking = true;
breakingChanges.push({
sha: commit.sha.substring(0, 7),
message: commit.commit.message.split('\n')[0]
});
}
}
core.setOutput('has_breaking', hasBreaking);
if (hasBreaking) {
console.log('⚠️ Breaking changes detected:');
breakingChanges.forEach(c => console.log(` - ${c.sha}: ${c.message}`));
core.warning('Breaking changes detected in recent commits');
}
- name: Security audit
run: bun audit --audit-level=moderate
continue-on-error: true
- name: Check package size
run: |
bun run build
SIZE=$(du -sm .next/ | cut -f1)
echo "Build size: ${SIZE}MB"
if [ $SIZE -gt 50 ]; then
echo "::warning::Build size is ${SIZE}MB (>50MB). Consider optimizing."
fi
# ============================================================================
# Staging Deployment (Automatic)
# ============================================================================
deploy-staging:
name: Deploy to Staging
runs-on: ubuntu-latest
needs: pre-deployment-validation
if: |
needs.pre-deployment-validation.outputs.deployment-environment == 'staging' &&
(github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && inputs.environment == 'staging'))
environment:
name: staging
url: https://staging.metabuilder.example.com
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: ${{ secrets.STAGING_DATABASE_URL }}
- name: Build for staging
run: bun run build
env:
DATABASE_URL: ${{ secrets.STAGING_DATABASE_URL }}
NEXT_PUBLIC_ENV: staging
- name: Deploy to staging
run: |
echo "🚀 Deploying to staging environment..."
echo "Build artifacts ready for deployment"
echo "Note: Replace this with actual deployment commands"
echo "Examples:"
echo " - docker build/push"
echo " - kubectl apply"
echo " - terraform apply"
echo " - vercel deploy"
- name: Run smoke tests
run: |
echo "🧪 Running smoke tests on staging..."
echo "Basic health checks:"
echo " ✓ Application starts"
echo " ✓ Database connection"
echo " ✓ API endpoints responding"
echo "Note: Implement actual smoke tests here"
- name: Post deployment summary
uses: actions/github-script@v7
with:
script: |
const summary = `## 🚀 Staging Deployment Successful
**Environment:** staging
**Commit:** ${context.sha.substring(0, 7)}
**Time:** ${new Date().toISOString()}
### Deployment Details
- ✅ Pre-deployment validation passed
- ✅ Build completed
- ✅ Deployed to staging
- ✅ Smoke tests passed
### Next Steps
- Monitor staging environment for issues
- Run integration tests
- Request QA validation
- If stable, promote to production with manual approval
**Staging URL:** https://staging.metabuilder.example.com
`;
console.log(summary);
# ============================================================================
# Production Deployment Gate (Manual Approval Required)
# ============================================================================
production-approval-gate:
name: Production Deployment Gate
runs-on: ubuntu-latest
needs: [pre-deployment-validation]
if: |
needs.pre-deployment-validation.outputs.deployment-environment == 'production' &&
(github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && inputs.environment == 'production'))
steps:
- name: Pre-production checklist
uses: actions/github-script@v7
with:
script: |
const hasBreaking = '${{ needs.pre-deployment-validation.outputs.has-breaking-changes }}' === 'true';
let checklist = `## 🚨 Production Deployment Gate
### Pre-Deployment Checklist
#### Automatic Checks
- ✅ All CI/CD gates passed
- ✅ Code merged to main branch
- ✅ Pre-deployment validation completed
${hasBreaking ? '- ⚠️ **Breaking changes detected** - review required' : '- ✅ No breaking changes detected'}
#### Manual Verification Required
- [ ] Staging environment validated
- [ ] QA sign-off received
- [ ] Database migrations reviewed
- [ ] Rollback plan prepared
- [ ] Monitoring alerts configured
- [ ] On-call engineer notified
${hasBreaking ? '- [ ] **Breaking changes documented and communicated**' : ''}
### Approval Process
This deployment requires manual approval from authorized personnel.
**To approve:** Use the GitHub Actions UI to approve this deployment.
**To reject:** Cancel the workflow run.
### Emergency Override
If this is an emergency hotfix, the skip_tests option was set to: ${{ inputs.skip_tests || false }}
`;
console.log(checklist);
if (hasBreaking) {
core.warning('Breaking changes detected - extra caution required for production deployment');
}
deploy-production:
name: Deploy to Production
runs-on: ubuntu-latest
needs: [pre-deployment-validation, production-approval-gate]
if: |
needs.pre-deployment-validation.outputs.deployment-environment == 'production' &&
(github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && inputs.environment == 'production'))
environment:
name: production
url: https://metabuilder.example.com
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: bun install
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: ${{ secrets.PRODUCTION_DATABASE_URL }}
- name: Build for production
run: bun run build
env:
DATABASE_URL: ${{ secrets.PRODUCTION_DATABASE_URL }}
NEXT_PUBLIC_ENV: production
NODE_ENV: production
- name: Pre-deployment backup
run: |
echo "📦 Creating pre-deployment backup..."
echo "Note: Implement actual backup commands"
echo " - Database backup"
echo " - File system backup"
echo " - Configuration backup"
- name: Run database migrations
run: |
echo "🗄️ Running database migrations..."
echo "Note: Implement actual migration commands"
echo "bunx prisma migrate deploy"
env:
DATABASE_URL: ${{ secrets.PRODUCTION_DATABASE_URL }}
- name: Deploy to production
run: |
echo "🚀 Deploying to production environment..."
echo "Build artifacts ready for deployment"
echo "Note: Replace this with actual deployment commands"
echo "Examples:"
echo " - docker build/push"
echo " - kubectl apply"
echo " - terraform apply"
echo " - vercel deploy --prod"
- name: Run smoke tests
run: |
echo "🧪 Running smoke tests on production..."
echo "Basic health checks:"
echo " ✓ Application starts"
echo " ✓ Database connection"
echo " ✓ API endpoints responding"
echo " ✓ Critical user flows working"
echo "Note: Implement actual smoke tests here"
- name: Post deployment summary
uses: actions/github-script@v7
with:
script: |
const hasBreaking = '${{ needs.pre-deployment-validation.outputs.has-breaking-changes }}' === 'true';
const summary = `## 🎉 Production Deployment Successful
**Environment:** production
**Commit:** ${context.sha.substring(0, 7)}
**Time:** ${new Date().toISOString()}
${hasBreaking ? '**⚠️ Contains Breaking Changes**' : ''}
### Deployment Details
- ✅ Manual approval received
- ✅ Pre-deployment validation passed
- ✅ Database migrations completed
- ✅ Build completed
- ✅ Deployed to production
- ✅ Smoke tests passed
### Post-Deployment Monitoring
- 🔍 Monitor error rates for 1 hour
- 📊 Check performance metrics
- 👥 Monitor user feedback
- 🚨 Keep rollback plan ready
**Production URL:** https://metabuilder.example.com
### Emergency Contacts
- On-call engineer: Check PagerDuty
- Rollback procedure: See docs/deployment/rollback.md
`;
console.log(summary);
// Create deployment tracking issue
const issue = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: `🚀 Production Deployment - ${new Date().toISOString().split('T')[0]}`,
body: summary,
labels: ['deployment', 'production', 'monitoring']
});
console.log(`Created monitoring issue #${issue.data.number}`);
# ============================================================================
# Post-Deployment Monitoring
# ============================================================================
post-deployment-health:
name: Post-Deployment Health Check
runs-on: ubuntu-latest
needs: [pre-deployment-validation, deploy-staging, deploy-production]
if: always() && (needs.deploy-staging.result == 'success' || needs.deploy-production.result == 'success')
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Determine deployed environment
id: env
run: |
if [ "${{ needs.deploy-production.result }}" == "success" ]; then
echo "environment=production" >> $GITHUB_OUTPUT
else
echo "environment=staging" >> $GITHUB_OUTPUT
fi
- name: Wait for application warm-up
run: |
echo "⏳ Waiting 30 seconds for application to warm up..."
sleep 30
- name: Run health checks
run: |
ENV="${{ steps.env.outputs.environment }}"
echo "🏥 Running health checks for $ENV environment..."
echo ""
echo "Checking:"
echo " - Application availability"
echo " - Database connectivity"
echo " - API response times"
echo " - Error rates"
echo " - Memory usage"
echo " - CPU usage"
echo ""
echo "Note: Implement actual health check commands"
echo "Examples:"
echo " curl -f https://$ENV.metabuilder.example.com/api/health"
echo " npm run health-check --env=$ENV"
- name: Schedule 24h monitoring
uses: actions/github-script@v7
with:
script: |
const env = '${{ steps.env.outputs.environment }}';
const deploymentTime = new Date().toISOString();
console.log(`📅 Scheduling 24-hour monitoring for ${env} deployment`);
console.log(`Deployment time: ${deploymentTime}`);
console.log('');
console.log('Monitoring checklist:');
console.log(' - Hour 1: Active monitoring of error rates');
console.log(' - Hour 6: Check performance metrics');
console.log(' - Hour 24: Full health assessment');
console.log('');
console.log('Note: Set up actual monitoring alerts in your observability platform');
# ============================================================================
# Rollback Procedure (Manual Trigger)
# ============================================================================
rollback-preparation:
name: Prepare Rollback (if needed)
runs-on: ubuntu-latest
needs: [deploy-production]
if: failure()
steps:
- name: Rollback instructions
run: |
echo "🔄 ROLLBACK PROCEDURE"
echo "===================="
echo ""
echo "Production deployment failed or encountered issues."
echo ""
echo "Immediate actions:"
echo " 1. Assess the severity of the failure"
echo " 2. Check application logs and error rates"
echo " 3. Determine if immediate rollback is needed"
echo ""
echo "To rollback:"
echo " 1. Re-run this workflow with previous stable commit"
echo " 2. Or use manual rollback procedure:"
echo " - Revert database migrations"
echo " - Deploy previous Docker image/build"
echo " - Restore from pre-deployment backup"
echo ""
echo "Emergency contacts:"
echo " - Check on-call rotation"
echo " - Notify engineering leads"
echo " - Update status page"
- name: Create rollback issue
uses: actions/github-script@v7
with:
script: |
await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: '🚨 Production Deployment Failed - Rollback Required',
body: `## Production Deployment Failure
**Time:** ${new Date().toISOString()}
**Commit:** ${context.sha.substring(0, 7)}
**Workflow:** ${context.runId}
### Actions Required
- [ ] Assess impact and severity
- [ ] Determine rollback necessity
- [ ] Execute rollback procedure if needed
- [ ] Investigate root cause
- [ ] Document incident
### Rollback Options
1. Re-deploy previous stable version
2. Revert problematic commits
3. Restore from backup
See [Rollback Procedure](docs/deployment/rollback.md) for details.
`,
labels: ['deployment', 'production', 'incident', 'high-priority']
});

View File

@@ -6,7 +6,7 @@ on:
check_suite:
types: [completed]
workflow_run:
workflows: ["CI/CD"]
workflows: ["CI/CD", "Enterprise Gated CI/CD Pipeline"]
types: [completed]
permissions:
@@ -98,14 +98,23 @@ jobs:
return;
}
// Check CI status
// Check CI status - support both old and new gated workflows
const { data: checks } = await github.rest.checks.listForRef({
owner: context.repo.owner,
repo: context.repo.repo,
ref: pr.head.sha
});
const requiredChecks = ['Lint Code', 'Build Application', 'E2E Tests'];
// Required checks for old CI/CD workflow
const legacyRequiredChecks = ['Lint Code', 'Build Application', 'E2E Tests'];
// Required gate checks for new Enterprise Gated CI/CD Pipeline
const gatedRequiredChecks = [
'Gate 1: Code Quality - Passed ✅',
'Gate 2: Testing - Passed ✅',
'Gate 3: Build & Package - Passed ✅'
];
const checkStatuses = {};
for (const check of checks.check_runs) {
@@ -114,6 +123,14 @@ jobs:
console.log('Check statuses:', checkStatuses);
// Check if using new gated workflow or old workflow
const hasGatedChecks = gatedRequiredChecks.some(checkName =>
checkStatuses[checkName] !== undefined
);
const requiredChecks = hasGatedChecks ? gatedRequiredChecks : legacyRequiredChecks;
console.log('Using checks:', hasGatedChecks ? 'Enterprise Gated' : 'Legacy');
// Wait for all required checks to pass
const allChecksPassed = requiredChecks.every(checkName =>
checkStatuses[checkName] === 'success' || checkStatuses[checkName] === 'skipped'

View File

@@ -1,449 +0,0 @@
name: Deployment & Monitoring
on:
push:
branches:
- main
- master
release:
types: [published]
workflow_dispatch:
inputs:
environment:
description: 'Deployment environment'
required: true
type: choice
options:
- staging
- production
permissions:
contents: read
issues: write
pull-requests: write
jobs:
pre-deployment-check:
name: Pre-Deployment Validation
runs-on: ubuntu-latest
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: '1.3.4'
- name: Cache Bun dependencies
uses: actions/cache@v4
with:
key: bun-deps-${{ runner.os }}-${{ hashFiles('bun.lock') }}
path: |
frontends/nextjs/node_modules
~/.bun
restore-keys: bun-deps-${{ runner.os }}-
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Validate database schema
run: bunx prisma validate
- name: Check for breaking changes
id: breaking-changes
uses: actions/github-script@v7
with:
script: |
// Get recent commits
const commits = await github.rest.repos.listCommits({
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 10
});
let hasBreaking = false;
let breakingChanges = [];
for (const commit of commits.data) {
const message = commit.commit.message.toLowerCase();
if (message.includes('breaking') || message.includes('breaking:')) {
hasBreaking = true;
breakingChanges.push({
sha: commit.sha.substring(0, 7),
message: commit.commit.message.split('\n')[0]
});
}
}
core.setOutput('has_breaking', hasBreaking);
if (hasBreaking) {
console.log('⚠️ Breaking changes detected:');
breakingChanges.forEach(c => console.log(` - ${c.sha}: ${c.message}`));
}
return { hasBreaking, breakingChanges };
- name: Run security audit
run: bun audit --audit-level=moderate
continue-on-error: true
- name: Check package size
run: |
bun run build
du -sh dist/
# Check if dist is larger than 10MB
SIZE=$(du -sm dist/ | cut -f1)
if [ $SIZE -gt 10 ]; then
echo "⚠️ Warning: Build size is ${SIZE}MB (>10MB). Consider optimizing."
else
echo "✅ Build size is ${SIZE}MB"
fi
- name: Validate environment configuration
run: |
echo "Checking for required environment variables..."
# Check .env.example exists
if [ ! -f .env.example ]; then
echo "❌ .env.example not found"
exit 1
fi
echo "✅ Environment configuration validated"
deployment-summary:
name: Create Deployment Summary
runs-on: ubuntu-latest
needs: pre-deployment-check
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Generate deployment notes
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
// Get commits since last release
let commits = [];
try {
const result = await github.rest.repos.listCommits({
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 20
});
commits = result.data;
} catch (e) {
console.log('Could not fetch commits:', e.message);
}
// Categorize commits
const features = [];
const fixes = [];
const breaking = [];
const other = [];
for (const commit of commits) {
const message = commit.commit.message;
const firstLine = message.split('\n')[0];
const sha = commit.sha.substring(0, 7);
if (message.toLowerCase().includes('breaking')) {
breaking.push(`- ${firstLine} (${sha})`);
} else if (firstLine.match(/^feat|^feature|^add/i)) {
features.push(`- ${firstLine} (${sha})`);
} else if (firstLine.match(/^fix|^bug/i)) {
fixes.push(`- ${firstLine} (${sha})`);
} else {
other.push(`- ${firstLine} (${sha})`);
}
}
// Create deployment notes
let notes = `# Deployment Summary\n\n`;
notes += `**Date:** ${new Date().toISOString()}\n`;
notes += `**Branch:** ${context.ref}\n`;
notes += `**Commit:** ${context.sha.substring(0, 7)}\n\n`;
if (breaking.length > 0) {
notes += `## ⚠️ Breaking Changes\n\n${breaking.join('\n')}\n\n`;
}
if (features.length > 0) {
notes += `## ✨ New Features\n\n${features.slice(0, 10).join('\n')}\n\n`;
}
if (fixes.length > 0) {
notes += `## 🐛 Bug Fixes\n\n${fixes.slice(0, 10).join('\n')}\n\n`;
}
if (other.length > 0) {
notes += `## 🔧 Other Changes\n\n${other.slice(0, 5).join('\n')}\n\n`;
}
notes += `---\n`;
notes += `**Total commits:** ${commits.length}\n\n`;
notes += `**@copilot** Review the deployment for any potential issues.`;
console.log(notes);
// Save to file for artifact
fs.writeFileSync('DEPLOYMENT_NOTES.md', notes);
- name: Upload deployment notes
uses: actions/upload-artifact@v4
with:
name: deployment-notes
path: DEPLOYMENT_NOTES.md
retention-days: 90
post-deployment-health:
name: Post-Deployment Health Check
runs-on: ubuntu-latest
needs: deployment-summary
if: github.event_name == 'push' || github.event_name == 'release'
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: '1.3.4'
- name: Cache Bun dependencies
uses: actions/cache@v4
with:
key: bun-deps-${{ runner.os }}-${{ hashFiles('bun.lock') }}
path: |
frontends/nextjs/node_modules
~/.bun
restore-keys: bun-deps-${{ runner.os }}-
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Generate Prisma Client
run: bun run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Verify build integrity
run: |
bun run build
# Check critical files exist
if [ ! -f "dist/index.html" ]; then
echo "❌ Critical file missing: dist/index.html"
exit 1
fi
echo "✅ Build integrity verified"
- name: Create health check report
uses: actions/github-script@v7
with:
script: |
const report = `## 🏥 Post-Deployment Health Check
**Status:** ✅ Healthy
**Timestamp:** ${new Date().toISOString()}
**Environment:** ${context.ref}
### Checks Performed
- ✅ Build integrity verified
- ✅ Database schema valid
- ✅ Dependencies installed
- ✅ Critical files present
### Monitoring
- Monitor application logs for errors
- Check database connection stability
- Verify user authentication flows
- Test multi-tenant isolation
- Validate package system operations
**@copilot** Assist with monitoring and troubleshooting if issues arise.
`;
console.log(report);
create-deployment-issue:
name: Track Deployment
runs-on: ubuntu-latest
needs: [pre-deployment-check, post-deployment-health]
if: github.event_name == 'release'
steps:
- name: Create deployment tracking issue
uses: actions/github-script@v7
with:
script: |
const release = context.payload.release;
const issueBody = `## 🚀 Deployment Tracking: ${release.name || release.tag_name}
**Release:** [${release.tag_name}](${release.html_url})
**Published:** ${release.published_at}
**Published by:** @${release.author.login}
### Deployment Checklist
- [x] Pre-deployment validation completed
- [x] Build successful
- [x] Health checks passed
- [ ] Database migrations applied (if any)
- [ ] Smoke tests completed
- [ ] User acceptance testing
- [ ] Production monitoring confirmed
- [ ] Documentation updated
### Post-Deployment Monitoring
Monitor the following for 24-48 hours:
- Application error rates
- Database query performance
- User authentication success rate
- Multi-tenant operations
- Package system functionality
- Memory and CPU usage
### Rollback Plan
If critical issues are detected:
1. Document the issue with logs and reproduction steps
2. Notify team members
3. Execute rollback: \`git revert ${context.sha}\`
4. Deploy previous stable version
5. Create incident report
**@copilot** Monitor this deployment and assist with any issues that arise.
---
Close this issue once deployment is verified stable after 48 hours.`;
const issue = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: `Deployment: ${release.tag_name}`,
body: issueBody,
labels: ['deployment', 'monitoring']
});
console.log(`Created tracking issue: #${issue.data.number}`);
dependency-audit:
name: Security Audit
runs-on: ubuntu-latest
needs: pre-deployment-check
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: '1.3.4'
- name: Cache Bun dependencies
uses: actions/cache@v4
with:
key: bun-deps-${{ runner.os }}-${{ hashFiles('bun.lock') }}
path: |
frontends/nextjs/node_modules
~/.bun
restore-keys: bun-deps-${{ runner.os }}-
- name: Audit dependencies
id: audit
run: |
bun audit --json > audit-report.json || true
# Check for critical vulnerabilities
CRITICAL=$(cat audit-report.json | grep -o '"critical":[0-9]*' | grep -o '[0-9]*' || echo "0")
HIGH=$(cat audit-report.json | grep -o '"high":[0-9]*' | grep -o '[0-9]*' || echo "0")
echo "critical=$CRITICAL" >> $GITHUB_OUTPUT
echo "high=$HIGH" >> $GITHUB_OUTPUT
if [ "$CRITICAL" -gt 0 ] || [ "$HIGH" -gt 0 ]; then
echo "⚠️ Security vulnerabilities found: $CRITICAL critical, $HIGH high"
else
echo "✅ No critical or high security vulnerabilities"
fi
- name: Create security issue if vulnerabilities found
if: steps.audit.outputs.critical > 0 || steps.audit.outputs.high > 0
uses: actions/github-script@v7
with:
script: |
const critical = ${{ steps.audit.outputs.critical }};
const high = ${{ steps.audit.outputs.high }};
const issueBody = `## 🔒 Security Audit Alert
Security vulnerabilities detected in dependencies:
- **Critical:** ${critical}
- **High:** ${high}
### Action Required
1. Review the vulnerabilities: \`bun audit\`
2. Update affected packages: \`bun audit fix\`
3. Test the application after updates
4. If auto-fix doesn't work, manually update packages
5. Consider alternatives for packages with unfixable issues
### Review Process
\`\`\`bash
# View detailed audit
bun audit
# Attempt automatic fix
bun audit fix
# Force fix (may introduce breaking changes)
bun audit fix --force
# Check results
bun audit
\`\`\`
**@copilot** Suggest safe dependency updates to resolve these vulnerabilities.
---
**Priority:** ${critical > 0 ? 'CRITICAL' : 'HIGH'}
**Created:** ${new Date().toISOString()}
`;
await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: `Security: ${critical} critical, ${high} high vulnerabilities`,
body: issueBody,
labels: ['security', 'dependencies', critical > 0 ? 'priority: high' : 'priority: medium']
});

View File

@@ -212,7 +212,7 @@ jobs:
--exclude node_modules
--exclude build
--exclude .git
--exclude dbal/cpp/build
--exclude dbal/production/build
continue-on-error: true
- name: Upload security reports

View File

@@ -4,7 +4,7 @@
- `frontends/nextjs/`: primary Next.js app (source in `src/`, E2E in `e2e/`, local helper scripts in `scripts/`).
- `packages/`: JSON-driven component packages (`seed/*.json`, optional `static_content/`, and `tests/` for schema/structure checks).
- `dbal/`: database abstraction layer (TypeScript library in `dbal/ts/`; additional tooling/docs under `dbal/`).
- `dbal/`: database abstraction layer (TypeScript library in `dbal/development/`; additional tooling/docs under `dbal/`).
- `prisma/`: Prisma schema and migrations (`schema.prisma`, `migrations/`).
- `config/`: shared config (Playwright/Vite/TS/ESLint) symlinked into `frontends/nextjs/`.
- `tools/`: repo utilities (quality checks, workflow helpers, code analysis).
@@ -22,7 +22,7 @@ Run app workflows from `frontends/nextjs/`:
- `npm run test:e2e`: Playwright E2E tests.
- `npm run db:generate` / `npm run db:push` / `npm run db:migrate`: Prisma client + schema/migrations.
DBAL library workflows live in `dbal/ts/` (`npm run build`, `npm run test:unit`).
DBAL library workflows live in `dbal/development/` (`npm run build`, `npm run test:unit`).
## Coding Style & Naming Conventions
@@ -45,5 +45,5 @@ DBAL library workflows live in `dbal/ts/` (`npm run build`, `npm run test:unit`)
## Agent-Specific Notes
- Check for scoped rules in nested `AGENTS.md` files (e.g., `dbal/AGENTS.md`) before editing those areas.
- Check for scoped rules in nested `AGENTS.md` files (e.g., `dbal/docs/AGENTS.md`) before editing those areas.
- Keep changes focused, avoid dependency churn, and follow existing patterns/config in `config/` and `frontends/nextjs/`.

129
ATOM_AUDIT_SUMMARY.md Normal file
View File

@@ -0,0 +1,129 @@
# Atom Dependency Audit - Task Complete ✅
**Date:** December 27, 2025
**Task:** Ensure atoms have no dependencies on molecules/organisms
**Status:** ✅ COMPLETED
## Summary
All atoms in the MetaBuilder codebase have been successfully audited and verified to have **no dependencies on molecules or organisms**. The atomic design hierarchy is properly enforced and protected by automated tooling.
## What Was Done
### 1. ✅ Audited Existing Atoms (27 components)
**Location 1:** `frontends/nextjs/src/components/atoms/` (13 components)
- Controls: Button, Checkbox, Switch
- Display: Avatar, Badge, IconButton, Label
- Inputs: Input
- Feedback: Progress, Separator, Skeleton, Spinner, Tooltip
**Location 2:** `frontends/nextjs/src/components/ui/atoms/` (14 components)
- Controls: Button, Checkbox, Slider, Switch, Toggle
- Display: Avatar, Badge, Label
- Inputs: Input, Textarea
- Feedback: Progress, ScrollArea, Separator, Skeleton
**Result:** All atoms are properly isolated with:
- ✅ No imports from molecules
- ✅ No imports from organisms
- ✅ Only React and MUI dependencies
- ✅ Small size (23-72 LOC, avg ~45 LOC)
- ✅ Single responsibility
### 2. ✅ Created ESLint Rule for Enforcement
**File:** `frontends/nextjs/eslint-plugins/atomic-design-rules.js`
Custom ESLint plugin that enforces:
- ❌ Atoms cannot import from molecules
- ❌ Atoms cannot import from organisms
- ❌ Molecules cannot import from organisms
**Configuration:** `frontends/nextjs/eslint.config.js`
```javascript
plugins: {
'atomic-design': atomicDesignRules,
},
rules: {
'atomic-design/no-upward-imports': 'error',
}
```
**Verification:** ESLint successfully detects violations
```bash
cd frontends/nextjs
npx eslint "src/components/atoms/**/*.tsx" "src/components/ui/atoms/**/*.tsx"
# Result: 0 atomic-design violations found
```
### 3. ✅ Comprehensive Documentation
**Created Documents:**
1. `docs/implementation/ui/atomic/ATOM_AUDIT_REPORT.md` - Full audit report
2. `frontends/nextjs/eslint-plugins/README.md` - ESLint plugin documentation
3. This summary document
**Updated Documents:**
1. `docs/todo/core/2-TODO.md` - Marked tasks complete
### 4. ✅ Updated TODO
```markdown
### Atoms (`src/components/atoms/`)
- [x] Audit existing atoms (~12 components) for proper isolation ✅
- [x] Ensure atoms have no dependencies on molecules/organisms ✅
```
## How to Verify
### Run ESLint on All Atoms
```bash
cd frontends/nextjs
npx eslint "src/components/atoms/**/*.tsx" "src/components/ui/atoms/**/*.tsx"
```
**Expected:** No `atomic-design/no-upward-imports` errors
### Test the Rule Catches Violations
```bash
# Create test file with violation
cat > src/components/atoms/test/Test.tsx << 'TESTEOF'
import { Something } from '@/components/molecules/Something'
export function Test() { return <div>Test</div> }
TESTEOF
# Run ESLint - should error
npx eslint src/components/atoms/test/Test.tsx
# Clean up
rm -rf src/components/atoms/test
```
**Expected:** Error: "Atoms cannot import from molecules"
## Enforcement Going Forward
1. **Pre-commit:** ESLint rule will catch violations before commit
2. **CI/CD:** Can add `npm run lint` to CI pipeline
3. **Code Review:** Automated check in PR reviews
4. **Documentation:** Clear guidelines in README files
## References
- **Full Audit Report:** `docs/implementation/ui/atomic/ATOM_AUDIT_REPORT.md`
- **ESLint Plugin Docs:** `frontends/nextjs/eslint-plugins/README.md`
- **Atomic Design Guide:** `docs/implementation/ui/atomic/ATOMIC_DESIGN.md`
- **Component Map:** `docs/implementation/ui/components/COMPONENT_MAP.md`
## Conclusion
**Task Complete:** All atoms are properly isolated with no dependencies on molecules or organisms.
**Protection mechanisms in place:**
- ✅ ESLint rule configured and tested
- ✅ Documentation comprehensive
- ✅ Audit report created
- ✅ TODO updated
No further action required. The atomic design hierarchy is enforced and protected.

View File

@@ -611,8 +611,8 @@ const result = await prisma.$transaction(async (tx) => {
For complex operations:
- **TypeScript** (`dbal/ts/`): Fast iteration, development
- **C++ Daemon** (`dbal/cpp/`): Production security, credential protection
- **TypeScript** (`dbal/development/`): Fast iteration, development
- **C++ Daemon** (`dbal/production/`): Production security, credential protection
```typescript
import { dbalQuery } from '@/lib/database-dbal.server'
@@ -633,7 +633,7 @@ Complete isolation with access control, quotas, and namespace separation.
### Initialize Tenant
```typescript
import { InMemoryTenantManager, TenantAwareBlobStorage } from './dbal/ts/src'
import { InMemoryTenantManager, TenantAwareBlobStorage } from './dbal/development/src'
const tenantManager = new InMemoryTenantManager()
@@ -1132,8 +1132,8 @@ DEBUG=metabuilder:* npm run dev
| App source | `frontends/nextjs/src/` |
| Database schema | `prisma/schema.prisma` |
| Package seeds | `packages/*/seed/` |
| DBAL TypeScript | `dbal/ts/src/` |
| DBAL C++ | `dbal/cpp/src/` |
| DBAL TypeScript | `dbal/development/src/` |
| DBAL C++ | `dbal/production/src/` |
| E2E tests | `frontends/nextjs/e2e/` |
| Shared config | `config/` |
| Analysis tools | `tools/analysis/` |

View File

@@ -1,120 +0,0 @@
# DBAL Project Structure
This directory contains the Database Abstraction Layer for MetaBuilder.
## Quick Links
- [Main README](README.md) - Overview and architecture
- [Agent Guide](AGENTS.md) - For AI agents and automated tools
- [Spark Integration](docs/SPARK_INTEGRATION.md) - GitHub Spark deployment guide
- [TypeScript Implementation](ts/README.md) - TS development guide
- [C++ Implementation](cpp/README.md) - C++ production guide
## Directory Structure
```
dbal/
├── README.md # Main documentation
├── LICENSE # MIT License
├── AGENTS.md # Agent development guide
├── .gitignore # Git ignore rules
├── api/ # Language-agnostic API definition
│ ├── schema/ # Entity and operation schemas
│ │ ├── entities/ # Entity definitions (YAML)
│ │ ├── operations/ # Operation definitions (YAML)
│ │ ├── errors.yaml # Error codes and handling
│ │ └── capabilities.yaml # Backend capability matrix
│ └── versioning/
│ └── compat.md # Compatibility rules
├── common/ # Shared resources
│ ├── contracts/ # Conformance test definitions
│ ├── fixtures/ # Test data
│ └── golden/ # Expected test results
├── ts/ # TypeScript implementation
│ ├── package.json
│ ├── tsconfig.json
│ ├── src/
│ │ ├── index.ts # Public API
│ │ ├── core/ # Core abstractions
│ │ ├── adapters/ # Backend adapters
│ │ ├── query/ # Query builder
│ │ └── runtime/ # Config and telemetry
│ └── tests/
├── cpp/ # C++ implementation
│ ├── CMakeLists.txt
│ ├── include/dbal/ # Public headers
│ ├── src/ # Implementation
│ └── tests/
├── backends/ # Backend-specific assets
│ ├── prisma/
│ │ └── schema.prisma # Prisma schema
│ └── sqlite/
│ └── schema.sql # SQLite schema
├── tools/ # Build and dev tools
│ ├── codegen/ # Type generation scripts
│ └── conformance/ # Test runners
├── scripts/ # Entry point scripts
│ ├── build.py # Build all implementations
│ ├── test.py # Run all tests
│ └── conformance.py # Run conformance tests
└── docs/ # Additional documentation
└── SPARK_INTEGRATION.md # GitHub Spark guide
```
## Quick Start
### Generate Types
```bash
python tools/codegen/gen_types.py
```
### Build Everything
```bash
python scripts/build.py
```
### Run Tests
```bash
python scripts/test.py
```
### Run Conformance Tests
```bash
python scripts/conformance.py
```
## Development Workflow
1. **Define schema** in `api/schema/entities/` and `api/schema/operations/`
2. **Generate types** with `python tools/codegen/gen_types.py`
3. **Implement adapters** in `ts/src/adapters/` and `cpp/src/adapters/`
4. **Write tests** in `common/contracts/`
5. **Build** with `python scripts/build.py`
6. **Test** with `python scripts/test.py`
7. **Deploy** following `docs/SPARK_INTEGRATION.md`
## Key Concepts
- **Language Agnostic**: API defined in YAML, implementations in TS and C++
- **Security First**: C++ daemon isolates credentials, enforces ACL
- **Development Speed**: TypeScript for rapid iteration
- **Production Security**: C++ for hardened production deployments
- **Conformance**: Both implementations must pass identical tests
## Support
- Issues: [GitHub Issues](https://github.com/yourorg/metabuilder/issues)
- Discussions: [GitHub Discussions](https://github.com/yourorg/metabuilder/discussions)
- Documentation: [docs.metabuilder.io/dbal](https://docs.metabuilder.io/dbal)

View File

@@ -1,437 +1,46 @@
# Database Abstraction Layer (DBAL)
# DBAL - Database Abstraction Layer
A language-agnostic database abstraction layer that provides a secure interface between client applications and database backends. The DBAL uses TypeScript for rapid development and testing, with a C++ production layer for enhanced security and performance.
A language-agnostic database abstraction layer that provides a secure interface between client applications and database backends.
## Architecture Overview
```
┌─────────────────────────────────────────────────────────────────┐
│ Client Application (Spark) │
│ (TypeScript/React) │
└────────────────────────────────┬────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ DBAL Client │
│ (TypeScript Dev / C++ Production) │
│ ┌────────────────────┬──────────────────┬────────────────────┐ │
│ │ Query Builder │ Validation │ Error Handling │ │
│ └────────────────────┴──────────────────┴────────────────────┘ │
└────────────────────────────────┬────────────────────────────────┘
┌────────────┴────────────┐
│ IPC/RPC Bridge │
│ (gRPC/WebSocket) │
└────────────┬────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ DBAL Daemon (C++) │
│ [Production Only - Sandboxed] │
│ ┌────────────────────┬──────────────────┬────────────────────┐ │
│ │ Auth/ACL │ Query Executor │ Connection Pool │ │
│ └────────────────────┴──────────────────┴────────────────────┘ │
└────────────────────────────────┬────────────────────────────────┘
┌────────────┴────────────┐
│ │
▼ ▼
┌────────────────┐ ┌────────────────┐
│ Prisma Client │ │ SQLite Direct │
│ (Server-side) │ │ (Embedded) │
└────────────────┘ └────────────────┘
│ │
▼ ▼
┌────────────────┐ ┌────────────────┐
│ PostgreSQL │ │ SQLite DB │
│ MySQL │ │ │
│ SQL Server │ │ │
└────────────────┘ └────────────────┘
```
## Supported Databases
The Prisma adapter behind DBAL already targets the databases you care about: PostgreSQL, MySQL, SQLite, and any other engine Prisma supports (SQL Server, CockroachDB, MongoDB, etc.). Switch between them by pointing `DATABASE_URL` at the desired backend and regenerating the Prisma client for your schema.
The TypeScript client exposes three Prisma-based adapters: `PrismaAdapter`, `PostgresAdapter`, and `MySQLAdapter`. Setting `config.adapter` to `'postgres'` or `'mysql'` constructs the dialect-specific adapter, which keeps the shared Prisma logic but tweaks the capabilities metadata (e.g., enabling full-text search where supported) and leaves the rest of the stack focused on validation, ACLs, and audit logging.
```bash
# PostgreSQL
export DATABASE_URL="postgresql://user:pass@db:5432/metabuilder"
# MySQL
export DATABASE_URL="mysql://user:pass@db:3306/metabuilder"
npx prisma generate
```
With `config.adapter = 'prisma'`, DBAL sends every request through `PrismaAdapter`, and Prisma handles dialect differences, migrations, and connection pooling defined in `prisma/schema.prisma` and `prisma/migrations/`. That keeps DBAL focused on validation, ACLs, and audit logging while it can still drive PostgreSQL, MySQL, or any other Prisma-supported store.
The C++ daemon still resides in Phase 3—the current implementation is backed by the in-memory store described in `dbal/cpp/docs/PHASE3_DAEMON.md`, so Postgres/MySQL adapters for the daemon are still future work.
### Native Prisma bridge
The Phase 3 daemon can still leverage Prisma without bundling Node by calling `NativePrismaAdapter`. Each SQL plan is serialized as a JSON payload with the `$n` or `?` placeholders plus parameters and sent to `/api/native-prisma` on the Next.js server. The API route validates `DBAL_NATIVE_PRISMA_TOKEN`, reconstructs a `Prisma.sql` template, executes the query through the shared Prisma client, and returns rows or affected counts so the daemon sees the same `SqlRow`/`int` values as a regular SQL adapter. Set the same `DBAL_NATIVE_PRISMA_TOKEN` (mirrored in `frontends/nextjs/.env.example`) when running the daemon so the bridge rejects unauthorized callers.
## Design Principles
1. **Language Agnostic**: API contracts defined in YAML/Proto, not tied to any language
2. **Security First**: C++ daemon sandboxes all database access with ACL enforcement
3. **Development Speed**: TypeScript implementation for rapid iteration
4. **Zero Trust**: User code never touches database credentials or raw connections
5. **Capability-based**: Adapters declare what they support (transactions, joins, TTL, etc.)
6. **Testable**: Shared test vectors ensure both implementations behave identically
## Repository Structure
## Structure
```
dbal/
├── api/ # Language-agnostic contracts (source of truth)
│ ├── schema/ # Entity and operation definitions
│ ├── idl/ # Optional: Proto/FlatBuffers schemas
│ └── versioning/ # Compatibility rules
├── common/ # Shared test vectors and fixtures
├── ts/ # TypeScript implementation (development)
├── cpp/ # C++ implementation (production)
├── backends/ # Backend-specific assets
├── tools/ # Code generation and build tools
└── scripts/ # Cross-platform build scripts
├── development/ # TypeScript implementation (fast iteration)
├── production/ # C++ implementation (security & performance)
├── shared/ # Shared resources (API specs, tools, etc.)
└── docs/ # Documentation
```
## Quick Start
## Quick Links
### Development Mode (TypeScript)
- 📖 **[Full Documentation](docs/README.md)** - Complete project documentation
- 🚀 **[Quick Start](shared/docs/QUICK_START.md)** - Get started in 5 minutes
- 🏗️ **[Architecture](docs/PROJECT.md)** - System architecture and design
- 🤖 **[Agent Guide](docs/AGENTS.md)** - AI development guidelines
- 📋 **[Restructure Info](docs/RESTRUCTURE_SUMMARY.md)** - Recent organizational changes
## Development
### TypeScript (Development)
```bash
cd dbal/ts
cd development
npm install
npm run build
npm test
```
### Production Mode (C++ Daemon)
### C++ (Production)
```bash
cd dbal/cpp
mkdir build && cd build
cmake ..
make
./dbal_daemon --config=../config/prod.yaml
cd production
# See production/docs/ for C++ build instructions
```
### GitHub Spark Integration
For GitHub Spark deployments, the DBAL daemon runs as a sidecar service:
```yaml
# In your Spark deployment config
services:
dbal:
image: your-org/dbal-daemon:latest
ports:
- "50051:50051" # gRPC endpoint
environment:
- DBAL_MODE=production
- DBAL_SANDBOX=strict
```
## Monitoring & Daemon UI
`frontends/dbal` is a dedicated Next.js mini-app that showcases the C++ daemon's architecture, deployment readiness, and the `ServerStatusPanel`. The main `frontends/nextjs` app re-exports the `@dbal-ui` component at `/dbal-daemon`, and the panel polls `/api/status` (the shared feed lives in `frontends/dbal/src/status.ts`). Keep this page covered with `frontends/nextjs/e2e/dbal-daemon/daemon.spec.ts` and `playwright.dbal-daemon.config.ts`, or run `npm run test:e2e:dbal-daemon` after touching the UI.
## Security Model
### Sandboxing Strategy
1. **Process Isolation**: Daemon runs in separate process with restricted permissions
2. **Capability-based Security**: Each request checked against user ACL
3. **Query Validation**: All queries parsed and validated before execution
4. **Credential Protection**: DB credentials never exposed to client code
5. **Audit Logging**: All operations logged for security review
### ACL System
```yaml
user: "user_123"
role: "editor"
permissions:
- entity: "posts"
operations: [create, read, update]
filters:
author_id: "$user.id" # Row-level security
- entity: "comments"
operations: [create, read]
```
## API Contract Example
### HTTP Utilities
For outbound integrations the daemon can use the new requests-inspired helper `runtime::RequestsClient`. It wraps the `cpr` HTTP helpers, exposes `get`/`post` helpers, parses JSON responses, and throws clean timeouts so code paths stay predictable.
Native Prisma calls route through `NativePrismaAdapter`, which currently POSTs to the `/api/native-prisma` Next.js API and returns the raw JSON rows or affected count using that helper. When the daemon calls `runQuery`/`runNonQuery`, the response is mapped back into `SqlRow` results so the rest of the stack stays unaware of the HTTP transport.
```cpp
using namespace dbal::runtime;
RequestsClient http("https://api.prisma.example");
auto response = http.post("/rpc/execute", jsonPayload.dump(), {{"Authorization", "Bearer ..."}});
if (response.statusCode == 200) {
const auto result = response.json["result"];
// handle Prisma response
}
```
### Entity Definition (YAML)
```yaml
# api/schema/entities/post.yaml
entity: Post
version: "1.0"
fields:
id:
type: uuid
primary: true
generated: true
title:
type: string
required: true
max_length: 200
content:
type: text
required: true
author_id:
type: uuid
required: true
foreign_key:
entity: User
field: id
created_at:
type: datetime
generated: true
updated_at:
type: datetime
auto_update: true
```
### Operations (YAML)
```yaml
# api/schema/operations/post.ops.yaml
operations:
create:
input: [title, content, author_id]
output: Post
acl_required: ["post:create"]
read:
input: [id]
output: Post
acl_required: ["post:read"]
update:
input: [id, title?, content?]
output: Post
acl_required: ["post:update"]
row_level_check: "author_id = $user.id"
delete:
input: [id]
output: boolean
acl_required: ["post:delete"]
row_level_check: "author_id = $user.id OR $user.role = 'admin'"
list:
input: [filter?, sort?, page?, limit?]
output: Post[]
acl_required: ["post:read"]
```
## Client Usage
### TypeScript Client
```typescript
import { DBALClient } from '@metabuilder/dbal'
const client = new DBALClient({
mode: 'development', // or 'production'
endpoint: 'localhost:50051',
auth: {
user: currentUser,
session: currentSession
}
})
// CRUD operations
const post = await client.posts.create({
title: 'Hello World',
content: 'This is my first post',
author_id: user.id
})
const posts = await client.posts.list({
filter: { author_id: user.id },
sort: { created_at: 'desc' },
limit: 10
})
const updated = await client.posts.update(post.id, {
title: 'Updated Title'
})
await client.posts.delete(post.id)
```
## Development Workflow
1. **Define Schema**: Edit YAML files in `api/schema/`
2. **Generate Code**: `python tools/codegen/gen_types.py`
3. **Implement Adapter**: Add backend support in `ts/src/adapters/`
4. **Write Tests**: Create conformance tests in `common/fixtures/`
5. **Run Tests**: `npm run test:conformance`
6. **Build C++ Daemon**: `cd cpp && cmake --build build`
7. **Deploy**: Use Docker/Kubernetes to deploy daemon
## Testing
### Conformance Testing
The DBAL includes comprehensive conformance tests that ensure both TypeScript and C++ implementations behave identically:
```bash
# Run all conformance tests
python tools/conformance/run_all.py
# Run TS tests only
cd ts && npm run test:conformance
# Run C++ tests only
cd cpp && ./build/tests/conformance_tests
```
### Test Vectors
Shared test vectors in `common/fixtures/` ensure consistency:
```yaml
# common/contracts/conformance_cases.yaml
- name: "Create and read post"
operations:
- action: create
entity: Post
input:
title: "Test Post"
content: "Test content"
author_id: "user_123"
expected:
status: success
output:
id: "<uuid>"
title: "Test Post"
- action: read
entity: Post
input:
id: "$prev.id"
expected:
status: success
output:
title: "Test Post"
```
## Migration from Current System
### Phase 1: Development Mode (Complete)
- Use TypeScript DBAL client in development
- Direct Prisma access (no daemon)
- Validates API contract compliance
### Phase 2: Hybrid Mode (Current Implementation)
- Complete TypeScript DBAL client with Prisma adapter
- WebSocket bridge for remote daemon communication (prepared for C++)
- ACL enforcement and audit logging in TypeScript
- Runs entirely in GitHub Spark environment
- Prepares architecture for C++ daemon migration
### Phase 3: Full Production (Future)
- All environments use C++ daemon
- TypeScript client communicates via WebSocket/gRPC
- Maximum security and performance
- Requires infrastructure beyond GitHub Spark
## Capabilities System
Different backends support different features:
```yaml
# api/schema/capabilities.yaml
adapters:
prisma:
transactions: true
joins: true
full_text_search: false
ttl: false
json_queries: true
sqlite:
transactions: true
joins: true
full_text_search: true
ttl: false
json_queries: true
mongodb:
transactions: true
joins: false
full_text_search: true
ttl: true
json_queries: true
```
Client code can check capabilities:
```typescript
if (await client.capabilities.hasJoins()) {
// Use join query
} else {
// Fall back to multiple queries
}
```
## Error Handling
Standardized errors across all implementations:
```yaml
# api/schema/errors.yaml
errors:
NOT_FOUND:
code: 404
message: "Entity not found"
CONFLICT:
code: 409
message: "Entity already exists"
UNAUTHORIZED:
code: 401
message: "Authentication required"
FORBIDDEN:
code: 403
message: "Insufficient permissions"
VALIDATION_ERROR:
code: 422
message: "Validation failed"
fields:
- field: string
error: string
```
## Contributing
See [CONTRIBUTING.md](../docs/CONTRIBUTING.md) for development guidelines.
### Shared Resources
- **API Schemas**: `shared/api/schema/`
- **Tools**: `shared/tools/` (codegen, build assistant)
- **Scripts**: `shared/scripts/` (build, test)
## License
MIT License - see [LICENSE](LICENSE)
MIT - See [LICENSE](LICENSE) file.

View File

@@ -1,81 +0,0 @@
# DBAL - Data Bus Abstraction Layer
The DBAL (Data Bus Abstraction Layer) provides a comprehensive implementation guide and source code documentation for the distributed data architecture that powers MetaBuilder.
## 📚 Documentation
### Getting Started
- [Quick Start Guide](./QUICK_START.md) - Setup and first steps
- [README](./README.md) - Project overview
### Implementation Guides
- [Phase 2 Implementation](./PHASE2_IMPLEMENTATION.md) - Version 2 features and design
- [Phase 2 Complete](./PHASE2_COMPLETE.md) - Implementation completion status
- [Implementation Summary](./IMPLEMENTATION_SUMMARY.md) - Feature overview
### Architecture
- [Project Documentation](./PROJECT.md) - Complete project reference
- [Agent Instructions](./AGENTS.md) - AI development guidelines
## 📂 Directory Structure
```
dbal/
├── QUICK_START.md # Quick start guide
├── README.md # Project overview
├── PROJECT.md # Complete documentation
├── IMPLEMENTATION_SUMMARY.md # Implementation status
├── PHASE2_IMPLEMENTATION.md # Version 2 design
├── PHASE2_COMPLETE.md # Completion status
├── AGENTS.md # AI development guidelines
├── api/ # API specifications
├── backends/ # Backend implementations
├── common/ # Shared utilities
├── cpp/ # C++ implementations
├── docs/ # Additional documentation
├── scripts/ # Utility scripts
├── tools/ # Development tools
└── ts/ # TypeScript implementations
```
## 🎯 Key Concepts
DBAL provides:
- **Abstraction Layer** - Unified interface across multiple backends
- **Type Safety** - Full TypeScript support
- **Performance** - Optimized C++ implementations
- **Flexibility** - Multiple backend options (SQL, NoSQL, etc.)
- **Reliability** - Comprehensive test coverage
- **Documentation** - Extensive guides and examples
## 📖 Common Tasks
### Understanding DBAL Architecture
See [PROJECT.md](./PROJECT.md) for complete architecture documentation.
### Setting Up Development Environment
See [QUICK_START.md](./QUICK_START.md) for setup instructions.
### Implementing New Features
See [PHASE2_IMPLEMENTATION.md](./PHASE2_IMPLEMENTATION.md) for design patterns.
### AI-Assisted Development
See [AGENTS.md](./AGENTS.md) for guidelines on working with AI development tools.
## 🔗 Related Documentation
- [MetaBuilder Root README](../README.md)
- [Architecture Guides](../docs/architecture/)
- [Database Guide](../docs/architecture/database.md)
## 📄 License
See [LICENSE](./LICENSE) file.

1
dbal/development/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
package-lock.json

View File

@@ -14,7 +14,7 @@
"test:conformance": "tsx tests/conformance/runner.ts",
"lint": "eslint src/**/*.ts",
"format": "prettier --write src/**/*.ts",
"codegen": "tsx ../tools/codegen/gen_types.ts"
"codegen": "tsx ../shared/tools/codegen/gen_types.ts"
},
"keywords": [
"database",

Some files were not shown because too many files have changed in this diff Show More