diff --git a/.github/workflows/README.md b/.github/workflows/README.md index f1519fbc1..4cf7dc10c 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -67,6 +67,42 @@ All workflows are designed to work seamlessly with **GitHub Copilot** to assist - Automatic progression through gates - Summary report with all gate results +**Best for:** Small to medium teams, straightforward workflows + +#### 1a. Enterprise Gated CI/CD Pipeline - Atomic (`gated-ci-atomic.yml`) 🆕 +**Triggered on:** Push to main/master/develop branches, Pull requests + +**Structure:** +- **Gate 1:** Code Quality - 7 atomic steps + - 1.1 Prisma Validation + - 1.2 TypeScript Check (+ strict mode analysis) + - 1.3 ESLint (+ any-type detection + ts-ignore detection) + - 1.4 Security Scan (+ dependency audit) + - 1.5 File Size Check + - 1.6 Code Complexity Analysis + - 1.7 Stub Implementation Detection +- **Gate 2:** Testing - 3 atomic steps + - 2.1 Unit Tests (+ coverage analysis) + - 2.2 E2E Tests + - 2.3 DBAL Daemon Tests +- **Gate 3:** Build & Package - 2 atomic steps + - 3.1 Application Build (+ bundle analysis) + - 3.2 Quality Metrics +- **Gate 4:** Review & Approval (Human review required) + +**Features:** +- **Atomic validation steps** for superior visualization +- Each tool from `/tools` runs as separate job +- **Gate artifacts** persisted between steps (30-day retention) +- Granular failure detection +- Parallel execution within gates +- Complete audit trail with JSON artifacts +- Individual step timing and status + +**Best for:** Large teams, enterprise compliance, audit requirements + +**Documentation:** See [Atomic Gated Workflow Architecture](../../docs/ATOMIC_GATED_WORKFLOW.md) + #### 2. Enterprise Gated Deployment (`gated-deployment.yml`) **Triggered on:** Push to main/master, Releases, Manual workflow dispatch diff --git a/.github/workflows/gated-ci-atomic.yml b/.github/workflows/gated-ci-atomic.yml new file mode 100644 index 000000000..258a09d95 --- /dev/null +++ b/.github/workflows/gated-ci-atomic.yml @@ -0,0 +1,1033 @@ +name: Enterprise Gated CI/CD Pipeline (Atomic) + +on: + push: + branches: [ main, master, develop ] + pull_request: + branches: [ main, master, develop ] + +permissions: + contents: read + pull-requests: write + checks: write + statuses: write + +# Enterprise Gated Tree Workflow with Atomic Steps +# Each validation tool runs as a separate step for better visualization +# Gate artifacts are persisted between stages using GitHub Actions artifacts +# Changes must pass through 5 gates before merge: +# Gate 1: Code Quality (lint, typecheck, security) +# Gate 2: Testing (unit, E2E) +# Gate 3: Build & Package +# Gate 4: Review & Approval +# Gate 5: Deployment (staging → production with manual approval) + +jobs: + # ============================================================================ + # GATE 1: Code Quality Gates - Atomic Steps + # ============================================================================ + + gate-1-start: + name: "Gate 1: Code Quality - Starting" + runs-on: ubuntu-latest + steps: + - name: Gate 1 checkpoint + run: | + echo "🚦 GATE 1: CODE QUALITY VALIDATION" + echo "================================================" + echo "Running atomic validation steps..." + echo "Status: IN PROGRESS" + + - name: Create gate artifacts directory + run: | + mkdir -p gate-artifacts/gate-1 + echo "started" > gate-artifacts/gate-1/status.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-1/start-time.txt + + - name: Upload gate start marker + uses: actions/upload-artifact@v4 + with: + name: gate-1-start + path: gate-artifacts/gate-1/ + + # Atomic Step 1.1: Prisma Validation + prisma-check: + name: "Gate 1.1: Validate Prisma Schema" + runs-on: ubuntu-latest + needs: gate-1-start + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Generate Prisma Client + run: bun run db:generate + env: + DATABASE_URL: file:./dev.db + + - name: Validate Prisma Schema + run: bunx prisma validate + env: + DATABASE_URL: file:./dev.db + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-1 + echo "${{ job.status }}" > gate-artifacts/gate-1/prisma-check.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-1/prisma-check-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-1-prisma-result + path: gate-artifacts/gate-1/ + + # Atomic Step 1.2: TypeScript Check + typecheck: + name: "Gate 1.2: TypeScript Type Check" + runs-on: ubuntu-latest + needs: prisma-check + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Generate Prisma Client + run: bun run db:generate + env: + DATABASE_URL: file:./dev.db + + - name: Run TypeScript type check + run: bun run typecheck + + - name: Run atomic TypeScript strict checker + run: | + cd ../.. + tsx tools/quality/code/check-typescript-strict.ts > gate-artifacts/typescript-strict.json || true + continue-on-error: true + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-1 + echo "${{ job.status }}" > gate-artifacts/gate-1/typecheck.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-1/typecheck-time.txt + cp gate-artifacts/typescript-strict.json gate-artifacts/gate-1/ || true + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-1-typecheck-result + path: gate-artifacts/gate-1/ + + # Atomic Step 1.3: ESLint + lint: + name: "Gate 1.3: Lint Code" + runs-on: ubuntu-latest + needs: prisma-check + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Generate Prisma Client + run: bun run db:generate + env: + DATABASE_URL: file:./dev.db + + - name: Run ESLint + run: bun run lint + + - name: Run atomic lint tools + run: | + mkdir -p ../../gate-artifacts/gate-1 + cd ../.. + + # Find any types + tsx tools/misc/lint/find-any-types.ts > gate-artifacts/gate-1/any-types.json || true + + # Find ts-ignore comments + tsx tools/misc/lint/find-ts-ignores.ts > gate-artifacts/gate-1/ts-ignores.json || true + continue-on-error: true + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-1 + echo "${{ job.status }}" > gate-artifacts/gate-1/lint.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-1/lint-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-1-lint-result + path: gate-artifacts/gate-1/ + + # Atomic Step 1.4: Security Scan + security-scan: + name: "Gate 1.4: Security Scan" + runs-on: ubuntu-latest + needs: prisma-check + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Run atomic security scanner + run: | + mkdir -p ../../gate-artifacts/gate-1 + cd ../.. + tsx tools/security/security-scanner.ts > gate-artifacts/gate-1/security-scan.json || true + continue-on-error: true + + - name: Run dependency audit + run: | + bun audit --json > ../../gate-artifacts/gate-1/audit-results.json 2>&1 || true + echo "Security audit completed" + continue-on-error: true + + - name: Parse audit results + run: | + cd ../.. + tsx tools/misc/metrics/parse-npm-audit.ts gate-artifacts/gate-1/audit-results.json > gate-artifacts/gate-1/audit-summary.json || true + continue-on-error: true + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-1 + echo "${{ job.status }}" > gate-artifacts/gate-1/security-scan.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-1/security-scan-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-1-security-result + path: gate-artifacts/gate-1/ + + # Atomic Step 1.5: File Size Check + file-size-check: + name: "Gate 1.5: File Size Check" + runs-on: ubuntu-latest + needs: prisma-check + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Run atomic file size checker + run: | + mkdir -p ../../gate-artifacts/gate-1 + cd ../.. + tsx tools/quality/files/check-file-sizes.ts > gate-artifacts/gate-1/file-sizes.json || true + continue-on-error: true + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-1 + echo "${{ job.status }}" > gate-artifacts/gate-1/file-size-check.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-1/file-size-check-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-1-filesize-result + path: gate-artifacts/gate-1/ + + # Atomic Step 1.6: Code Complexity Check + code-complexity-check: + name: "Gate 1.6: Code Complexity Check" + runs-on: ubuntu-latest + needs: prisma-check + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Run atomic code complexity checker + run: | + mkdir -p ../../gate-artifacts/gate-1 + cd ../.. + tsx tools/quality/code/check-code-complexity.ts > gate-artifacts/gate-1/complexity.json || true + continue-on-error: true + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-1 + echo "${{ job.status }}" > gate-artifacts/gate-1/complexity-check.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-1/complexity-check-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-1-complexity-result + path: gate-artifacts/gate-1/ + + # Atomic Step 1.7: Stub Detection + stub-detection: + name: "Gate 1.7: Detect Stub Implementations" + runs-on: ubuntu-latest + needs: prisma-check + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Run atomic stub detector + run: | + mkdir -p ../../gate-artifacts/gate-1 + cd ../.. + tsx tools/detection/detect-stub-implementations.ts > gate-artifacts/gate-1/stubs.json || true + continue-on-error: true + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-1 + echo "${{ job.status }}" > gate-artifacts/gate-1/stub-detection.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-1/stub-detection-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-1-stub-result + path: gate-artifacts/gate-1/ + + gate-1-complete: + name: "Gate 1: Code Quality - Passed ✅" + runs-on: ubuntu-latest + needs: [prisma-check, typecheck, lint, security-scan, file-size-check, code-complexity-check, stub-detection] + steps: + - name: Download all gate 1 artifacts + uses: actions/download-artifact@v4 + with: + pattern: gate-1-* + path: gate-artifacts/ + merge-multiple: true + + - name: Generate Gate 1 summary + run: | + echo "✅ GATE 1 PASSED: CODE QUALITY" + echo "================================================" + echo "Atomic validation steps completed:" + echo "✓ 1.1 Prisma schema validated" + echo "✓ 1.2 TypeScript types checked" + echo "✓ 1.3 Code linted" + echo "✓ 1.4 Security scan completed" + echo "✓ 1.5 File sizes checked" + echo "✓ 1.6 Code complexity analyzed" + echo "✓ 1.7 Stub implementations detected" + echo "" + echo "Gate artifacts preserved for audit trail" + echo "Proceeding to Gate 2: Testing..." + + - name: Create consolidated gate report + run: | + mkdir -p gate-artifacts/gate-1 + echo "completed" > gate-artifacts/gate-1/status.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-1/end-time.txt + + # List all validation results + ls -la gate-artifacts/gate-1/ || true + + - name: Upload consolidated gate 1 report + uses: actions/upload-artifact@v4 + with: + name: gate-1-complete-report + path: gate-artifacts/ + + # ============================================================================ + # GATE 2: Testing Gates - Atomic Steps + # ============================================================================ + + gate-2-start: + name: "Gate 2: Testing - Starting" + runs-on: ubuntu-latest + needs: gate-1-complete + steps: + - name: Gate 2 checkpoint + run: | + echo "🚦 GATE 2: TESTING VALIDATION" + echo "================================================" + echo "Running atomic test steps..." + echo "Status: IN PROGRESS" + + - name: Create gate artifacts directory + run: | + mkdir -p gate-artifacts/gate-2 + echo "started" > gate-artifacts/gate-2/status.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-2/start-time.txt + + - name: Upload gate start marker + uses: actions/upload-artifact@v4 + with: + name: gate-2-start + path: gate-artifacts/gate-2/ + + # Atomic Step 2.1: Unit Tests + test-unit: + name: "Gate 2.1: Unit Tests" + runs-on: ubuntu-latest + needs: gate-2-start + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Generate Prisma Client + run: bun run db:generate + env: + DATABASE_URL: file:./dev.db + + - name: Run unit tests + run: bun run test:unit + env: + DATABASE_URL: file:./dev.db + + - name: Generate test coverage report + run: | + mkdir -p ../../gate-artifacts/gate-2 + cd ../.. + node tools/generation/generate-test-coverage-report.js > gate-artifacts/gate-2/coverage-report.json || true + continue-on-error: true + + - name: Check function coverage + run: | + cd ../.. + node tools/quality/code/check-function-coverage.js > gate-artifacts/gate-2/function-coverage.json || true + continue-on-error: true + + - name: Upload coverage report + if: always() + uses: actions/upload-artifact@v4 + with: + name: coverage-report + path: frontends/nextjs/coverage/ + retention-days: 7 + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-2 + echo "${{ job.status }}" > gate-artifacts/gate-2/test-unit.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-2/test-unit-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-2-unit-result + path: gate-artifacts/gate-2/ + + # Atomic Step 2.2: E2E Tests + test-e2e: + name: "Gate 2.2: E2E Tests" + runs-on: ubuntu-latest + needs: gate-2-start + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Generate Prisma Client + run: bun run db:generate + env: + DATABASE_URL: file:./dev.db + + - name: Install Playwright Browsers + run: bunx playwright install --with-deps chromium + + - name: Run Playwright tests + run: bun run test:e2e + env: + DATABASE_URL: file:./dev.db + + - name: Upload test results + if: always() + uses: actions/upload-artifact@v4 + with: + name: playwright-report + path: frontends/nextjs/playwright-report/ + retention-days: 7 + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-2 + echo "${{ job.status }}" > gate-artifacts/gate-2/test-e2e.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-2/test-e2e-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-2-e2e-result + path: gate-artifacts/gate-2/ + + # Atomic Step 2.3: DBAL Daemon Tests + test-dbal-daemon: + name: "Gate 2.3: DBAL Daemon E2E" + runs-on: ubuntu-latest + needs: gate-2-start + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Generate Prisma Client + run: bun run db:generate + env: + DATABASE_URL: file:./dev.db + + - name: Install Playwright Browsers + run: bunx playwright install --with-deps chromium + + - name: Run DBAL daemon suite + run: bun run test:e2e:dbal-daemon + env: + DATABASE_URL: file:./dev.db + + - name: Upload daemon test report + if: always() + uses: actions/upload-artifact@v4 + with: + name: playwright-report-dbal-daemon + path: frontends/nextjs/playwright-report/ + retention-days: 7 + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-2 + echo "${{ job.status }}" > gate-artifacts/gate-2/test-dbal-daemon.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-2/test-dbal-daemon-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-2-dbal-result + path: gate-artifacts/gate-2/ + + gate-2-complete: + name: "Gate 2: Testing - Passed ✅" + runs-on: ubuntu-latest + needs: [test-unit, test-e2e, test-dbal-daemon] + steps: + - name: Download all gate 2 artifacts + uses: actions/download-artifact@v4 + with: + pattern: gate-2-* + path: gate-artifacts/ + merge-multiple: true + + - name: Generate Gate 2 summary + run: | + echo "✅ GATE 2 PASSED: TESTING" + echo "================================================" + echo "Atomic test steps completed:" + echo "✓ 2.1 Unit tests passed" + echo "✓ 2.2 E2E tests passed" + echo "✓ 2.3 DBAL daemon tests passed" + echo "" + echo "Gate artifacts preserved for audit trail" + echo "Proceeding to Gate 3: Build & Package..." + + - name: Create consolidated gate report + run: | + mkdir -p gate-artifacts/gate-2 + echo "completed" > gate-artifacts/gate-2/status.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-2/end-time.txt + ls -la gate-artifacts/gate-2/ || true + + - name: Upload consolidated gate 2 report + uses: actions/upload-artifact@v4 + with: + name: gate-2-complete-report + path: gate-artifacts/ + + # ============================================================================ + # GATE 3: Build & Package Gates - Atomic Steps + # ============================================================================ + + gate-3-start: + name: "Gate 3: Build & Package - Starting" + runs-on: ubuntu-latest + needs: gate-2-complete + steps: + - name: Gate 3 checkpoint + run: | + echo "🚦 GATE 3: BUILD & PACKAGE VALIDATION" + echo "================================================" + echo "Running atomic build steps..." + echo "Status: IN PROGRESS" + + - name: Create gate artifacts directory + run: | + mkdir -p gate-artifacts/gate-3 + echo "started" > gate-artifacts/gate-3/status.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-3/start-time.txt + + - name: Upload gate start marker + uses: actions/upload-artifact@v4 + with: + name: gate-3-start + path: gate-artifacts/gate-3/ + + # Atomic Step 3.1: Build Application + build: + name: "Gate 3.1: Build Application" + runs-on: ubuntu-latest + needs: gate-3-start + defaults: + run: + working-directory: frontends/nextjs + outputs: + build-success: ${{ steps.build-step.outcome }} + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Generate Prisma Client + run: bun run db:generate + env: + DATABASE_URL: file:./dev.db + + - name: Build + id: build-step + run: bun run build + env: + DATABASE_URL: file:./dev.db + + - name: Analyze bundle size + run: | + mkdir -p ../../gate-artifacts/gate-3 + cd ../.. + tsx tools/analysis/bundle/analyze-bundle-size.ts > gate-artifacts/gate-3/bundle-size.json || true + continue-on-error: true + + - name: Upload build artifacts + uses: actions/upload-artifact@v4 + with: + name: dist + path: frontends/nextjs/.next/ + retention-days: 7 + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-3 + echo "${{ job.status }}" > gate-artifacts/gate-3/build.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-3/build-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-3-build-result + path: gate-artifacts/gate-3/ + + # Atomic Step 3.2: Quality Metrics + quality-check: + name: "Gate 3.2: Code Quality Metrics" + runs-on: ubuntu-latest + needs: gate-3-start + if: github.event_name == 'pull_request' + defaults: + run: + working-directory: frontends/nextjs + steps: + - name: Checkout code + uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Install dependencies + run: bun install + + - name: Generate Prisma Client + run: bun run db:generate + env: + DATABASE_URL: file:./dev.db + + - name: Check for console.log statements + run: | + if git diff origin/${{ github.base_ref }}...HEAD -- '*.ts' '*.tsx' '*.js' '*.jsx' | grep -E '^\+.*console\.(log|debug|info)'; then + echo "⚠️ Found console.log statements in the changes" + echo "Please remove console.log statements before merging" + exit 1 + fi + continue-on-error: true + + - name: Check for TODO comments + run: | + TODO_COUNT=$(git diff origin/${{ github.base_ref }}...HEAD -- '*.ts' '*.tsx' '*.js' '*.jsx' | grep -E '^\+.*TODO|FIXME' | wc -l) + if [ $TODO_COUNT -gt 0 ]; then + echo "⚠️ Found $TODO_COUNT TODO/FIXME comments in the changes" + echo "Please address TODO comments before merging or create issues for them" + fi + continue-on-error: true + + - name: Generate quality summary + run: | + mkdir -p ../../gate-artifacts/gate-3 + cd ../.. + tsx tools/generation/generate-quality-summary.ts > gate-artifacts/gate-3/quality-summary.json || true + continue-on-error: true + + - name: Record validation result + if: always() + run: | + mkdir -p gate-artifacts/gate-3 + echo "${{ job.status }}" > gate-artifacts/gate-3/quality-check.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-3/quality-check-time.txt + + - name: Upload validation result + if: always() + uses: actions/upload-artifact@v4 + with: + name: gate-3-quality-result + path: gate-artifacts/gate-3/ + + gate-3-complete: + name: "Gate 3: Build & Package - Passed ✅" + runs-on: ubuntu-latest + needs: [build, quality-check] + if: always() && needs.build.result == 'success' && (needs.quality-check.result == 'success' || needs.quality-check.result == 'skipped') + steps: + - name: Download all gate 3 artifacts + uses: actions/download-artifact@v4 + with: + pattern: gate-3-* + path: gate-artifacts/ + merge-multiple: true + + - name: Generate Gate 3 summary + run: | + echo "✅ GATE 3 PASSED: BUILD & PACKAGE" + echo "================================================" + echo "Atomic build steps completed:" + echo "✓ 3.1 Application built successfully" + echo "✓ 3.2 Quality metrics validated" + echo "" + echo "Gate artifacts preserved for audit trail" + echo "Proceeding to Gate 4: Review & Approval..." + + - name: Create consolidated gate report + run: | + mkdir -p gate-artifacts/gate-3 + echo "completed" > gate-artifacts/gate-3/status.txt + echo "$(date -Iseconds)" > gate-artifacts/gate-3/end-time.txt + ls -la gate-artifacts/gate-3/ || true + + - name: Upload consolidated gate 3 report + uses: actions/upload-artifact@v4 + with: + name: gate-3-complete-report + path: gate-artifacts/ + + # ============================================================================ + # GATE 4: Review & Approval Gate (PR only) + # ============================================================================ + + gate-4-review-required: + name: "Gate 4: Review & Approval Required" + runs-on: ubuntu-latest + needs: gate-3-complete + if: github.event_name == 'pull_request' + steps: + - name: Check PR approval status + uses: actions/github-script@v7 + with: + script: | + const { data: reviews } = await github.rest.pulls.listReviews({ + owner: context.repo.owner, + repo: context.repo.repo, + pull_number: context.issue.number + }); + + const latestReviews = {}; + for (const review of reviews) { + latestReviews[review.user.login] = review.state; + } + + const hasApproval = Object.values(latestReviews).includes('APPROVED'); + const hasRequestChanges = Object.values(latestReviews).includes('CHANGES_REQUESTED'); + + console.log('Review Status:'); + console.log('=============='); + console.log('Approvals:', Object.values(latestReviews).filter(s => s === 'APPROVED').length); + console.log('Change Requests:', Object.values(latestReviews).filter(s => s === 'CHANGES_REQUESTED').length); + + if (hasRequestChanges) { + core.setFailed('❌ Changes requested - PR cannot proceed to deployment'); + } else if (!hasApproval) { + core.notice('⏳ PR approval required before merge - this gate will pass when approved'); + } else { + console.log('✅ PR approved - gate passed'); + } + + gate-4-complete: + name: "Gate 4: Review & Approval - Status" + runs-on: ubuntu-latest + needs: gate-4-review-required + if: always() && github.event_name == 'pull_request' + steps: + - name: Gate 4 status + run: | + echo "🚦 GATE 4: REVIEW & APPROVAL" + echo "================================================" + echo "Note: This gate requires human approval" + echo "PR must be approved by reviewers before auto-merge" + echo "" + if [ "${{ needs.gate-4-review-required.result }}" == "success" ]; then + echo "✅ Review approval received" + echo "Proceeding to Gate 5: Deployment (post-merge)..." + else + echo "⏳ Awaiting review approval" + echo "Gate will complete when PR is approved" + fi + + # ============================================================================ + # GATE 5: Deployment Gate (post-merge, main branch only) + # ============================================================================ + + gate-5-deployment-ready: + name: "Gate 5: Deployment Ready" + runs-on: ubuntu-latest + needs: gate-3-complete + if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master') + steps: + - name: Deployment gate checkpoint + run: | + echo "🚦 GATE 5: DEPLOYMENT VALIDATION" + echo "================================================" + echo "Code merged to main branch" + echo "Ready for staging deployment" + echo "" + echo "✅ ALL GATES PASSED" + echo "================================================" + echo "✓ Gate 1: Code Quality (7 atomic steps)" + echo "✓ Gate 2: Testing (3 atomic steps)" + echo "✓ Gate 3: Build & Package (2 atomic steps)" + echo "✓ Gate 4: Review & Approval" + echo "✓ Gate 5: Ready for Deployment" + echo "" + echo "Note: Production deployment requires manual approval" + echo "Use workflow_dispatch with environment='production'" + + # ============================================================================ + # Summary Report with Gate Artifacts + # ============================================================================ + + gates-summary: + name: "🎯 Gates Summary with Audit Trail" + runs-on: ubuntu-latest + needs: [gate-1-complete, gate-2-complete, gate-3-complete] + if: always() + steps: + - name: Download all gate artifacts + uses: actions/download-artifact@v4 + with: + pattern: gate-*-complete-report + path: all-gate-artifacts/ + merge-multiple: true + + - name: Generate comprehensive gates report + uses: actions/github-script@v7 + with: + script: | + const fs = require('fs'); + const gates = [ + { name: 'Gate 1: Code Quality (Atomic)', status: '${{ needs.gate-1-complete.result }}', steps: 7 }, + { name: 'Gate 2: Testing (Atomic)', status: '${{ needs.gate-2-complete.result }}', steps: 3 }, + { name: 'Gate 3: Build & Package (Atomic)', status: '${{ needs.gate-3-complete.result }}', steps: 2 } + ]; + + let summary = '## 🚦 Enterprise Gated CI/CD Pipeline Summary (Atomic)\n\n'; + summary += '### Gate Results\n\n'; + + for (const gate of gates) { + const icon = gate.status === 'success' ? '✅' : + gate.status === 'failure' ? '❌' : + gate.status === 'skipped' ? '⏭️' : '⏳'; + summary += `${icon} **${gate.name}**: ${gate.status} (${gate.steps} atomic steps)\n`; + } + + summary += '\n### Atomic Step Visualization\n\n'; + summary += 'Each gate consists of individual atomic validation steps for better visibility:\n\n'; + summary += '**Gate 1 Steps:**\n'; + summary += '- 1.1 Prisma Validation\n'; + summary += '- 1.2 TypeScript Check\n'; + summary += '- 1.3 ESLint\n'; + summary += '- 1.4 Security Scan\n'; + summary += '- 1.5 File Size Check\n'; + summary += '- 1.6 Code Complexity\n'; + summary += '- 1.7 Stub Detection\n\n'; + + summary += '**Gate 2 Steps:**\n'; + summary += '- 2.1 Unit Tests\n'; + summary += '- 2.2 E2E Tests\n'; + summary += '- 2.3 DBAL Daemon Tests\n\n'; + + summary += '**Gate 3 Steps:**\n'; + summary += '- 3.1 Application Build\n'; + summary += '- 3.2 Quality Metrics\n\n'; + + summary += '### Gate Artifacts\n\n'; + summary += 'All validation results are preserved as artifacts for audit trail:\n'; + summary += '- Security scan results\n'; + summary += '- Code complexity analysis\n'; + summary += '- Test coverage reports\n'; + summary += '- Bundle size analysis\n'; + summary += '- Quality metrics\n\n'; + + if (context.eventName === 'pull_request') { + summary += '### Next Steps\n'; + summary += '- ✅ All CI gates passed with atomic validation\n'; + summary += '- ⏳ Awaiting PR approval (Gate 4)\n'; + summary += '- 📋 Once approved, PR will auto-merge\n'; + summary += '- 🚀 Deployment gates (Gate 5) run after merge to main\n'; + } + + console.log(summary); + + // Post comment on PR if applicable + if (context.eventName === 'pull_request') { + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.issue.number, + body: summary + }); + } + + - name: Upload complete audit trail + uses: actions/upload-artifact@v4 + with: + name: complete-gate-audit-trail + path: all-gate-artifacts/ + retention-days: 30 diff --git a/docs/ATOMIC_GATED_WORKFLOW.md b/docs/ATOMIC_GATED_WORKFLOW.md new file mode 100644 index 000000000..04cd496da --- /dev/null +++ b/docs/ATOMIC_GATED_WORKFLOW.md @@ -0,0 +1,320 @@ +# Atomic Gated Workflow Architecture + +## Overview + +The atomic gated workflow breaks down each gate into individual, granular validation steps. This provides superior visibility into the CI/CD pipeline and creates a comprehensive audit trail. + +## Atomic Steps by Gate + +### Gate 1: Code Quality (7 Atomic Steps) + +Each validation runs as a separate job with its own status: + +1. **1.1 Prisma Validation** - Database schema validation +2. **1.2 TypeScript Check** - Type checking + strict mode analysis +3. **1.3 ESLint** - Linting + any-type detection + ts-ignore detection +4. **1.4 Security Scan** - Security pattern detection + dependency audit +5. **1.5 File Size Check** - File size limits enforcement +6. **1.6 Code Complexity** - Complexity analysis +7. **1.7 Stub Detection** - Incomplete implementation detection + +### Gate 2: Testing (3 Atomic Steps) + +1. **2.1 Unit Tests** - Unit test execution + coverage analysis +2. **2.2 E2E Tests** - Playwright end-to-end tests +3. **2.3 DBAL Daemon Tests** - Database abstraction layer tests + +### Gate 3: Build & Package (2 Atomic Steps) + +1. **3.1 Application Build** - Production build + bundle analysis +2. **3.2 Quality Metrics** - console.log detection + TODO analysis + quality summary + +## Atomic Validation Tools Used + +The workflow leverages existing atomic tools from the `/tools` directory: + +### Quality Tools +- `tools/quality/code/check-typescript-strict.ts` - TypeScript strict mode checker +- `tools/quality/code/check-code-complexity.ts` - Code complexity analyzer +- `tools/quality/code/check-function-coverage.js` - Function test coverage +- `tools/quality/files/check-file-sizes.ts` - File size validator + +### Security Tools +- `tools/security/security-scanner.ts` - Security pattern detector +- `tools/misc/metrics/parse-npm-audit.ts` - Audit result parser + +### Detection Tools +- `tools/detection/detect-stub-implementations.ts` - Stub implementation detector +- `tools/detection/detect-circular-dependencies.ts` - Circular dependency detector +- `tools/detection/detect-code-duplication.ts` - Code duplication detector + +### Lint Tools +- `tools/misc/lint/find-any-types.ts` - Any type usage detector +- `tools/misc/lint/find-ts-ignores.ts` - ts-ignore comment detector + +### Analysis Tools +- `tools/analysis/bundle/analyze-bundle-size.ts` - Bundle size analyzer +- `tools/analysis/bundle/analyze-dependency-tree.ts` - Dependency tree analyzer +- `tools/analysis/code/analyze-function-metrics.ts` - Function metrics analyzer + +### Generation Tools +- `tools/generation/generate-quality-summary.ts` - Quality summary generator +- `tools/generation/generate-test-coverage-report.js` - Coverage report generator + +## Gate Artifacts & Persistent Storage + +Each atomic step generates artifacts that are persisted using GitHub Actions artifacts: + +### Artifact Structure + +``` +gate-artifacts/ +├── gate-1/ # Gate 1 artifacts +│ ├── status.txt # Gate status +│ ├── start-time.txt # Gate start timestamp +│ ├── end-time.txt # Gate end timestamp +│ ├── prisma-check.txt # Step 1.1 result +│ ├── prisma-check-time.txt # Step 1.1 timestamp +│ ├── typecheck.txt # Step 1.2 result +│ ├── typescript-strict.json # TypeScript strict analysis +│ ├── lint.txt # Step 1.3 result +│ ├── any-types.json # Any type usage +│ ├── ts-ignores.json # ts-ignore comments +│ ├── security-scan.txt # Step 1.4 result +│ ├── security-scan.json # Security issues +│ ├── audit-results.json # npm audit results +│ ├── audit-summary.json # Parsed audit summary +│ ├── file-size-check.txt # Step 1.5 result +│ ├── file-sizes.json # File size analysis +│ ├── complexity-check.txt # Step 1.6 result +│ ├── complexity.json # Complexity analysis +│ ├── stub-detection.txt # Step 1.7 result +│ └── stubs.json # Stub implementations +├── gate-2/ # Gate 2 artifacts +│ ├── status.txt +│ ├── start-time.txt +│ ├── end-time.txt +│ ├── test-unit.txt # Step 2.1 result +│ ├── test-unit-time.txt +│ ├── coverage-report.json # Test coverage +│ ├── function-coverage.json # Function coverage +│ ├── test-e2e.txt # Step 2.2 result +│ ├── test-e2e-time.txt +│ ├── test-dbal-daemon.txt # Step 2.3 result +│ └── test-dbal-daemon-time.txt +├── gate-3/ # Gate 3 artifacts +│ ├── status.txt +│ ├── start-time.txt +│ ├── end-time.txt +│ ├── build.txt # Step 3.1 result +│ ├── build-time.txt +│ ├── bundle-size.json # Bundle analysis +│ ├── quality-check.txt # Step 3.2 result +│ ├── quality-check-time.txt +│ └── quality-summary.json # Quality metrics +└── complete-gate-audit-trail/ # Consolidated report + └── [all artifacts merged] +``` + +### Artifact Retention + +- **Individual step results**: 7 days +- **Complete audit trail**: 30 days +- **Test reports (coverage, playwright)**: 7 days +- **Build artifacts**: 7 days + +## Benefits of Atomic Steps + +### 1. Superior Visualization + +Each validation step appears as a separate job in the GitHub Actions UI, providing: +- Clear visual progress through gates +- Individual step status (✓ or ✗) +- Precise failure identification +- Step-by-step execution time + +### 2. Granular Failure Detection + +When a gate fails, you immediately see: +- Which specific validation failed +- Exact time of failure +- Detailed logs for that step only +- No need to dig through monolithic logs + +### 3. Parallel Execution + +Within each gate, independent steps run in parallel: +- Gate 1: All 7 quality checks run simultaneously +- Gate 2: All 3 test suites run simultaneously +- Gate 3: Build and quality check run simultaneously + +### 4. Audit Trail + +Complete artifact chain provides: +- Forensic analysis of what was checked +- Historical trend analysis +- Compliance documentation +- Debugging historical issues + +### 5. Progressive Enhancement + +Easy to add new atomic steps: +- Add new validation tool to `/tools` +- Add new step to workflow +- Artifact collection happens automatically +- No disruption to existing steps + +## Usage + +### Running Locally with Act + +Test individual atomic steps: + +```bash +# Test specific atomic step +npm run act -- -j prisma-check -W ../.github/workflows/gated-ci-atomic.yml +npm run act -- -j security-scan -W ../.github/workflows/gated-ci-atomic.yml +npm run act -- -j stub-detection -W ../.github/workflows/gated-ci-atomic.yml + +# Test complete gate +npm run act -- -j gate-1-complete -W ../.github/workflows/gated-ci-atomic.yml +npm run act -- -j gate-2-complete -W ../.github/workflows/gated-ci-atomic.yml + +# Test full workflow +npm run act pull_request -W ../.github/workflows/gated-ci-atomic.yml +``` + +### Viewing Gate Artifacts + +After workflow completion: + +1. Navigate to workflow run in GitHub Actions +2. Scroll to "Artifacts" section at bottom +3. Download artifact bundles: + - `gate-1-complete-report` - All Gate 1 validation results + - `gate-2-complete-report` - All Gate 2 test results + - `gate-3-complete-report` - All Gate 3 build results + - `complete-gate-audit-trail` - Complete audit trail (30 days) + +### Analyzing Results + +Each JSON artifact can be analyzed: + +```bash +# Security scan results +cat gate-1/security-scan.json | jq '.issues[] | select(.severity == "critical")' + +# Stub implementations +cat gate-1/stubs.json | jq '.stubs[] | select(.severity == "high")' + +# Test coverage +cat gate-2/coverage-report.json | jq '.summary' + +# Bundle size +cat gate-3/bundle-size.json | jq '.totalSize' + +# Quality summary +cat gate-3/quality-summary.json | jq '.score' +``` + +## Comparison: Monolithic vs Atomic + +### Monolithic Approach (Original) + +``` +Gate 1: Code Quality [RUNNING] + ├─ All validation in one job + ├─ Failure = need to read full log + └─ No intermediate artifacts +``` + +### Atomic Approach (New) + +``` +Gate 1: Code Quality + ├─ 1.1 Prisma Validation ✓ [artifact] + ├─ 1.2 TypeScript Check ✓ [artifact] + ├─ 1.3 ESLint ✗ [artifact + detailed error] + ├─ 1.4 Security Scan ✓ [artifact] + ├─ 1.5 File Size Check ✓ [artifact] + ├─ 1.6 Code Complexity ✓ [artifact] + └─ 1.7 Stub Detection ✓ [artifact] +``` + +**Immediate benefit**: You know ESLint failed, not something else. + +## Workflow Selection + +The repository now has two gated workflows: + +### `gated-ci.yml` - Consolidated Gates +- Fewer jobs (simpler for small teams) +- Faster execution (less orchestration overhead) +- Good for: Small teams, simple projects + +### `gated-ci-atomic.yml` - Atomic Gates +- More jobs (better visibility) +- Detailed audit trail +- Individual step artifacts +- Good for: Large teams, compliance requirements, complex projects + +**Recommendation**: Use `gated-ci-atomic.yml` for better visualization and audit trail. + +## Migration from Original Gated Workflow + +To switch to atomic workflow: + +1. **Update branch protection rules**: Change required checks to atomic step names +2. **Update auto-merge workflow**: Add atomic step names to check list +3. **Test with sample PR**: Verify all atomic steps run correctly +4. **Monitor first few PRs**: Ensure parallel execution works as expected + +## Future Enhancements + +Potential additions to atomic workflow: + +1. **Artifact Dashboard**: Web UI for browsing gate artifacts +2. **Trend Analysis**: Historical charts of validation metrics +3. **Smart Retry**: Automatic retry of flaky atomic steps +4. **Conditional Steps**: Skip irrelevant validations based on changed files +5. **Custom Tools**: Add project-specific atomic validators +6. **Performance Budgets**: Enforce performance metrics per atomic step +7. **Notification Hooks**: Slack/email alerts for specific atomic step failures + +## Troubleshooting + +### Artifact Not Found + +If gate artifacts are missing: +- Check artifact retention period (7-30 days) +- Verify step completed (check job logs) +- Ensure upload step ran (check for upload errors) + +### Step Takes Too Long + +If an atomic step is slow: +- Check if parallel execution is enabled +- Review tool implementation for efficiency +- Consider caching dependencies +- Split into smaller atomic steps if possible + +### Validation Tool Fails + +If atomic validation tool crashes: +- Steps have `continue-on-error: true` for resilience +- Check tool logs in step output +- Verify tool has required dependencies +- Test tool locally: `tsx tools/path/to/tool.ts` + +## Related Documentation + +- [Enterprise Gated Workflow](ENTERPRISE_GATED_WORKFLOW.md) - Original gated workflow +- [Testing Gated Workflows](guides/TESTING_GATED_WORKFLOWS.md) - Local testing guide +- [Legacy Pipeline Cruft Report](LEGACY_PIPELINE_CRUFT_REPORT.md) - Cleanup analysis +- [Tools README](../tools/README.md) - Available atomic validation tools + +--- + +**Version:** 1.0.0 +**Last Updated:** December 27, 2025 +**Status:** Active - Recommended for Enterprise Use