git/metabuilder
1
0
Fork 0
mirror of https://github.com/johndoe6345789/metabuilder.git synced 2026-04-24 13:54:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #1443 from johndoe6345789/copilot/refactor-pipeline-structure

Browse Source 
Consolidate 3 gated workflows into single unified pipeline
This commit is contained in:
johndoe6345789
2026-01-09 01:11:43 +00:00
committed by GitHub
parent f86e637da1 149c0f8715
commit 72e367c209
6 changed files with 1804 additions and 2116 deletions
Download Patch File Download Diff File Expand all files Collapse all files

157
.github/workflows/README.md vendored
View File

@@ -4,35 +4,41 @@ This directory contains automated workflows for CI/CD, code quality, and compreh
## 🚦 Enterprise Gated Tree Workflow
MetaBuilder uses an **Enterprise Gated Tree Workflow** that ensures all code changes pass through multiple validation gates before being merged and deployed.
MetaBuilder uses a **Unified Enterprise Gated Pipeline** that consolidates all CI/CD, deployment, and development assistance into a single workflow with clear gate visualization.
**📖 Complete Guide:** [Enterprise Gated Workflow Documentation](../../docs/ENTERPRISE_GATED_WORKFLOW.md)
### Quick Overview
All PRs must pass through 5 sequential gates:
All PRs and deployments flow through 6 sequential gates in a single workflow:
1. **Gate 1: Code Quality** - Prisma, TypeScript, Lint, Security
2. **Gate 2: Testing** - Unit, E2E, DBAL Daemon tests
3. **Gate 3: Build & Package** - Application build, quality metrics
4. **Gate 4: Review & Approval** - Human code review (1 approval required)
5. **Gate 5: Deployment** - Staging (auto) → Production (manual approval)
1. **Gate 1: Code Quality** - Prisma, TypeScript, Lint, Security (7 atomic steps)
2. **Gate 2: Testing** - Unit, E2E, DBAL Daemon tests (3 atomic steps)
3. **Gate 3: Build & Package** - Application build, quality metrics (2 atomic steps)
4. **Gate 4: Development Assistance** - Architectural feedback, Copilot interaction (PR only)
5. **Gate 5: Staging Deployment** - Automatic deployment to staging (main branch push)
6. **Gate 6: Production Deployment** - Manual approval required (release/workflow_dispatch)
**Key Benefits:**
- ✅ **Single unified workflow** - No confusion about which pipeline runs what
- ✅ Sequential gates prevent wasted resources
- ✅ Tree structure for clear visualization of all validation steps
- ✅ Automatic merge after approval
- ✅ Manual approval required for production
- ✅ Clear visibility of gate status on PRs
- ✅ Audit trail for all deployments
- ✅ Conditional execution based on trigger (PR vs push vs release)
- ✅ Complete audit trail for all deployments
### Legacy Workflow Cleanup
### Pipeline Consolidation (Jan 2026)
**Deprecated and Removed (Dec 2025):**
- ❌ `ci/ci.yml` - Replaced by `gated-ci.yml` (100% redundant)
- ❌ `quality/deployment.yml` - Replaced by `gated-deployment.yml` (100% redundant)
**Consolidated into `gated-pipeline.yml`:**
- ✅ `gated-ci.yml` (1048 lines) - CI with gates 1-5
- ✅ `gated-deployment.yml` (617 lines) - Deployment workflows
- ✅ `development.yml` (360 lines) - Development assistance
**Modified:**
- ⚡ `development.yml` - Refactored to remove redundant quality checks, kept unique Copilot features
**Result:** Single 1287-line workflow with all functionality preserved and no duplication.
**Previous Deprecated and Removed (Dec 2025):**
- ❌ `ci/ci.yml` - Replaced by gated workflows
- ❌ `quality/deployment.yml` - Replaced by gated workflows
See [Legacy Pipeline Cruft Report](../../docs/LEGACY_PIPELINE_CRUFT_REPORT.md) for analysis.
@@ -50,23 +56,12 @@ All workflows are designed to work seamlessly with **GitHub Copilot** to assist
## Workflows Overview
### 🚦 Enterprise Gated Workflows (New)
### 🚦 Enterprise Gated Workflow (Unified)
#### Issue and PR Triage (`triage.yml`) 🆕
**Triggered on:** Issues (opened/edited/reopened) and Pull Requests (opened/reopened/synchronize/edited)
#### Enterprise Gated Pipeline (`gated-pipeline.yml`) 🆕
**Triggered on:** Push to main/master/develop, Pull requests, Releases, Manual dispatch, Issue comments
**Purpose:** Quickly categorize inbound work so reviewers know what to look at first.
- Auto-applies labels for type (bug/enhancement/docs/security/testing/performance) and area (frontend/backend/database/workflows/documentation)
- Sets a default priority and highlights beginner-friendly issues
- Flags missing information (repro steps, expected/actual results, versions) with a checklist comment
- For PRs, labels areas touched, estimates risk based on change size and critical paths, and prompts for test plans/screenshots/linked issues
- Mentions **@copilot** to sanity-check the triage with GitHub-native AI (no external Codex webhooks)
This workflow runs alongside the existing PR management jobs to keep triage lightweight while preserving the richer checks in the gated pipelines.
#### 1. Enterprise Gated CI/CD Pipeline (`gated-ci.yml`)
**Triggered on:** Push to main/master/develop branches, Pull requests
**Consolidates:** All CI/CD, deployment, and development assistance in one workflow
**Structure:**
- **Gate 1:** Code Quality - 7 validation steps
@@ -83,13 +78,24 @@ This workflow runs alongside the existing PR management jobs to keep triage ligh
- 2.3 DBAL Daemon Tests
- **Gate 3:** Build & Package - 2 validation steps
- 3.1 Application Build (+ bundle analysis)
- 3.2 Quality Metrics
- **Gate 4:** Review & Approval (Human review required)
- **Gate 5:** Deployment (post-merge, automatic staging)
- 3.2 Quality Metrics (PR only)
- **Gate 4:** Development Assistance (PR only)
- 4.1 Code metrics analysis
- 4.2 Architectural compliance
- 4.3 Refactoring suggestions
- 4.4 Copilot interaction handler
- **Gate 5:** Staging Deployment (main branch push)
- Automatic deployment to staging environment
- Smoke tests and health checks
- **Gate 6:** Production Deployment (release/manual)
- Manual approval gate
- Production deployment with health monitoring
- Deployment tracking issue creation
**Features:**
- Individual validation steps for superior visualization
- **Gate artifacts** persisted between steps (30-day retention)
- Conditional execution based on trigger type
- Granular failure detection
- Parallel execution within gates
- Complete audit trail with JSON artifacts
@@ -98,42 +104,32 @@ This workflow runs alongside the existing PR management jobs to keep triage ligh
- Clear gate status reporting on PRs
- Summary report with all gate results
#### 2. Enterprise Gated Deployment (`gated-deployment.yml`)
**Triggered on:** Push to main/master, Releases, Manual workflow dispatch
### 🔄 Supporting Workflows
**Environments:**
- **Staging:** Automatic deployment after merge to main
- **Production:** Manual approval required
#### Issue and PR Triage (`triage.yml`)
**Triggered on:** Issues (opened/edited/reopened) and Pull Requests (opened/reopened/synchronize/edited)
**Features:**
- Pre-deployment validation (schema, security, size)
- Breaking change detection and warnings
- Environment-specific deployment paths
- Post-deployment health checks
- Automatic deployment tracking issues
- Rollback preparation and procedures
**Purpose:** Quickly categorize inbound work so reviewers know what to look at first.
**Gate 5:** Deployment gate ensures only reviewed code reaches production
- Auto-applies labels for type (bug/enhancement/docs/security/testing/performance) and area (frontend/backend/database/workflows/documentation)
- Sets a default priority and highlights beginner-friendly issues
- Flags missing information (repro steps, expected/actual results, versions) with a checklist comment
- For PRs, labels areas touched, estimates risk based on change size and critical paths, and prompts for test plans/screenshots/linked issues
- Mentions **@copilot** to sanity-check the triage with GitHub-native AI
### 🔄 Legacy Workflows (Still Active)
This workflow runs alongside the gated pipeline to provide quick triage feedback.
#### 3. CI/CD Workflow (`ci/ci.yml`) - ❌ REMOVED
### 🗑️ Legacy Workflows (Removed)
#### CI/CD Workflow (`ci/ci.yml`) - ❌ REMOVED
**Status:** Deprecated and removed (Dec 2025)
**Reason:** 100% functionality superseded by `gated-ci.yml`
**Reason:** 100% functionality superseded by gated pipeline
**Jobs:** ~~Prisma Check, Lint, Build, E2E Tests, Quality Check~~
**Replacement:** Use `gated-ci.yml` for all CI/CD operations
**Triggered on:** Push to main/master/develop branches, Pull requests
**Replacement:** Consolidated into `gated-pipeline.yml`
**Jobs:**
- **Prisma Check**: Validates database schema and generates Prisma client
- **Lint**: Runs ESLint to check code quality
- **Build**: Builds the application and uploads artifacts
- **E2E Tests**: Runs Playwright end-to-end tests
- **Quality Check**: Checks for console.log statements and TODO comments
### 4. Automated Code Review (`code-review.yml`)
### 3. Automated Code Review (`pr/code-review.yml`)
**Triggered on:** Pull request opened, synchronized, or reopened
**Features:**
@@ -150,21 +146,21 @@ This workflow runs alongside the existing PR management jobs to keep triage ligh
- ✅ React best practices
- ✅ File size warnings
### 5. Auto Merge (`auto-merge.yml`) - Updated for Gated Workflow
### 4. Auto Merge (`pr/auto-merge.yml`) - Updated for Gated Pipeline
**Triggered on:** PR approval, CI workflow completion
**Features:**
- Automatically merges PRs when:
- PR is approved by reviewers
- All gates pass (supports both gated and legacy CI checks)
- All gates pass in unified gated pipeline
- No merge conflicts
- PR is not in draft
- **Automatically deletes the branch** after successful merge
- Uses squash merge strategy
- Posts comments about merge status
- **Updated:** Now supports Enterprise Gated CI/CD Pipeline checks
- **Updated:** Now supports unified Enterprise Gated Pipeline checks
### 6. Issue Triage (`issue-triage.yml`)
### 5. Issue Triage (`issue-triage.yml`)
**Triggered on:** New issues opened, issues labeled
**Features:**
@@ -176,7 +172,7 @@ This workflow runs alongside the existing PR management jobs to keep triage ligh
- Suggests automated fix attempts for simple issues
- Can create fix branches automatically with `create-pr` label
### 7. PR Management (`pr-management.yml`)
### 6. PR Management (`pr/pr-management.yml`)
**Triggered on:** PR opened, synchronized, labeled
**Features:**
@@ -188,7 +184,7 @@ This workflow runs alongside the existing PR management jobs to keep triage ligh
- Links related issues automatically
- Posts comments on related issues
### 8. Merge Conflict Check (`merge-conflict-check.yml`)
### 7. Merge Conflict Check (`pr/merge-conflict-check.yml`)
**Triggered on:** PR opened/synchronized, push to main/master
**Features:**
@@ -197,7 +193,7 @@ This workflow runs alongside the existing PR management jobs to keep triage ligh
- Adds/removes `merge-conflict` label
- Fails CI if conflicts exist
### 9. Planning & Design (`planning.yml`) 🆕
### 8. Planning & Design (`quality/planning.yml`) 🆕
**Triggered on:** Issues opened or labeled with enhancement/feature-request
**Features:**
@@ -211,28 +207,7 @@ This workflow runs alongside the existing PR management jobs to keep triage ligh
**SDLC Phase:** Planning & Design
### 10. Development Assistance (`development.yml`) 🆕 - Refactored
**Triggered on:** Pull request updates, @copilot mentions
**Features:**
- **Architectural Compliance Feedback**: Monitors declarative ratio and component sizes
- **@copilot Interaction Handler**: Responds to @copilot mentions with context-aware guidance
- **Refactoring Suggestions**: Identifies opportunities for improvement
- Provides architectural reminders and best practices
**Note:** Refactored to remove redundant quality checks (lint/build now in gated-ci.yml)
**SDLC Phase:** Development
### 11. Deployment & Monitoring (`deployment.yml`) - ❌ REMOVED
**Status:** Deprecated and removed (Dec 2025)
**Reason:** 100% functionality superseded by `gated-deployment.yml` with improvements
**Jobs:** ~~Pre-Deployment Validation, Deployment Summary, Post-Deployment Health Checks~~
**Replacement:** Use `gated-deployment.yml` for all deployment operations
### 12. Code Size Limits (`size-limits.yml`)
### 9. Code Size Limits (`quality/size-limits.yml`)
**Triggered on:** Pull requests, pushes to main (when source files change)
**Features:**
@@ -246,11 +221,11 @@ This workflow runs alongside the existing PR management jobs to keep triage ligh
```
┌─────────────┐
│ Planning │ ← planning.yml (Architecture Review, PRD Check)
│ Planning │ ← quality/planning.yml (Architecture Review, PRD Check)
└──────┬──────┘
↓
┌─────────────┐
│ Development │ ← development.yml (Quality Feedback, Refactoring)
│ Development │ ← gated-pipeline.yml Gate 4 (Dev Feedback, Copilot)
└──────┬──────┘
↓
┌─────────────┐

360
.github/workflows/development.yml vendored
View File

@@ -1,360 +0,0 @@
name: Development Assistance
on:
pull_request:
types: [opened, synchronize, ready_for_review]
issue_comment:
types: [created]
permissions:
contents: read
issues: write
pull-requests: write
jobs:
code-quality-feedback:
name: Continuous Quality Feedback
runs-on: ubuntu-latest
if: |
github.event_name == 'pull_request' && !github.event.pull_request.draft
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Analyze code metrics (no redundant checks)
id: quality
run: |
# Note: Lint/build/tests are handled by gated-ci.yml
# This job only collects metrics for architectural feedback
# Count TypeScript files and their sizes
TOTAL_TS_FILES=$(find src -name "*.ts" -o -name "*.tsx" 2>/dev/null | wc -l)
LARGE_FILES=$(find src -name "*.ts" -o -name "*.tsx" -exec wc -l {} \; 2>/dev/null | awk '$1 > 150 {print $2}' | wc -l)
echo "total_ts_files=$TOTAL_TS_FILES" >> $GITHUB_OUTPUT
echo "large_files=$LARGE_FILES" >> $GITHUB_OUTPUT
# Check for declarative vs imperative balance
JSON_FILES=$(find src packages -name "*.json" 2>/dev/null | wc -l)
LUA_SCRIPTS=$(find src packages -name "*.lua" 2>/dev/null | wc -l)
echo "json_files=$JSON_FILES" >> $GITHUB_OUTPUT
echo "lua_scripts=$LUA_SCRIPTS" >> $GITHUB_OUTPUT
- name: Check architectural compliance
id: architecture
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const path = require('path');
let issues = [];
let suggestions = [];
// Get changed files
let changedFiles = [];
if (context.eventName === 'pull_request') {
const { data: files } = await github.rest.pulls.listFiles({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number,
});
changedFiles = files.map(f => f.filename);
}
// Check for hardcoded components outside ui/
const hardcodedComponents = changedFiles.filter(f =>
f.endsWith('.tsx') &&
f.includes('src/components/') &&
!f.includes('src/components/ui/') &&
!f.includes('src/components/shared/') &&
!['RenderComponent', 'FieldRenderer', 'GenericPage'].some(g => f.includes(g))
);
if (hardcodedComponents.length > 0) {
suggestions.push(`Consider if these components could be declarative: ${hardcodedComponents.join(', ')}`);
}
// Check for database changes without seed data
const schemaChanged = changedFiles.some(f => f.includes('schema.prisma'));
const seedChanged = changedFiles.some(f => f.includes('seed'));
if (schemaChanged && !seedChanged) {
suggestions.push('Database schema changed but no seed data updates detected. Consider updating seed data.');
}
// Check for new routes without PageRoutes table updates
const routeFiles = changedFiles.filter(f => f.includes('Route') || f.includes('route'));
if (routeFiles.length > 0) {
suggestions.push('Route changes detected. Ensure PageRoutes table is updated for dynamic routing.');
}
// Check for large TypeScript files
const largeFiles = parseInt('${{ steps.quality.outputs.large_files }}');
if (largeFiles > 0) {
issues.push(`${largeFiles} TypeScript files exceed 150 lines. Consider breaking them into smaller components.`);
}
return { issues, suggestions };
- name: Provide development feedback
if: github.event_name == 'pull_request'
uses: actions/github-script@v7
with:
script: |
const analysis = JSON.parse('${{ steps.architecture.outputs.result }}');
const totalFiles = parseInt('${{ steps.quality.outputs.total_ts_files }}');
const largeFiles = parseInt('${{ steps.quality.outputs.large_files }}');
const jsonFiles = parseInt('${{ steps.quality.outputs.json_files }}');
const luaScripts = parseInt('${{ steps.quality.outputs.lua_scripts }}');
let comment = `## 💻 Development Quality Feedback\n\n`;
comment += `### 📊 Code Metrics\n\n`;
comment += `- TypeScript files: ${totalFiles}\n`;
comment += `- Files >150 LOC: ${largeFiles} ${largeFiles > 0 ? '⚠️' : '✅'}\n`;
comment += `- JSON config files: ${jsonFiles}\n`;
comment += `- Lua scripts: ${luaScripts}\n`;
comment += `- Declarative ratio: ${((jsonFiles + luaScripts) / Math.max(totalFiles, 1) * 100).toFixed(1)}%\n\n`;
if (analysis.issues.length > 0) {
comment += `### ⚠️ Architectural Issues\n\n`;
analysis.issues.forEach(issue => comment += `- ${issue}\n`);
comment += '\n';
}
if (analysis.suggestions.length > 0) {
comment += `### 💡 Suggestions\n\n`;
analysis.suggestions.forEach(suggestion => comment += `- ${suggestion}\n`);
comment += '\n';
}
comment += `### 🎯 Project Goals Reminder\n\n`;
comment += `- **Declarative First:** Prefer JSON + Lua over TypeScript\n`;
comment += `- **Component Size:** Keep files under 150 LOC\n`;
comment += `- **Generic Renderers:** Use RenderComponent for dynamic components\n`;
comment += `- **Database-Driven:** Store configuration in database, not code\n`;
comment += `- **Package-Based:** Organize features as importable packages\n\n`;
comment += `**@copilot** can help refactor code to better align with these principles.\n\n`;
comment += `📖 See [Architecture Guidelines](/.github/copilot-instructions.md)`;
// Check if we already commented
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const botComment = comments.find(c =>
c.user.type === 'Bot' && c.body.includes('Development Quality Feedback')
);
if (botComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: comment
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: comment
});
}
copilot-interaction:
name: Handle Copilot Mentions
runs-on: ubuntu-latest
if: |
github.event_name == 'issue_comment' &&
contains(github.event.comment.body, '@copilot')
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Parse Copilot request
uses: actions/github-script@v7
with:
script: |
const comment = context.payload.comment.body.toLowerCase();
const issue = context.payload.issue;
let response = `## 🤖 Copilot Assistance\n\n`;
// Determine what the user is asking for
if (comment.includes('implement') || comment.includes('fix this')) {
response += `To implement this with Copilot assistance:\n\n`;
response += `1. **Create a branch:** \`git checkout -b feature/issue-${issue.number}\`\n`;
response += `2. **Use Copilot in your IDE** to generate code with context from:\n`;
response += ` - [Copilot Instructions](/.github/copilot-instructions.md)\n`;
response += ` - [PRD.md](/PRD.md)\n`;
response += ` - Existing package structure in \`/packages/\`\n`;
response += `3. **Follow the architectural principles:**\n`;
response += ` - Declarative over imperative\n`;
response += ` - Database-driven configuration\n`;
response += ` - Generic renderers vs hardcoded components\n`;
response += `4. **Test your changes:** \`npm run lint && npm run test:e2e\`\n`;
response += `5. **Create a PR** - The automated workflows will review it\n\n`;
}
if (comment.includes('review') || comment.includes('check')) {
response += `Copilot can review this through:\n\n`;
response += `- **Automated Code Review** workflow (runs on PRs)\n`;
response += `- **Development Assistance** workflow (runs on pushes)\n`;
response += `- **Planning & Design** workflow (runs on feature requests)\n\n`;
response += `Create a PR to trigger comprehensive review!\n\n`;
}
if (comment.includes('architecture') || comment.includes('design')) {
response += `### 🏗️ Architectural Guidance\n\n`;
response += `MetaBuilder follows these principles:\n\n`;
response += `1. **5-Level Architecture:** User → Admin → God → SuperGod levels\n`;
response += `2. **Multi-Tenant:** Isolated tenant instances with independent configs\n`;
response += `3. **Declarative Components:** JSON config + Lua scripts, not TSX\n`;
response += `4. **Package System:** Self-contained, importable feature bundles\n`;
response += `5. **Database-First:** All config in Prisma, not hardcoded\n\n`;
response += `📖 Full details: [PRD.md](/PRD.md)\n\n`;
}
if (comment.includes('test') || comment.includes('e2e')) {
response += `### 🧪 Testing with Copilot\n\n`;
response += `\`\`\`bash\n`;
response += `# Run E2E tests\n`;
response += `npm run test:e2e\n\n`;
response += `# Run with UI\n`;
response += `npm run test:e2e:ui\n\n`;
response += `# Run linter\n`;
response += `npm run lint\n`;
response += `\`\`\`\n\n`;
response += `Use Copilot in your IDE to:\n`;
response += `- Generate test cases based on user stories\n`;
response += `- Write Playwright selectors and assertions\n`;
response += `- Create mock data for tests\n\n`;
}
if (comment.includes('help') || (!comment.includes('implement') && !comment.includes('review') && !comment.includes('architecture') && !comment.includes('test'))) {
response += `### 🆘 How to Use Copilot\n\n`;
response += `Mention **@copilot** in comments with:\n\n`;
response += `- \`@copilot implement this\` - Get implementation guidance\n`;
response += `- \`@copilot review this\` - Request code review\n`;
response += `- \`@copilot architecture\` - Get architectural guidance\n`;
response += `- \`@copilot test this\` - Get testing guidance\n`;
response += `- \`@copilot fix this issue\` - Request automated fix\n\n`;
response += `**In your IDE:**\n`;
response += `- Use GitHub Copilot with context from [Copilot Instructions](/.github/copilot-instructions.md)\n`;
response += `- Reference the [PRD](/PRD.md) when prompting\n`;
response += `- Follow patterns from existing packages in \`/packages/\`\n\n`;
}
response += `---\n`;
response += `*This is an automated response. For detailed Copilot assistance, use the extension in your IDE with project context.*`;
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
body: response
});
suggest-refactoring:
name: Suggest Refactoring Opportunities
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' && !github.event.pull_request.draft
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Analyze refactoring opportunities
uses: actions/github-script@v7
with:
script: |
const { data: files } = await github.rest.pulls.listFiles({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number,
});
let opportunities = [];
// Look for opportunities in changed files
for (const file of files) {
const patch = file.patch || '';
// Check for repeated code patterns
if (patch.split('\n').length > 100) {
opportunities.push({
file: file.filename,
type: 'Size',
suggestion: 'Large changeset - consider breaking into smaller PRs or extracting common utilities'
});
}
// Check for hardcoded values
if (patch.match(/['"][A-Z_]{3,}['"]\s*:/)) {
opportunities.push({
file: file.filename,
type: 'Configuration',
suggestion: 'Hardcoded constants detected - consider moving to database configuration'
});
}
// Check for new TSX components
if (file.filename.includes('components/') && file.filename.endsWith('.tsx') && file.status === 'added') {
opportunities.push({
file: file.filename,
type: 'Architecture',
suggestion: 'New component added - could this be implemented declaratively with JSON + Lua?'
});
}
// Check for inline styles or complex class strings
if (patch.includes('style={{') || patch.match(/className="[^"]{50,}"/)) {
opportunities.push({
file: file.filename,
type: 'Styling',
suggestion: 'Complex styling detected - consider extracting to theme configuration'
});
}
}
if (opportunities.length > 0) {
let comment = `## 🔄 Refactoring Opportunities\n\n`;
comment += `**@copilot** identified potential improvements:\n\n`;
const grouped = {};
opportunities.forEach(opp => {
if (!grouped[opp.type]) grouped[opp.type] = [];
grouped[opp.type].push(opp);
});
for (const [type, opps] of Object.entries(grouped)) {
comment += `### ${type}\n\n`;
opps.forEach(opp => {
comment += `- **${opp.file}**: ${opp.suggestion}\n`;
});
comment += '\n';
}
comment += `---\n`;
comment += `These are suggestions, not requirements. Consider them as part of continuous improvement.\n\n`;
comment += `Use **@copilot** in your IDE to help implement these refactorings.`;
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: comment
});
}

617
.github/workflows/gated-deployment.yml vendored
View File

@@ -1,617 +0,0 @@
name: Enterprise Gated Deployment
on:
push:
branches:
- main
- master
release:
types: [published]
workflow_dispatch:
inputs:
environment:
description: 'Target deployment environment'
required: true
type: choice
options:
- staging
- production
skip_tests:
description: 'Skip pre-deployment tests (emergency only)'
required: false
type: boolean
default: false
permissions:
contents: read
issues: write
pull-requests: write
deployments: write
# Enterprise Deployment with Environment Gates
# Staging: Automatic deployment after main branch push
# Production: Requires manual approval
jobs:
# ============================================================================
# Pre-Deployment Validation
# ============================================================================
pre-deployment-validation:
name: Pre-Deployment Checks
runs-on: ubuntu-latest
defaults:
run:
working-directory: frontends/nextjs
outputs:
has-breaking-changes: ${{ steps.breaking.outputs.has_breaking }}
deployment-environment: ${{ steps.determine-env.outputs.environment }}
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Determine target environment
id: determine-env
run: |
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
echo "environment=${{ inputs.environment }}" >> $GITHUB_OUTPUT
elif [ "${{ github.event_name }}" == "release" ]; then
echo "environment=production" >> $GITHUB_OUTPUT
else
echo "environment=staging" >> $GITHUB_OUTPUT
fi
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 20
- name: Install dependencies
run: npm install
- name: Generate Prisma Client
run: npm run db:generate
env:
DATABASE_URL: file:./dev.db
- name: Validate database schema
run: npx prisma validate --schema=../../prisma/schema.prisma
env:
DATABASE_URL: file:./dev.db
- name: Check for breaking changes
id: breaking
uses: actions/github-script@v7
with:
script: |
const commits = await github.rest.repos.listCommits({
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 10
});
let hasBreaking = false;
let breakingChanges = [];
for (const commit of commits.data) {
const message = commit.commit.message.toLowerCase();
if (message.includes('breaking') || message.includes('breaking:') || message.startsWith('!')) {
hasBreaking = true;
breakingChanges.push({
sha: commit.sha.substring(0, 7),
message: commit.commit.message.split('\n')[0]
});
}
}
core.setOutput('has_breaking', hasBreaking);
if (hasBreaking) {
console.log('⚠️ Breaking changes detected:');
breakingChanges.forEach(c => console.log(` - ${c.sha}: ${c.message}`));
core.warning('Breaking changes detected in recent commits');
}
- name: Security audit
run: npm audit --audit-level=moderate
continue-on-error: true
- name: Check package size
run: |
npm run build
SIZE=$(du -sm .next/ | cut -f1)
echo "Build size: ${SIZE}MB"
if [ $SIZE -gt 50 ]; then
echo "::warning::Build size is ${SIZE}MB (>50MB). Consider optimizing."
fi
# ============================================================================
# Staging Deployment (Automatic)
# ============================================================================
deploy-staging:
name: Deploy to Staging
runs-on: ubuntu-latest
needs: pre-deployment-validation
if: |
needs.pre-deployment-validation.outputs.deployment-environment == 'staging' &&
(github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && inputs.environment == 'staging'))
environment:
name: staging
url: https://staging.metabuilder.example.com
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 20
- name: Install dependencies
run: npm install
- name: Generate Prisma Client
run: npm run db:generate
env:
DATABASE_URL: ${{ secrets.STAGING_DATABASE_URL }}
- name: Build for staging
run: npm run build
env:
DATABASE_URL: ${{ secrets.STAGING_DATABASE_URL }}
NEXT_PUBLIC_ENV: staging
- name: Deploy to staging
run: |
echo "🚀 Deploying to staging environment..."
echo "Build artifacts ready for deployment"
echo "Note: Replace this with actual deployment commands"
echo "Examples:"
echo " - docker build/push"
echo " - kubectl apply"
echo " - terraform apply"
echo " - vercel deploy"
- name: Run smoke tests
run: |
echo "🧪 Running smoke tests on staging..."
echo "Basic health checks:"
echo " ✓ Application starts"
echo " ✓ Database connection"
echo " ✓ API endpoints responding"
echo "Note: Implement actual smoke tests here"
- name: Post deployment summary
uses: actions/github-script@v7
with:
script: |
const summary = `## 🚀 Staging Deployment Successful
**Environment:** staging
**Commit:** ${context.sha.substring(0, 7)}
**Time:** ${new Date().toISOString()}
### Deployment Details
- ✅ Pre-deployment validation passed
- ✅ Build completed
- ✅ Deployed to staging
- ✅ Smoke tests passed
### Next Steps
- Monitor staging environment for issues
- Run integration tests
- Request QA validation
- If stable, promote to production with manual approval
**Staging URL:** https://staging.metabuilder.example.com
`;
console.log(summary);
# ============================================================================
# Production Deployment Gate (Manual Approval Required)
# ============================================================================
production-approval-gate:
name: Production Deployment Gate
runs-on: ubuntu-latest
needs: [pre-deployment-validation]
if: |
needs.pre-deployment-validation.outputs.deployment-environment == 'production' &&
(github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && inputs.environment == 'production'))
steps:
- name: Pre-production checklist
uses: actions/github-script@v7
with:
script: |
const hasBreaking = '${{ needs.pre-deployment-validation.outputs.has-breaking-changes }}' === 'true';
let checklist = `## 🚨 Production Deployment Gate
### Pre-Deployment Checklist
#### Automatic Checks
- ✅ All CI/CD gates passed
- ✅ Code merged to main branch
- ✅ Pre-deployment validation completed
${hasBreaking ? '- ⚠️ **Breaking changes detected** - review required' : '- ✅ No breaking changes detected'}
#### Manual Verification Required
- [ ] Staging environment validated
- [ ] QA sign-off received
- [ ] Database migrations reviewed
- [ ] Rollback plan prepared
- [ ] Monitoring alerts configured
- [ ] On-call engineer notified
${hasBreaking ? '- [ ] **Breaking changes documented and communicated**' : ''}
### Approval Process
This deployment requires manual approval from authorized personnel.
**To approve:** Use the GitHub Actions UI to approve this deployment.
**To reject:** Cancel the workflow run.
### Emergency Override
If this is an emergency hotfix, the skip_tests option was set to: ${{ inputs.skip_tests || false }}
`;
console.log(checklist);
if (hasBreaking) {
core.warning('Breaking changes detected - extra caution required for production deployment');
}
deploy-production:
name: Deploy to Production
runs-on: ubuntu-latest
needs: [pre-deployment-validation, production-approval-gate]
if: |
needs.pre-deployment-validation.outputs.deployment-environment == 'production' &&
(github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && inputs.environment == 'production'))
environment:
name: production
url: https://metabuilder.example.com
defaults:
run:
working-directory: frontends/nextjs
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 20
- name: Install dependencies
run: npm install
- name: Generate Prisma Client
run: npm run db:generate
env:
DATABASE_URL: ${{ secrets.PRODUCTION_DATABASE_URL }}
- name: Build for production
run: npm run build
env:
DATABASE_URL: ${{ secrets.PRODUCTION_DATABASE_URL }}
NEXT_PUBLIC_ENV: production
NODE_ENV: production
- name: Pre-deployment backup
run: |
echo "📦 Creating pre-deployment backup..."
echo "Note: Implement actual backup commands"
echo " - Database backup"
echo " - File system backup"
echo " - Configuration backup"
- name: Run database migrations
run: |
echo "🗄️ Running database migrations..."
echo "Note: Implement actual migration commands"
echo "npx prisma migrate deploy"
env:
DATABASE_URL: ${{ secrets.PRODUCTION_DATABASE_URL }}
- name: Deploy to production
run: |
echo "🚀 Deploying to production environment..."
echo "Build artifacts ready for deployment"
echo "Note: Replace this with actual deployment commands"
echo "Examples:"
echo " - docker build/push"
echo " - kubectl apply"
echo " - terraform apply"
echo " - vercel deploy --prod"
- name: Run smoke tests
run: |
echo "🧪 Running smoke tests on production..."
echo "Basic health checks:"
echo " ✓ Application starts"
echo " ✓ Database connection"
echo " ✓ API endpoints responding"
echo " ✓ Critical user flows working"
echo "Note: Implement actual smoke tests here"
- name: Post deployment summary
uses: actions/github-script@v7
with:
script: |
const hasBreaking = '${{ needs.pre-deployment-validation.outputs.has-breaking-changes }}' === 'true';
const summary = `## 🎉 Production Deployment Successful
**Environment:** production
**Commit:** ${context.sha.substring(0, 7)}
**Time:** ${new Date().toISOString()}
${hasBreaking ? '**⚠️ Contains Breaking Changes**' : ''}
### Deployment Details
- ✅ Manual approval received
- ✅ Pre-deployment validation passed
- ✅ Database migrations completed
- ✅ Build completed
- ✅ Deployed to production
- ✅ Smoke tests passed
### Post-Deployment Monitoring
- 🔍 Monitor error rates for 1 hour
- 📊 Check performance metrics
- 👥 Monitor user feedback
- 🚨 Keep rollback plan ready
**Production URL:** https://metabuilder.example.com
### Emergency Contacts
- On-call engineer: Check PagerDuty
- Rollback procedure: See docs/deployment/rollback.md
`;
console.log(summary);
// Create deployment tracking issue
const issue = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: `🚀 Production Deployment - ${new Date().toISOString().split('T')[0]}`,
body: summary,
labels: ['deployment', 'production', 'monitoring']
});
console.log(`Created monitoring issue #${issue.data.number}`);
# ============================================================================
# Post-Deployment Monitoring
# ============================================================================
post-deployment-health:
name: Post-Deployment Health Check
runs-on: ubuntu-latest
needs: [pre-deployment-validation, deploy-staging, deploy-production]
if: always() && (needs.deploy-staging.result == 'success' || needs.deploy-production.result == 'success')
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Determine deployed environment
id: env
run: |
if [ "${{ needs.deploy-production.result }}" == "success" ]; then
echo "environment=production" >> $GITHUB_OUTPUT
else
echo "environment=staging" >> $GITHUB_OUTPUT
fi
- name: Wait for application warm-up
run: |
echo "⏳ Waiting 30 seconds for application to warm up..."
sleep 30
- name: Run health checks
run: |
ENV="${{ steps.env.outputs.environment }}"
echo "🏥 Running health checks for $ENV environment..."
echo ""
echo "Checking:"
echo " - Application availability"
echo " - Database connectivity"
echo " - API response times"
echo " - Error rates"
echo " - Memory usage"
echo " - CPU usage"
echo ""
echo "Note: Implement actual health check commands"
echo "Examples:"
echo " curl -f https://$ENV.metabuilder.example.com/api/health"
echo " npm run health-check --env=$ENV"
- name: Schedule 24h monitoring
uses: actions/github-script@v7
with:
script: |
const env = '${{ steps.env.outputs.environment }}';
const deploymentTime = new Date().toISOString();
console.log(`📅 Scheduling 24-hour monitoring for ${env} deployment`);
console.log(`Deployment time: ${deploymentTime}`);
console.log('');
console.log('Monitoring checklist:');
console.log(' - Hour 1: Active monitoring of error rates');
console.log(' - Hour 6: Check performance metrics');
console.log(' - Hour 24: Full health assessment');
console.log('');
console.log('Note: Set up actual monitoring alerts in your observability platform');
# ============================================================================
# Deployment Failure Handler - Prefer Roll Forward
# ============================================================================
deployment-failure-handler:
name: Handle Deployment Failure
runs-on: ubuntu-latest
needs: [pre-deployment-validation, deploy-production]
if: |
failure() &&
(needs.pre-deployment-validation.result == 'failure' || needs.deploy-production.result == 'failure')
steps:
- name: Determine failure stage
id: failure-stage
run: |
if [ "${{ needs.pre-deployment-validation.result }}" == "failure" ]; then
echo "stage=pre-deployment" >> $GITHUB_OUTPUT
echo "severity=low" >> $GITHUB_OUTPUT
else
echo "stage=production" >> $GITHUB_OUTPUT
echo "severity=high" >> $GITHUB_OUTPUT
fi
- name: Display roll-forward guidance
run: |
echo "⚡ DEPLOYMENT FAILURE DETECTED"
echo "================================"
echo ""
echo "Failure Stage: ${{ steps.failure-stage.outputs.stage }}"
echo "Severity: ${{ steps.failure-stage.outputs.severity }}"
echo ""
echo "🎯 RECOMMENDED APPROACH: ROLL FORWARD"
echo "────────────────────────────────────────"
echo ""
echo "Rolling forward is preferred because it:"
echo " ✅ Fixes the root cause permanently"
echo " ✅ Maintains forward progress"
echo " ✅ Builds team capability"
echo " ✅ Prevents recurrence"
echo ""
echo "Steps to roll forward:"
echo " 1. Review failure logs (link below)"
echo " 2. Identify and fix the root cause"
echo " 3. Test the fix locally"
echo " 4. Push fix to trigger new deployment"
echo ""
echo "⚠️ ROLLBACK ONLY IF:"
echo "────────────────────────"
echo " • Production is actively broken"
echo " • Users are experiencing outages"
echo " • Critical security vulnerability"
echo " • Data integrity at risk"
echo ""
if [ "${{ steps.failure-stage.outputs.stage }}" == "pre-deployment" ]; then
echo "✅ GOOD NEWS: Failure occurred pre-deployment"
echo " → Production is NOT affected"
echo " → Safe to fix and retry"
echo " → No rollback needed"
else
echo "🚨 Production deployment failed"
echo " → Assess production impact immediately"
echo " → Check monitoring dashboards"
echo " → Verify user-facing functionality"
fi
- name: Create fix-forward issue
uses: actions/github-script@v7
with:
script: |
const stage = '${{ steps.failure-stage.outputs.stage }}';
const severity = '${{ steps.failure-stage.outputs.severity }}';
const isProd = stage === 'production';
const title = isProd
? '🚨 Production Deployment Failed - Fix Required'
: '⚠️ Pre-Deployment Validation Failed';
const body = `## Deployment Failure - ${stage === 'production' ? 'Production' : 'Pre-Deployment'}
**Time:** ${new Date().toISOString()}
**Commit:** ${context.sha.substring(0, 7)}
**Workflow Run:** [View Logs](${context.payload.repository.html_url}/actions/runs/${context.runId})
**Failure Stage:** ${stage}
**Severity:** ${severity}
${!isProd ? '✅ **Good News:** Production is NOT affected. The failure occurred during pre-deployment checks.\n' : '🚨 **Alert:** Production deployment failed. Assess impact immediately.\n'}
### 🎯 Recommended Action: Roll Forward (Fix and Re-deploy)
Rolling forward is the preferred approach because it:
- ✅ Fixes the root cause permanently
- ✅ Maintains development momentum
- ✅ Prevents the same issue from recurring
- ✅ Builds team problem-solving skills
### 📋 Fix-Forward Checklist
- [ ] **Investigate:** Review [workflow logs](${context.payload.repository.html_url}/actions/runs/${context.runId})
- [ ] **Diagnose:** Identify root cause of failure
- [ ] **Fix:** Implement fix in a new branch/commit
- [ ] **Test:** Verify fix locally (run relevant tests/builds)
- [ ] **Deploy:** Push fix to trigger new deployment
- [ ] **Verify:** Monitor deployment and confirm success
- [ ] **Document:** Update this issue with resolution details
${isProd ? `
### 🚨 Production Impact Assessment
**Before proceeding, verify:**
- [ ] Check monitoring dashboards for errors/alerts
- [ ] Verify critical user flows are working
- [ ] Check application logs for issues
- [ ] Assess if immediate rollback is needed
` : ''}
### ⚠️ When to Rollback Instead
**Only rollback if:**
- 🔴 Production is actively broken with user impact
- 🔴 Critical security vulnerability exposed
- 🔴 Data integrity at risk
- 🔴 Cannot fix forward within acceptable timeframe
${isProd ? `
### 🔄 Rollback Procedure (if absolutely necessary)
1. **Re-run workflow** with previous stable commit SHA
2. **OR use manual rollback:**
- Rollback specific migration: \`npx prisma migrate resolve --rolled-back MIGRATION_NAME --schema=prisma/schema.prisma\`
- Deploy previous Docker image/build
- Restore from pre-deployment backup if needed
- ⚠️ Avoid \`prisma migrate reset\` in production (causes data loss)
3. **Notify:** Update team and status page
4. **Document:** Create post-mortem issue
See [Rollback Procedure](docs/deployment/rollback.md) for details.
` : `
### 💡 Common Pre-Deployment Failures
- **Prisma Generate:** Check schema.prisma syntax and DATABASE_URL
- **Build Failure:** Review TypeScript errors or missing dependencies
- **Test Failure:** Fix failing tests or update test snapshots
- **Lint Errors:** Run \`npm run lint:fix\` locally
`}
### 📚 Resources
- [Workflow Run Logs](${context.payload.repository.html_url}/actions/runs/${context.runId})
- [Commit Details](${context.payload.repository.html_url}/commit/${context.sha})
- [Deployment Documentation](docs/deployment/)
`;
const labels = isProd
? ['deployment', 'production', 'incident', 'high-priority', 'fix-forward']
: ['deployment', 'pre-deployment', 'ci-failure', 'fix-forward'];
await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: title,
body: body,
labels: labels
});

2335
.github/workflows/gated-ci.yml → .github/workflows/gated-pipeline.yml vendored
View File

File diff suppressed because it is too large Load Diff

184
CONSOLIDATION_SUMMARY.md Normal file
View File

@@ -0,0 +1,184 @@
# Pipeline Consolidation Summary
## Problem Statement
> "I asked a bot to merge 2 big pipelines, now I got 3 pipelines wtf instead of 1. I do like the tree (gated) structure as it gives good visability."
The repository had evolved to have 3 separate pipeline files:
1. `gated-ci.yml` (1048 lines)
2. `gated-deployment.yml` (617 lines)
3. `development.yml` (360 lines)
This created confusion about which pipeline does what and had some duplication.
## Solution Implemented
✅ **Consolidated all 3 workflows into 1 unified `gated-pipeline.yml`**
### What Was Changed
**Files Removed:**
- ❌ `.github/workflows/gated-ci.yml`
- ❌ `.github/workflows/gated-deployment.yml`
- ❌ `.github/workflows/development.yml`
**Files Added:**
- ✅ `.github/workflows/gated-pipeline.yml` (1287 lines)
**Files Updated:**
- ✅ `.github/workflows/README.md` - Updated documentation
- ✅ `docs/PIPELINE_CONSOLIDATION.md` - Comprehensive consolidation guide
### Results
| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Number of files | 3 | 1 | **67% reduction** |
| Total lines | 2025 | 1287 | **36% reduction** |
| Functionality | 100% | 100% | **0% lost** |
| Tree visibility | ✅ | ✅ | **Preserved** |
| Clarity | ❌ Confusing | ✅ Clear | **Much better** |
## Unified Pipeline Structure
The new `gated-pipeline.yml` contains 6 sequential gates with conditional execution:
```
┌──────────────────────────────────────────────────┐
│ Gate 1: Code Quality (7 atomic steps) │
│ ├─ Prisma validation │
│ ├─ TypeScript type checking │
│ ├─ ESLint │
│ ├─ Security scan │
│ ├─ File size check │
│ ├─ Code complexity analysis │
│ └─ Stub detection │
└─────────────────┬────────────────────────────────┘
↓
┌──────────────────────────────────────────────────┐
│ Gate 2: Testing (3 atomic steps) │
│ ├─ Unit tests (with coverage) │
│ ├─ E2E tests (Playwright) │
│ └─ DBAL daemon tests │
└─────────────────┬────────────────────────────────┘
↓
┌──────────────────────────────────────────────────┐
│ Gate 3: Build & Package (2 steps) │
│ ├─ Application build │
│ └─ Quality metrics (PR only) │
└─────────────────┬────────────────────────────────┘
↓
┌──────────────────────────────────────────────────┐
│ Gate 4: Development Assistance (PR only) │
│ ├─ Code metrics analysis │
│ ├─ Architectural compliance check │
│ ├─ Development feedback │
│ └─ Copilot interaction handler │
└─────────────────┬────────────────────────────────┘
↓
┌──────────────────────────────────────────────────┐
│ Gate 5: Staging Deployment (main push only) │
│ ├─ Build for staging │
│ ├─ Deploy to staging environment │
│ └─ Smoke tests │
└─────────────────┬────────────────────────────────┘
↓
┌──────────────────────────────────────────────────┐
│ Gate 6: Production Deployment (release/manual) │
│ ├─ Manual approval gate │
│ ├─ Build for production │
│ ├─ Deploy to production │
│ ├─ Smoke tests │
│ └─ Create deployment tracking issue │
└──────────────────────────────────────────────────┘
```
## Conditional Execution
The unified pipeline intelligently runs different gates based on the trigger:
| Trigger Event | Gates Executed | Use Case |
|---------------|----------------|----------|
| Pull Request | Gates 1-4 | Code review and validation |
| Push to main | Gates 1-3 + 5 | CI validation + staging deployment |
| Release | Gates 1-6 | Full flow including production |
| Manual dispatch | Gates 1-6 | On-demand deployment |
| Issue comment (@copilot) | Gate 4 only | Copilot assistance |
## Key Benefits
### 1. Single Source of Truth
- ✅ One workflow file to maintain instead of three
- ✅ No confusion about which pipeline handles what
- ✅ Easier to understand the complete flow
- ✅ Simpler to debug issues
### 2. Eliminated Duplication
- ✅ No redundant quality checks
- ✅ Consolidated triggers and permissions
- ✅ Unified artifact management
- ✅ Single audit trail
### 3. Maintained Tree Structure
- ✅ Clear gate hierarchy preserved
- ✅ Individual atomic steps visible in GitHub UI
- ✅ Easy to identify specific failures
- ✅ Excellent visibility as requested
### 4. Conditional Logic
- ✅ Smart execution based on trigger type
- ✅ Skips irrelevant gates automatically
- ✅ Saves CI/CD resources
- ✅ Faster feedback loops
### 5. Complete Feature Integration
- ✅ All CI checks from gated-ci.yml
- ✅ All deployment logic from gated-deployment.yml
- ✅ All dev assistance from development.yml
- ✅ No functionality lost
## Documentation
Complete documentation available at:
- **Consolidation Guide:** `docs/PIPELINE_CONSOLIDATION.md`
- **Workflow README:** `.github/workflows/README.md`
- **This Summary:** `CONSOLIDATION_SUMMARY.md`
## Testing Recommendations
1. **Pull Request Flow**
- Open a PR to main/develop
- Verify Gates 1-4 execute
- Check that development feedback is posted
2. **Main Branch Push**
- Merge a PR to main
- Verify Gates 1-3 + Gate 5 execute
- Confirm staging deployment succeeds
3. **Release Flow**
- Create a new release
- Verify all Gates 1-6 execute
- Confirm production approval gate works
4. **Manual Deployment**
- Trigger workflow_dispatch
- Select production environment
- Verify manual approval required
5. **Copilot Interaction**
- Comment `@copilot help` on an issue
- Verify Gate 4 responds with guidance
## Migration Complete
✅ All 3 workflows successfully consolidated
✅ Tree structure preserved for visibility
✅ Documentation updated
✅ No functionality lost
✅ 36% reduction in code
✅ Much clearer and easier to maintain
---
**Result:** Successfully solved the problem of having 3 confusing pipelines by consolidating them into 1 clear, unified workflow while maintaining the tree (gated) structure for excellent visibility! 🎉

267
docs/PIPELINE_CONSOLIDATION.md Normal file
View File

@@ -0,0 +1,267 @@
# Pipeline Consolidation - Jan 2026
## Problem Statement
The repository had 3 separate pipeline files that were confusing and had some duplication:
- `gated-ci.yml` (1048 lines) - CI with gates 1-5
- `gated-deployment.yml` (617 lines) - Deployment workflows
- `development.yml` (360 lines) - Development assistance
**Issue:** "I asked a bot to merge 2 big pipelines, now I got 3 pipelines wtf instead of 1"
## Solution
Consolidated all 3 workflows into a single unified pipeline file: `gated-pipeline.yml` (1287 lines)
### Before: 3 Separate Workflows
```
┌─────────────────────────────────────┐
│ gated-ci.yml (1048 lines) │
│ ┌───────────────────────────────┐ │
│ │ Gate 1: Code Quality (7 steps)│ │
│ └───────────────────────────────┘ │
│ ┌───────────────────────────────┐ │
│ │ Gate 2: Testing (3 steps) │ │
│ └───────────────────────────────┘ │
│ ┌───────────────────────────────┐ │
│ │ Gate 3: Build (2 steps) │ │
│ └───────────────────────────────┘ │
│ ┌───────────────────────────────┐ │
│ │ Gate 4: Review & Approval │ │
│ └───────────────────────────────┘ │
│ ┌───────────────────────────────┐ │
│ │ Gate 5: Deployment (partial) │ │
│ └───────────────────────────────┘ │
└─────────────────────────────────────┘
⚠️ DUPLICATION ⚠️
┌─────────────────────────────────────┐
│ gated-deployment.yml (617 lines) │
│ ┌───────────────────────────────┐ │
│ │ Pre-deployment Validation │ │
│ └───────────────────────────────┘ │
│ ┌───────────────────────────────┐ │
│ │ Staging Deployment │ │
│ └───────────────────────────────┘ │
│ ┌───────────────────────────────┐ │
│ │ Production Gate & Deploy │ │
│ └───────────────────────────────┘ │
└─────────────────────────────────────┘
⚠️ SEPARATION ⚠️
┌─────────────────────────────────────┐
│ development.yml (360 lines) │
│ ┌───────────────────────────────┐ │
│ │ Quality Feedback │ │
│ └───────────────────────────────┘ │
│ ┌───────────────────────────────┐ │
│ │ Copilot Interaction │ │
│ └───────────────────────────────┘ │
│ ┌───────────────────────────────┐ │
│ │ Refactoring Suggestions │ │
│ └───────────────────────────────┘ │
└─────────────────────────────────────┘
Total: 2025 lines across 3 files
```
### After: 1 Unified Workflow
```
┌───────────────────────────────────────────────────────────┐
│ gated-pipeline.yml (1287 lines) │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Gate 1: Code Quality (7 atomic steps) │ │
│ │ ├─ 1.1 Prisma Validation │ │
│ │ ├─ 1.2 TypeScript Check │ │
│ │ ├─ 1.3 ESLint │ │
│ │ ├─ 1.4 Security Scan │ │
│ │ ├─ 1.5 File Size Check │ │
│ │ ├─ 1.6 Code Complexity │ │
│ │ └─ 1.7 Stub Detection │ │
│ └─────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Gate 2: Testing (3 atomic steps) │ │
│ │ ├─ 2.1 Unit Tests │ │
│ │ ├─ 2.2 E2E Tests │ │
│ │ └─ 2.3 DBAL Daemon Tests │ │
│ └─────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Gate 3: Build & Package (2 atomic steps) │ │
│ │ ├─ 3.1 Application Build │ │
│ │ └─ 3.2 Quality Metrics (PR only) │ │
│ └─────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Gate 4: Development Assistance (PR only) │ │
│ │ ├─ 4.1 Code Metrics Analysis │ │
│ │ ├─ 4.2 Architectural Compliance │ │
│ │ ├─ 4.3 Development Feedback │ │
│ │ └─ 4.4 Copilot Interaction Handler │ │
│ └─────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Gate 5: Staging Deployment (main push only) │ │
│ │ ├─ Build for staging │ │
│ │ ├─ Deploy to staging │ │
│ │ └─ Smoke tests │ │
│ └─────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Gate 6: Production Deploy (release/manual) │ │
│ │ ├─ Production approval gate │ │
│ │ ├─ Build for production │ │
│ │ ├─ Deploy to production │ │
│ │ ├─ Smoke tests │ │
│ │ └─ Create tracking issue │ │
│ └─────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Gates Summary │ │
│ │ └─ Complete audit trail (30-day retention) │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
└───────────────────────────────────────────────────────────┘
Total: 1287 lines in 1 file
✅ Net reduction: 738 lines (36% smaller)
✅ Eliminated duplication
✅ Maintained tree structure for visibility
```
## Benefits
### 1. Single Source of Truth
- ✅ One workflow file to maintain
- ✅ No confusion about which pipeline does what
- ✅ Easier to understand the complete flow
### 2. Conditional Execution
The unified pipeline intelligently runs different gates based on the trigger:
| Trigger | Gates Executed |
|---------|----------------|
| PR to main/develop | Gates 1-4 (Code → Test → Build → Dev Feedback) |
| Push to main | Gates 1-5 (Code → Test → Build → Skip Gate 4 → Staging Deploy) |
| Release | Gates 1-6 (Full flow including Production) |
| Manual dispatch | Gates 1-6 (Full flow with manual approval) |
| Issue comment (@copilot) | Gate 4 only (Copilot interaction) |
### 3. Tree Structure Preserved
Each gate contains multiple atomic steps, providing excellent visibility:
- Individual step status visible in GitHub UI
- Easy to identify which specific check failed
- Parallel execution within gates for efficiency
- Sequential execution between gates to save resources
### 4. No Redundant Steps
- Development feedback integrated into gate flow
- No duplicate quality checks
- Deployment workflows properly sequenced
- All conditional logic unified
### 5. Cleaner Artifact Management
- Single artifact namespace
- Consistent naming: `gate-X-*-result`
- Complete audit trail: `complete-gate-audit-trail`
- 30-day retention for all gate results
## Migration Path
### Files Removed
```bash
rm .github/workflows/gated-ci.yml
rm .github/workflows/gated-deployment.yml
rm .github/workflows/development.yml
```
### File Added
```bash
.github/workflows/gated-pipeline.yml # New unified workflow
```
### Documentation Updated
- `.github/workflows/README.md` - Updated to reference unified pipeline
- No changes needed to external docs (no direct references to old files)
## Triggers Consolidated
### Previous (3 files)
```yaml
# gated-ci.yml
on:
push:
branches: [ main, master, develop ]
pull_request:
branches: [ main, master, develop ]
# gated-deployment.yml
on:
push:
branches: [ main, master ]
release:
types: [published]
workflow_dispatch:
inputs: ...
# development.yml
on:
pull_request:
types: [opened, synchronize, ready_for_review]
issue_comment:
types: [created]
```
### Current (1 file)
```yaml
# gated-pipeline.yml - All triggers in one place
on:
push:
branches: [ main, master, develop ]
pull_request:
branches: [ main, master, develop ]
types: [opened, synchronize, ready_for_review]
release:
types: [published]
issue_comment:
types: [created]
workflow_dispatch:
inputs:
environment: ...
skip_tests: ...
```
## Permissions Consolidated
All permissions unified in one place:
```yaml
permissions:
contents: read # Read repo content
pull-requests: write # Comment on PRs
checks: write # Update check status
statuses: write # Update commit status
issues: write # Create tracking issues
deployments: write # Manage deployments
```
## Testing the Unified Pipeline
1. **PR Flow** - Open a PR, verify gates 1-4 run
2. **Main Push** - Merge PR, verify gates 1-3 + gate 5 (staging) run
3. **Release Flow** - Create release, verify gates 1-6 run with approval
4. **Manual Dispatch** - Trigger manually, verify production deployment
5. **Copilot Interaction** - Comment `@copilot help`, verify gate 4 responds
## Result
✨ **Problem Solved:** Successfully merged 3 pipelines into 1 unified workflow while maintaining the tree (gated) structure for excellent visibility.
### Metrics
- **Lines of code:** 2025 → 1287 (36% reduction)
- **Number of files:** 3 → 1 (67% reduction)
- **Maintenance complexity:** HIGH → LOW
- **Clarity:** CONFUSING → CLEAR
The unified pipeline provides a single, clear view of the entire CI/CD process from code quality through production deployment.