metabuilder/docs/stub-detection

Stub Implementation Detection Guide

Overview

Stub detection automatically identifies incomplete, placeholder, or mock implementations in the codebase. This prevents accidental deployment of unfinished code and helps teams track what still needs to be completed.

What is a Stub?

A stub is a function or component that:

• Throws "not implemented" errors
• Returns placeholder values (null, empty objects, mock data)
• Contains TODO/FIXME comments indicating incomplete work
• Only logs to console without doing real work
• Has minimal/empty implementation
• Contains placeholder text in UI

Detection Methods

1. Pattern-Based Detection

Scans code for specific patterns that indicate stubs:

Error Throwing

// ❌ Detected as stub
export function processData(data) {
  throw new Error('not implemented')
}

Console Logging Only

// ❌ Detected as stub
export function validateEmail(email) {
  console.log('validating:', email) // Nothing else
}

Null/Undefined Returns

// ❌ Detected as stub
export function fetchData() {
  return null
}

Placeholder Text

// ❌ Detected as stub
function Dashboard() {
  return <div>TODO: Build dashboard</div>
}

Mock Data with Markers

// ❌ Detected as stub
export function getUsers() {
  return [ // stub data
    { id: 1, name: 'John' }
  ]
}

2. Completeness Analysis

Analyzes implementation quality:

• Logical Lines: Actual code (excluding returns and comments)
• Return Lines: How many return statements
• JSX Lines: For components, how much actual UI
• Completeness Score: 0-100% based on implementation density

Scoring:

• 0%: Throws not implemented error
• 0-30%: Critical (only logging, returning null, no real logic)
• 30-60%: Medium (minimal implementation, mostly mock data)
• 60-80%: Low (mostly complete, some placeholders)
• 80-100%: Good (real implementation)

3. Code Indicators

Detects special markers:

Indicator	Severity	Meaning
throw new Error('not implemented')	Critical	Explicitly unimplemented
// TODO: or // FIXME:	Medium	Known issue, incomplete
console.log() only	High	Debug logging, not real code
// mock, // stub	Medium	Explicitly marked placeholder
return null	Low	Empty return value
<> </>	High	Empty component fragment

Workflow Integration

Automatic Detection

The detect-stubs.yml workflow runs:

• On every PR (to flag new stubs)
• On every push to main/master
• Weekly scheduled (Monday midnight)

PR Comments

Posts a summary comment showing:

• Total stubs found
• Severity breakdown
• Critical issues with file/line/function
• Recommendations for fixes

Example:

## 🔍 Stub Implementation Detection Report

### Summary
- Pattern-Based Stubs: 5
- Low Completeness Items: 3
- Average Completeness: 72%

### Severity Breakdown
- 🔴 Critical: 2
- 🟠 Medium: 2
- 🟡 Low: 1

### 🔴 Critical Issues Found
| File | Line | Function | Type |
|------|------|----------|------|
| src/api/users.ts | 42 | fetchUsers | throws-not-implemented |
| src/components/Dashboard.tsx | 15 | Dashboard | empty-fragment |

Local Usage

Run Detection Locally

# Pattern-based stub detection
npx tsx scripts/detect-stub-implementations.ts

# Completeness analysis
npx tsx scripts/analyze-implementation-completeness.ts

# Generate detailed report
npx tsx scripts/generate-stub-report.ts

# All three
npm run detect-stubs

Output Formats

All scripts output JSON for programmatic access:

Pattern Detection Output

{
  "totalStubsFound": 5,
  "bySeverity": {
    "high": 2,
    "medium": 2,
    "low": 1
  },
  "byType": {
    "not-implemented": 2,
    "todo-comment": 1,
    "console-log-only": 1,
    "returns-empty-value": 1
  },
  "criticalIssues": [
    {
      "file": "src/api/users.ts",
      "line": 42,
      "function": "fetchUsers",
      "type": "not-implemented"
    }
  ]
}

Completeness Analysis Output

{
  "totalAnalyzed": 45,
  "bySeverity": {
    "critical": 2,
    "high": 5,
    "medium": 8,
    "low": 12
  },
  "flagTypes": {
    "has-todo-comments": 5,
    "throws-not-implemented": 2,
    "only-console-log": 1
  },
  "averageCompleteness": 72.4,
  "criticalStubs": [
    {
      "file": "src/api/users.ts",
      "line": 42,
      "name": "fetchUsers",
      "type": "function",
      "flags": ["throws-not-implemented"],
      "summary": "Potential stub: throws-not-implemented"
    }
  ]
}

Common Stub Patterns & Fixes

Pattern 1: Not Implemented Error

Problem: Function explicitly throws unimplemented error Severity: 🔴 Critical Fix: Implement the function

// ❌ Before
export async function fetchUserData(id: string) {
  throw new Error('not implemented')
}

// ✅ After
export async function fetchUserData(id: string): Promise<User> {
  const response = await fetch(`/api/users/${id}`)
  if (!response.ok) throw new Error(`Failed to fetch user ${id}`)
  return response.json()
}

Pattern 2: Console Logging Only

Problem: Function only logs without doing real work Severity: 🟠 High Fix: Replace logging with actual implementation

// ❌ Before
export function calculateTotal(items: Item[]): number {
  console.log('calculating total for:', items)
  // nothing else
}

// ✅ After
export function calculateTotal(items: Item[]): number {
  return items.reduce((sum, item) => sum + item.price, 0)
}

Pattern 3: Return Null/Undefined

Problem: Function returns null or undefined Severity: 🟡 Medium Fix: Return actual data or use types to force implementation

// ❌ Before
export function getConfig() {
  return null // TODO: load config
}

// ✅ After
export function getConfig(): Config {
  return configLoader.load()
}

Pattern 4: Placeholder Component

Problem: Component renders placeholder text instead of UI Severity: 🟠 High Fix: Implement the component properly

// ❌ Before
export function Dashboard() {
  return <div>TODO: Build dashboard</div>
}

// ✅ After
export function Dashboard() {
  return (
    <div className="dashboard">
      <Header />
      <Sidebar />
      <MainContent />
    </div>
  )
}

Pattern 5: Mock Data

Problem: Function returns hardcoded mock data Severity: 🟠 Medium Fix: Replace with real data source

// ❌ Before
export function getUsers(): User[] {
  return [ // mock data
    { id: 1, name: 'John', email: 'john@example.com' },
    { id: 2, name: 'Jane', email: 'jane@example.com' }
  ]
}

// ✅ After
export async function getUsers(): Promise<User[]> {
  const data = await Database.query('SELECT * FROM users')
  return data.map(row => new User(row))
}

Pattern 6: TODO Comments

Problem: Code has TODO/FIXME comments indicating incomplete work Severity: 🟡 Medium/Low Fix: Create GitHub issue and remove TODO from code

// ❌ Before
export function validateEmail(email: string): boolean {
  // TODO: Add real email validation
  return true
}

// ✅ After (create issue #123 first)
export function validateEmail(email: string): boolean {
  const emailRegex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/
  return emailRegex.test(email)
}

Pattern 7: Empty Component

Problem: Component has minimal or empty JSX Severity: 🟠 High Fix: Implement the component's UI

// ❌ Before
export function UserList() {
  return <div></div>
}

// ✅ After
export function UserList() {
  const users = useQuery('users')
  return (
    <div className="user-list">
      {users.map(user => (
        <UserCard key={user.id} user={user} />
      ))}
    </div>
  )
}

Best Practices

1. Never Leave Stubs in Main Branch

# Before merging, ensure no stubs in changed files
git diff HEAD~1 -- 'src/**/*.ts' | grep -i 'todo\|not implemented'

2. Use Types to Force Implementation

// ❌ Function can return anything
function getValue() {
  // oops, forgot to implement
}

// ✅ Type forces implementation
function getValue(): string {
  // TypeScript error: missing return
  // MUST implement
}

3. Create Issues Instead of TODOs

// ❌ Don't do this
function processData(data) {
  // TODO: add caching (#TBD)
  return compute(data)
}

// ✅ Do this
// Implemented cache per issue #456
const cache = new LRUCache()
function processData(data) {
  return cache.getOrCompute(data, () => compute(data))
}

4. Test Stubs Explicitly

// Make stubs fail tests explicitly
describe('getUsers', () => {
  it('should fetch users from API', async () => {
    // This WILL fail if implementation is missing
    const users = await getUsers()
    expect(users).toBeDefined()
    expect(users.length).toBeGreaterThan(0)
    expect(users[0].name).toBeDefined()
  })
})

5. Use Linting to Prevent Stubs

Add ESLint rules to .eslintrc.json:

{
  "rules": {
    "no-console": ["error", { "allow": ["warn", "error"] }],
    "no-throw-literal": "error",
    "no-empty-function": ["error", { "allow": ["arrowFunctions"] }]
  }
}

Interpreting the Report

Severity Levels

• 🔴 Critical (Completeness: 0%)

  • Blocks production
  • Must fix immediately
  • Examples: throw new Error('not implemented')

• 🟠 High (Completeness: 10-30%)

  • Should fix before merge
  • Likely causes bugs
  • Examples: Console logging only, empty components

• 🟡 Medium (Completeness: 40-70%)

  • Should fix soon
  • Partial implementation
  • Examples: Mock data, basic structure

• 🟢 Low (Completeness: 80-99%)

  • Nice to fix
  • Mostly complete
  • Examples: Minor TODOs, edge cases

Metrics Explained

Metric	Meaning
Logical Lines	Lines that do actual work (not returns/comments)
Return Lines	Number of return statements
JSX Lines	For components, lines rendering UI
Completeness	Estimated % of implementation done
Flags	Detected stub indicators

Artifacts

The workflow uploads three artifacts:

1. stub-patterns.json - Raw pattern detection data
2. implementation-analysis.json - Completeness scoring
3. stub-report.md - Formatted markdown report
4. changed-stubs.txt - Stubs in changed files (PRs only)

Available for 30 days after workflow run.

CI/CD Integration

Failing Builds

The workflow runs with continue-on-error: true, so stubs don't block merging. However, you can:

1. Enable stricter linting to fail on TODOs
2. Require test coverage (which fails with stubs)
3. Use branch protection rules to require review of detected stubs

GitHub Checks

• Posts check run with stub detection results
• Shows in PR checks section
• Includes summary of critical issues

Troubleshooting

"No stubs found" but code looks incomplete

• Patterns may not match your specific style
• Check individual function with completeness analyzer
• Consider adding custom patterns to detection script

Too many false positives

• Edit STUB_PATTERNS in detect-stub-implementations.ts
• Adjust completeness thresholds in analyze-implementation-completeness.ts
• Add file/function exclusions

Stubs not showing in PR comment

• Check workflow permissions (needs pull-requests: write)
• Verify GitHub API isn't rate limited
• Check workflow logs for errors

Extension Points

Add Custom Patterns

Edit scripts/detect-stub-implementations.ts:

const STUB_PATTERNS = [
  // ... existing patterns
  {
    name: 'Your custom pattern',
    pattern: /your regex here/,
    type: 'custom-stub',
    severity: 'high',
    description: 'Your description'
  }
]

Add Custom Analysis

Create new script in scripts/:

npx tsx scripts/my-custom-analysis.ts > my-report.json

Then add to workflow in .github/workflows/detect-stubs.yml.

References

• Workflow Definition
• Detection Script
• Completeness Analyzer
• Report Generator

---

Last updated: December 25, 2025