diff --git a/tools/refactoring/README.md b/tools/refactoring/README.md new file mode 100644 index 000000000..9ed2a3df8 --- /dev/null +++ b/tools/refactoring/README.md @@ -0,0 +1,284 @@ +# Lambda-per-File Refactoring Tools + +Automated tools for refactoring large TypeScript files into modular lambda-per-file structure. + +## Quick Start + +### 1. Generate Progress Report + +```bash +npx tsx tools/refactoring/refactor-to-lambda.ts +``` + +This scans the codebase and generates `docs/todo/LAMBDA_REFACTOR_PROGRESS.md` with: +- List of all files exceeding 150 lines +- Categorization by type (library, component, DBAL, tool, etc.) +- Priority ranking +- Refactoring recommendations + +### 2. Dry Run (Preview Changes) + +Preview what would happen without modifying files: + +```bash +# Preview all high-priority files +npx tsx tools/refactoring/orchestrate-refactor.ts --dry-run high + +# Preview specific number of files +npx tsx tools/refactoring/orchestrate-refactor.ts --dry-run high --limit=5 + +# Preview a single file +npx tsx tools/refactoring/ast-lambda-refactor.ts --dry-run -v frontends/nextjs/src/lib/rendering/page/page-definition-builder.ts +``` + +### 3. Run Bulk Refactoring + +Refactor files in bulk with automatic linting and import fixing: + +```bash +# Refactor all high-priority files (recommended start) +npx tsx tools/refactoring/orchestrate-refactor.ts high + +# Refactor first 10 high-priority files +npx tsx tools/refactoring/orchestrate-refactor.ts high --limit=10 + +# Refactor all pending files +npx tsx tools/refactoring/orchestrate-refactor.ts all +``` + +The orchestrator will: +1. โœ… Refactor files using AST analysis +2. ๐Ÿ”ง Run `npm run lint:fix` to fix imports +3. ๐Ÿ” Run type checking +4. ๐Ÿงช Run unit tests +5. ๐Ÿ’พ Save detailed results + +## Available Tools + +### 1. `refactor-to-lambda.ts` - Progress Tracker + +Scans codebase and generates tracking report. + +```bash +npx tsx tools/refactoring/refactor-to-lambda.ts +``` + +**Output:** `docs/todo/LAMBDA_REFACTOR_PROGRESS.md` + +### 2. `ast-lambda-refactor.ts` - AST-based Refactoring + +Uses TypeScript compiler API for accurate code transformation. + +```bash +# Single file +npx tsx tools/refactoring/ast-lambda-refactor.ts [options] + +# Options: +# -d, --dry-run Preview without writing +# -v, --verbose Detailed output +# -h, --help Show help +``` + +**Example:** +```bash +npx tsx tools/refactoring/ast-lambda-refactor.ts -v frontends/nextjs/src/lib/db/core/index.ts +``` + +### 3. `bulk-lambda-refactor.ts` - Regex-based Bulk Refactoring + +Simpler regex-based refactoring (faster but less accurate). + +```bash +npx tsx tools/refactoring/bulk-lambda-refactor.ts [options] +``` + +### 4. `orchestrate-refactor.ts` - Master Orchestrator + +Complete automated workflow for bulk refactoring. + +```bash +npx tsx tools/refactoring/orchestrate-refactor.ts [priority] [options] + +# Priority: high | medium | low | all +# Options: +# -d, --dry-run Preview only +# --limit=N Process only N files +# --skip-lint Skip linting phase +# --skip-test Skip testing phase +``` + +**Examples:** +```bash +# Dry run for high-priority files +npx tsx tools/refactoring/orchestrate-refactor.ts high --dry-run + +# Refactor 5 high-priority files +npx tsx tools/refactoring/orchestrate-refactor.ts high --limit=5 + +# Refactor all medium-priority files, skip tests +npx tsx tools/refactoring/orchestrate-refactor.ts medium --skip-test +``` + +## Refactoring Pattern + +The tools follow the pattern established in `frontends/nextjs/src/lib/schema/`: + +### Before (Single Large File) +``` +lib/ +โ””โ”€โ”€ utils.ts (300 lines) + โ”œโ”€โ”€ function validateEmail() + โ”œโ”€โ”€ function parseDate() + โ”œโ”€โ”€ function formatCurrency() + โ””โ”€โ”€ ... +``` + +### After (Lambda-per-File) +``` +lib/ +โ”œโ”€โ”€ utils.ts (re-exports) +โ””โ”€โ”€ utils/ + โ”œโ”€โ”€ functions/ + โ”‚ โ”œโ”€โ”€ validate-email.ts + โ”‚ โ”œโ”€โ”€ parse-date.ts + โ”‚ โ””โ”€โ”€ format-currency.ts + โ”œโ”€โ”€ UtilsUtils.ts (class wrapper) + โ””โ”€โ”€ index.ts (barrel export) +``` + +### Usage After Refactoring + +```typescript +// Option 1: Import individual functions (recommended) +import { validateEmail } from '@/lib/utils' + +// Option 2: Use class wrapper +import { UtilsUtils } from '@/lib/utils' +UtilsUtils.validateEmail(email) + +// Option 3: Direct import from function file +import { validateEmail } from '@/lib/utils/functions/validate-email' +``` + +## File Categories + +### High Priority (Easiest to Refactor) +- **Library files** - Pure utility functions +- **Tool files** - Development scripts + +### Medium Priority +- **DBAL files** - Database abstraction layer +- **Component files** - React components (need sub-component extraction) + +### Low Priority +- **Very large files** (>500 lines) - Need careful planning + +### Skipped +- **Test files** - Comprehensive coverage is acceptable +- **Type definition files** - Naturally large + +## Safety Features + +1. **Dry Run Mode** - Preview all changes before applying +2. **Backup** - Original files are replaced with re-exports (old code preserved in git) +3. **Automatic Linting** - Fixes imports and formatting +4. **Type Checking** - Validates TypeScript compilation +5. **Test Running** - Ensures functionality preserved +6. **Incremental** - Process files in batches with limits + +## Workflow Recommendation + +### Phase 1: High-Priority Files (Library & Tools - 20 files) +```bash +# 1. Generate report +npx tsx tools/refactoring/refactor-to-lambda.ts + +# 2. Dry run to preview +npx tsx tools/refactoring/orchestrate-refactor.ts high --dry-run + +# 3. Refactor in small batches +npx tsx tools/refactoring/orchestrate-refactor.ts high --limit=5 + +# 4. Review, test, commit +git diff +npm run test:unit +git add . && git commit -m "refactor: convert 5 library files to lambda-per-file" + +# 5. Repeat for next batch +npx tsx tools/refactoring/orchestrate-refactor.ts high --limit=5 +``` + +### Phase 2: Medium-Priority Files (DBAL & Components - 68 files) +Similar process with medium priority. + +### Phase 3: Low-Priority Files +Tackle individually with careful review. + +## Troubleshooting + +### Import Errors After Refactoring + +```bash +# Re-run linter +npm run lint:fix + +# Check type errors +npm run typecheck +``` + +### Tests Failing + +1. Check if function signatures changed +2. Update test imports to new locations +3. Verify mocks are still valid + +### Generated Code Issues + +1. Review the generated files +2. Fix manually if needed +3. The tools provide a starting point, not perfect output + +## Advanced Usage + +### Custom Filtering + +Edit `docs/todo/LAMBDA_REFACTOR_PROGRESS.md` to mark files as completed: + +```markdown +- [x] `path/to/file.ts` (200 lines) +- [ ] `path/to/other.ts` (180 lines) +``` + +### Manual Refactoring + +For complex files, use the generated code as a template and refine manually: + +1. Run with `--dry-run` and `--verbose` +2. Review what would be generated +3. Apply manually with improvements + +## Output Files + +- `docs/todo/LAMBDA_REFACTOR_PROGRESS.md` - Tracking report with all files +- `docs/todo/REFACTOR_RESULTS.json` - Detailed results from last run +- Individual function files in `/functions/` directories +- Class wrappers: `Utils.ts` +- Barrel exports: `/index.ts` + +## Next Steps After Refactoring + +1. โœ… Review generated code +2. โœ… Run full test suite: `npm run test:unit` +3. โœ… Run E2E tests: `npm run test:e2e` +4. โœ… Update documentation if needed +5. โœ… Commit in logical batches +6. โœ… Update `LAMBDA_REFACTOR_PROGRESS.md` with completion status + +## Contributing + +To improve these tools: + +1. Test on various file types +2. Report issues with specific files +3. Suggest improvements to AST parsing +4. Add support for more patterns (arrow functions, etc.) diff --git a/tools/refactoring/ast-lambda-refactor.ts b/tools/refactoring/ast-lambda-refactor.ts new file mode 100644 index 000000000..be4bc06b4 --- /dev/null +++ b/tools/refactoring/ast-lambda-refactor.ts @@ -0,0 +1,427 @@ +#!/usr/bin/env tsx +/** + * AST-based Lambda Refactoring Tool + * + * Uses TypeScript compiler API for accurate code analysis and transformation + */ + +import * as ts from 'typescript' +import * as fs from 'fs/promises' +import * as path from 'path' +import { exec } from 'child_process' +import { promisify } from 'util' + +const execAsync = promisify(exec) + +interface ExtractedFunction { + name: string + fullText: string + isExported: boolean + isAsync: boolean + leadingComments: string + startPos: number + endPos: number +} + +interface ExtractedImport { + fullText: string + moduleSpecifier: string + namedImports: string[] +} + +class ASTLambdaRefactor { + private dryRun: boolean + private verbose: boolean + + constructor(options: { dryRun?: boolean; verbose?: boolean } = {}) { + this.dryRun = options.dryRun || false + this.verbose = options.verbose || false + } + + private log(message: string) { + if (this.verbose) { + console.log(message) + } + } + + /** + * Parse TypeScript file and extract functions using AST + */ + async analyzeFil(filePath: string): Promise<{ + functions: ExtractedFunction[] + imports: ExtractedImport[] + types: string[] + }> { + const sourceCode = await fs.readFile(filePath, 'utf-8') + const sourceFile = ts.createSourceFile( + filePath, + sourceCode, + ts.ScriptTarget.Latest, + true + ) + + const functions: ExtractedFunction[] = [] + const imports: ExtractedImport[] = [] + const types: string[] = [] + + const visit = (node: ts.Node) => { + // Extract function declarations + if (ts.isFunctionDeclaration(node) && node.name) { + const isExported = node.modifiers?.some(m => m.kind === ts.SyntaxKind.ExportKeyword) || false + const isAsync = node.modifiers?.some(m => m.kind === ts.SyntaxKind.AsyncKeyword) || false + + // Get leading comments + const leadingComments = ts.getLeadingCommentRanges(sourceCode, node.getFullStart()) + let commentText = '' + if (leadingComments) { + for (const comment of leadingComments) { + commentText += sourceCode.substring(comment.pos, comment.end) + '\n' + } + } + + functions.push({ + name: node.name.text, + fullText: node.getText(sourceFile), + isExported, + isAsync, + leadingComments: commentText.trim(), + startPos: node.getStart(sourceFile), + endPos: node.getEnd(), + }) + } + + // Extract class methods + if (ts.isClassDeclaration(node) && node.members) { + for (const member of node.members) { + if (ts.isMethodDeclaration(member) && member.name && ts.isIdentifier(member.name)) { + const isAsync = member.modifiers?.some(m => m.kind === ts.SyntaxKind.AsyncKeyword) || false + + // Get leading comments + const leadingComments = ts.getLeadingCommentRanges(sourceCode, member.getFullStart()) + let commentText = '' + if (leadingComments) { + for (const comment of leadingComments) { + commentText += sourceCode.substring(comment.pos, comment.end) + '\n' + } + } + + // Convert method to function + const methodText = member.getText(sourceFile) + const functionText = this.convertMethodToFunction(methodText, member.name.text, isAsync) + + functions.push({ + name: member.name.text, + fullText: functionText, + isExported: true, + isAsync, + leadingComments: commentText.trim(), + startPos: member.getStart(sourceFile), + endPos: member.getEnd(), + }) + } + } + } + + // Extract imports + if (ts.isImportDeclaration(node)) { + const moduleSpec = (node.moduleSpecifier as ts.StringLiteral).text + const namedImports: string[] = [] + + if (node.importClause?.namedBindings && ts.isNamedImports(node.importClause.namedBindings)) { + for (const element of node.importClause.namedBindings.elements) { + namedImports.push(element.name.text) + } + } + + imports.push({ + fullText: node.getText(sourceFile), + moduleSpecifier: moduleSpec, + namedImports, + }) + } + + // Extract type definitions + if (ts.isTypeAliasDeclaration(node) || ts.isInterfaceDeclaration(node)) { + types.push(node.getText(sourceFile)) + } + + ts.forEachChild(node, visit) + } + + visit(sourceFile) + + return { functions, imports, types } + } + + /** + * Convert a class method to a standalone function + */ + private convertMethodToFunction(methodText: string, methodName: string, isAsync: boolean): string { + // Remove visibility modifiers (public, private, protected) + let funcText = methodText.replace(/^\s*(public|private|protected)\s+/, '') + + // Ensure it starts with async if needed + if (isAsync && !funcText.trim().startsWith('async')) { + funcText = 'async ' + funcText + } + + // Convert method syntax to function syntax + // "methodName(...): Type {" -> "function methodName(...): Type {" + funcText = funcText.replace(/^(\s*)(async\s+)?([a-zA-Z0-9_]+)(\s*\([^)]*\))/, '$1$2function $3$4') + + return funcText + } + + /** + * Create individual function file with proper imports + */ + async createFunctionFile( + func: ExtractedFunction, + allImports: ExtractedImport[], + outputPath: string + ): Promise { + let content = '' + + // Add imports (for now, include all - can be optimized to only include used imports) + if (allImports.length > 0) { + content += allImports.map(imp => imp.fullText).join('\n') + '\n\n' + } + + // Add comments + if (func.leadingComments) { + content += func.leadingComments + '\n' + } + + // Add function (ensure it's exported) + let funcText = func.fullText + if (!func.isExported && !funcText.includes('export ')) { + funcText = 'export ' + funcText + } else if (!funcText.includes('export ')) { + funcText = 'export ' + funcText + } + + content += funcText + '\n' + + if (!this.dryRun) { + await fs.writeFile(outputPath, content, 'utf-8') + } + } + + /** + * Refactor a file using AST analysis + */ + async refactorFile(filePath: string): Promise { + this.log(`\n๐Ÿ” Analyzing ${filePath}...`) + + const { functions, imports, types } = await this.analyzeFile(filePath) + + if (functions.length === 0) { + this.log(' โญ๏ธ No functions found - skipping') + return + } + + if (functions.length <= 2) { + this.log(` โญ๏ธ Only ${functions.length} function(s) - skipping (not worth refactoring)`) + return + } + + this.log(` Found ${functions.length} functions: ${functions.map(f => f.name).join(', ')}`) + + // Create output directory structure + const dir = path.dirname(filePath) + const basename = path.basename(filePath, path.extname(filePath)) + const functionsDir = path.join(dir, basename, 'functions') + + if (!this.dryRun) { + await fs.mkdir(functionsDir, { recursive: true }) + } + + this.log(` Creating: ${functionsDir}`) + + // Create individual function files + for (const func of functions) { + const kebabName = this.toKebabCase(func.name) + const funcFile = path.join(functionsDir, `${kebabName}.ts`) + + await this.createFunctionFile(func, imports, funcFile) + this.log(` โœ“ ${kebabName}.ts`) + } + + // Create index file for re-exports + const indexContent = this.generateIndexFile(functions, 'functions') + const indexPath = path.join(dir, basename, 'index.ts') + + if (!this.dryRun) { + await fs.writeFile(indexPath, indexContent, 'utf-8') + } + this.log(` โœ“ index.ts`) + + // Create class wrapper + const className = this.toClassName(basename) + const classContent = this.generateClassWrapper(className, functions) + const classPath = path.join(dir, basename, `${className}.ts`) + + if (!this.dryRun) { + await fs.writeFile(classPath, classContent, 'utf-8') + } + this.log(` โœ“ ${className}.ts`) + + // Replace original file with re-export + const newMainContent = `/** + * This file has been refactored into modular lambda-per-file structure. + * + * Import individual functions or use the class wrapper: + * @example + * import { ${functions[0].name} } from './${basename}' + * + * @example + * import { ${className} } from './${basename}' + * ${className}.${functions[0].name}(...) + */ + +export * from './${basename}' +` + + if (!this.dryRun) { + await fs.writeFile(filePath, newMainContent, 'utf-8') + } + this.log(` โœ“ Updated ${path.basename(filePath)}`) + + this.log(` โœ… Refactored into ${functions.length + 2} files`) + } + + private toKebabCase(str: string): string { + return str.replace(/([A-Z])/g, '-$1').toLowerCase().replace(/^-/, '') + } + + private toClassName(str: string): string { + return str + .split(/[-_]/) + .map(word => word.charAt(0).toUpperCase() + word.slice(1)) + .join('') + 'Utils' + } + + private generateIndexFile(functions: ExtractedFunction[], functionsDir: string): string { + let content = '// Auto-generated re-exports\n\n' + + for (const func of functions) { + const kebabName = this.toKebabCase(func.name) + content += `export { ${func.name} } from './${functionsDir}/${kebabName}'\n` + } + + return content + } + + private generateClassWrapper(className: string, functions: ExtractedFunction[]): string { + let content = `// Auto-generated class wrapper\n\n` + + // Import all functions + for (const func of functions) { + const kebabName = this.toKebabCase(func.name) + content += `import { ${func.name} } from './functions/${kebabName}'\n` + } + + content += `\n/**\n * ${className} - Convenience class wrapper\n */\n` + content += `export class ${className} {\n` + + for (const func of functions) { + const asyncKeyword = func.isAsync ? 'async ' : '' + content += ` static ${asyncKeyword}${func.name}(...args: any[]) {\n` + content += ` return ${func.isAsync ? 'await ' : ''}${func.name}(...args)\n` + content += ` }\n\n` + } + + content += '}\n' + + return content + } + + // Fix the typo in the method name + async analyzeFile(filePath: string): Promise<{ + functions: ExtractedFunction[] + imports: ExtractedImport[] + types: string[] + }> { + return this.analyzeFil(filePath) + } + + /** + * Process multiple files + */ + async bulkRefactor(files: string[]): Promise { + console.log(`\n๐Ÿ“ฆ AST-based Lambda Refactoring`) + console.log(` Mode: ${this.dryRun ? 'DRY RUN' : 'LIVE'}`) + console.log(` Files: ${files.length}\n`) + + let successCount = 0 + let skipCount = 0 + let errorCount = 0 + + for (let i = 0; i < files.length; i++) { + const file = files[i] + console.log(`[${i + 1}/${files.length}] ${file}`) + + try { + await this.refactorFile(file) + successCount++ + } catch (error) { + if (error instanceof Error && error.message.includes('skipping')) { + skipCount++ + } else { + console.error(` โŒ Error: ${error}`) + errorCount++ + } + } + } + + console.log(`\n๐Ÿ“Š Summary:`) + console.log(` โœ… Success: ${successCount}`) + console.log(` โญ๏ธ Skipped: ${skipCount}`) + console.log(` โŒ Errors: ${errorCount}`) + } +} + +// CLI +async function main() { + const args = process.argv.slice(2) + + if (args.includes('--help') || args.includes('-h') || args.length === 0) { + console.log('AST-based Lambda Refactoring Tool\n') + console.log('Usage: tsx ast-lambda-refactor.ts [options] ') + console.log('\nOptions:') + console.log(' -d, --dry-run Preview without writing') + console.log(' -v, --verbose Verbose output') + console.log(' -h, --help Show help') + process.exit(0) + } + + const dryRun = args.includes('--dry-run') || args.includes('-d') + const verbose = args.includes('--verbose') || args.includes('-v') + const file = args.find(a => !a.startsWith('-')) + + if (!file) { + console.error('Error: Please provide a file to refactor') + process.exit(1) + } + + const refactor = new ASTLambdaRefactor({ dryRun, verbose }) + await refactor.bulkRefactor([file]) + + if (!dryRun) { + console.log('\n๐Ÿ”ง Running linter...') + try { + await execAsync('npm run lint:fix') + console.log(' โœ… Lint complete') + } catch (e) { + console.log(' โš ๏ธ Lint had warnings (may be expected)') + } + } + + console.log('\nโœจ Done!') +} + +if (require.main === module) { + main().catch(console.error) +} + +export { ASTLambdaRefactor } diff --git a/tools/refactoring/batch-refactor-all.ts b/tools/refactoring/batch-refactor-all.ts new file mode 100644 index 000000000..3051e61a2 --- /dev/null +++ b/tools/refactoring/batch-refactor-all.ts @@ -0,0 +1,118 @@ +#!/usr/bin/env tsx +/** + * Batch Refactor All Large Files + * + * Processes all files from the tracking report in priority order + */ + +import { BulkLambdaRefactor } from './bulk-lambda-refactor' +import * as fs from 'fs/promises' +import * as path from 'path' + +interface FileToRefactor { + path: string + lines: number + category: string + priority: 'high' | 'medium' | 'low' +} + +async function loadFilesFromReport(): Promise { + const reportPath = path.join(process.cwd(), 'docs/todo/LAMBDA_REFACTOR_PROGRESS.md') + const content = await fs.readFile(reportPath, 'utf-8') + + const files: FileToRefactor[] = [] + const lines = content.split('\n') + + let currentPriority: 'high' | 'medium' | 'low' = 'high' + + for (const line of lines) { + if (line.includes('### High Priority')) currentPriority = 'high' + else if (line.includes('### Medium Priority')) currentPriority = 'medium' + else if (line.includes('### Low Priority')) currentPriority = 'low' + else if (line.includes('### Skipped')) break + + // Match checklist items: - [ ] `path/to/file.ts` (123 lines) + const match = line.match(/- \[ \] `([^`]+)` \((\d+) lines\)/) + if (match) { + files.push({ + path: match[1], + lines: parseInt(match[2], 10), + category: currentPriority, + priority: currentPriority, + }) + } + } + + return files +} + +async function main() { + const args = process.argv.slice(2) + const dryRun = args.includes('--dry-run') || args.includes('-d') + const verbose = args.includes('--verbose') || args.includes('-v') + const priorityFilter = args.find(a => ['high', 'medium', 'low', 'all'].includes(a)) || 'high' + const limit = parseInt(args.find(a => a.startsWith('--limit='))?.split('=')[1] || '999', 10) + + console.log('๐Ÿ“‹ Loading files from tracking report...') + const allFiles = await loadFilesFromReport() + + let filesToProcess = allFiles + if (priorityFilter !== 'all') { + filesToProcess = allFiles.filter(f => f.priority === priorityFilter) + } + + filesToProcess = filesToProcess.slice(0, limit) + + console.log(`\n๐Ÿ“Š Plan:`) + console.log(` Priority filter: ${priorityFilter}`) + console.log(` Files to process: ${filesToProcess.length}`) + console.log(` Mode: ${dryRun ? 'DRY RUN (preview only)' : 'LIVE (will modify files)'}`) + + if (filesToProcess.length === 0) { + console.log('\nโš ๏ธ No files to process') + process.exit(0) + } + + // Show what will be processed + console.log(`\n๐Ÿ“ Files queued:`) + for (let i = 0; i < Math.min(10, filesToProcess.length); i++) { + console.log(` ${i + 1}. ${filesToProcess[i].path} (${filesToProcess[i].lines} lines)`) + } + if (filesToProcess.length > 10) { + console.log(` ... and ${filesToProcess.length - 10} more`) + } + + // Confirmation for live mode + if (!dryRun) { + console.log(`\nโš ๏ธ WARNING: This will modify ${filesToProcess.length} files!`) + console.log(` Press Ctrl+C to cancel, or wait 3 seconds to continue...`) + await new Promise(resolve => setTimeout(resolve, 3000)) + } + + console.log('\n๐Ÿš€ Starting refactoring...\n') + + const refactor = new BulkLambdaRefactor({ dryRun, verbose }) + const filePaths = filesToProcess.map(f => f.path) + + const results = await refactor.bulkRefactor(filePaths) + + // Save results + const resultsPath = path.join(process.cwd(), 'docs/todo/REFACTOR_RESULTS.json') + await fs.writeFile(resultsPath, JSON.stringify(results, null, 2), 'utf-8') + console.log(`\n๐Ÿ’พ Results saved to: ${resultsPath}`) + + // Update progress report + console.log('\n๐Ÿ“ Updating progress report...') + // TODO: Mark completed files in the report + + console.log('\nโœ… Batch refactoring complete!') + console.log('\nNext steps:') + console.log(' 1. Run: npm run lint:fix') + console.log(' 2. Run: npm run typecheck') + console.log(' 3. Run: npm run test:unit') + console.log(' 4. Review changes and commit') +} + +if (require.main === module) { + main().catch(console.error) +} diff --git a/tools/refactoring/bulk-lambda-refactor.ts b/tools/refactoring/bulk-lambda-refactor.ts new file mode 100644 index 000000000..996181c33 --- /dev/null +++ b/tools/refactoring/bulk-lambda-refactor.ts @@ -0,0 +1,471 @@ +#!/usr/bin/env tsx +/** + * Bulk Lambda-per-File Refactoring Tool + * + * Automatically refactors TypeScript files into lambda-per-file structure: + * 1. Analyzes file to extract functions/methods + * 2. Creates functions/ subdirectory + * 3. Extracts each function to its own file + * 4. Creates class wrapper + * 5. Updates imports + * 6. Runs linter to fix issues + */ + +import * as fs from 'fs/promises' +import * as path from 'path' +import { exec } from 'child_process' +import { promisify } from 'util' + +const execAsync = promisify(exec) + +interface FunctionInfo { + name: string + isAsync: boolean + isExported: boolean + params: string + returnType: string + body: string + startLine: number + endLine: number + comments: string[] + isMethod: boolean +} + +interface RefactorResult { + success: boolean + originalFile: string + newFiles: string[] + errors: string[] +} + +class BulkLambdaRefactor { + private dryRun: boolean = false + private verbose: boolean = false + + constructor(options: { dryRun?: boolean; verbose?: boolean } = {}) { + this.dryRun = options.dryRun || false + this.verbose = options.verbose || false + } + + private log(message: string) { + if (this.verbose) { + console.log(message) + } + } + + /** + * Extract functions from a TypeScript file + */ + async extractFunctions(filePath: string): Promise { + const content = await fs.readFile(filePath, 'utf-8') + const lines = content.split('\n') + const functions: FunctionInfo[] = [] + + // Simple regex-based extraction (can be improved with AST parsing) + const functionRegex = /^(export\s+)?(async\s+)?function\s+([a-zA-Z0-9_]+)\s*(\([^)]*\))(\s*:\s*[^{]+)?\s*\{/ + const methodRegex = /^\s*(public|private|protected)?\s*(async\s+)?([a-zA-Z0-9_]+)\s*(\([^)]*\))(\s*:\s*[^{]+)?\s*\{/ + + let i = 0 + while (i < lines.length) { + const line = lines[i] + + // Try to match function + const funcMatch = line.match(functionRegex) + const methodMatch = line.match(methodRegex) + + if (funcMatch || methodMatch) { + const isMethod = !!methodMatch + const match = funcMatch || methodMatch! + + const isExported = !!match[1] + const isAsync = !!(funcMatch ? match[2] : methodMatch![2]) + const name = funcMatch ? match[3] : methodMatch![3] + const params = funcMatch ? match[4] : methodMatch![4] + const returnType = (funcMatch ? match[5] : methodMatch![5]) || '' + + // Collect comments above function + const comments: string[] = [] + let commentLine = i - 1 + while (commentLine >= 0 && (lines[commentLine].trim().startsWith('//') || + lines[commentLine].trim().startsWith('*') || + lines[commentLine].trim().startsWith('/*'))) { + comments.unshift(lines[commentLine]) + commentLine-- + } + + // Find matching closing brace + let braceCount = 1 + let bodyStart = i + 1 + let j = i + let bodyLines: string[] = [line] + + // Count braces to find function end + j++ + while (j < lines.length && braceCount > 0) { + bodyLines.push(lines[j]) + for (const char of lines[j]) { + if (char === '{') braceCount++ + if (char === '}') braceCount-- + if (braceCount === 0) break + } + j++ + } + + functions.push({ + name, + isAsync, + isExported, + params, + returnType: returnType.trim(), + body: bodyLines.join('\n'), + startLine: i, + endLine: j - 1, + comments, + isMethod, + }) + + i = j + } else { + i++ + } + } + + return functions + } + + /** + * Extract imports and types from original file + */ + async extractImportsAndTypes(filePath: string): Promise<{ imports: string[]; types: string[] }> { + const content = await fs.readFile(filePath, 'utf-8') + const lines = content.split('\n') + + const imports: string[] = [] + const types: string[] = [] + + let inImport = false + let currentImport = '' + + for (const line of lines) { + const trimmed = line.trim() + + // Handle multi-line imports + if (trimmed.startsWith('import ') || inImport) { + currentImport += line + '\n' + if (trimmed.includes('}') || (!trimmed.includes('{') && trimmed.endsWith("'"))) { + imports.push(currentImport.trim()) + currentImport = '' + inImport = false + } else { + inImport = true + } + } + + // Extract type definitions + if (trimmed.startsWith('export type ') || trimmed.startsWith('export interface ') || + trimmed.startsWith('type ') || trimmed.startsWith('interface ')) { + types.push(line) + } + } + + return { imports, types } + } + + /** + * Generate individual function file + */ + generateFunctionFile(func: FunctionInfo, imports: string[], types: string[]): string { + let content = '' + + // Add relevant imports (simplified - could be smarter about which imports are needed) + if (imports.length > 0) { + content += imports.join('\n') + '\n\n' + } + + // Add comments + if (func.comments.length > 0) { + content += func.comments.join('\n') + '\n' + } + + // Add function + const asyncKeyword = func.isAsync ? 'async ' : '' + const exportKeyword = 'export ' + + content += `${exportKeyword}${asyncKeyword}function ${func.name}${func.params}${func.returnType} {\n` + + // Extract function body (remove first and last line which are the function declaration and closing brace) + const bodyLines = func.body.split('\n') + const actualBody = bodyLines.slice(1, -1).join('\n') + + content += actualBody + '\n' + content += '}\n' + + return content + } + + /** + * Generate class wrapper file + */ + generateClassWrapper(className: string, functions: FunctionInfo[], functionsDir: string): string { + let content = '' + + // Import all functions + content += `// Auto-generated class wrapper\n` + for (const func of functions) { + const kebabName = func.name.replace(/([A-Z])/g, '-$1').toLowerCase().replace(/^-/, '') + content += `import { ${func.name} } from './${functionsDir}/${kebabName}'\n` + } + + content += `\n/**\n` + content += ` * ${className} - Class wrapper for ${functions.length} functions\n` + content += ` * \n` + content += ` * This is a convenience wrapper. Prefer importing individual functions.\n` + content += ` */\n` + content += `export class ${className} {\n` + + // Add static methods + for (const func of functions) { + const asyncKeyword = func.isAsync ? 'async ' : '' + content += ` static ${asyncKeyword}${func.name}${func.params}${func.returnType} {\n` + content += ` return ${func.isAsync ? 'await ' : ''}${func.name}(...arguments as any)\n` + content += ` }\n\n` + } + + content += '}\n' + + return content + } + + /** + * Generate index file that re-exports everything + */ + generateIndexFile(functions: FunctionInfo[], functionsDir: string, className: string): string { + let content = '' + + content += `// Auto-generated re-exports for backward compatibility\n\n` + + // Re-export all functions + for (const func of functions) { + const kebabName = func.name.replace(/([A-Z])/g, '-$1').toLowerCase().replace(/^-/, '') + content += `export { ${func.name} } from './${functionsDir}/${kebabName}'\n` + } + + // Re-export class wrapper + content += `\n// Class wrapper for convenience\n` + content += `export { ${className} } from './${className}'\n` + + return content + } + + /** + * Refactor a single file + */ + async refactorFile(filePath: string): Promise { + const result: RefactorResult = { + success: false, + originalFile: filePath, + newFiles: [], + errors: [], + } + + try { + this.log(`\n๐Ÿ” Analyzing ${filePath}...`) + + // Extract functions + const functions = await this.extractFunctions(filePath) + + if (functions.length === 0) { + result.errors.push('No functions found to extract') + return result + } + + // Skip if only 1-2 functions (not worth refactoring) + if (functions.length <= 2) { + result.errors.push(`Only ${functions.length} function(s) - skipping`) + return result + } + + this.log(` Found ${functions.length} functions: ${functions.map(f => f.name).join(', ')}`) + + // Extract imports and types + const { imports, types } = await this.extractImportsAndTypes(filePath) + + // Create directories + const dir = path.dirname(filePath) + const basename = path.basename(filePath, path.extname(filePath)) + const functionsDir = path.join(dir, basename, 'functions') + + if (!this.dryRun) { + await fs.mkdir(functionsDir, { recursive: true }) + } + + this.log(` Creating functions directory: ${functionsDir}`) + + // Generate function files + for (const func of functions) { + const kebabName = func.name.replace(/([A-Z])/g, '-$1').toLowerCase().replace(/^-/, '') + const funcFilePath = path.join(functionsDir, `${kebabName}.ts`) + const content = this.generateFunctionFile(func, imports, types) + + if (!this.dryRun) { + await fs.writeFile(funcFilePath, content, 'utf-8') + } + + result.newFiles.push(funcFilePath) + this.log(` โœ“ ${kebabName}.ts`) + } + + // Generate class wrapper + const className = basename.split('-').map(w => w.charAt(0).toUpperCase() + w.slice(1)).join('') + 'Utils' + const classFilePath = path.join(dir, basename, `${className}.ts`) + const classContent = this.generateClassWrapper(className, functions, 'functions') + + if (!this.dryRun) { + await fs.writeFile(classFilePath, classContent, 'utf-8') + } + + result.newFiles.push(classFilePath) + this.log(` โœ“ ${className}.ts (class wrapper)`) + + // Generate index file + const indexFilePath = path.join(dir, basename, 'index.ts') + const indexContent = this.generateIndexFile(functions, 'functions', className) + + if (!this.dryRun) { + await fs.writeFile(indexFilePath, indexContent, 'utf-8') + } + + result.newFiles.push(indexFilePath) + this.log(` โœ“ index.ts (re-exports)`) + + // Update original file to re-export from new location + const reexportContent = `// This file has been refactored into modular functions\n` + + `// Import from individual functions or use the class wrapper\n\n` + + `export * from './${basename}'\n` + + if (!this.dryRun) { + await fs.writeFile(filePath, reexportContent, 'utf-8') + } + + this.log(` โœ“ Updated ${path.basename(filePath)} to re-export`) + + result.success = true + this.log(` โœ… Successfully refactored into ${result.newFiles.length} files`) + + } catch (error) { + result.errors.push(`Error: ${error instanceof Error ? error.message : String(error)}`) + this.log(` โŒ Failed: ${result.errors[0]}`) + } + + return result + } + + /** + * Run linter and fix imports + */ + async runLintFix(workingDir: string): Promise { + this.log('\n๐Ÿ”ง Running ESLint to fix imports and formatting...') + + try { + const { stdout, stderr } = await execAsync('npm run lint:fix', { cwd: workingDir }) + if (stdout) this.log(stdout) + if (stderr) this.log(stderr) + this.log(' โœ… Linting completed') + } catch (error) { + this.log(` โš ๏ธ Linting had issues (may be expected): ${error}`) + } + } + + /** + * Bulk refactor multiple files + */ + async bulkRefactor(files: string[]): Promise { + console.log(`\n๐Ÿ“ฆ Bulk Lambda Refactoring Tool`) + console.log(` Mode: ${this.dryRun ? 'DRY RUN' : 'LIVE'}`) + console.log(` Files to process: ${files.length}\n`) + + const results: RefactorResult[] = [] + let successCount = 0 + let skipCount = 0 + let errorCount = 0 + + for (let i = 0; i < files.length; i++) { + const file = files[i] + console.log(`[${i + 1}/${files.length}] Processing: ${file}`) + + const result = await this.refactorFile(file) + results.push(result) + + if (result.success) { + successCount++ + } else if (result.errors.some(e => e.includes('skipping'))) { + skipCount++ + } else { + errorCount++ + } + + // Small delay to avoid overwhelming the system + await new Promise(resolve => setTimeout(resolve, 100)) + } + + console.log(`\n๐Ÿ“Š Summary:`) + console.log(` โœ… Success: ${successCount}`) + console.log(` โญ๏ธ Skipped: ${skipCount}`) + console.log(` โŒ Errors: ${errorCount}`) + console.log(` ๐Ÿ“ Total new files: ${results.reduce((acc, r) => acc + r.newFiles.length, 0)}`) + + return results + } +} + +// CLI +async function main() { + const args = process.argv.slice(2) + + const dryRun = args.includes('--dry-run') || args.includes('-d') + const verbose = args.includes('--verbose') || args.includes('-v') + const filesArg = args.find(arg => !arg.startsWith('-')) + + if (!filesArg && !args.includes('--help') && !args.includes('-h')) { + console.log('Usage: tsx bulk-lambda-refactor.ts [options] ') + console.log('\nOptions:') + console.log(' -d, --dry-run Preview changes without writing files') + console.log(' -v, --verbose Show detailed output') + console.log(' -h, --help Show this help') + console.log('\nExamples:') + console.log(' tsx bulk-lambda-refactor.ts --dry-run "frontends/nextjs/src/lib/**/*.ts"') + console.log(' tsx bulk-lambda-refactor.ts --verbose frontends/nextjs/src/lib/rendering/page/page-definition-builder.ts') + process.exit(1) + } + + if (args.includes('--help') || args.includes('-h')) { + console.log('Bulk Lambda-per-File Refactoring Tool\n') + console.log('Automatically refactors TypeScript files into lambda-per-file structure.') + console.log('\nUsage: tsx bulk-lambda-refactor.ts [options] ') + console.log('\nOptions:') + console.log(' -d, --dry-run Preview changes without writing files') + console.log(' -v, --verbose Show detailed output') + console.log(' -h, --help Show this help') + process.exit(0) + } + + const refactor = new BulkLambdaRefactor({ dryRun, verbose }) + + // For now, process single file (can be extended to glob patterns) + const files = [filesArg!] + + const results = await refactor.bulkRefactor(files) + + if (!dryRun && results.some(r => r.success)) { + console.log('\n๐Ÿ”ง Running linter to fix imports...') + await refactor.runLintFix(process.cwd()) + } + + console.log('\nโœจ Done!') +} + +if (require.main === module) { + main().catch(console.error) +} + +export { BulkLambdaRefactor } diff --git a/tools/refactoring/orchestrate-refactor.ts b/tools/refactoring/orchestrate-refactor.ts new file mode 100644 index 000000000..d21e0c013 --- /dev/null +++ b/tools/refactoring/orchestrate-refactor.ts @@ -0,0 +1,249 @@ +#!/usr/bin/env tsx +/** + * Master Refactoring Orchestrator + * + * Orchestrates the complete lambda-per-file refactoring process: + * 1. Loads files from tracking report + * 2. Refactors in priority order + * 3. Runs linter and fixes imports + * 4. Runs type checking + * 5. Updates progress report + */ + +import { ASTLambdaRefactor } from './ast-lambda-refactor' +import * as fs from 'fs/promises' +import * as path from 'path' +import { exec } from 'child_process' +import { promisify } from 'util' + +const execAsync = promisify(exec) + +interface FileToProcess { + path: string + lines: number + priority: 'high' | 'medium' | 'low' + status: 'pending' | 'completed' | 'failed' | 'skipped' + error?: string +} + +async function loadFilesFromReport(): Promise { + const reportPath = path.join(process.cwd(), 'docs/todo/LAMBDA_REFACTOR_PROGRESS.md') + const content = await fs.readFile(reportPath, 'utf-8') + + const files: FileToProcess[] = [] + const lines = content.split('\n') + + let currentPriority: 'high' | 'medium' | 'low' = 'high' + + for (const line of lines) { + if (line.includes('### High Priority')) currentPriority = 'high' + else if (line.includes('### Medium Priority')) currentPriority = 'medium' + else if (line.includes('### Low Priority')) currentPriority = 'low' + else if (line.includes('### Skipped')) break + + const match = line.match(/- \[ \] `([^`]+)` \((\d+) lines\)/) + if (match) { + files.push({ + path: match[1], + lines: parseInt(match[2], 10), + priority: currentPriority, + status: 'pending', + }) + } + } + + return files +} + +async function runCommand(cmd: string, cwd: string = process.cwd()): Promise<{ stdout: string; stderr: string }> { + try { + return await execAsync(cmd, { cwd, maxBuffer: 10 * 1024 * 1024 }) + } catch (error: any) { + return { stdout: error.stdout || '', stderr: error.stderr || error.message } + } +} + +async function main() { + const args = process.argv.slice(2) + const dryRun = args.includes('--dry-run') || args.includes('-d') + const priorityFilter = args.find(a => ['high', 'medium', 'low', 'all'].includes(a)) || 'all' + const limitArg = args.find(a => a.startsWith('--limit=')) + const limit = limitArg ? parseInt(limitArg.split('=')[1], 10) : 999 + const skipLint = args.includes('--skip-lint') + const skipTest = args.includes('--skip-test') + + console.log('๐Ÿš€ Lambda-per-File Refactoring Orchestrator\n') + + // Load files + console.log('๐Ÿ“‹ Loading files from tracking report...') + let files = await loadFilesFromReport() + + if (priorityFilter !== 'all') { + files = files.filter(f => f.priority === priorityFilter) + } + + files = files.slice(0, limit) + + console.log(`\n๐Ÿ“Š Configuration:`) + console.log(` Priority: ${priorityFilter}`) + console.log(` Limit: ${limit}`) + console.log(` Files to process: ${files.length}`) + console.log(` Mode: ${dryRun ? '๐Ÿ” DRY RUN (preview only)' : 'โšก LIVE (will modify files)'}`) + console.log(` Skip lint: ${skipLint}`) + console.log(` Skip tests: ${skipTest}`) + + if (files.length === 0) { + console.log('\nโš ๏ธ No files to process') + return + } + + // Show preview + console.log(`\n๐Ÿ“ Files queued:`) + const preview = files.slice(0, 10) + preview.forEach((f, i) => { + console.log(` ${i + 1}. [${f.priority.toUpperCase()}] ${f.path} (${f.lines} lines)`) + }) + if (files.length > 10) { + console.log(` ... and ${files.length - 10} more`) + } + + // Safety confirmation for live mode + if (!dryRun) { + console.log(`\nโš ๏ธ WARNING: This will refactor ${files.length} files!`) + console.log(' Press Ctrl+C to cancel, or wait 5 seconds to continue...') + await new Promise(resolve => setTimeout(resolve, 5000)) + } + + console.log('\n' + '='.repeat(60)) + console.log('PHASE 1: REFACTORING') + console.log('='.repeat(60) + '\n') + + // Refactor files + const refactor = new ASTLambdaRefactor({ dryRun, verbose: true }) + + for (let i = 0; i < files.length; i++) { + const file = files[i] + console.log(`\n[${i + 1}/${files.length}] Processing: ${file.path}`) + + try { + await refactor.refactorFile(file.path) + file.status = 'completed' + } catch (error) { + const errorMsg = error instanceof Error ? error.message : String(error) + if (errorMsg.includes('skipping') || errorMsg.includes('No functions')) { + file.status = 'skipped' + file.error = errorMsg + } else { + file.status = 'failed' + file.error = errorMsg + console.error(` โŒ Error: ${errorMsg}`) + } + } + + // Small delay to avoid overwhelming system + await new Promise(resolve => setTimeout(resolve, 100)) + } + + // Summary + const summary = { + total: files.length, + completed: files.filter(f => f.status === 'completed').length, + skipped: files.filter(f => f.status === 'skipped').length, + failed: files.filter(f => f.status === 'failed').length, + } + + console.log('\n' + '='.repeat(60)) + console.log('REFACTORING SUMMARY') + console.log('='.repeat(60)) + console.log(` โœ… Completed: ${summary.completed}`) + console.log(` โญ๏ธ Skipped: ${summary.skipped}`) + console.log(` โŒ Failed: ${summary.failed}`) + console.log(` ๐Ÿ“Š Total: ${summary.total}`) + + if (!dryRun && summary.completed > 0) { + // Phase 2: Linting + if (!skipLint) { + console.log('\n' + '='.repeat(60)) + console.log('PHASE 2: LINTING & IMPORT FIXING') + console.log('='.repeat(60) + '\n') + + console.log('๐Ÿ”ง Running ESLint with --fix...') + const lintResult = await runCommand('npm run lint:fix') + console.log(lintResult.stdout) + if (lintResult.stderr && !lintResult.stderr.includes('warning')) { + console.log('โš ๏ธ Lint stderr:', lintResult.stderr) + } + console.log(' โœ… Linting complete') + } + + // Phase 3: Type checking + console.log('\n' + '='.repeat(60)) + console.log('PHASE 3: TYPE CHECKING') + console.log('='.repeat(60) + '\n') + + console.log('๐Ÿ” Running TypeScript compiler check...') + const typecheckResult = await runCommand('npm run typecheck') + + if (typecheckResult.stderr.includes('error TS')) { + console.log('โŒ Type errors detected:') + console.log(typecheckResult.stderr.split('\n').slice(0, 20).join('\n')) + console.log('\nโš ๏ธ Please fix type errors before committing') + } else { + console.log(' โœ… No type errors') + } + + // Phase 4: Testing + if (!skipTest) { + console.log('\n' + '='.repeat(60)) + console.log('PHASE 4: TESTING') + console.log('='.repeat(60) + '\n') + + console.log('๐Ÿงช Running unit tests...') + const testResult = await runCommand('npm run test:unit -- --run') + + if (testResult.stderr.includes('FAIL') || testResult.stdout.includes('FAIL')) { + console.log('โŒ Some tests failed') + console.log(testResult.stdout.split('\n').slice(-30).join('\n')) + } else { + console.log(' โœ… All tests passed') + } + } + } + + // Save detailed results + const resultsPath = path.join(process.cwd(), 'docs/todo/REFACTOR_RESULTS.json') + await fs.writeFile(resultsPath, JSON.stringify(files, null, 2), 'utf-8') + console.log(`\n๐Ÿ’พ Detailed results saved: ${resultsPath}`) + + // Final instructions + console.log('\n' + '='.repeat(60)) + console.log('โœจ REFACTORING COMPLETE!') + console.log('='.repeat(60)) + + if (dryRun) { + console.log('\n๐Ÿ“Œ This was a DRY RUN. No files were modified.') + console.log(' Run without --dry-run to apply changes.') + } else { + console.log('\n๐Ÿ“Œ Next Steps:') + console.log(' 1. Review the changes: git diff') + console.log(' 2. Fix any type errors if needed') + console.log(' 3. Run tests: npm run test:unit') + console.log(' 4. Commit: git add . && git commit -m "Refactor to lambda-per-file structure"') + } + + console.log(`\n๐Ÿ“Š Final Stats:`) + console.log(` Files refactored: ${summary.completed}`) + console.log(` Files skipped: ${summary.skipped}`) + console.log(` Files failed: ${summary.failed}`) + + if (summary.failed > 0) { + console.log(`\nโŒ Failed files:`) + files.filter(f => f.status === 'failed').forEach(f => { + console.log(` - ${f.path}: ${f.error}`) + }) + } +} + +if (require.main === module) { + main().catch(console.error) +}