Document workflow engine and add reorganization summary

Co-authored-by: johndoe6345789 <224850594+johndoe6345789@users.noreply.github.com>

REORGANIZATION_SUMMARY.md

@@ -0,0 +1,163 @@
+# Repository Reorganization Summary
+
+## Overview
+This document summarizes the reorganization of the AutoMetabuilder repository, focusing on organizing workflow plugins and converting additional functionality into workflow plugins (dogfooding approach).
+
+## Changes Made
+
+### 1. Workflow Plugins Organized into Subdirectories
+
+All 78 existing workflow plugins have been organized from a flat structure into 12 category-based subdirectories:
+
+```
+backend/autometabuilder/workflow/plugins/
+├── backend/      - Backend infrastructure (11 plugins)
+├── core/         - Core workflow orchestration (7 plugins)
+├── tools/        - Tool execution (7 plugins)
+├── logic/        - Logic operations (9 plugins)
+├── list/         - List operations (7 plugins)
+├── dict/         - Dictionary operations (6 plugins)
+├── string/       - String manipulation (8 plugins)
+├── math/         - Math operations (10 plugins)
+├── convert/      - Type conversions (7 plugins)
+├── control/      - Control flow (1 plugin)
+├── var/          - Variable management (4 plugins)
+└── utils/        - Utilities (7 plugins)
+```
+
+**Total: 84 plugins** (78 existing + 6 new)
+
+### 2. New Workflow Plugins Created (Dogfooding)
+
+Six new workflow plugins were created to expose backend functionality as workflow nodes:
+
+#### Backend Plugins
+- **`backend.parse_cli_args`** - Parse command line arguments (from `cli_args.py`)
+- **`backend.load_env`** - Load environment variables (from `env_loader.py`)
+- **`backend.load_tool_registry`** - Load tool registry (from `tool_registry_loader.py`)
+
+#### Tools Plugins
+- **`tools.run_docker`** - Run commands in Docker containers (from `docker_utils.py`)
+
+#### Utils Plugins
+- **`utils.check_mvp`** - Check if MVP is reached (from `roadmap_utils.py`)
+- **`utils.update_roadmap`** - Update roadmap file (from `roadmap_utils.py`)
+
+### 3. Import Path Updates
+
+- Updated `plugin_map.json` with new subdirectory-based paths (84 entries)
+- Fixed relative imports in 18 plugin files to account for new directory structure
+- All plugin imports use `....` to reach the parent `autometabuilder` package
+
+Example:
+```python
+# Before: from ...metadata_loader import load_metadata
+# After:  from ....metadata_loader import load_metadata
+```
+
+### 4. Workflow Engine Documentation
+
+Enhanced `workflow/__init__.py` to document the workflow engine architecture:
+- Core execution modules
+- N8N support modules
+- Plugin system
+- Utility modules
+
+### 5. Plugin Documentation Updates
+
+Updated `plugins/README.md` to reflect:
+- New directory structure with plugin counts
+- Documentation for 6 new plugins
+- Clear organization by category
+
+## Benefits
+
+### 1. Better Organization
+- Plugins are now grouped by functionality
+- Easy to find related plugins
+- Clear separation of concerns
+- Scalable structure for future plugins
+
+### 2. Dogfooding Success
+- Core backend functionality now available as workflow nodes
+- Demonstrates the power of the workflow plugin system
+- Enables declarative backend initialization
+- CLI argument parsing can be part of workflows
+
+### 3. Maintainability
+- Each subdirectory has its own `__init__.py`
+- Import paths clearly show module relationships
+- Easier to add new plugins (just place in appropriate directory)
+- Better IDE support with organized structure
+
+### 4. Consistency
+- All plugins follow the same pattern
+- Consistent naming conventions maintained
+- Uniform documentation style
+
+## Files Modified
+
+### Created (19 files)
+- 12 `__init__.py` files (one per subdirectory)
+- 6 new workflow plugin files
+- 1 reorganization summary document (this file)
+
+### Modified (3 files)
+- `plugin_map.json` - Updated all 84 plugin paths
+- `plugins/README.md` - Added directory structure and new plugin docs
+- `workflow/__init__.py` - Enhanced module documentation
+
+### Moved (78 files)
+- All existing workflow plugins moved to subdirectories
+- 18 plugins had imports updated
+
+## Testing
+
+- Plugin loading tested successfully (84 plugins registered)
+- Existing unit tests pass
+- Plugin registry correctly loads from new paths
+- Import resolution works correctly
+
+## Remaining Structure
+
+The `backend/autometabuilder` directory still contains ~25 Python files:
+- Core application files (`main.py`, `app_runner.py`)
+- Utility/loader modules (used by plugins)
+- Service modules (`github_service.py`, `openai_client.py`)
+- Configuration loaders
+
+These files remain in place as they are:
+1. Core infrastructure needed by the application
+2. Utilities imported by workflow plugins
+3. Service classes that manage state
+
+This is the appropriate organization - workflow plugins expose functionality, while the underlying implementation remains in the backend package.
+
+## Plugin Count by Category
+
+| Category | Count | Description |
+|----------|-------|-------------|
+| backend  | 11    | Backend initialization and infrastructure |
+| core     | 7     | Core workflow orchestration |
+| tools    | 7     | Development tools and Docker |
+| logic    | 9     | Boolean logic and comparisons |
+| list     | 7     | Array/list operations |
+| dict     | 6     | Dictionary/object operations |
+| string   | 8     | String manipulation |
+| math     | 10    | Mathematical operations |
+| convert  | 7     | Type conversions |
+| control  | 1     | Control flow (switch) |
+| var      | 4     | Variable management |
+| utils    | 7     | General utilities |
+| **Total** | **84** | |
+
+## Next Steps
+
+This reorganization provides a solid foundation for:
+1. Adding more workflow plugins easily
+2. Building complex declarative workflows
+3. Visual workflow editors (n8n-compatible)
+4. Low-code workflow development
+5. Full dogfooding of the workflow system
+
+The workflow plugin system is now production-ready with excellent organization and comprehensive coverage of software development primitives.

backend/autometabuilder/workflow/__init__.py

@@ -1 +1,32 @@
-"""Workflow package."""
+"""Workflow engine package.
+
+This package provides a declarative workflow engine that executes n8n-style workflows.
+
+Core Modules:
+    engine.py - Main workflow engine and execution coordinator
+    runtime.py - Runtime context and state management
+    
+Execution:
+    n8n_executor.py - N8N workflow format executor
+    node_executor.py - Individual node execution
+    execution_order.py - Topological sort for node execution order
+    loop_executor.py - Loop iteration execution
+    
+N8N Support:
+    n8n_schema.py - N8N workflow schema definitions
+    n8n_converter.py - Convert legacy workflows to N8N format
+    workflow_adapter.py - Workflow format adapter
+    
+Plugin System:
+    plugin_loader.py - Plugin loading utilities
+    plugin_registry.py - Plugin registration and discovery
+    plugin_map.json - Plugin name to module path mapping
+    plugins/ - Organized workflow plugins by category
+    
+Utilities:
+    input_resolver.py - Input value resolution and variable binding
+    value_helpers.py - Value type checking and conversion helpers
+    tool_runner.py - Tool execution wrapper
+    tool_calls_handler.py - AI tool calls processing
+"""
+