mirror of
https://github.com/johndoe6345789/metabuilder.git
synced 2026-04-24 13:54:57 +00:00
johndoe6345789
38ab84b632
Implemented two production-ready TypeScript modules for the workflow executor:
1. plugin-validator.ts (1023 lines)
- Schema validation against plugin metadata
- JSON schema validation with type checking
- Plugin compatibility checking (versions, dependencies, credentials)
- Pre-execution validation (parameters, credentials, context)
- Error type mapping with structured ErrorType enum
- Comprehensive JSDoc documentation
- Singleton pattern for global validator instance
2. error-recovery.ts (791 lines)
- Error recovery strategies: fallback, skip, retry, fail
- Exponential backoff with configurable multiplier and max delay
- Comprehensive metrics tracking:
* Error counts by type, node type, and strategy
* Recovery success/failure tracking
* Average recovery time calculation
* Error state persistence (up to 500 states)
- Recovery attempt recording with detailed audit trail
- Error statistics and reporting
- Singleton pattern for global recovery manager instance
- Full JSDoc with parameter and return documentation
Key Features:
- Multi-tenant awareness in error context tracking
- Jitter in backoff calculations to prevent thundering herd
- Structured error mapping for robust error handling
- Memory-bounded history tracking (MAX_RECOVERY_HISTORY, MAX_ERROR_STATES)
- Production-ready error handling with recoverable/non-recoverable classification
- Comprehensive metrics export for monitoring and debugging
Testing: Compiles cleanly with TypeScript ES2020 target
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Workflow Executor Runtimes
This folder contains language-specific runtime executors for the workflow engine.
Structure
executor/
├── cpp/ # C++ runtime (high-performance)
├── python/ # Python runtime (AI/ML capabilities)
└── ts/ # TypeScript runtime + core engine
├── executor/ # DAG executor
├── registry/ # Plugin registry
├── utils/ # Priority queue, template engine
├── types.ts # Type definitions
└── index.ts # Main exports
Purpose
Each runtime provides the execution environment for plugins written in that language:
TypeScript Runtime (ts/)
- Contains the core engine (DAG executor, registry, utils)
- Default runtime for orchestration
- Direct JavaScript/TypeScript execution
- Full type safety
- Fastest startup time
Python Runtime (python/)
- Child process execution
- AI/ML library access (TensorFlow, PyTorch, transformers)
- Data science capabilities (pandas, numpy)
- NLP processing (spaCy, NLTK)
C++ Runtime (cpp/)
- Native FFI bindings
- 100-1000x faster than TypeScript
- Low memory footprint
- Ideal for bulk data processing
How It Works
┌─────────────────────────────────────────┐
│ DAGExecutor (TypeScript Core) │
│ - Orchestrates workflow execution │
│ - Resolves dependencies │
│ - Manages execution state │
└─────────────────┬───────────────────────┘
│
┌─────────┼─────────┐
│ │ │
↓ ↓ ↓
┌────────┬────────┬────────┐
│ TS │ C++ │ Python │
│Runtime │Runtime │Runtime │
└────────┴────────┴────────┘
│ │ │
↓ ↓ ↓
┌────────┬────────┬────────┐
│Direct │Native │Child │
│Import │FFI │Process │
└────────┴────────┴────────┘
Adding a New Runtime
- Create folder:
executor/{language}/ - Implement
PluginLoaderinterface - Register loader in
ts/registry/node-executor-registry.ts - Add plugins to
plugins/{language}/