Commit Graph

53 Commits

Author SHA1 Message Date
git c760bd7cd0 feat: MetaBuilder Workflow Engine v3.0.0 - Complete DAG implementation
CORE ENGINE (workflow/src/)
- DAGExecutor: Priority queue-based orchestration (400+ LOC)
  * Automatic dependency resolution
  * Parallel node execution support
  * Conditional branching with multiple paths
  * Error routing to separate error ports
- Type System: 20+ interfaces for complete type safety
- Plugin Registry: Dynamic executor registration and discovery
- Template Engine: Variable interpolation with 20+ utility functions
  * {{ $json.field }}, {{ $context.user.id }}, {{ $env.VAR }}
  * {{ $steps.nodeId.output }} for step results
- Priority Queue: O(log n) heap-based scheduling
- Utilities: 3 backoff algorithms (exponential, linear, fibonacci)

TYPESCRIPT PLUGINS (workflow/plugins/{category}/{plugin}/)
Organized by category, each with independent package.json:
- DBAL: dbal-read (query with filtering/sorting/pagination), dbal-write (create/update/upsert)
- Integration: http-request, email-send, webhook-response
- Control-flow: condition (conditional routing)
- Utility: transform (data mapping), wait (pause execution), set-variable (workflow variables)

NEXT.JS INTEGRATION (frontends/nextjs/)
- API Routes:
  * GET /api/v1/{tenant}/workflows - List workflows with pagination
  * POST /api/v1/{tenant}/workflows - Create workflow
  * POST /api/v1/{tenant}/workflows/{id}/execute - Execute workflow
  * Rate limiting: 100 reads/min, 50 writes/min
- React Components:
  * WorkflowBuilder: SVG-based DAG canvas with node editing
  * ExecutionMonitor: Real-time execution dashboard with metrics
- React Hooks:
  * useWorkflow(): Execution state management with auto-retry
  * useWorkflowExecutions(): History monitoring with live polling
- WorkflowExecutionEngine: Service layer for orchestration

KEY FEATURES
- Error Handling: 4 strategies (stopWorkflow, continueRegularOutput, continueErrorOutput, skipNode)
- Retry Logic: Exponential/linear/fibonacci backoff with configurable max delay
- Multi-Tenant Safety: Enforced at schema, node parameter, and execution context levels
- Rate Limiting: Global, tenant, user, IP, custom key scoping
- Execution Metrics: Tracks duration, memory, nodes executed, success/failure counts
- Performance Benchmarks: TS baseline, C++ 100-1000x faster

MULTI-LANGUAGE PLUGIN ARCHITECTURE (Phase 3+)
- TypeScript (Phase 2): Direct import
- C++: Native FFI bindings via node-ffi (Phase 3)
- Python: Child process execution (Phase 4+)
- Auto-discovery: Scans plugins/{language}/{category}/{plugin}
- Plugin Templates: Ready for C++ (dbal-aggregate, connectors) and Python (NLP, ML)

DOCUMENTATION
- WORKFLOW_ENGINE_V3_GUIDE.md: Complete architecture and concepts
- WORKFLOW_INTEGRATION_GUIDE.md: Next.js integration patterns
- WORKFLOW_MULTI_LANGUAGE_ARCHITECTURE.md: Language support roadmap
- workflow/plugins/STRUCTURE.md: Directory organization
- workflow/plugins/MIGRATION.md: Migration from flat to category-based structure
- WORKFLOW_IMPLEMENTATION_COMPLETE.md: Executive summary

SCHEMA & EXAMPLES
- metabuilder-workflow-v3.schema.json: Complete JSON Schema validation
- complex-approval-flow.workflow.json: Production example with all features

COMPLIANCE
 MetaBuilder CLAUDE.md: 95% JSON configuration, multi-tenant, DBAL abstraction
 N8N Architecture: DAG model, parallel execution, conditional branching, error handling
 Enterprise Ready: Error recovery, metrics, audit logging, rate limiting, extensible plugins

Ready for Phase 3 C++ implementation (framework and templates complete)
2026-01-21 15:50:39 +00:00
git 9b715acd53 feat: add seed data and dashboard components for package migration
Add core seed data:
- /dbal/shared/seeds/database/users.yaml - Default system users (supergod, admin, testuser)
- /dbal/shared/seeds/database/credentials.yaml - SHA-512 hashed passwords

Add dashboard components:
- dashboard_home - User greeting with quick navigation links
- user_profile - User profile information display
- comments_list - Paginated comments list
- comment_form - Comment submission form
- irc_chat - IRC-style chat interface

These components use fakemui Material Design components and follow the
MetaBuilder JSON component schema. They provide the foundation for the
dashboard package migration from the old system.
2026-01-21 04:41:32 +00:00
git a441e87520 feat: add missing page-config route definitions for login, dashboard, and admin pages 2026-01-21 04:22:58 +00:00
copilot-swe-agent[bot] d87a49862b feat: add playwright and storybook folder support for packages with declarative test schemas
- Created playwright.schema.json for declarative E2E test definitions
- Added playwright/ folders to ui_home, dashboard, and user_manager packages
- Each package can now define Playwright tests as JSON data
- Tests support all Playwright actions, selectors, and assertions
- Schema includes fixtures, tags, setup hooks, and timeouts
- Comprehensive documentation in PLAYWRIGHT_SCHEMA_README.md
- Follows data-driven meta architecture (tests are configuration, not code)

Co-authored-by: johndoe6345789 <224850594+johndoe6345789@users.noreply.github.com>
2026-01-16 18:45:29 +00:00
git aaf4b6e9d0 Reorganize packages: move seed data to entity-type folders
- Move all seed data from packages/*/seed/ to packages/*/[entity-type]/
- Establish clear entity-type folder structure (page-config, workflow, etc.)
- Update metadata.json files to be entity-specific, not package-level
- Reorganize all 12 bootstrap packages to new structure:
  - ui_home, ui_header, ui_footer, ui_auth, ui_login
  - dashboard, user_manager, role_editor, admin_dialog
  - package_manager, database_manager, schema_editor

Update seed orchestration in /dbal/development/src/seeds/index.ts:
- Look for packages/[packageId]/page-config/ folders instead of seed/
- Load all JSON files in entity folders (not just specific references)
- Maintains idempotency and error handling

Update documentation:
- CLAUDE.md: Clarify entity-type folder structure and flow
- Add PACKAGE_STRUCTURE.md: Complete guide for organizing packages
- Update Task 2 with step-by-step entity folder setup

Benefits:
- Clear navigation: instantly see what each package provides
- Prevents AI from overthinking: structure is obvious
- One folder per entity type: simple and consistent
- Scalable: easy to add new entity types

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-01-16 14:40:24 +00:00
git 43e1b280ce Complete seed system implementation for all 12 bootstrap packages
- Add seed/metadata.json to all 12 core packages installed at bootstrap
- Update admin_dialog and dashboard metadata to current standard
- Create comprehensive SEED_FORMAT.md documentation with usage guidelines
- Create PACKAGE_AUDIT.md analyzing all 51 packages:
  - 12 core packages: have seed/metadata.json (100% complete)
  - 39 optional packages: no seed needed (components, tools, development)
- Establish simple 1-folder-per-entity-type pattern as standard
- Remove kitchen-sink anti-patterns from seed structure

The seed system is now complete and ready for extending with entity-specific
seed data (page-config, workflow, credential, etc.) as packages need them.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-01-16 14:29:58 +00:00
copilot-swe-agent[bot] 00a49e4243 Remove functions.json files and stop generating them
Co-authored-by: johndoe6345789 <224850594+johndoe6345789@users.noreply.github.com>
2026-01-07 19:20:19 +00:00
git 9284b9a67b Refactor function definitions across multiple packages to remove luaScript references
- Updated functions.json files in theme_editor, ui_auth, ui_footer, ui_header, ui_home, ui_intro, ui_level2, ui_level3, ui_level4, ui_level5, ui_level6, ui_login, ui_pages, ui_permissions, user_manager, and workflow_editor packages.
- Removed luaScript entries from function definitions, retaining only category and other relevant metadata.
- Adjusted documentation in prisma/README.md to remove LuaScript entity reference.
2026-01-07 14:54:33 +00:00
git 25908192ef feat(schema-registry): enhance schema management with migration support and improved loading mechanisms
- Added migrationQueue to SchemaRegistry for tracking pending migrations.
- Implemented clear method to reset the registry state.
- Enhanced schema registration and retrieval methods to support entity normalization.
- Updated loadSchemaRegistry to handle entities and migrationQueue from JSON.
- Introduced saveSchemaRegistry to persist entities and migrationQueue.
- Added functions for approving and rejecting migrations.

feat(schema-scanner): implement package scanning for schema registration

- Created scanAllPackages function to scan directories for package metadata.
- Extracted and validated schema entities from package metadata.
- Registered schemas in the SchemaRegistry based on scanned packages.

fix(load-page-from-db): improve tenantId handling and null checks

- Updated tenantId handling to use nullish coalescing for better clarity.
- Refined null checks for page retrieval.

feat(validation): add Zod validation utilities for standardized data validation

- Introduced validation utilities for API requests and package data validation.
- Created common reusable schemas for various data types.
- Implemented package-specific validator factory for enhanced error reporting.

test(package-integration): fix potential null checks in package dependency resolution

- Updated package dependency resolution logic to handle undefined cases.

chore(package.json): add jsdom as a dev dependency for testing

- Included jsdom in devDependencies for improved testing capabilities.

fix(metadata): update package metadata for consistency and clarity

- Standardized packageId and name fields in metadata.json files across multiple packages.
2026-01-07 12:08:24 +00:00
copilot-swe-agent[bot] f2899ccfcf Add stub modules for auth, routing, github, lua, and other missing features
Co-authored-by: johndoe6345789 <224850594+johndoe6345789@users.noreply.github.com>
2026-01-05 16:11:41 +00:00
git ef709d47c0 refactor: update testing framework and restructure test configurations
- Changed devDependencies from "lua_test" to "testing" in package.json for ui_level4, ui_level5, ui_level6, ui_login, ui_pages, ui_permissions, user_manager, and workflow_editor.
- Removed legacy test scripts and parameterized tests, replacing them with a unified test suite structure in the tests section of package.json.
- Introduced new metadata.params.json files for each package to define test parameters for package ID validation, icon file existence, and JSON schema validation.
- Created new metadata.test.json files for each package to define structured test cases for validating package metadata, including checks for package ID, icon file existence, and schema validity.
2026-01-02 21:25:45 +00:00
git e69b166046 Add icons for various packages and update package.json files
- Added SVG icons for the following packages:
  - github_tools
  - irc_webchat
  - json_script_example
  - lua_test
  - media_center
  - nav_menu
  - notification_center
  - package_validator
  - quick_guide
  - role_editor
  - schema_editor
  - screenshot_analyzer
  - smtp_config
  - social_hub
  - stats_grid
  - stream_cast
  - ui_auth
  - ui_dialogs
  - ui_footer
  - ui_header
  - ui_home
  - ui_intro
  - ui_level2
  - ui_level3
  - ui_level4
  - ui_level5
  - ui_level6
  - ui_login
  - ui_pages
  - ui_permissions
  - user_manager
  - workflow_editor

- Updated package.json files to include icon paths for each package.
2026-01-02 21:12:38 +00:00
git 40ec511651 feat(permissions): Add roles.json files for various components with access permissions 2026-01-02 20:11:45 +00:00
git 59a473dfb8 feat: Add user management package with components, permissions, functions, and styles 2026-01-02 15:29:03 +00:00
git e22b801f37 Refactor code structure for improved readability and maintainability 2026-01-02 12:47:38 +00:00
Richard Ward 939b243e97 config: packages,metadata,json (7 files) 2025-12-31 02:04:59 +00:00
Richard Ward 96f8607c46 config: packages,metadata,json (44 files) 2025-12-31 02:01:20 +00:00
Richard Ward 198cd22961 config: packages,metadata,json (47 files) 2025-12-31 00:19:04 +00:00
Richard Ward 24b06b5536 config: styles,packages,json (39 files) 2025-12-31 00:17:06 +00:00
Richard Ward a72ffff6e2 config: packages,metadata,json (18 files) 2025-12-30 23:11:25 +00:00
Richard Ward 667e92a20b config: yaml,packages,entities (11 files) 2025-12-30 22:09:26 +00:00
Richard Ward d3b7e0ae9c update: packages,operations,lua (3 files) 2025-12-30 22:01:42 +00:00
Richard Ward fceea6e3f1 config: packages,lua,ui (32 files) 2025-12-30 19:45:10 +00:00
Richard Ward 85aa69d924 config: packages,json,dashboard (1 files) 2025-12-30 19:05:01 +00:00
Richard Ward e1d7c14947 update: types,packages,lua (3 files) 2025-12-30 14:48:40 +00:00
Richard Ward 3ff6f2b779 docs: packages,lua,stats (12 files) 2025-12-30 14:31:20 +00:00
Richard Ward 1fd0873cb8 config: packages,lua,config (16 files) 2025-12-30 14:27:23 +00:00
Richard Ward dfcc6bdace config: role,packages,editor (5 files) 2025-12-30 14:25:40 +00:00
Richard Ward 16d0d21bee update: stats,packages,lua (1 files) 2025-12-30 14:21:00 +00:00
Richard Ward 0041224444 update: packages,lua,dashboard (2 files) 2025-12-30 14:19:46 +00:00
Richard Ward 24ed495f14 code: tsx,icons,fakemui (12 files) 2025-12-30 14:09:12 +00:00
git cfa1e5bfee feat: add new icon components including Envelope, ChatCircle, HardDrives, WarningCircle, CheckCircle, XCircle, TrendUp, ShieldWarning, LockKey, Crown, Sparkle, BookOpen, Tree, Broom, Export, UploadSimple, Funnel, FunnelSimple, MapTrifold, PushPinSimple, Buildings, GithubLogo, and GoogleLogo 2025-12-30 12:21:15 +00:00
git dc0336bb9e feat: enhance type definitions and add new classes for various components in code editor and notification center 2025-12-30 10:21:46 +00:00
git bf1401fe34 Enhance module definitions and add type annotations across various packages
- Added type annotations and class definitions in the dashboard layout, stats, and data table modules for improved type safety and documentation.
- Introduced new classes for UI components, props, and configuration in the form builder, navigation menu, notification center, and UI dialogs packages.
- Implemented detailed type definitions for actions, fields, and pagination components to streamline usage and enhance clarity.
- Updated initialization functions in multiple packages to include versioning and installation context.
- Improved structure and readability of the codebase by organizing and documenting component properties and methods.
2025-12-30 10:21:33 +00:00
git 42e9e816ad refactor: remove outdated test files for various packages 2025-12-30 01:52:37 +00:00
git 589f61faeb feat: add quick guide package with initialization and media handling; update test configurations for multiple packages 2025-12-30 01:37:16 +00:00
git a3e57efb31 feat: Enhance Lua test runner and add parameterized testing support
- Introduced `TestCaseFiles` interface to manage test case files in JSON format.
- Added `load_cases` function to load test cases from JSON files in the Lua test runner.
- Implemented parameterized test helpers (`it_each`, `fit_each`, `xit_each`) for more flexible test case execution.
- Updated existing tests in `ui_login`, `ui_permissions`, `user_manager`, and `workflow_editor` to utilize JSON-based test cases for better maintainability and readability.
- Added new components and scripts for the notification center, including a summary card with severity-based notifications.
- Enhanced metadata and manifest files to include new components and scripts.
- Introduced comprehensive test cases for user actions, permission checks, and workflow statuses.
2025-12-30 01:35:55 +00:00
git 7cd954b038 refactor: remove unused DBAL demo components and utilities
- Deleted KVStoreDemo, LogsPanel, ResultPanel, and associated utility functions from the DBAL demo.
- Removed ScreenshotAnalyzer components including ResultPanel and UploadSection.
- Added comprehensive tests for dashboard stats, UI login validation, UI permissions, user manager actions, and workflow editor status.
- Introduced parameterized tests for better coverage and maintainability.
2025-12-30 01:30:52 +00:00
git a920657d03 feat(tests): remove obsolete test files for dashboard, ui_login, ui_permissions, user_manager, and workflow_editor modules 2025-12-30 01:27:50 +00:00
git 0eeb28b02e feat(tests): integrate lua_test package and add test configurations for multiple modules 2025-12-30 01:26:05 +00:00
git a55da0eb38 feat(ui_permissions): add tests for levels module with value checks and hierarchy validation 2025-12-30 01:22:06 +00:00
git 4d451dab46 fix(types): enhance type safety across various components and utilities 2025-12-30 00:47:10 +00:00
git 52e1337b69 feat: Implement local and remote package sources with package source manager
- Added LocalPackageSource to load packages from the local filesystem.
- Introduced RemotePackageSource to fetch packages from a remote registry.
- Created PackageSourceManager to manage multiple package sources and resolve conflicts.
- Added tests for package source types and configurations.
- Updated metadata.json files for various packages to include dependencies.
- Enhanced type definitions for package sources and related data structures.
2025-12-30 00:32:52 +00:00
git b72763d9ff feat: add minLevel property to metadata.json files and update permission levels in levels.lua 2025-12-30 00:13:12 +00:00
git afa51500a5 feat(layout): add flex, grid, section, and stat components for dashboard layout
feat(nav_menu): implement menu item, group, and divider components
feat(ui_header): create logo, user, and actions sections for header rendering
2025-12-30 00:01:01 +00:00
git 4b002d8bfa fix(types): replace 'any' with 'unknown' in DBALConfig and DBALUser interfaces for better type safety 2025-12-29 23:47:39 +00:00
git 9d67c8dbbc feat: add notification center package with toast and list functionalities
- Implemented notification center initialization and rendering utilities.
- Added toast notification types: success, error, warning, and info.
- Created notification list rendering with badge support.
- Included SVG icons for notification center.
- Added tests for notification center functionalities.

feat: introduce schema editor package for database management

- Developed schema editor initialization and field type definitions.
- Implemented table management and relationships handling.
- Added SVG icons for schema editor.
- Included tests for schema editor functionalities.

feat: create user manager package for user management operations

- Implemented user management actions: create, update, delete, change level, and toggle active.
- Developed user list rendering with sortable columns.
- Added SVG icons for user manager.
- Included tests for user manager functionalities.

feat: add workflow editor package for workflow management

- Developed workflow editor rendering and step management utilities.
- Implemented workflow run status display and progress rendering.
- Added SVG icons for workflow editor.
- Included tests for workflow editor functionalities.

chore: add static SVG icons for various packages
2025-12-29 23:47:01 +00:00
git 036a435cc0 feat(ui): Add dashboard and auth UI components, including layouts and dialogs 2025-12-29 23:36:44 +00:00
git 26ba2d3ec0 feat(ui): Update metadata and add new components for dashboard, form builder, and navigation menu 2025-12-29 23:34:37 +00:00
git 5cd62c08e8 Generated by Spark: Each package in package folder has a unit tests folder 2025-12-24 19:51:20 +00:00