mirror of
https://github.com/johndoe6345789/AutoMetabuilder.git
synced 2026-04-25 06:15:01 +00:00
Merge pull request #11 from johndoe6345789/copilot/migrate-to-workflow-plugins
Migrate web data access and Flask server setup to workflow plugins
This commit is contained in:
312
WEB_PLUGIN_MIGRATION.md
Normal file
312
WEB_PLUGIN_MIGRATION.md
Normal file
@@ -0,0 +1,312 @@
|
||||
# Web to Workflow Plugins Migration
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the migration of web data access functions and Flask server setup from `autometabuilder/web/data` to workflow plugins in `autometabuilder/workflow/plugins/web`.
|
||||
|
||||
## Migration Summary
|
||||
|
||||
**Total Plugins Created:** 24
|
||||
**Plugin Map Updated:** 91 → 115 total plugins
|
||||
**New Plugin Category:** `web.*`
|
||||
|
||||
## Why This Migration?
|
||||
|
||||
This migration follows the established pattern of converting core backend functionality into reusable workflow plugins, enabling:
|
||||
|
||||
1. **Declarative Configuration**: Web operations can be composed in workflow definitions
|
||||
2. **Visual Workflow Editing**: Operations can be visualized and edited graphically
|
||||
3. **Composability**: Web plugins can be combined with other workflow plugins
|
||||
4. **Testability**: Individual operations are isolated and testable
|
||||
5. **Consistency**: All backend operations follow the same plugin architecture
|
||||
|
||||
## Plugin Categories
|
||||
|
||||
### 1. Environment Management (2 plugins)
|
||||
|
||||
| Plugin | Source | Description |
|
||||
|--------|--------|-------------|
|
||||
| `web.get_env_vars` | `web/data/env.py` | Load environment variables from .env file |
|
||||
| `web.persist_env_vars` | `web/data/env.py` | Write environment variables to .env file |
|
||||
|
||||
### 2. File I/O Operations (3 plugins)
|
||||
|
||||
| Plugin | Source | Description |
|
||||
|--------|--------|-------------|
|
||||
| `web.read_json` | `web/data/json_utils.py` | Read and parse JSON files |
|
||||
| `web.get_recent_logs` | `web/data/logs.py` | Retrieve recent log entries |
|
||||
| `web.load_messages` | `web/data/messages_io.py` | Load translation messages from path |
|
||||
|
||||
### 3. Translation Management (8 plugins)
|
||||
|
||||
| Plugin | Source | Description |
|
||||
|--------|--------|-------------|
|
||||
| `web.list_translations` | `web/data/translations.py` | List all available translations |
|
||||
| `web.load_translation` | `web/data/translations.py` | Load a specific language translation |
|
||||
| `web.create_translation` | `web/data/translations.py` | Create a new translation |
|
||||
| `web.update_translation` | `web/data/translations.py` | Update existing translation |
|
||||
| `web.delete_translation` | `web/data/translations.py` | Delete a translation |
|
||||
| `web.get_ui_messages` | `web/data/translations.py` | Get UI messages with fallback |
|
||||
| `web.write_messages_dir` | `web/data/messages_io.py` | Write messages to directory structure |
|
||||
|
||||
### 4. Navigation & Metadata (2 plugins)
|
||||
|
||||
| Plugin | Source | Description |
|
||||
|--------|--------|-------------|
|
||||
| `web.get_navigation_items` | `web/data/navigation.py` | Get navigation menu items |
|
||||
|
||||
### 5. Prompt Management (3 plugins)
|
||||
|
||||
| Plugin | Source | Description |
|
||||
|--------|--------|-------------|
|
||||
| `web.get_prompt_content` | `web/data/prompt.py` | Read prompt content from file |
|
||||
| `web.write_prompt` | `web/data/prompt.py` | Write prompt content to file |
|
||||
| `web.build_prompt_yaml` | `web/data/prompt.py` | Build YAML prompt from components |
|
||||
|
||||
### 6. Workflow Operations (4 plugins)
|
||||
|
||||
| Plugin | Source | Description |
|
||||
|--------|--------|-------------|
|
||||
| `web.get_workflow_content` | `web/data/workflow.py` | Read workflow JSON content |
|
||||
| `web.write_workflow` | `web/data/workflow.py` | Write workflow JSON content |
|
||||
| `web.load_workflow_packages` | `web/data/workflow.py` | Load all workflow packages |
|
||||
| `web.summarize_workflow_packages` | `web/data/workflow.py` | Create package summaries |
|
||||
|
||||
### 7. Flask Server Setup (4 plugins)
|
||||
|
||||
| Plugin | Source | Description |
|
||||
|--------|--------|-------------|
|
||||
| `web.create_flask_app` | New | Create and configure Flask app |
|
||||
| `web.register_blueprint` | New | Register Flask blueprints |
|
||||
| `web.start_server` | `web/server.py` | Start Flask web server |
|
||||
| `web.build_context` | `web/routes/context.py` | Build complete API context |
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Example 1: Loading Environment Variables
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "load_env",
|
||||
"type": "web.get_env_vars",
|
||||
"name": "Load Environment Variables",
|
||||
"parameters": {}
|
||||
}
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```json
|
||||
{
|
||||
"result": {
|
||||
"OPENAI_API_KEY": "sk-...",
|
||||
"GITHUB_TOKEN": "ghp_...",
|
||||
"LOG_LEVEL": "INFO"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Example 2: Building Prompt YAML
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "build_prompt",
|
||||
"type": "web.build_prompt_yaml",
|
||||
"name": "Build Prompt",
|
||||
"parameters": {
|
||||
"system_content": "You are a helpful coding assistant",
|
||||
"user_content": "Help me write clean code",
|
||||
"model": "openai/gpt-4o"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```yaml
|
||||
messages:
|
||||
- role: system
|
||||
content: >-
|
||||
You are a helpful coding assistant
|
||||
- role: user
|
||||
content: >-
|
||||
Help me write clean code
|
||||
model: openai/gpt-4o
|
||||
```
|
||||
|
||||
### Example 3: Setting Up Flask Server
|
||||
|
||||
A workflow can now configure and start the Flask server:
|
||||
|
||||
```json
|
||||
{
|
||||
"nodes": [
|
||||
{
|
||||
"id": "create_app",
|
||||
"type": "web.create_flask_app",
|
||||
"name": "Create Flask App",
|
||||
"parameters": {
|
||||
"name": "autometabuilder",
|
||||
"config": {
|
||||
"JSON_SORT_KEYS": false
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "register_context",
|
||||
"type": "web.register_blueprint",
|
||||
"name": "Register Context Routes",
|
||||
"parameters": {
|
||||
"blueprint_path": "autometabuilder.web.routes.context.context_bp"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "register_navigation",
|
||||
"type": "web.register_blueprint",
|
||||
"name": "Register Navigation Routes",
|
||||
"parameters": {
|
||||
"blueprint_path": "autometabuilder.web.routes.navigation.navigation_bp"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "start_server",
|
||||
"type": "web.start_server",
|
||||
"name": "Start Web Server",
|
||||
"parameters": {
|
||||
"host": "0.0.0.0",
|
||||
"port": 8000
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Example 4: Translation Management
|
||||
|
||||
```json
|
||||
{
|
||||
"nodes": [
|
||||
{
|
||||
"id": "list_langs",
|
||||
"type": "web.list_translations",
|
||||
"name": "List Available Languages"
|
||||
},
|
||||
{
|
||||
"id": "load_en",
|
||||
"type": "web.load_translation",
|
||||
"name": "Load English",
|
||||
"parameters": {
|
||||
"lang": "en"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "create_es",
|
||||
"type": "web.create_translation",
|
||||
"name": "Create Spanish Translation",
|
||||
"parameters": {
|
||||
"lang": "es"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Plugin Architecture
|
||||
|
||||
All web plugins follow the standard workflow plugin pattern:
|
||||
|
||||
```python
|
||||
def run(runtime, inputs):
|
||||
"""
|
||||
Plugin implementation.
|
||||
|
||||
Args:
|
||||
runtime: WorkflowRuntime instance with context and store
|
||||
inputs: Dictionary of input parameters
|
||||
|
||||
Returns:
|
||||
Dictionary with 'result' key or 'error' key
|
||||
"""
|
||||
# Implementation
|
||||
return {"result": value}
|
||||
```
|
||||
|
||||
### Runtime Context
|
||||
|
||||
Flask-related plugins use the runtime context to share the Flask app instance:
|
||||
|
||||
- `web.create_flask_app` stores app in `runtime.context["flask_app"]`
|
||||
- `web.register_blueprint` retrieves app from context
|
||||
- `web.start_server` retrieves app from context
|
||||
|
||||
## Testing
|
||||
|
||||
A comprehensive test suite has been added in `backend/tests/test_web_plugins.py` with tests for:
|
||||
|
||||
- Plugin map registration
|
||||
- JSON file reading
|
||||
- Prompt YAML building
|
||||
- Flask app creation
|
||||
- Blueprint registration
|
||||
- UI message loading
|
||||
- Translation management
|
||||
- Workflow package operations
|
||||
- Context building
|
||||
|
||||
Run tests with:
|
||||
```bash
|
||||
PYTHONPATH=backend poetry run pytest backend/tests/test_web_plugins.py -v
|
||||
```
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
**Important:** This migration adds new workflow plugins but **does not remove** existing web/data modules. The original functions remain in place and continue to work as before. The workflow plugins are thin wrappers that call the existing functions.
|
||||
|
||||
This means:
|
||||
- ✅ Existing code using `autometabuilder.web.data` continues to work
|
||||
- ✅ Flask routes continue to function normally
|
||||
- ✅ New workflows can use the plugin system
|
||||
- ✅ No breaking changes
|
||||
|
||||
## Integration with Existing Systems
|
||||
|
||||
The web plugins integrate seamlessly with:
|
||||
|
||||
1. **Backend Plugins**: Can be combined with `backend.*` plugins in workflows
|
||||
2. **Core Plugins**: Can work alongside `core.*` AI and tool execution plugins
|
||||
3. **Data Plugins**: Can use `dict.*`, `list.*`, `string.*` for data manipulation
|
||||
4. **Control Flow**: Can use `control.*` and `logic.*` for conditional logic
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
Potential additions to the web plugin category:
|
||||
|
||||
1. **Route Handlers**: Create plugins for individual route handlers
|
||||
2. **Middleware**: Workflow plugins for Flask middleware
|
||||
3. **Session Management**: Plugins for session operations
|
||||
4. **Authentication**: Login/logout workflow plugins
|
||||
5. **WebSocket Support**: Real-time communication plugins
|
||||
6. **Static File Serving**: Asset management plugins
|
||||
|
||||
## Files Changed
|
||||
|
||||
### New Files (27)
|
||||
- `backend/autometabuilder/workflow/plugins/web/__init__.py`
|
||||
- `backend/autometabuilder/workflow/plugins/web/web_*.py` (24 plugin files)
|
||||
- `backend/tests/test_web_plugins.py`
|
||||
|
||||
### Modified Files (1)
|
||||
- `backend/autometabuilder/workflow/plugin_map.json` (added 24 entries)
|
||||
|
||||
## Conclusion
|
||||
|
||||
This migration successfully converts web data access and Flask server operations into workflow plugins, following the established pattern used for backend plugins. The system now has 115 total plugins covering:
|
||||
|
||||
- Backend initialization (13 plugins)
|
||||
- Core AI operations (7 plugins)
|
||||
- Data manipulation (40 plugins)
|
||||
- Logic and control flow (11 plugins)
|
||||
- Testing utilities (5 plugins)
|
||||
- Tool execution (7 plugins)
|
||||
- Utility operations (8 plugins)
|
||||
- **Web operations (24 plugins)** ← New!
|
||||
|
||||
This enables fully declarative workflow-based configuration of the entire AutoMetabuilder system, from backend initialization to web server setup.
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"name": "web_server_bootstrap",
|
||||
"version": "1.0.0",
|
||||
"description": "Bootstrap the Flask web server with all routes",
|
||||
"keywords": ["web", "flask", "server", "bootstrap"],
|
||||
"license": "MIT",
|
||||
"category": "infrastructure"
|
||||
}
|
||||
@@ -0,0 +1,233 @@
|
||||
{
|
||||
"name": "Web Server Bootstrap",
|
||||
"active": false,
|
||||
"nodes": [
|
||||
{
|
||||
"id": "configure_logging",
|
||||
"name": "Configure Logging",
|
||||
"type": "backend.configure_logging",
|
||||
"typeVersion": 1,
|
||||
"position": [0, 0],
|
||||
"parameters": {}
|
||||
},
|
||||
{
|
||||
"id": "load_env",
|
||||
"name": "Load Environment",
|
||||
"type": "backend.load_env",
|
||||
"typeVersion": 1,
|
||||
"position": [300, 0],
|
||||
"parameters": {}
|
||||
},
|
||||
{
|
||||
"id": "create_app",
|
||||
"name": "Create Flask App",
|
||||
"type": "web.create_flask_app",
|
||||
"typeVersion": 1,
|
||||
"position": [600, 0],
|
||||
"parameters": {
|
||||
"name": "autometabuilder",
|
||||
"config": {
|
||||
"JSON_SORT_KEYS": false
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "register_context",
|
||||
"name": "Register Context Routes",
|
||||
"type": "web.register_blueprint",
|
||||
"typeVersion": 1,
|
||||
"position": [900, -150],
|
||||
"parameters": {
|
||||
"blueprint_path": "autometabuilder.web.routes.context.context_bp"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "register_run",
|
||||
"name": "Register Run Routes",
|
||||
"type": "web.register_blueprint",
|
||||
"typeVersion": 1,
|
||||
"position": [900, -50],
|
||||
"parameters": {
|
||||
"blueprint_path": "autometabuilder.web.routes.run.run_bp"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "register_prompt",
|
||||
"name": "Register Prompt Routes",
|
||||
"type": "web.register_blueprint",
|
||||
"typeVersion": 1,
|
||||
"position": [900, 50],
|
||||
"parameters": {
|
||||
"blueprint_path": "autometabuilder.web.routes.prompt.prompt_bp"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "register_settings",
|
||||
"name": "Register Settings Routes",
|
||||
"type": "web.register_blueprint",
|
||||
"typeVersion": 1,
|
||||
"position": [900, 150],
|
||||
"parameters": {
|
||||
"blueprint_path": "autometabuilder.web.routes.settings.settings_bp"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "register_translations",
|
||||
"name": "Register Translation Routes",
|
||||
"type": "web.register_blueprint",
|
||||
"typeVersion": 1,
|
||||
"position": [900, 250],
|
||||
"parameters": {
|
||||
"blueprint_path": "autometabuilder.web.routes.translations.translations_bp"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "register_navigation",
|
||||
"name": "Register Navigation Routes",
|
||||
"type": "web.register_blueprint",
|
||||
"typeVersion": 1,
|
||||
"position": [900, 350],
|
||||
"parameters": {
|
||||
"blueprint_path": "autometabuilder.web.routes.navigation.navigation_bp"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "start_server",
|
||||
"name": "Start Web Server",
|
||||
"type": "web.start_server",
|
||||
"typeVersion": 1,
|
||||
"position": [1200, 100],
|
||||
"parameters": {
|
||||
"host": "0.0.0.0",
|
||||
"port": 8000,
|
||||
"debug": false
|
||||
}
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Configure Logging": {
|
||||
"main": {
|
||||
"0": [
|
||||
{
|
||||
"node": "Load Environment",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"Load Environment": {
|
||||
"main": {
|
||||
"0": [
|
||||
{
|
||||
"node": "Create Flask App",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"Create Flask App": {
|
||||
"main": {
|
||||
"0": [
|
||||
{
|
||||
"node": "Register Context Routes",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
},
|
||||
{
|
||||
"node": "Register Run Routes",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
},
|
||||
{
|
||||
"node": "Register Prompt Routes",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
},
|
||||
{
|
||||
"node": "Register Settings Routes",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
},
|
||||
{
|
||||
"node": "Register Translation Routes",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
},
|
||||
{
|
||||
"node": "Register Navigation Routes",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"Register Context Routes": {
|
||||
"main": {
|
||||
"0": [
|
||||
{
|
||||
"node": "Start Web Server",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"Register Run Routes": {
|
||||
"main": {
|
||||
"0": [
|
||||
{
|
||||
"node": "Start Web Server",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"Register Prompt Routes": {
|
||||
"main": {
|
||||
"0": [
|
||||
{
|
||||
"node": "Start Web Server",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"Register Settings Routes": {
|
||||
"main": {
|
||||
"0": [
|
||||
{
|
||||
"node": "Start Web Server",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"Register Translation Routes": {
|
||||
"main": {
|
||||
"0": [
|
||||
{
|
||||
"node": "Start Web Server",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"Register Navigation Routes": {
|
||||
"main": {
|
||||
"0": [
|
||||
{
|
||||
"node": "Start Web Server",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -89,5 +89,29 @@
|
||||
"var.delete": "autometabuilder.workflow.plugins.var.var_delete.run",
|
||||
"var.exists": "autometabuilder.workflow.plugins.var.var_exists.run",
|
||||
"var.get": "autometabuilder.workflow.plugins.var.var_get.run",
|
||||
"var.set": "autometabuilder.workflow.plugins.var.var_set.run"
|
||||
"var.set": "autometabuilder.workflow.plugins.var.var_set.run",
|
||||
"web.build_context": "autometabuilder.workflow.plugins.web.web_build_context.run",
|
||||
"web.build_prompt_yaml": "autometabuilder.workflow.plugins.web.web_build_prompt_yaml.run",
|
||||
"web.create_flask_app": "autometabuilder.workflow.plugins.web.web_create_flask_app.run",
|
||||
"web.create_translation": "autometabuilder.workflow.plugins.web.web_create_translation.run",
|
||||
"web.delete_translation": "autometabuilder.workflow.plugins.web.web_delete_translation.run",
|
||||
"web.get_env_vars": "autometabuilder.workflow.plugins.web.web_get_env_vars.run",
|
||||
"web.get_navigation_items": "autometabuilder.workflow.plugins.web.web_get_navigation_items.run",
|
||||
"web.get_prompt_content": "autometabuilder.workflow.plugins.web.web_get_prompt_content.run",
|
||||
"web.get_recent_logs": "autometabuilder.workflow.plugins.web.web_get_recent_logs.run",
|
||||
"web.get_ui_messages": "autometabuilder.workflow.plugins.web.web_get_ui_messages.run",
|
||||
"web.get_workflow_content": "autometabuilder.workflow.plugins.web.web_get_workflow_content.run",
|
||||
"web.list_translations": "autometabuilder.workflow.plugins.web.web_list_translations.run",
|
||||
"web.load_messages": "autometabuilder.workflow.plugins.web.web_load_messages.run",
|
||||
"web.load_translation": "autometabuilder.workflow.plugins.web.web_load_translation.run",
|
||||
"web.load_workflow_packages": "autometabuilder.workflow.plugins.web.web_load_workflow_packages.run",
|
||||
"web.persist_env_vars": "autometabuilder.workflow.plugins.web.web_persist_env_vars.run",
|
||||
"web.read_json": "autometabuilder.workflow.plugins.web.web_read_json.run",
|
||||
"web.register_blueprint": "autometabuilder.workflow.plugins.web.web_register_blueprint.run",
|
||||
"web.start_server": "autometabuilder.workflow.plugins.web.web_start_server.run",
|
||||
"web.summarize_workflow_packages": "autometabuilder.workflow.plugins.web.web_summarize_workflow_packages.run",
|
||||
"web.update_translation": "autometabuilder.workflow.plugins.web.web_update_translation.run",
|
||||
"web.write_messages_dir": "autometabuilder.workflow.plugins.web.web_write_messages_dir.run",
|
||||
"web.write_prompt": "autometabuilder.workflow.plugins.web.web_write_prompt.run",
|
||||
"web.write_workflow": "autometabuilder.workflow.plugins.web.web_write_workflow.run"
|
||||
}
|
||||
|
||||
1
backend/autometabuilder/workflow/plugins/web/__init__.py
Normal file
1
backend/autometabuilder/workflow/plugins/web/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Web data access workflow plugins."""
|
||||
@@ -0,0 +1,17 @@
|
||||
"""Workflow plugin: build context for API."""
|
||||
import os
|
||||
from ....web.routes.context import build_context
|
||||
|
||||
|
||||
def run(_runtime, _inputs):
|
||||
"""
|
||||
Build the complete context object for the web UI.
|
||||
|
||||
This includes logs, env vars, translations, metadata, navigation,
|
||||
prompt, workflow, packages, messages, and status.
|
||||
|
||||
Returns:
|
||||
dict: Complete context object
|
||||
"""
|
||||
context = build_context()
|
||||
return {"result": context}
|
||||
@@ -0,0 +1,12 @@
|
||||
"""Workflow plugin: build prompt YAML."""
|
||||
from ....web.data.prompt import build_prompt_yaml
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Build prompt YAML from system and user content."""
|
||||
system_content = inputs.get("system_content")
|
||||
user_content = inputs.get("user_content")
|
||||
model = inputs.get("model")
|
||||
|
||||
yaml_content = build_prompt_yaml(system_content, user_content, model)
|
||||
return {"result": yaml_content}
|
||||
@@ -0,0 +1,28 @@
|
||||
"""Workflow plugin: create Flask app."""
|
||||
from flask import Flask
|
||||
|
||||
|
||||
def run(runtime, inputs):
|
||||
"""
|
||||
Create a Flask application instance.
|
||||
|
||||
Inputs:
|
||||
name: Application name (default: __name__)
|
||||
config: Dictionary of Flask configuration options
|
||||
|
||||
Returns:
|
||||
dict: Contains the Flask app in result
|
||||
"""
|
||||
name = inputs.get("name", "__main__")
|
||||
config = inputs.get("config", {})
|
||||
|
||||
app = Flask(name)
|
||||
|
||||
# Apply configuration
|
||||
for key, value in config.items():
|
||||
app.config[key] = value
|
||||
|
||||
# Store app in runtime context for other plugins to use
|
||||
runtime.context["flask_app"] = app
|
||||
|
||||
return {"result": app, "message": "Flask app created"}
|
||||
@@ -0,0 +1,12 @@
|
||||
"""Workflow plugin: create translation."""
|
||||
from ....web.data.translations import create_translation
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Create a new translation."""
|
||||
lang = inputs.get("lang")
|
||||
if not lang:
|
||||
return {"error": "lang is required"}
|
||||
|
||||
created = create_translation(lang)
|
||||
return {"result": created}
|
||||
@@ -0,0 +1,12 @@
|
||||
"""Workflow plugin: delete translation."""
|
||||
from ....web.data.translations import delete_translation
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Delete a translation."""
|
||||
lang = inputs.get("lang")
|
||||
if not lang:
|
||||
return {"error": "lang is required"}
|
||||
|
||||
deleted = delete_translation(lang)
|
||||
return {"result": deleted}
|
||||
@@ -0,0 +1,8 @@
|
||||
"""Workflow plugin: get environment variables."""
|
||||
from ....web.data.env import get_env_vars
|
||||
|
||||
|
||||
def run(_runtime, _inputs):
|
||||
"""Get environment variables from .env file."""
|
||||
env_vars = get_env_vars()
|
||||
return {"result": env_vars}
|
||||
@@ -0,0 +1,8 @@
|
||||
"""Workflow plugin: get navigation items."""
|
||||
from ....web.data.navigation import get_navigation_items
|
||||
|
||||
|
||||
def run(_runtime, _inputs):
|
||||
"""Get navigation items."""
|
||||
items = get_navigation_items()
|
||||
return {"result": items}
|
||||
@@ -0,0 +1,8 @@
|
||||
"""Workflow plugin: get prompt content."""
|
||||
from ....web.data.prompt import get_prompt_content
|
||||
|
||||
|
||||
def run(_runtime, _inputs):
|
||||
"""Get prompt content from prompt file."""
|
||||
content = get_prompt_content()
|
||||
return {"result": content}
|
||||
@@ -0,0 +1,9 @@
|
||||
"""Workflow plugin: get recent logs."""
|
||||
from ....web.data.logs import get_recent_logs
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Get recent log entries."""
|
||||
lines = inputs.get("lines", 50)
|
||||
logs = get_recent_logs(lines)
|
||||
return {"result": logs}
|
||||
@@ -0,0 +1,17 @@
|
||||
"""Workflow plugin: get UI messages."""
|
||||
from ....web.data.translations import get_ui_messages
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""
|
||||
Get UI messages for a specific language with fallback to English.
|
||||
|
||||
Inputs:
|
||||
lang: Language code (default: en)
|
||||
|
||||
Returns:
|
||||
dict: UI messages with __lang key indicating the language
|
||||
"""
|
||||
lang = inputs.get("lang", "en")
|
||||
messages = get_ui_messages(lang)
|
||||
return {"result": messages}
|
||||
@@ -0,0 +1,8 @@
|
||||
"""Workflow plugin: get workflow content."""
|
||||
from ....web.data.workflow import get_workflow_content
|
||||
|
||||
|
||||
def run(_runtime, _inputs):
|
||||
"""Get workflow content from workflow file."""
|
||||
content = get_workflow_content()
|
||||
return {"result": content}
|
||||
@@ -0,0 +1,8 @@
|
||||
"""Workflow plugin: list translations."""
|
||||
from ....web.data.translations import list_translations
|
||||
|
||||
|
||||
def run(_runtime, _inputs):
|
||||
"""List all available translations."""
|
||||
translations = list_translations()
|
||||
return {"result": translations}
|
||||
@@ -0,0 +1,13 @@
|
||||
"""Workflow plugin: load messages."""
|
||||
from pathlib import Path
|
||||
from ....web.data.messages_io import load_messages
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Load translation messages from path."""
|
||||
path = inputs.get("path")
|
||||
if not path:
|
||||
return {"error": "path is required"}
|
||||
|
||||
messages = load_messages(Path(path))
|
||||
return {"result": messages}
|
||||
@@ -0,0 +1,9 @@
|
||||
"""Workflow plugin: load translation."""
|
||||
from ....web.data.translations import load_translation
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Load translation for a specific language."""
|
||||
lang = inputs.get("lang", "en")
|
||||
translation = load_translation(lang)
|
||||
return {"result": translation}
|
||||
@@ -0,0 +1,8 @@
|
||||
"""Workflow plugin: load workflow packages."""
|
||||
from ....web.data.workflow import load_workflow_packages
|
||||
|
||||
|
||||
def run(_runtime, _inputs):
|
||||
"""Load all workflow packages."""
|
||||
packages = load_workflow_packages()
|
||||
return {"result": packages}
|
||||
@@ -0,0 +1,9 @@
|
||||
"""Workflow plugin: persist environment variables."""
|
||||
from ....web.data.env import persist_env_vars
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Persist environment variables to .env file."""
|
||||
updates = inputs.get("updates", {})
|
||||
persist_env_vars(updates)
|
||||
return {"result": "Environment variables persisted"}
|
||||
@@ -0,0 +1,13 @@
|
||||
"""Workflow plugin: read JSON file."""
|
||||
from pathlib import Path
|
||||
from ....web.data.json_utils import read_json
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Read JSON file."""
|
||||
path = inputs.get("path")
|
||||
if not path:
|
||||
return {"error": "path is required"}
|
||||
|
||||
json_data = read_json(Path(path))
|
||||
return {"result": json_data}
|
||||
@@ -0,0 +1,29 @@
|
||||
"""Workflow plugin: register Flask blueprint."""
|
||||
|
||||
|
||||
def run(runtime, inputs):
|
||||
"""
|
||||
Register a Flask blueprint with the Flask app.
|
||||
|
||||
Inputs:
|
||||
blueprint_path: Dotted path to the blueprint (e.g., "autometabuilder.web.routes.context.context_bp")
|
||||
|
||||
Returns:
|
||||
dict: Success indicator
|
||||
"""
|
||||
from ....loaders.callable_loader import load_callable
|
||||
|
||||
app = runtime.context.get("flask_app")
|
||||
if not app:
|
||||
return {"error": "Flask app not found in context. Run web.create_flask_app first."}
|
||||
|
||||
blueprint_path = inputs.get("blueprint_path")
|
||||
if not blueprint_path:
|
||||
return {"error": "blueprint_path is required"}
|
||||
|
||||
try:
|
||||
blueprint = load_callable(blueprint_path)
|
||||
app.register_blueprint(blueprint)
|
||||
return {"result": f"Blueprint {blueprint_path} registered"}
|
||||
except Exception as e:
|
||||
return {"error": f"Failed to register blueprint: {str(e)}"}
|
||||
@@ -0,0 +1,27 @@
|
||||
"""Workflow plugin: start Flask server."""
|
||||
|
||||
|
||||
def run(runtime, inputs):
|
||||
"""
|
||||
Start the Flask web server.
|
||||
|
||||
Inputs:
|
||||
host: Host address (default: 0.0.0.0)
|
||||
port: Port number (default: 8000)
|
||||
debug: Enable debug mode (default: False)
|
||||
|
||||
Returns:
|
||||
dict: Success indicator (note: this blocks until server stops)
|
||||
"""
|
||||
app = runtime.context.get("flask_app")
|
||||
if not app:
|
||||
return {"error": "Flask app not found in context. Run web.create_flask_app first."}
|
||||
|
||||
host = inputs.get("host", "0.0.0.0")
|
||||
port = inputs.get("port", 8000)
|
||||
debug = inputs.get("debug", False)
|
||||
|
||||
# This will block until the server is stopped
|
||||
app.run(host=host, port=port, debug=debug)
|
||||
|
||||
return {"result": "Server stopped"}
|
||||
@@ -0,0 +1,9 @@
|
||||
"""Workflow plugin: summarize workflow packages."""
|
||||
from ....web.data.workflow import summarize_workflow_packages
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Summarize workflow packages."""
|
||||
packages = inputs.get("packages", [])
|
||||
summary = summarize_workflow_packages(packages)
|
||||
return {"result": summary}
|
||||
@@ -0,0 +1,14 @@
|
||||
"""Workflow plugin: update translation."""
|
||||
from ....web.data.translations import update_translation
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Update an existing translation."""
|
||||
lang = inputs.get("lang")
|
||||
payload = inputs.get("payload", {})
|
||||
|
||||
if not lang:
|
||||
return {"error": "lang is required"}
|
||||
|
||||
updated = update_translation(lang, payload)
|
||||
return {"result": updated}
|
||||
@@ -0,0 +1,15 @@
|
||||
"""Workflow plugin: write messages directory."""
|
||||
from pathlib import Path
|
||||
from ....web.data.messages_io import write_messages_dir
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Write messages to directory."""
|
||||
base_dir = inputs.get("base_dir")
|
||||
payload_content = inputs.get("payload_content", {})
|
||||
|
||||
if not base_dir:
|
||||
return {"error": "base_dir is required"}
|
||||
|
||||
write_messages_dir(Path(base_dir), payload_content)
|
||||
return {"result": "Messages written successfully"}
|
||||
@@ -0,0 +1,9 @@
|
||||
"""Workflow plugin: write prompt."""
|
||||
from ....web.data.prompt import write_prompt
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Write prompt content to file."""
|
||||
content = inputs.get("content", "")
|
||||
write_prompt(content)
|
||||
return {"result": "Prompt written successfully"}
|
||||
@@ -0,0 +1,9 @@
|
||||
"""Workflow plugin: write workflow."""
|
||||
from ....web.data.workflow import write_workflow
|
||||
|
||||
|
||||
def run(_runtime, inputs):
|
||||
"""Write workflow content to file."""
|
||||
content = inputs.get("content", "")
|
||||
write_workflow(content)
|
||||
return {"result": "Workflow written successfully"}
|
||||
242
backend/tests/test_web_plugins.py
Normal file
242
backend/tests/test_web_plugins.py
Normal file
@@ -0,0 +1,242 @@
|
||||
"""Tests for web workflow plugins."""
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from autometabuilder.workflow.plugin_registry import PluginRegistry, load_plugin_map
|
||||
from autometabuilder.workflow.runtime import WorkflowRuntime
|
||||
|
||||
|
||||
class MockLogger:
|
||||
"""Mock logger for testing."""
|
||||
def info(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def debug(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def error(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
|
||||
def create_test_runtime():
|
||||
"""Create a test runtime with empty context."""
|
||||
logger = MockLogger()
|
||||
return WorkflowRuntime(context={}, store={}, tool_runner=None, logger=logger)
|
||||
|
||||
|
||||
def test_plugin_map_includes_web_plugins():
|
||||
"""Test that plugin map includes all new web plugins."""
|
||||
plugin_map = load_plugin_map()
|
||||
|
||||
# Test web data plugins
|
||||
assert "web.get_env_vars" in plugin_map
|
||||
assert "web.persist_env_vars" in plugin_map
|
||||
assert "web.get_recent_logs" in plugin_map
|
||||
assert "web.read_json" in plugin_map
|
||||
assert "web.load_messages" in plugin_map
|
||||
assert "web.write_messages_dir" in plugin_map
|
||||
assert "web.get_navigation_items" in plugin_map
|
||||
assert "web.get_prompt_content" in plugin_map
|
||||
assert "web.write_prompt" in plugin_map
|
||||
assert "web.build_prompt_yaml" in plugin_map
|
||||
assert "web.get_workflow_content" in plugin_map
|
||||
assert "web.write_workflow" in plugin_map
|
||||
assert "web.load_workflow_packages" in plugin_map
|
||||
assert "web.summarize_workflow_packages" in plugin_map
|
||||
|
||||
# Test translation plugins
|
||||
assert "web.load_translation" in plugin_map
|
||||
assert "web.list_translations" in plugin_map
|
||||
assert "web.create_translation" in plugin_map
|
||||
assert "web.update_translation" in plugin_map
|
||||
assert "web.delete_translation" in plugin_map
|
||||
assert "web.get_ui_messages" in plugin_map
|
||||
|
||||
# Test Flask/server plugins
|
||||
assert "web.create_flask_app" in plugin_map
|
||||
assert "web.register_blueprint" in plugin_map
|
||||
assert "web.start_server" in plugin_map
|
||||
assert "web.build_context" in plugin_map
|
||||
|
||||
|
||||
def test_web_read_json_plugin():
|
||||
"""Test web.read_json plugin."""
|
||||
plugin_map = load_plugin_map()
|
||||
registry = PluginRegistry(plugin_map)
|
||||
runtime = create_test_runtime()
|
||||
|
||||
plugin = registry.get("web.read_json")
|
||||
assert plugin is not None
|
||||
|
||||
# Test with non-existent file (should return empty dict)
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f:
|
||||
f.write('{"test": "value"}')
|
||||
temp_path = f.name
|
||||
|
||||
try:
|
||||
result = plugin(runtime, {"path": temp_path})
|
||||
assert "result" in result
|
||||
assert result["result"]["test"] == "value"
|
||||
finally:
|
||||
os.unlink(temp_path)
|
||||
|
||||
|
||||
def test_web_build_prompt_yaml_plugin():
|
||||
"""Test web.build_prompt_yaml plugin."""
|
||||
plugin_map = load_plugin_map()
|
||||
registry = PluginRegistry(plugin_map)
|
||||
runtime = create_test_runtime()
|
||||
|
||||
plugin = registry.get("web.build_prompt_yaml")
|
||||
assert plugin is not None
|
||||
|
||||
result = plugin(runtime, {
|
||||
"system_content": "You are a helpful assistant",
|
||||
"user_content": "Help me with coding",
|
||||
"model": "openai/gpt-4o"
|
||||
})
|
||||
|
||||
assert "result" in result
|
||||
yaml_content = result["result"]
|
||||
assert "messages:" in yaml_content
|
||||
assert "role: system" in yaml_content
|
||||
assert "role: user" in yaml_content
|
||||
assert "model: openai/gpt-4o" in yaml_content
|
||||
|
||||
|
||||
def test_web_create_flask_app_plugin():
|
||||
"""Test web.create_flask_app plugin."""
|
||||
plugin_map = load_plugin_map()
|
||||
registry = PluginRegistry(plugin_map)
|
||||
runtime = create_test_runtime()
|
||||
|
||||
plugin = registry.get("web.create_flask_app")
|
||||
assert plugin is not None
|
||||
|
||||
result = plugin(runtime, {
|
||||
"name": "test_app",
|
||||
"config": {"JSON_SORT_KEYS": False}
|
||||
})
|
||||
|
||||
assert "result" in result
|
||||
assert runtime.context.get("flask_app") is not None
|
||||
|
||||
app = runtime.context["flask_app"]
|
||||
assert app.config["JSON_SORT_KEYS"] is False
|
||||
|
||||
|
||||
def test_web_register_blueprint_plugin():
|
||||
"""Test web.register_blueprint plugin."""
|
||||
plugin_map = load_plugin_map()
|
||||
registry = PluginRegistry(plugin_map)
|
||||
runtime = create_test_runtime()
|
||||
|
||||
# First create a Flask app
|
||||
create_app_plugin = registry.get("web.create_flask_app")
|
||||
create_app_plugin(runtime, {"name": "test_app"})
|
||||
|
||||
# Now test registering a blueprint
|
||||
plugin = registry.get("web.register_blueprint")
|
||||
assert plugin is not None
|
||||
|
||||
result = plugin(runtime, {
|
||||
"blueprint_path": "autometabuilder.web.routes.context.context_bp"
|
||||
})
|
||||
|
||||
assert "result" in result
|
||||
assert "registered" in result["result"]
|
||||
|
||||
|
||||
def test_web_get_ui_messages_plugin():
|
||||
"""Test web.get_ui_messages plugin."""
|
||||
plugin_map = load_plugin_map()
|
||||
registry = PluginRegistry(plugin_map)
|
||||
runtime = create_test_runtime()
|
||||
|
||||
plugin = registry.get("web.get_ui_messages")
|
||||
assert plugin is not None
|
||||
|
||||
result = plugin(runtime, {"lang": "en"})
|
||||
|
||||
assert "result" in result
|
||||
assert isinstance(result["result"], dict)
|
||||
assert result["result"].get("__lang") == "en"
|
||||
|
||||
|
||||
def test_web_list_translations_plugin():
|
||||
"""Test web.list_translations plugin."""
|
||||
plugin_map = load_plugin_map()
|
||||
registry = PluginRegistry(plugin_map)
|
||||
runtime = create_test_runtime()
|
||||
|
||||
plugin = registry.get("web.list_translations")
|
||||
assert plugin is not None
|
||||
|
||||
result = plugin(runtime, {})
|
||||
|
||||
assert "result" in result
|
||||
assert isinstance(result["result"], dict)
|
||||
|
||||
|
||||
def test_web_load_workflow_packages_plugin():
|
||||
"""Test web.load_workflow_packages plugin."""
|
||||
plugin_map = load_plugin_map()
|
||||
registry = PluginRegistry(plugin_map)
|
||||
runtime = create_test_runtime()
|
||||
|
||||
plugin = registry.get("web.load_workflow_packages")
|
||||
assert plugin is not None
|
||||
|
||||
result = plugin(runtime, {})
|
||||
|
||||
assert "result" in result
|
||||
assert isinstance(result["result"], list)
|
||||
|
||||
|
||||
def test_web_summarize_workflow_packages_plugin():
|
||||
"""Test web.summarize_workflow_packages plugin."""
|
||||
plugin_map = load_plugin_map()
|
||||
registry = PluginRegistry(plugin_map)
|
||||
runtime = create_test_runtime()
|
||||
|
||||
plugin = registry.get("web.summarize_workflow_packages")
|
||||
assert plugin is not None
|
||||
|
||||
packages = [
|
||||
{
|
||||
"id": "test_pkg",
|
||||
"name": "Test Package",
|
||||
"description": "A test package",
|
||||
"version": "1.0.0"
|
||||
}
|
||||
]
|
||||
|
||||
result = plugin(runtime, {"packages": packages})
|
||||
|
||||
assert "result" in result
|
||||
assert isinstance(result["result"], list)
|
||||
assert len(result["result"]) == 1
|
||||
assert result["result"][0]["id"] == "test_pkg"
|
||||
|
||||
|
||||
def test_web_build_context_plugin():
|
||||
"""Test web.build_context plugin."""
|
||||
plugin_map = load_plugin_map()
|
||||
registry = PluginRegistry(plugin_map)
|
||||
runtime = create_test_runtime()
|
||||
|
||||
plugin = registry.get("web.build_context")
|
||||
assert plugin is not None
|
||||
|
||||
result = plugin(runtime, {})
|
||||
|
||||
assert "result" in result
|
||||
context = result["result"]
|
||||
|
||||
# Verify expected keys in context
|
||||
assert "logs" in context
|
||||
assert "env_vars" in context
|
||||
assert "translations" in context
|
||||
assert "metadata" in context
|
||||
assert "navigation" in context
|
||||
assert "status" in context
|
||||
Reference in New Issue
Block a user