metabuilder/workflowui/docs/README.md

WorkflowUI - Visual Workflow Editor

Modern, production-grade visual workflow editor inspired by n8n, built for MetaBuilder's DAG executor system.

🎯 Overview

WorkflowUI is a full-featured workflow editor with:

• Visual DAG Editor: Drag-and-drop node-based workflow construction with React Flow
• Real-Time Collaboration: Live updates via Redux and WebSockets
• Offline-First: IndexedDB for local workflow storage and sync
• Plugin Ecosystem: Support for custom node types and extensions
• Type-Safe: Full TypeScript with strict mode enabled

🏗️ Architecture

Frontend (Next.js + React)
  ├─ UI Components (FakeMUI)
  ├─ Redux Store (state management)
  ├─ React Flow (DAG visualization)
  └─ IndexedDB (offline storage)
       │
       ▼
Backend (Flask + Python)
  ├─ Workflow execution
  ├─ Plugin management
  ├─ Database persistence
  └─ WebSocket server
       │
       ▼
MetaBuilder DAG Executor
  ├─ Node execution
  ├─ Error recovery
  ├─ Multi-tenant support
  └─ Plugin registry

📦 Tech Stack

Layer	Technology	Purpose
Frontend	Next.js 14	Server-side rendering, API routes
UI Framework	FakeMUI	Material UI compatible components
State	Redux + Redux Toolkit	Centralized state management
Flow Editor	React Flow	DAG visualization and manipulation
Styling	SCSS	Component scoped styles
Storage	IndexedDB (Dexie)	Offline workflow persistence
HTTP	Axios	API client with interceptors
Backend	Flask	Python workflow execution
Database	SQLAlchemy + PostgreSQL/SQLite	Persistent workflow storage
Validation	n8n JSON Schema	Workflow and node validation

🚀 Quick Start

Prerequisites

Node.js 18+
Python 3.11+
npm or yarn

Installation

# Install frontend dependencies
npm install

# Install backend dependencies
pip install -r backend/requirements.txt

# Initialize IndexedDB
npm run db:init

Development

# Start both frontend and backend
npm run dev:all

# Or separately:
npm run dev          # Frontend on http://localhost:3000
npm run backend      # Backend on http://localhost:5000

Database Setup

The backend uses SQLAlchemy for database persistence:

# Default: SQLite (development)
# The database file is created automatically at backend/workflows.db

# PostgreSQL (production)
export DATABASE_URL=postgresql://user:password@localhost/workflows
npm run backend

# Create tables
python -c "from server_sqlalchemy import app, db; app.app_context().push(); db.create_all()"

Build

npm run build
npm start

📂 Folder Structure

workflowui/
├── src/
│   ├── app/                    # Next.js app directory
│   │   ├── layout.tsx          # Root layout
│   │   ├── page.tsx            # Dashboard page
│   │   ├── editor/[id].tsx     # Workflow editor
│   │   └── api/                # Next.js API routes
│   │
│   ├── components/             # React components
│   │   ├── Editor/             # Workflow editor components
│   │   │   ├── Canvas.tsx      # React Flow canvas
│   │   │   ├── Toolbar.tsx     # Editor toolbar
│   │   │   ├── NodePanel.tsx   # Node configuration panel
│   │   │   └── Properties.tsx  # Node properties panel
│   │   ├── Nodes/              # Custom node types
│   │   │   ├── BaseNode.tsx    # Base node wrapper
│   │   │   ├── PlaywrightNode.tsx
│   │   │   ├── StorybookNode.tsx
│   │   │   └── CustomNode.tsx
│   │   ├── UI/                 # Shared UI components
│   │   │   ├── Button.tsx
│   │   │   ├── Modal.tsx
│   │   │   ├── Sidebar.tsx
│   │   │   └── Toolbar.tsx
│   │   └── Layout/             # Layout components
│   │       ├── Header.tsx
│   │       └── MainLayout.tsx
│   │
│   ├── store/                  # Redux store
│   │   ├── store.ts            # Store configuration
│   │   ├── slices/
│   │   │   ├── workflowSlice.ts
│   │   │   ├── editorSlice.ts
│   │   │   ├── nodesSlice.ts
│   │   │   ├── connectionSlice.ts
│   │   │   └── uiSlice.ts
│   │   └── types.ts            # Store type definitions
│   │
│   ├── services/               # API/backend services
│   │   ├── workflowService.ts  # Workflow API calls
│   │   ├── nodeService.ts      # Node registry API
│   │   ├── executionService.ts # Workflow execution
│   │   └── storageService.ts   # IndexedDB persistence
│   │
│   ├── db/                     # IndexedDB schema & operations
│   │   ├── schema.ts           # Dexie schema
│   │   ├── workflows.ts        # Workflow queries
│   │   ├── nodes.ts            # Node cache
│   │   └── cache.ts            # Cache operations
│   │
│   ├── hooks/                  # React hooks
│   │   ├── useWorkflow.ts      # Workflow state hook
│   │   ├── useEditor.ts        # Editor state hook
│   │   ├── useNodes.ts         # Node operations hook
│   │   ├── useConnection.ts    # Connection hook
│   │   └── useStorage.ts       # IndexedDB hook
│   │
│   ├── types/                  # TypeScript types
│   │   ├── workflow.ts         # Workflow types
│   │   ├── node.ts             # Node types
│   │   ├── connection.ts       # Connection types
│   │   └── index.ts            # Export all types
│   │
│   ├── utils/                  # Utility functions
│   │   ├── dagValidation.ts    # DAG validation
│   │   ├── nodeFactory.ts      # Node creation factory
│   │   ├── layoutEngine.ts     # Auto-layout algorithm
│   │   ├── jsonValidator.ts    # JSON Schema validation
│   │   └── transformers.ts     # n8n format converters
│   │
│   └── styles/                 # Global styles
│       ├── globals.scss        # Global styles
│       ├── variables.scss      # Design tokens
│       └── mixins.scss         # SCSS mixins
│
├── backend/                    # Flask backend
│   ├── server.py               # Flask app
│   ├── requirements.txt        # Python dependencies
│   ├── routes/
│   │   ├── workflows.py        # Workflow endpoints
│   │   ├── execution.py        # Execution endpoints
│   │   └── nodes.py            # Node registry
│   ├── models/
│   │   ├── workflow.py         # Workflow model
│   │   └── execution.py        # Execution model
│   ├── services/
│   │   ├── executor.py         # Workflow execution
│   │   └── storage.py          # Persistence
│   └── utils/
│       ├── validation.py       # Workflow validation
│       └── converters.py       # Format converters
│
├── workflows/                  # Sample workflows
│   ├── e2e-testing.json
│   ├── documentation.json
│   └── complex-pipeline.json
│
├── stories/                    # Storybook stories
│   ├── Editor.stories.tsx
│   ├── Nodes.stories.tsx
│   └── UI.stories.tsx
│
├── scripts/                    # Build/setup scripts
│   ├── init-db.js              # Initialize IndexedDB
│   └── migrate-db.js           # Database migrations
│
├── public/                     # Static assets
│   └── icons/
│
├── tsconfig.json               # TypeScript config
├── next.config.js              # Next.js config
├── jest.config.js              # Jest config
├── .storybook/                 # Storybook config
└── README.md

🎨 Key Components

1. Canvas (React Flow)

// Drag-and-drop DAG editor with zoom, pan, selection
<WorkflowCanvas>
  <ReactFlowProvider>
    <ReactFlow nodes={nodes} edges={edges} onConnect={onConnect}>
      <Background />
      <Controls />
      <MiniMap />
    </ReactFlow>
  </ReactFlowProvider>
</WorkflowCanvas>

2. Node Types

// Playwright node example
<PlaywrightNode
  id="test_chromium"
  data={{
    browser: "chromium",
    baseUrl: "http://localhost:3000",
    headless: true
  }}
/>

3. Redux Store

// Central state management
store/slices:
  - workflowSlice: Current workflow, metadata
  - editorSlice: Canvas zoom, pan, selection
  - nodesSlice: Node registry, templates
  - connectionSlice: Edge creation state
  - uiSlice: Modals, panels, notifications

4. IndexedDB Storage

// Offline-first storage with Dexie
db.workflows.put(workflow)
db.workflows.get(id)
db.workflows.toArray()
db.syncWithServer() // Auto-sync when online

5. Backend API

# Flask endpoints
POST   /api/workflows              # Create workflow
GET    /api/workflows/<id>         # Get workflow
PUT    /api/workflows/<id>         # Update workflow
DELETE /api/workflows/<id>         # Delete workflow
POST   /api/workflows/<id>/execute # Execute workflow
GET    /api/nodes                  # Get node registry

🔌 Plugin System

Custom node types can be added:

// Define custom node
const CustomNode: NodeType = {
  id: 'my.custom',
  name: 'My Custom Node',
  category: 'custom',
  parameters: {
    field1: { type: 'string', required: true },
    field2: { type: 'number', required: false }
  }
};

// Register in node registry
registerNodeType(CustomNode);

📊 Features

Visual Editing

• ✅ Drag-and-drop node creation
• ✅ Connection drawing with validation
• ✅ Node selection and multi-select
• ✅ Undo/redo support
• ✅ Auto-layout (horizontal, vertical, hierarchical)
• ✅ Zoom and pan controls
• ✅ Minimap navigation

Workflow Management

• ✅ Create/edit/delete workflows
• ✅ Version control integration
• ✅ Workflow templates
• ✅ Import/export (JSON, n8n format)
• ✅ Workflow validation (DAG constraints)
• ✅ Multi-tenant support

Execution

• ✅ Dry-run execution
• ✅ Execution history
• ✅ Result visualization
• ✅ Error reporting
• ✅ Step-by-step debugging
• ✅ Performance metrics

Developer Experience

• ✅ Hot reload
• ✅ Redux DevTools integration
• ✅ Storybook for component development
• ✅ TypeScript strict mode
• ✅ Comprehensive error messages
• ✅ Debug mode

🧪 Testing

# Run tests
npm test

# Watch mode
npm test:watch

# Component development
npm run storybook

📚 Documentation

• Architecture Guide
• Component API
• Redux Store Design
• IndexedDB Schema
• Node Type Reference
• API Reference

🚀 Deployment

# Build for production
npm run build

# Start production server
npm start

Deployed with:

• Vercel (Next.js frontend)
• Heroku/Railway (Flask backend)
• MongoDB Atlas (persistent storage)

🤝 Contributing

1. Create feature branch: git checkout -b feature/name
2. Make changes with tests
3. Submit PR with description

📄 License

Part of MetaBuilder project

🔗 Related

• MetaBuilder
• DAG Executor
• Workflow Plugins
• n8n Format