Restructure dbal folder: development, production, and shared

Co-authored-by: johndoe6345789 <224850594+johndoe6345789@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2025-12-27 13:30:57 +00:00
parent b2dee2d870
commit 2180f608fb
470 changed files with 358 additions and 331 deletions

View File

@@ -16,8 +16,8 @@ body:
label: DBAL Implementation
description: Which DBAL implementation is affected?
options:
- TypeScript SDK (dbal/ts/)
- C++ Daemon (dbal/cpp/)
- TypeScript SDK (dbal/development/)
- C++ Daemon (dbal/production/)
- Both implementations
- YAML Contracts (api/schema/)
- Conformance Tests

View File

@@ -94,7 +94,7 @@ Report issues with the Database Abstraction Layer.
**Best For:**
- DBAL TypeScript SDK issues (`dbal/ts/`)
- DBAL C++ daemon issues (`dbal/cpp/`)
- DBAL C++ daemon issues (`dbal/production/`)
- YAML contract problems (`api/schema/`)
- Conformance test failures
- Implementation inconsistencies
@@ -285,7 +285,7 @@ Packages follow strict conventions:
### DBAL (Database Abstraction Layer)
- TypeScript implementation: `dbal/ts/` (development)
- C++ implementation: `dbal/cpp/` (production)
- C++ implementation: `dbal/production/` (production)
- YAML contracts: `api/schema/` (source of truth)
- Always update YAML first
- Run conformance tests: `python tools/conformance/run_all.py`

View File

@@ -190,7 +190,7 @@ if (user.level >= 3) { // Admin and above
## DBAL-Specific Guidance
**TypeScript DBAL**: Fast iteration, development use. Located in `dbal/ts/src/`.
**C++ DBAL Daemon**: Production security, credential protection. Located in `dbal/cpp/src/`.
**C++ DBAL Daemon**: Production security, credential protection. Located in `dbal/production/src/`.
**Conformance Tests**: Guarantee both implementations behave identically. Update `common/contracts/` when changing YAML schemas.
If fixing a DBAL bug:

View File

@@ -4,7 +4,7 @@ Run DBAL commands from `dbal/`.
Add a new entity to the DBAL following the API-first approach:
1. **Define entity** in `dbal/api/schema/entities/{name}.yaml`:
1. **Define entity** in `dbal/shared/api/schema/entities/{name}.yaml`:
```yaml
entity: EntityName
version: "1.0"
@@ -13,14 +13,14 @@ fields:
# Add fields...
```
2. **Define operations** in `dbal/api/schema/operations/{name}.ops.yaml`
2. **Define operations** in `dbal/shared/api/schema/operations/{name}.ops.yaml`
3. **Generate types**: `python tools/codegen/gen_types.py`
4. **Implement adapters** in both:
- `dbal/ts/src/adapters/`
- `dbal/cpp/src/adapters/`
- `dbal/development/src/adapters/`
- `dbal/production/src/adapters/`
5. **Add conformance tests** in `dbal/common/contracts/{name}_tests.yaml`
5. **Add conformance tests** in `dbal/shared/common/contracts/{name}_tests.yaml`
6. **Verify**: `python tools/conformance/run_all.py`

View File

@@ -36,4 +36,4 @@ static async getNewEntities(filter: { tenantId: string }) {
```
## 4. Update DBAL (if applicable)
Add entity to `dbal/api/schema/entities/`
Add entity to `dbal/shared/api/schema/entities/`

View File

@@ -10,7 +10,7 @@ Run app commands from `frontends/nextjs/` unless a step says otherwise.
npm run db:generate && npm run db:push
```
2. **DBAL contracts**: If new entity/operation, update YAML in `dbal/api/schema/`
2. **DBAL contracts**: If new entity/operation, update YAML in `dbal/shared/api/schema/`
3. **Database layer**: Add methods to `Database` class in `src/lib/database.ts`

View File

@@ -19,7 +19,7 @@ Use this as the default workflow when starting work in this repo.
## Where Work Lives
- Next.js app: `frontends/nextjs/` (source in `src/`, E2E in `e2e/`, local scripts in `scripts/`).
- Component packages: `packages/` (seed JSON under `packages/*/seed/`, optional `static_content/`, schema checks in `packages/*/tests/`).
- DBAL: `dbal/` (TypeScript library in `dbal/ts/`).
- DBAL: `dbal/` (TypeScript library in `dbal/development/`).
- Prisma schema/migrations: `prisma/` (`schema.prisma`, `migrations/`).
- Shared config: `config/` (symlinked into `frontends/nextjs/`).
- Repo utilities: `tools/` (quality checks, workflow helpers, code analysis).
@@ -41,7 +41,7 @@ Run app workflows from `frontends/nextjs/`:
- Validate: `npx prisma validate`
- Coverage output: `frontends/nextjs/coverage/`
DBAL workflows live in `dbal/ts/` (`npm run build`, `npm run test:unit`).
DBAL workflows live in `dbal/development/` (`npm run build`, `npm run test:unit`).
## Source + Tests
- TypeScript + ESM. Prefer `@/…` imports inside `frontends/nextjs/src/`.

View File

@@ -5,7 +5,7 @@ Before implementing, analyze the feature requirements:
1. **Check existing docs**: `docs/architecture/` for design patterns
2. **Identify affected areas**:
- Database schema changes? → `prisma/schema.prisma`
- New API/DBAL operations? → `dbal/api/schema/`
- New API/DBAL operations? → `dbal/shared/api/schema/`
- UI components? → Use declarative `RenderComponent`
- Business logic? → Consider Lua script in `packages/*/seed/scripts/`

View File

@@ -4,14 +4,14 @@ on:
push:
branches: [ main, develop ]
paths:
- 'dbal/cpp/**'
- 'dbal/tools/cpp-build-assistant.cjs'
- 'dbal/production/**'
- 'dbal/shared/tools/cpp-build-assistant.cjs'
- '.github/workflows/cpp-build.yml'
pull_request:
branches: [ main, develop ]
paths:
- 'dbal/cpp/**'
- 'dbal/tools/cpp-build-assistant.cjs'
- 'dbal/production/**'
- 'dbal/shared/tools/cpp-build-assistant.cjs'
- '.github/workflows/cpp-build.yml'
workflow_dispatch:
@@ -33,7 +33,7 @@ jobs:
- name: Check if C++ sources exist
id: check
run: |
if [ -d "dbal/cpp/src" ] && [ "$(find dbal/cpp/src -name '*.cpp' | wc -l)" -gt 0 ]; then
if [ -d "dbal/production/src" ] && [ "$(find dbal/production/src -name '*.cpp' | wc -l)" -gt 0 ]; then
echo "has_sources=true" >> $GITHUB_OUTPUT
echo "✓ C++ source files found"
else
@@ -112,8 +112,8 @@ jobs:
with:
name: dbal-daemon-linux
path: |
dbal/cpp/build/dbal_daemon
dbal/cpp/build/*.so
dbal/production/build/dbal_daemon
dbal/production/build/*.so
retention-days: 7
build-macos:
@@ -151,7 +151,7 @@ jobs:
CMAKE_BUILD_TYPE: ${{ matrix.build_type }}
run: |
if [ "${{ matrix.build_type }}" = "Debug" ]; then
node dbal/tools/cpp-build-assistant.cjs full --debug
node dbal/shared/tools/cpp-build-assistant.cjs full --debug
else
bun run cpp:full
fi
@@ -165,8 +165,8 @@ jobs:
with:
name: dbal-daemon-macos
path: |
dbal/cpp/build/dbal_daemon
dbal/cpp/build/*.dylib
dbal/production/build/dbal_daemon
dbal/production/build/*.dylib
retention-days: 7
build-windows:
@@ -206,7 +206,7 @@ jobs:
shell: bash
run: |
if [ "${{ matrix.build_type }}" = "Debug" ]; then
node dbal/tools/cpp-build-assistant.cjs full --debug
node dbal/shared/tools/cpp-build-assistant.cjs full --debug
else
bun run cpp:full
fi
@@ -220,8 +220,8 @@ jobs:
with:
name: dbal-daemon-windows
path: |
dbal/cpp/build/dbal_daemon.exe
dbal/cpp/build/*.dll
dbal/production/build/dbal_daemon.exe
dbal/production/build/*.dll
retention-days: 7
code-quality:
@@ -255,13 +255,13 @@ jobs:
run: |
cppcheck --enable=all --inconclusive --error-exitcode=1 \
--suppress=missingIncludeSystem \
-I dbal/cpp/include \
dbal/cpp/src/
-I dbal/production/include \
dbal/production/src/
continue-on-error: true
- name: Check formatting
run: |
find dbal/cpp/src dbal/cpp/include -name '*.cpp' -o -name '*.hpp' | \
find dbal/production/src dbal/production/include -name '*.cpp' -o -name '*.hpp' | \
xargs clang-format --dry-run --Werror
continue-on-error: true
@@ -288,15 +288,15 @@ jobs:
uses: actions/download-artifact@v4
with:
name: dbal-daemon-linux
path: dbal/cpp/build/
path: dbal/production/build/
- name: Make daemon executable
run: chmod +x dbal/cpp/build/dbal_daemon
run: chmod +x dbal/production/build/dbal_daemon
- name: Run integration tests
run: |
# Start C++ daemon
./dbal/cpp/build/dbal_daemon &
./dbal/production/build/dbal_daemon &
DAEMON_PID=$!
sleep 2

View File

@@ -212,7 +212,7 @@ jobs:
--exclude node_modules
--exclude build
--exclude .git
--exclude dbal/cpp/build
--exclude dbal/production/build
continue-on-error: true
- name: Upload security reports

View File

@@ -4,7 +4,7 @@
- `frontends/nextjs/`: primary Next.js app (source in `src/`, E2E in `e2e/`, local helper scripts in `scripts/`).
- `packages/`: JSON-driven component packages (`seed/*.json`, optional `static_content/`, and `tests/` for schema/structure checks).
- `dbal/`: database abstraction layer (TypeScript library in `dbal/ts/`; additional tooling/docs under `dbal/`).
- `dbal/`: database abstraction layer (TypeScript library in `dbal/development/`; additional tooling/docs under `dbal/`).
- `prisma/`: Prisma schema and migrations (`schema.prisma`, `migrations/`).
- `config/`: shared config (Playwright/Vite/TS/ESLint) symlinked into `frontends/nextjs/`.
- `tools/`: repo utilities (quality checks, workflow helpers, code analysis).
@@ -22,7 +22,7 @@ Run app workflows from `frontends/nextjs/`:
- `npm run test:e2e`: Playwright E2E tests.
- `npm run db:generate` / `npm run db:push` / `npm run db:migrate`: Prisma client + schema/migrations.
DBAL library workflows live in `dbal/ts/` (`npm run build`, `npm run test:unit`).
DBAL library workflows live in `dbal/development/` (`npm run build`, `npm run test:unit`).
## Coding Style & Naming Conventions

View File

@@ -611,8 +611,8 @@ const result = await prisma.$transaction(async (tx) => {
For complex operations:
- **TypeScript** (`dbal/ts/`): Fast iteration, development
- **C++ Daemon** (`dbal/cpp/`): Production security, credential protection
- **TypeScript** (`dbal/development/`): Fast iteration, development
- **C++ Daemon** (`dbal/production/`): Production security, credential protection
```typescript
import { dbalQuery } from '@/lib/database-dbal.server'
@@ -633,7 +633,7 @@ Complete isolation with access control, quotas, and namespace separation.
### Initialize Tenant
```typescript
import { InMemoryTenantManager, TenantAwareBlobStorage } from './dbal/ts/src'
import { InMemoryTenantManager, TenantAwareBlobStorage } from './dbal/development/src'
const tenantManager = new InMemoryTenantManager()
@@ -1132,8 +1132,8 @@ DEBUG=metabuilder:* npm run dev
| App source | `frontends/nextjs/src/` |
| Database schema | `prisma/schema.prisma` |
| Package seeds | `packages/*/seed/` |
| DBAL TypeScript | `dbal/ts/src/` |
| DBAL C++ | `dbal/cpp/src/` |
| DBAL TypeScript | `dbal/development/src/` |
| DBAL C++ | `dbal/production/src/` |
| E2E tests | `frontends/nextjs/e2e/` |
| Shared config | `config/` |
| Analysis tools | `tools/analysis/` |

View File

@@ -415,7 +415,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: cd dbal/ts && npm ci
- run: cd dbal/development && npm ci
- run: npm run test:unit
- run: npm run test:integration
@@ -423,7 +423,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: cd dbal/cpp && cmake -B build && cmake --build build
- run: cd dbal/production && cmake -B build && cmake --build build
- run: ./build/tests/unit_tests
- run: ./build/tests/integration_tests
@@ -432,7 +432,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: python dbal/tools/conformance/run_all.py
- run: python dbal/shared/tools/conformance/run_all.py
```
### Pre-commit Hooks
@@ -440,7 +440,7 @@ jobs:
```bash
# .git/hooks/pre-commit
#!/bin/bash
cd dbal/api/schema
cd dbal/shared/api/schema
if git diff --cached --name-only | grep -q "\.yaml$"; then
echo "YAML schema changed, regenerating types..."
python ../../tools/codegen/gen_types.py
@@ -510,7 +510,7 @@ version: '3.8'
services:
dbal-daemon:
build: ./dbal/cpp
build: ./dbal/production
container_name: dbal-daemon
ports:
- "50051:50051"

View File

@@ -6,9 +6,9 @@ This directory contains the Database Abstraction Layer for MetaBuilder.
- [Main README](README.md) - Overview and architecture
- [Agent Guide](AGENTS.md) - For AI agents and automated tools
- [Spark Integration](docs/SPARK_INTEGRATION.md) - GitHub Spark deployment guide
- [TypeScript Implementation](ts/README.md) - TS development guide
- [C++ Implementation](cpp/README.md) - C++ production guide
- [Spark Integration](shared/docs/SPARK_INTEGRATION.md) - GitHub Spark deployment guide
- [TypeScript Implementation](development/README.md) - TS development guide
- [C++ Implementation](production/README.md) - C++ production guide
## Directory Structure
@@ -19,15 +19,42 @@ dbal/
├── AGENTS.md # Agent development guide
├── .gitignore # Git ignore rules
├── api/ # Language-agnostic API definition
│ ├── schema/ # Entity and operation schemas
│ ├── entities/ # Entity definitions (YAML)
│ ├── operations/ # Operation definitions (YAML)
│ │ ├── errors.yaml # Error codes and handling
│ │ └── capabilities.yaml # Backend capability matrix
│ └── versioning/
│ └── compat.md # Compatibility rules
├── development/ # TypeScript implementation (fast iteration)
│ ├── src/ # Source code
│ ├── tests/ # Test suites
└── package.json # NPM configuration
├── production/ # C++ implementation (security & performance)
│ ├── src/ # Source code
│ ├── include/ # Header files
│ ├── tests/ # Test suites
│ ├── docs/ # C++ specific documentation
│ └── CMakeLists.txt # CMake configuration
└── shared/ # Shared resources
├── api/ # Language-agnostic API definition
│ ├── schema/ # Entity and operation schemas
│ │ ├── entities/ # Entity definitions (YAML)
│ │ ├── operations/ # Operation definitions (YAML)
│ │ ├── errors.yaml # Error codes and handling
│ │ └── capabilities.yaml # Backend capability matrix
│ └── versioning/
│ └── compat.md # Compatibility rules
├── backends/ # Database backend schemas
│ ├── prisma/ # Prisma ORM schemas
│ └── sqlite/ # SQLite schemas
├── common/ # Shared utilities
│ └── contracts/ # Conformance test contracts
├── tools/ # Development tools
│ ├── codegen/ # Code generation from YAML
│ └── conformance/ # Conformance test runners
├── scripts/ # Automation scripts
└── docs/ # Additional documentation
```
├── common/ # Shared resources
│ ├── contracts/ # Conformance test definitions
│ ├── fixtures/ # Test data
@@ -74,36 +101,36 @@ dbal/
### Generate Types
```bash
python tools/codegen/gen_types.py
python shared/tools/codegen/gen_types.py
```
### Build Everything
```bash
python scripts/build.py
python shared/scripts/build.py
```
### Run Tests
```bash
python scripts/test.py
python shared/scripts/test.py
```
### Run Conformance Tests
```bash
python scripts/conformance.py
python shared/scripts/conformance.py
```
## Development Workflow
1. **Define schema** in `api/schema/entities/` and `api/schema/operations/`
2. **Generate types** with `python tools/codegen/gen_types.py`
3. **Implement adapters** in `ts/src/adapters/` and `cpp/src/adapters/`
4. **Write tests** in `common/contracts/`
5. **Build** with `python scripts/build.py`
6. **Test** with `python scripts/test.py`
7. **Deploy** following `docs/SPARK_INTEGRATION.md`
1. **Define schema** in `shared/api/schema/entities/` and `shared/api/schema/operations/`
2. **Generate types** with `python shared/tools/codegen/gen_types.py`
3. **Implement adapters** in `development/src/adapters/` and `production/src/adapters/`
4. **Write tests** in `shared/common/contracts/`
5. **Build** with `python shared/scripts/build.py`
6. **Test** with `python shared/scripts/test.py`
7. **Deploy** following `shared/docs/SPARK_INTEGRATION.md`
## Key Concepts

View File

@@ -66,7 +66,7 @@ npx prisma generate
With `config.adapter = 'prisma'`, DBAL sends every request through `PrismaAdapter`, and Prisma handles dialect differences, migrations, and connection pooling defined in `prisma/schema.prisma` and `prisma/migrations/`. That keeps DBAL focused on validation, ACLs, and audit logging while it can still drive PostgreSQL, MySQL, or any other Prisma-supported store.
The C++ daemon still resides in Phase 3—the current implementation is backed by the in-memory store described in `dbal/cpp/docs/PHASE3_DAEMON.md`, so Postgres/MySQL adapters for the daemon are still future work.
The C++ daemon still resides in Phase 3—the current implementation is backed by the in-memory store described in `dbal/production/docs/PHASE3_DAEMON.md`, so Postgres/MySQL adapters for the daemon are still future work.
### Native Prisma bridge
@@ -102,7 +102,7 @@ dbal/
### Development Mode (TypeScript)
```bash
cd dbal/ts
cd dbal/development
npm install
npm run build
npm test
@@ -111,7 +111,7 @@ npm test
### Production Mode (C++ Daemon)
```bash
cd dbal/cpp
cd dbal/production
mkdir build && cd build
cmake ..
make

View File

@@ -6,14 +6,14 @@ The DBAL (Data Bus Abstraction Layer) provides a comprehensive implementation gu
### Getting Started
- [Quick Start Guide](./QUICK_START.md) - Setup and first steps
- [Quick Start Guide](./shared/docs/QUICK_START.md) - Setup and first steps
- [README](./README.md) - Project overview
### Implementation Guides
- [Phase 2 Implementation](./PHASE2_IMPLEMENTATION.md) - Version 2 features and design
- [Phase 2 Complete](./PHASE2_COMPLETE.md) - Implementation completion status
- [Implementation Summary](./IMPLEMENTATION_SUMMARY.md) - Feature overview
- [Phase 2 Implementation](./shared/docs/PHASE2_IMPLEMENTATION.md) - Version 2 features and design
- [Phase 2 Complete](./shared/docs/PHASE2_COMPLETE.md) - Implementation completion status
- [Implementation Summary](./shared/docs/IMPLEMENTATION_SUMMARY.md) - Feature overview
### Architecture
@@ -24,21 +24,21 @@ The DBAL (Data Bus Abstraction Layer) provides a comprehensive implementation gu
```
dbal/
├── QUICK_START.md # Quick start guide
├── README.md # Project overview
├── README_INDEX.md # Documentation index
├── PROJECT.md # Complete documentation
├── IMPLEMENTATION_SUMMARY.md # Implementation status
├── PHASE2_IMPLEMENTATION.md # Version 2 design
├── PHASE2_COMPLETE.md # Completion status
├── AGENTS.md # AI development guidelines
├── api/ # API specifications
├── backends/ # Backend implementations
├── common/ # Shared utilities
├── cpp/ # C++ implementations
── docs/ # Additional documentation
├── scripts/ # Utility scripts
├── tools/ # Development tools
└── ts/ # TypeScript implementations
├── LICENSE # MIT License
├── .gitignore # Git ignore rules
├── development/ # TypeScript implementation (fast iteration)
├── production/ # C++ implementation (security & performance)
── shared/ # Shared resources
├── api/ # API specifications (YAML contracts)
├── backends/ # Backend implementations (Prisma, SQLite)
├── common/ # Shared utilities and contracts
├── docs/ # Additional documentation
├── scripts/ # Utility scripts
└── tools/ # Development tools (codegen, build assistant)
```
## 🎯 Key Concepts
@@ -60,11 +60,11 @@ See [PROJECT.md](./PROJECT.md) for complete architecture documentation.
### Setting Up Development Environment
See [QUICK_START.md](./QUICK_START.md) for setup instructions.
See [QUICK_START.md](./shared/docs/QUICK_START.md) for setup instructions.
### Implementing New Features
See [PHASE2_IMPLEMENTATION.md](./PHASE2_IMPLEMENTATION.md) for design patterns.
See [PHASE2_IMPLEMENTATION.md](./shared/docs/PHASE2_IMPLEMENTATION.md) for design patterns.
### AI-Assisted Development

View File

@@ -14,7 +14,7 @@
"test:conformance": "tsx tests/conformance/runner.ts",
"lint": "eslint src/**/*.ts",
"format": "prettier --write src/**/*.ts",
"codegen": "tsx ../tools/codegen/gen_types.ts"
"codegen": "tsx ../shared/tools/codegen/gen_types.ts"
},
"keywords": [
"database",

Some files were not shown because too many files have changed in this diff Show More