metabuilder/codegen/docs/PACKAGES_REFACTOR.md

Packages Folder Refactor Summary

Changes Made

1. Removed Workspaces Configuration

• File: package.json
• Change: Removed the workspaces section that was causing npm workspace protocol issues
• Before:

  "workspaces": {
    "packages": ["packages/*"]
  }
  

• After: Section completely removed

2. Changed Dependency Protocol

• File: package.json
• Package: @github/spark
• Before: Used workspace protocol (implicit with workspaces config)
• After: Using explicit file path protocol: "@github/spark": "file:./packages/spark-tools"
• Reason: The workspace:* protocol is not supported in standard npm and causes Docker build failures

3. Removed Workspace-Specific Overrides

• File: package.json
• Removed:

  "@github/spark": {
    "react": "^19.0.0",
    "vite": "^7.3.1"
  },
  "@local/spark-wrapper": {
    "react": "^19.0.0"
  }
  

• Reason: These were specific to workspace configurations and are no longer needed

4. Updated Dockerfile

• File: Dockerfile
• Change: Explicitly copy the packages folder before npm ci
• New approach:

  COPY package*.json ./
  COPY packages ./packages
  RUN npm ci --include=optional
  

• Reason: Ensures packages are available when npm tries to resolve the file: protocol dependency

5. Updated .gitignore

• File: .gitignore
• Added exception: !packages/*/dist
• Reason: The built dist folders in packages need to be committed for Docker builds to work

Why These Changes Were Needed

The Problem

The previous setup used npm workspaces with the workspace:* protocol, which:

1. Is a pnpm/yarn feature not fully supported by npm
2. Causes Docker build failures with error: npm error Unsupported URL Type "workspace:"
3. Creates complexity in CI/CD pipelines

The Solution

By removing workspaces and using explicit file: protocol:

1. Standard npm can handle the dependency
2. Docker builds work without special workspace handling
3. The spark-tools package remains functional
4. No changes needed to imports in source code

What Stays The Same

• The packages/spark-tools folder remains intact
• All imports from @github/spark continue to work
• The spark runtime features (useKV hook, spark global, vite plugins) are unchanged
• Source code requires no modifications

Next Steps

Run npm install to regenerate the package-lock.json file with the new configuration. This will:

1. Remove workspace references from package-lock.json
2. Properly link @github/spark using file: protocol
3. Ensure Docker builds will succeed

Testing

After running npm install, verify:

1. npm run build works locally
2. Docker build succeeds: docker build -t test .
3. All spark features work (useKV, spark.llm, spark.kv, etc.)