metabuilder/frontends/codegen/docs/QEMU_INTEGRATION.md

# QEMU Integration for Multi-Architecture Builds

## Overview

QEMU has been successfully integrated into all CI/CD pipelines to enable multi-architecture Docker image builds. This allows the application to run on both AMD64 (x86_64) and ARM64 (aarch64) platforms.

## What is QEMU?

QEMU (Quick Emulator) is an open-source machine emulator that enables cross-platform compilation. When building Docker images, QEMU allows us to:

- Build images for multiple CPU architectures from a single build environment
- Support cloud providers that use ARM-based instances (AWS Graviton, Azure ARM VMs, etc.)
- Enable deployment to edge devices and IoT platforms
- Reduce infrastructure costs by leveraging ARM instances

## Supported Platforms

All CI/CD pipelines now build Docker images for:

- **linux/amd64** - Intel/AMD 64-bit processors (standard cloud instances)
- **linux/arm64** - ARM 64-bit processors (AWS Graviton, Apple Silicon, Raspberry Pi 4/5)

## Implementation Details

### GitHub Actions (.github/workflows/ci.yml)

The GitHub Actions workflow uses official Docker actions:

```yaml
- name: Set up QEMU
  uses: docker/setup-qemu-action@v3
  with:
    platforms: linux/amd64,linux/arm64

- name: Set up Docker Buildx
  uses: docker/setup-buildx-action@v3
  with:
    platforms: linux/amd64,linux/arm64

- name: Build and push Docker image
  uses: docker/build-push-action@v5
  with:
    platforms: linux/amd64,linux/arm64
```

### CircleCI (.circleci/config.yml)

CircleCI uses the multiarch QEMU static binaries and Docker Buildx:

```yaml
- run:
    name: Install QEMU
    command: |
      docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

- run:
    name: Set up Docker Buildx
    command: |
      docker buildx create --name multiarch --driver docker-container --use
      docker buildx inspect --bootstrap

- run:
    name: Build multi-arch Docker image
    command: |
      docker buildx build \
        --platform linux/amd64,linux/arm64 \
        --push \
        .
```

### GitLab CI (.gitlab-ci.yml)

GitLab CI uses Docker-in-Docker with QEMU setup:

```yaml
before_script:
  - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  - docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
  - docker buildx create --name multiarch --driver docker-container --use
  - docker buildx inspect --bootstrap

script:
  - docker buildx build --platform linux/amd64,linux/arm64 --push .
```

### Jenkins (Jenkinsfile)

Jenkins uses shell commands to set up QEMU and Buildx:

```groovy
sh '''
    docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
    docker buildx create --name multiarch --driver docker-container --use
    docker buildx inspect --bootstrap
'''

sh """
    docker buildx build \
        --platform linux/amd64,linux/arm64 \
        --push \
        .
"""
```

## Benefits

### 1. **Cost Optimization**
- ARM-based instances are typically 20-40% cheaper than x86 instances
- AWS Graviton2/3 instances offer better price-performance ratio

### 2. **Performance**
- Native ARM support for Apple Silicon development machines
- Optimized performance on ARM-based cloud infrastructure

### 3. **Flexibility**
- Deploy to any cloud provider regardless of architecture
- Support edge computing and IoT deployments

### 4. **Future-Proofing**
- ARM adoption is growing in data centers
- Single build process for multiple architectures

## Usage

### Pulling Images

Images are now multi-architecture manifests. Docker automatically pulls the correct architecture:

```bash
# Automatically pulls the correct architecture for your system
docker pull ghcr.io/your-org/your-repo:latest

# Explicitly specify architecture
docker pull --platform linux/amd64 ghcr.io/your-org/your-repo:latest
docker pull --platform linux/arm64 ghcr.io/your-org/your-repo:latest
```

### Inspecting Image Manifests

Check which architectures are available:

```bash
docker manifest inspect ghcr.io/your-org/your-repo:latest
```

### Local Multi-Arch Builds

Build multi-architecture images locally:

```bash
# Set up QEMU (one-time setup)
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

# Create buildx builder
docker buildx create --name multiarch --driver docker-container --use
docker buildx inspect --bootstrap

# Build for multiple architectures
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --tag your-image:latest \
  --push \
  .
```

## Performance Considerations

### Build Times

Multi-architecture builds take longer than single-architecture builds:

- **AMD64 only**: ~5-10 minutes
- **AMD64 + ARM64**: ~10-20 minutes

The cross-platform emulation (AMD64 building ARM64) adds overhead, but this is acceptable for the benefits gained.

### Optimization Tips

1. **Use Build Cache**: All pipelines leverage Docker layer caching
2. **Parallel Builds**: Consider splitting AMD64 and ARM64 into separate jobs for faster builds
3. **Conditional Builds**: Only build multi-arch for production/main branches

## Troubleshooting

### QEMU Not Available

If you see errors about QEMU not being installed:

```bash
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
```

### Buildx Not Found

Install Docker Buildx plugin:

```bash
docker buildx install
docker buildx create --use
```

### Platform Not Supported

Ensure your Docker version supports multi-platform builds:

```bash
docker buildx version
# Should show version 0.10.0 or higher
```

### Build Failures on Specific Architecture

Test building for a specific platform:

```bash
# Test AMD64
docker buildx build --platform linux/amd64 -t test:amd64 .

# Test ARM64
docker buildx build --platform linux/arm64 -t test:arm64 .
```

## Verification

### Check Pipeline Logs

Look for these indicators in CI/CD logs:

```
Setting up QEMU
Installing multiarch/qemu-user-static
Building for linux/amd64, linux/arm64
Successfully pushed multi-arch manifest
```

### Verify Image Manifest

After a successful build:

```bash
docker manifest inspect ghcr.io/your-org/your-repo:latest | grep "architecture"
```

Should show both `amd64` and `arm64`.

## References

- [Docker Buildx Documentation](https://docs.docker.com/buildx/working-with-buildx/)
- [QEMU User Static](https://github.com/multiarch/qemu-user-static)
- [Multi-Platform Images](https://docs.docker.com/build/building/multi-platform/)
- [GitHub Actions Docker Build Push](https://github.com/docker/build-push-action)

## Next Steps

Consider adding additional architectures:

- **linux/arm/v7** - 32-bit ARM (Raspberry Pi 3 and older)
- **linux/ppc64le** - IBM POWER architecture
- **linux/s390x** - IBM Z mainframe architecture

To add more platforms, update the `--platform` flag in all CI/CD configurations:

```bash
--platform linux/amd64,linux/arm64,linux/arm/v7
```

## Support

For issues or questions about multi-architecture builds:

1. Check the CI/CD pipeline logs for specific error messages
2. Verify Docker Buildx and QEMU are properly installed
3. Test builds locally before pushing to CI/CD
4. Review the Docker Buildx documentation for advanced configuration