Generated by Spark: Add qemu to CI/CD processes.

2026-04-24 13:44:54 +00:00 · 2026-01-17 14:40:27 +00:00
parent 6c043807ff
commit 33ab37bd18
11 changed files with 1269 additions and 19 deletions
--- a/scripts/QEMU_SCRIPTS_README.md
+++ b/scripts/QEMU_SCRIPTS_README.md
@@ -0,0 +1,298 @@
+# Multi-Architecture Build Scripts
+
+This directory contains helper scripts for QEMU multi-architecture Docker builds.
+
+## Scripts
+
+### 🚀 build-multiarch.sh
+
+Builds multi-architecture Docker images locally using QEMU and Docker Buildx.
+
+**Usage:**
+```bash
+# Make executable
+chmod +x scripts/build-multiarch.sh
+
+# Build for local testing (loads into Docker)
+./scripts/build-multiarch.sh myapp latest
+
+# Build and push to registry
+./scripts/build-multiarch.sh myapp latest "linux/amd64,linux/arm64" ghcr.io --push
+
+# Custom platforms
+./scripts/build-multiarch.sh myapp v1.0 "linux/amd64,linux/arm64,linux/arm/v7" ghcr.io --push
+```
+
+**Parameters:**
+1. `IMAGE_NAME` - Docker image name (default: `myapp`)
+2. `IMAGE_TAG` - Image tag (default: `latest`)
+3. `PLATFORMS` - Comma-separated platforms (default: `linux/amd64,linux/arm64`)
+4. `REGISTRY` - Container registry (default: `ghcr.io`)
+5. `--push` - Push to registry (omit to load locally)
+
+**Features:**
+- ✅ Automatic QEMU setup
+- ✅ Buildx builder configuration
+- ✅ Color-coded output
+- ✅ Error handling
+- ✅ Progress indicators
+
+### 🔍 validate-qemu.sh
+
+Validates QEMU installation and multi-architecture build capabilities.
+
+**Usage:**
+```bash
+# Make executable
+chmod +x scripts/validate-qemu.sh
+
+# Run validation
+./scripts/validate-qemu.sh
+```
+
+**Checks:**
+- ✅ Docker installation
+- ✅ Docker Buildx availability
+- ✅ QEMU installation
+- ✅ QEMU binaries functionality
+- ✅ Buildx builder setup
+- ✅ Platform support (AMD64, ARM64)
+- ✅ Test builds for each platform
+- ✅ CI/CD configuration validation
+
+**Exit Codes:**
+- `0` - All validations passed
+- `1` - One or more validations failed
+
+## Quick Start
+
+### First Time Setup
+
+```bash
+# 1. Make scripts executable
+chmod +x scripts/*.sh
+
+# 2. Validate your environment
+./scripts/validate-qemu.sh
+
+# 3. Build your first multi-arch image
+./scripts/build-multiarch.sh codeforge latest
+```
+
+### Local Development
+
+```bash
+# Build and load into local Docker (AMD64 only for speed)
+./scripts/build-multiarch.sh myapp dev
+
+# Run the image
+docker run -p 80:80 ghcr.io/myapp:dev
+```
+
+### Production Release
+
+```bash
+# Build multi-arch and push to registry
+./scripts/build-multiarch.sh codeforge v1.2.3 "linux/amd64,linux/arm64" ghcr.io --push
+
+# Verify the manifest
+docker manifest inspect ghcr.io/codeforge:v1.2.3
+```
+
+## Supported Platforms
+
+### Default Platforms
+- `linux/amd64` - Intel/AMD 64-bit (x86_64)
+- `linux/arm64` - ARM 64-bit (aarch64)
+
+### Additional Platforms (Optional)
+- `linux/arm/v7` - ARM 32-bit (armv7l) - Raspberry Pi 3 and older
+- `linux/arm/v6` - ARM 32-bit (armv6l) - Raspberry Pi Zero
+- `linux/ppc64le` - IBM POWER (Little Endian)
+- `linux/s390x` - IBM Z mainframe
+- `linux/386` - Intel/AMD 32-bit (i386)
+
+## CI/CD Integration
+
+These scripts are reference implementations. The actual CI/CD pipelines use:
+
+### GitHub Actions
+```yaml
+- name: Set up QEMU
+  uses: docker/setup-qemu-action@v3
+  with:
+    platforms: linux/amd64,linux/arm64
+```
+
+### CircleCI
+```yaml
+- run:
+    name: Install QEMU
+    command: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
+```
+
+### GitLab CI
+```yaml
+before_script:
+  - docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
+  - docker buildx create --name multiarch --driver docker-container --use
+```
+
+### Jenkins
+```groovy
+sh 'docker run --rm --privileged multiarch/qemu-user-static --reset -p yes'
+sh 'docker buildx create --name multiarch --driver docker-container --use'
+```
+
+## Troubleshooting
+
+### Permission Denied
+
+```bash
+# Run with sudo
+sudo ./scripts/build-multiarch.sh myapp latest
+
+# Or add your user to docker group
+sudo usermod -aG docker $USER
+newgrp docker
+```
+
+### QEMU Not Found
+
+```bash
+# Manually install QEMU
+docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
+```
+
+### Buildx Not Available
+
+```bash
+# Install Docker Buildx
+docker buildx install
+
+# Verify installation
+docker buildx version
+```
+
+### Slow Builds
+
+Cross-compilation (especially AMD64 → ARM64) is slower than native builds. This is normal.
+
+**Optimization tips:**
+- Use build cache: `--cache-from type=gha --cache-to type=gha,mode=max`
+- Build single platform for development: remove `--platform` or specify one arch
+- Use native runners: GitHub Actions has ARM64 runners available
+
+### Platform Not Supported
+
+Some base images don't support all platforms. Check the base image documentation.
+
+```bash
+# Check available platforms for an image
+docker manifest inspect alpine:latest
+```
+
+## Performance Benchmarks
+
+Approximate build times for a typical web application:
+
+| Configuration | Time | Notes |
+|--------------|------|-------|
+| AMD64 only (native) | 5-8 min | Fastest |
+| ARM64 only (emulated) | 10-15 min | Cross-compiled on AMD64 |
+| AMD64 + ARM64 | 15-20 min | Both platforms |
+| AMD64 + ARM64 + ARMv7 | 20-30 min | Three platforms |
+
+## Environment Variables
+
+Scripts support these environment variables:
+
+```bash
+# Docker registry credentials
+export DOCKER_USERNAME="your-username"
+export DOCKER_PASSWORD="your-token"
+
+# Custom registry
+export REGISTRY="ghcr.io"
+
+# Build options
+export DOCKER_BUILDKIT=1
+export BUILDKIT_PROGRESS=plain
+```
+
+## Examples
+
+### Example 1: Development Build
+
+```bash
+# Quick local build for testing
+./scripts/build-multiarch.sh myapp dev
+
+# Test the image
+docker run -p 3000:80 ghcr.io/myapp:dev
+curl http://localhost:3000
+```
+
+### Example 2: Staging Release
+
+```bash
+# Build and push to staging
+./scripts/build-multiarch.sh myapp staging "linux/amd64,linux/arm64" ghcr.io --push
+
+# Deploy on staging server
+docker pull ghcr.io/myapp:staging
+docker run -d -p 80:80 ghcr.io/myapp:staging
+```
+
+### Example 3: Production Release
+
+```bash
+# Build with version tag
+./scripts/build-multiarch.sh myapp v2.1.0 "linux/amd64,linux/arm64" ghcr.io --push
+
+# Also tag as latest
+docker tag ghcr.io/myapp:v2.1.0 ghcr.io/myapp:latest
+docker push ghcr.io/myapp:latest
+```
+
+### Example 4: IoT/Edge Devices
+
+```bash
+# Build for Raspberry Pi (ARMv7 + ARM64)
+./scripts/build-multiarch.sh iot-app v1.0 "linux/arm64,linux/arm/v7" ghcr.io --push
+
+# Pull on Raspberry Pi
+docker pull ghcr.io/iot-app:v1.0
+docker run ghcr.io/iot-app:v1.0
+```
+
+## Additional Resources
+
+- [QEMU Integration Guide](../QEMU_INTEGRATION.md) - Full documentation
+- [CI/CD Summary](../QEMU_CI_CD_SUMMARY.md) - Implementation details
+- [Docker Buildx](https://docs.docker.com/buildx/) - Official documentation
+- [QEMU User Static](https://github.com/multiarch/qemu-user-static) - QEMU binaries
+
+## Contributing
+
+When adding new scripts:
+
+1. Follow the existing script structure
+2. Add color-coded output for better UX
+3. Include error handling with meaningful messages
+4. Document usage and parameters
+5. Update this README
+
+## Support
+
+For issues with multi-architecture builds:
+
+1. Run validation: `./scripts/validate-qemu.sh`
+2. Check Docker version: `docker --version` (v20.10+ recommended)
+3. Verify QEMU: `docker run --rm multiarch/qemu-user-static --version`
+4. Review logs for specific error messages
+
+---
+
+*Last Updated: 2024*
+*Maintained by: Development Team*
--- a/scripts/build-multiarch.sh
+++ b/scripts/build-multiarch.sh
@@ -0,0 +1,130 @@
+#!/bin/bash
+
+# Multi-Architecture Docker Build Script
+# This script demonstrates how to build multi-arch images locally with QEMU
+
+set -e
+
+echo "🚀 Multi-Architecture Docker Build with QEMU"
+echo "=============================================="
+echo ""
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+# Configuration
+IMAGE_NAME="${1:-myapp}"
+IMAGE_TAG="${2:-latest}"
+PLATFORMS="${3:-linux/amd64,linux/arm64}"
+REGISTRY="${4:-ghcr.io}"
+
+echo "📋 Configuration:"
+echo "  Image Name: $IMAGE_NAME"
+echo "  Image Tag: $IMAGE_TAG"
+echo "  Platforms: $PLATFORMS"
+echo "  Registry: $REGISTRY"
+echo ""
+
+# Check if Docker is installed
+if ! command -v docker &> /dev/null; then
+    echo -e "${RED}❌ Docker is not installed${NC}"
+    exit 1
+fi
+
+echo -e "${GREEN}✅ Docker is installed${NC}"
+
+# Check if Docker Buildx is available
+if ! docker buildx version &> /dev/null; then
+    echo -e "${RED}❌ Docker Buildx is not available${NC}"
+    echo "Installing Docker Buildx..."
+    docker buildx install
+fi
+
+echo -e "${GREEN}✅ Docker Buildx is available${NC}"
+
+# Set up QEMU
+echo ""
+echo "🔧 Setting up QEMU for multi-architecture builds..."
+docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
+
+if [ $? -eq 0 ]; then
+    echo -e "${GREEN}✅ QEMU setup successful${NC}"
+else
+    echo -e "${RED}❌ QEMU setup failed${NC}"
+    exit 1
+fi
+
+# Create or use existing buildx builder
+echo ""
+echo "🔧 Setting up Docker Buildx builder..."
+if docker buildx inspect multiarch &> /dev/null; then
+    echo -e "${YELLOW}⚠️  Builder 'multiarch' already exists, using existing${NC}"
+    docker buildx use multiarch
+else
+    docker buildx create --name multiarch --driver docker-container --use
+    echo -e "${GREEN}✅ Created new builder 'multiarch'${NC}"
+fi
+
+docker buildx inspect --bootstrap
+
+# Build the multi-architecture image
+echo ""
+echo "🏗️  Building multi-architecture Docker image..."
+echo "  This may take several minutes..."
+echo ""
+
+BUILD_ARGS=""
+if [ "$5" = "--push" ]; then
+    BUILD_ARGS="--push"
+    echo "  Will push to registry after build"
+else
+    BUILD_ARGS="--load"
+    echo "  Will load into local Docker daemon (single platform)"
+    # When loading, we can only build for one platform
+    PLATFORMS="linux/amd64"
+    echo -e "${YELLOW}⚠️  Loading locally, building only for linux/amd64${NC}"
+fi
+
+docker buildx build \
+    --platform $PLATFORMS \
+    --tag $REGISTRY/$IMAGE_NAME:$IMAGE_TAG \
+    $BUILD_ARGS \
+    .
+
+if [ $? -eq 0 ]; then
+    echo ""
+    echo -e "${GREEN}✅ Build successful!${NC}"
+    echo ""
+    echo "📦 Built images:"
+    echo "  $REGISTRY/$IMAGE_NAME:$IMAGE_TAG"
+    echo "  Platforms: $PLATFORMS"
+    
+    if [ "$5" = "--push" ]; then
+        echo ""
+        echo "🎉 Images pushed to registry!"
+        echo ""
+        echo "To pull the image:"
+        echo "  docker pull $REGISTRY/$IMAGE_NAME:$IMAGE_TAG"
+    else
+        echo ""
+        echo "🎉 Image loaded into local Docker!"
+        echo ""
+        echo "To run the image:"
+        echo "  docker run -p 80:80 $REGISTRY/$IMAGE_NAME:$IMAGE_TAG"
+    fi
+    
+    echo ""
+    echo "To inspect the manifest:"
+    echo "  docker manifest inspect $REGISTRY/$IMAGE_NAME:$IMAGE_TAG"
+else
+    echo ""
+    echo -e "${RED}❌ Build failed${NC}"
+    exit 1
+fi
+
+echo ""
+echo "=============================================="
+echo "✨ Build process complete!"
--- a/scripts/validate-qemu.sh
+++ b/scripts/validate-qemu.sh
@@ -0,0 +1,224 @@
+#!/bin/bash
+
+# QEMU Multi-Architecture Validation Script
+# This script validates that QEMU is properly configured and multi-arch builds work
+
+set -e
+
+echo "🔍 QEMU Multi-Architecture Validation"
+echo "======================================"
+echo ""
+
+# Colors
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+NC='\033[0m'
+
+PASSED=0
+FAILED=0
+
+# Function to print test results
+print_result() {
+    if [ $1 -eq 0 ]; then
+        echo -e "${GREEN}✅ PASS${NC}: $2"
+        ((PASSED++))
+    else
+        echo -e "${RED}❌ FAIL${NC}: $2"
+        ((FAILED++))
+    fi
+}
+
+echo -e "${BLUE}Step 1: Checking Docker installation${NC}"
+echo "--------------------------------------"
+
+if command -v docker &> /dev/null; then
+    DOCKER_VERSION=$(docker --version)
+    print_result 0 "Docker is installed: $DOCKER_VERSION"
+else
+    print_result 1 "Docker is not installed"
+    exit 1
+fi
+
+echo ""
+echo -e "${BLUE}Step 2: Checking Docker Buildx${NC}"
+echo "--------------------------------------"
+
+if docker buildx version &> /dev/null; then
+    BUILDX_VERSION=$(docker buildx version)
+    print_result 0 "Docker Buildx is available: $BUILDX_VERSION"
+else
+    print_result 1 "Docker Buildx is not available"
+    echo "Installing Docker Buildx..."
+    docker buildx install
+fi
+
+echo ""
+echo -e "${BLUE}Step 3: Setting up QEMU${NC}"
+echo "--------------------------------------"
+
+echo "Installing QEMU user static binaries..."
+if docker run --rm --privileged multiarch/qemu-user-static --reset -p yes > /dev/null 2>&1; then
+    print_result 0 "QEMU installation successful"
+else
+    print_result 1 "QEMU installation failed"
+fi
+
+echo ""
+echo -e "${BLUE}Step 4: Checking QEMU binaries${NC}"
+echo "--------------------------------------"
+
+if docker run --rm multiarch/qemu-user-static --version > /dev/null 2>&1; then
+    QEMU_VERSION=$(docker run --rm multiarch/qemu-user-static --version | head -n 1)
+    print_result 0 "QEMU binaries are functional: $QEMU_VERSION"
+else
+    print_result 1 "QEMU binaries not accessible"
+fi
+
+echo ""
+echo -e "${BLUE}Step 5: Setting up Buildx builder${NC}"
+echo "--------------------------------------"
+
+if docker buildx inspect multiarch &> /dev/null; then
+    echo "Builder 'multiarch' already exists"
+    docker buildx use multiarch
+    print_result 0 "Using existing builder 'multiarch'"
+else
+    if docker buildx create --name multiarch --driver docker-container --use > /dev/null 2>&1; then
+        print_result 0 "Created new builder 'multiarch'"
+    else
+        print_result 1 "Failed to create builder"
+    fi
+fi
+
+if docker buildx inspect --bootstrap > /dev/null 2>&1; then
+    print_result 0 "Builder bootstrap successful"
+else
+    print_result 1 "Builder bootstrap failed"
+fi
+
+echo ""
+echo -e "${BLUE}Step 6: Checking supported platforms${NC}"
+echo "--------------------------------------"
+
+PLATFORMS=$(docker buildx inspect multiarch | grep "Platforms:" | cut -d: -f2)
+echo "Available platforms:$PLATFORMS"
+
+if echo "$PLATFORMS" | grep -q "linux/amd64"; then
+    print_result 0 "AMD64 platform supported"
+else
+    print_result 1 "AMD64 platform not supported"
+fi
+
+if echo "$PLATFORMS" | grep -q "linux/arm64"; then
+    print_result 0 "ARM64 platform supported"
+else
+    print_result 1 "ARM64 platform not supported"
+fi
+
+echo ""
+echo -e "${BLUE}Step 7: Testing multi-arch build (dry run)${NC}"
+echo "--------------------------------------"
+
+# Create a simple test Dockerfile
+TEST_DIR=$(mktemp -d)
+cat > "$TEST_DIR/Dockerfile" << 'EOF'
+FROM alpine:latest
+RUN echo "Architecture: $(uname -m)"
+CMD ["echo", "Multi-arch test successful"]
+EOF
+
+echo "Testing build for linux/amd64..."
+if docker buildx build --platform linux/amd64 -t test-qemu:amd64 "$TEST_DIR" > /dev/null 2>&1; then
+    print_result 0 "AMD64 build successful"
+else
+    print_result 1 "AMD64 build failed"
+fi
+
+echo "Testing build for linux/arm64..."
+if docker buildx build --platform linux/arm64 -t test-qemu:arm64 "$TEST_DIR" > /dev/null 2>&1; then
+    print_result 0 "ARM64 build successful (cross-compiled)"
+else
+    print_result 1 "ARM64 build failed"
+fi
+
+echo "Testing multi-platform build..."
+if docker buildx build --platform linux/amd64,linux/arm64 -t test-qemu:multi "$TEST_DIR" > /dev/null 2>&1; then
+    print_result 0 "Multi-platform build successful"
+else
+    print_result 1 "Multi-platform build failed"
+fi
+
+# Cleanup
+rm -rf "$TEST_DIR"
+
+echo ""
+echo -e "${BLUE}Step 8: Validating CI/CD configurations${NC}"
+echo "--------------------------------------"
+
+# Check GitHub Actions
+if grep -q "docker/setup-qemu-action" .github/workflows/ci.yml 2>/dev/null; then
+    print_result 0 "GitHub Actions CI has QEMU configured"
+else
+    print_result 1 "GitHub Actions CI missing QEMU"
+fi
+
+if grep -q "docker/setup-qemu-action" .github/workflows/release.yml 2>/dev/null; then
+    print_result 0 "GitHub Actions Release has QEMU configured"
+else
+    print_result 1 "GitHub Actions Release missing QEMU"
+fi
+
+# Check CircleCI
+if grep -q "multiarch/qemu-user-static" .circleci/config.yml 2>/dev/null; then
+    print_result 0 "CircleCI has QEMU configured"
+else
+    print_result 1 "CircleCI missing QEMU"
+fi
+
+# Check GitLab CI
+if grep -q "multiarch/qemu-user-static" .gitlab-ci.yml 2>/dev/null; then
+    print_result 0 "GitLab CI has QEMU configured"
+else
+    print_result 1 "GitLab CI missing QEMU"
+fi
+
+# Check Jenkins
+if grep -q "multiarch/qemu-user-static" Jenkinsfile 2>/dev/null; then
+    print_result 0 "Jenkins has QEMU configured"
+else
+    print_result 1 "Jenkins missing QEMU"
+fi
+
+echo ""
+echo "======================================"
+echo -e "${BLUE}Validation Summary${NC}"
+echo "======================================"
+echo ""
+echo -e "${GREEN}Passed: $PASSED${NC}"
+echo -e "${RED}Failed: $FAILED${NC}"
+echo ""
+
+if [ $FAILED -eq 0 ]; then
+    echo -e "${GREEN}🎉 All validations passed!${NC}"
+    echo ""
+    echo "Your system is ready for multi-architecture builds."
+    echo ""
+    echo "Next steps:"
+    echo "  1. Run: ./scripts/build-multiarch.sh myapp latest"
+    echo "  2. Or push to CI/CD and watch multi-arch builds happen automatically"
+    echo ""
+    exit 0
+else
+    echo -e "${RED}⚠️  Some validations failed${NC}"
+    echo ""
+    echo "Please review the failures above and fix them before proceeding."
+    echo ""
+    echo "Common fixes:"
+    echo "  - Install Docker: https://docs.docker.com/get-docker/"
+    echo "  - Update Docker to latest version"
+    echo "  - Run with sudo if permission denied"
+    echo ""
+    exit 1
+fi