From bab922bcd02594f8a67413b2af5f48224ce560eb Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sun, 28 Dec 2025 18:32:04 +0000 Subject: [PATCH] Add comprehensive QEMU UEFI testing documentation Co-authored-by: johndoe6345789 <224850594+johndoe6345789@users.noreply.github.com> --- README.md | 13 +- docs/BUILD.md | 55 +++++- docs/QEMU_TESTING.md | 462 +++++++++++++++++++++++++++++++++++++++++++ docs/TESTING.md | 37 +++- 4 files changed, 555 insertions(+), 12 deletions(-) create mode 100644 docs/QEMU_TESTING.md diff --git a/README.md b/README.md index 99ce0ec..7c29f45 100644 --- a/README.md +++ b/README.md @@ -57,11 +57,20 @@ See [docs/ROADMAP.md](docs/ROADMAP.md) for detailed phase breakdown. ```bash make all # Build bootloader, kernel, and userspace make test # Run unit tests -make qemu # Test in QEMU +make qemu # Test in QEMU with UEFI firmware make clean # Clean build artifacts ``` -See [docs/BUILD.md](docs/BUILD.md) for detailed build instructions. +**QEMU UEFI Testing**: +```bash +make qemu # Boot in QEMU with UEFI (headless) +make qemu QEMU_DISPLAY=gtk # Boot with graphical display +make qemu-debug # Boot with debug output +make qemu-gdb # Boot with GDB debugging +make qemu-uefi-test # Test UEFI firmware setup +``` + +See [docs/BUILD.md](docs/BUILD.md) for detailed build instructions and [docs/TESTING.md](docs/TESTING.md) for testing guide. ## Documentation diff --git a/docs/BUILD.md b/docs/BUILD.md index 873c790..a67f278 100644 --- a/docs/BUILD.md +++ b/docs/BUILD.md @@ -88,30 +88,79 @@ This creates `build/metalos.img` - a bootable disk image containing: ## Testing in QEMU -### Boot MetalOS +MetalOS includes comprehensive QEMU support with UEFI firmware (OVMF) for testing. + +### Prerequisites + +Ensure QEMU and OVMF are installed: + +```bash +# Ubuntu/Debian +sudo apt-get install qemu-system-x86 ovmf mtools + +# Arch Linux +sudo pacman -S qemu-full edk2-ovmf mtools + +# Fedora +sudo dnf install qemu-system-x86 edk2-ovmf mtools +``` + +### Boot MetalOS in QEMU ```bash make qemu ``` +This will: +1. Build bootloader and kernel (or use placeholders if build fails) +2. Create a bootable FAT32 disk image with UEFI boot structure +3. Launch QEMU with OVMF UEFI firmware in headless mode +4. Boot the system (will drop to UEFI shell until bootloader is complete) + +**Display Options**: By default, QEMU runs in headless mode (no graphics). To use graphical display: + +```bash +# Use GTK display (if available) +make qemu QEMU_DISPLAY=gtk + +# Use SDL display (if available) +make qemu QEMU_DISPLAY=sdl +``` + ### Boot with Debug Output ```bash make qemu-debug ``` +This includes CPU interrupt and reset debugging output. + ### Boot with GDB Support +For debugging with GDB: + ```bash -# Terminal 1 +# Terminal 1 - Start QEMU with GDB server make qemu-gdb -# Terminal 2 +# Terminal 2 - Connect GDB gdb kernel/metalos.bin (gdb) target remote localhost:1234 (gdb) continue ``` +QEMU will wait for GDB connection before starting execution. + +### Test QEMU UEFI Setup + +To verify QEMU and OVMF are properly installed without needing a bootable OS image: + +```bash +make qemu-uefi-test +``` + +This boots directly to the UEFI shell, confirming your QEMU+OVMF setup works correctly. + ## Testing on Real Hardware ⚠️ **WARNING**: Testing on real hardware can be risky. Always backup your data. diff --git a/docs/QEMU_TESTING.md b/docs/QEMU_TESTING.md new file mode 100644 index 0000000..30589c9 --- /dev/null +++ b/docs/QEMU_TESTING.md @@ -0,0 +1,462 @@ +# QEMU UEFI Testing Guide + +This guide covers testing MetalOS in QEMU with UEFI firmware (OVMF). + +## Overview + +MetalOS uses QEMU with OVMF (Open Virtual Machine Firmware) to test UEFI boot functionality without requiring physical hardware. This enables rapid development and testing of the bootloader and kernel. + +## Prerequisites + +### Required Software + +1. **QEMU** (version 6.0 or later recommended) + - Provides x86_64 virtualization + - Supports UEFI firmware + +2. **OVMF** (UEFI firmware for QEMU) + - Open-source UEFI implementation + - Required for UEFI boot in QEMU + +3. **mtools** (for disk image creation) + - Creates FAT32 filesystems + - Manipulates MS-DOS filesystems + +### Installation + +**Ubuntu/Debian**: +```bash +sudo apt-get update +sudo apt-get install qemu-system-x86 ovmf mtools xorriso +``` + +**Arch Linux**: +```bash +sudo pacman -S qemu-full edk2-ovmf mtools xorriso +``` + +**Fedora**: +```bash +sudo dnf install qemu-system-x86 edk2-ovmf mtools xorriso +``` + +**macOS**: +```bash +brew install qemu mtools xorriso +# Note: OVMF may need manual installation +``` + +## Quick Start + +### Basic QEMU Boot + +```bash +# Build and boot MetalOS in QEMU +make qemu +``` + +This command: +1. Builds the bootloader and kernel (or uses placeholders) +2. Creates a FAT32 disk image with UEFI boot structure +3. Launches QEMU with OVMF firmware +4. Boots the system in headless mode + +**Expected Output**: +``` +Building bootloader... +Building kernel... +Creating bootable image... +Using mtools to create FAT32 disk image... +Success! Created build/metalos.img +Starting QEMU with UEFI firmware... +Using OVMF firmware: /usr/share/ovmf/OVMF.fd +Display mode: none (set QEMU_DISPLAY=gtk for graphical) +[UEFI boot messages...] +``` + +### Verify UEFI Setup + +To verify that QEMU and OVMF are properly installed: + +```bash +make qemu-uefi-test +``` + +This boots directly to the UEFI shell without any OS image, confirming your setup works. + +## Display Modes + +MetalOS supports multiple display modes for different use cases: + +### Headless Mode (Default) + +```bash +make qemu +# or explicitly +make qemu QEMU_DISPLAY=none +``` + +**Use cases**: +- CI/CD pipelines +- Headless servers +- Automated testing +- Serial console debugging + +**Characteristics**: +- No graphical window +- Serial output only +- Works in any environment + +### Graphical Mode - GTK + +```bash +make qemu QEMU_DISPLAY=gtk +``` + +**Use cases**: +- Interactive development +- Visual debugging +- UI testing + +**Characteristics**: +- Native GTK window +- Full graphics support +- Requires X11/Wayland + +### Graphical Mode - SDL + +```bash +make qemu QEMU_DISPLAY=sdl +``` + +**Use cases**: +- Cross-platform development +- Alternative to GTK + +**Characteristics**: +- SDL window +- Good compatibility +- Lighter than GTK + +### Text Mode - Curses + +```bash +make qemu QEMU_DISPLAY=curses +``` + +**Use cases**: +- SSH sessions +- Text-only environments + +**Characteristics**: +- Terminal-based UI +- No X11 required +- Works over SSH + +## Advanced Testing + +### Debug Mode + +Boot with CPU interrupt and reset debugging: + +```bash +make qemu-debug +``` + +**Additional debug output includes**: +- CPU interrupts +- CPU resets +- Exception handling +- Hardware events + +### GDB Debugging + +Debug the kernel with GDB: + +**Terminal 1** - Start QEMU with GDB server: +```bash +make qemu-gdb +``` + +QEMU will wait for GDB connection on `localhost:1234`. + +**Terminal 2** - Connect GDB: +```bash +gdb kernel/metalos.bin +(gdb) target remote localhost:1234 +(gdb) break kernel_main +(gdb) continue +``` + +**Useful GDB commands**: +```gdb +# Set breakpoints +break *0x100000 # Break at address +break kernel_main # Break at function + +# Examine memory +x/10i $rip # Disassemble 10 instructions +x/10x 0x100000 # Show 10 hex values + +# Step through code +stepi # Step one instruction +nexti # Step over calls +continue # Continue execution + +# Register inspection +info registers # Show all registers +print $rax # Print specific register +``` + +## Disk Image Details + +### Image Structure + +The `build/metalos.img` file contains: + +``` +metalos.img (64 MB FAT32 disk) +├── EFI/ +│ └── BOOT/ +│ └── bootx64.efi # UEFI bootloader +└── metalos.bin # Kernel binary +``` + +### Inspecting the Image + +View the disk image contents: + +```bash +# List files in the image +mdir -i build/metalos.img ::/ + +# List EFI boot directory +mdir -i build/metalos.img ::/EFI/BOOT/ + +# Extract a file +mcopy -i build/metalos.img ::/metalos.bin extracted_kernel.bin + +# Get image info +file build/metalos.img +``` + +### Manual Image Creation + +If you need to manually create or modify the image: + +```bash +# Create empty 64MB image +dd if=/dev/zero of=disk.img bs=1M count=64 + +# Format as FAT32 +mformat -i disk.img -F -v "METALOS" :: + +# Create directory structure +mmd -i disk.img ::/EFI +mmd -i disk.img ::/EFI/BOOT + +# Copy files +mcopy -i disk.img bootloader/bootx64.efi ::/EFI/BOOT/ +mcopy -i disk.img kernel/metalos.bin ::/ +``` + +## OVMF Firmware + +### Firmware Locations + +The Makefile automatically detects OVMF firmware in these locations: + +- `/usr/share/OVMF/OVMF_CODE.fd` (Ubuntu/Debian) +- `/usr/share/ovmf/OVMF.fd` (Ubuntu/Debian alternative) +- `/usr/share/edk2-ovmf/x64/OVMF_CODE.fd` (Arch Linux) +- `/usr/share/qemu/ovmf-x86_64.bin` (Other systems) + +### Firmware Variants + +**OVMF_CODE.fd**: Code-only firmware (read-only) +- Used with `-drive if=pflash,readonly=on` +- Best for testing +- Variables stored separately + +**OVMF.fd**: Combined code and variables +- Single file for firmware +- Simpler setup +- Variables reset on each boot + +**OVMF.secboot.fd**: Secure Boot enabled +- For secure boot testing +- Requires signed bootloader + +## Troubleshooting + +### QEMU Won't Start + +**Issue**: `gtk initialization failed` or `Could not initialize SDL` + +**Solution**: Use headless mode: +```bash +make qemu QEMU_DISPLAY=none +``` + +### OVMF Not Found + +**Issue**: `Error: OVMF UEFI firmware not found!` + +**Solution**: Install OVMF package: +```bash +# Ubuntu/Debian +sudo apt-get install ovmf + +# Arch Linux +sudo pacman -S edk2-ovmf +``` + +### Image Creation Fails + +**Issue**: `mformat: command not found` + +**Solution**: Install mtools: +```bash +# Ubuntu/Debian +sudo apt-get install mtools + +# Arch Linux +sudo pacman -S mtools +``` + +### Boots to UEFI Shell + +**Expected Behavior**: During development, booting to UEFI shell is normal until the bootloader is complete. + +**What it means**: +- ✅ QEMU and OVMF working correctly +- ✅ Disk image recognized (shows as FS0) +- ⚠️ Bootloader not yet functional (expected in Phase 1) + +**Next steps**: Implement Phase 2 (UEFI Bootloader) + +### Serial Output Missing + +**Issue**: No serial console output + +**Solution**: Ensure serial port is configured correctly: +- Output goes to stdio by default +- Check that `-serial stdio` is in QEMU command +- Verify kernel writes to serial port + +### QEMU Hangs + +**Issue**: QEMU appears to hang during boot + +**Possible causes**: +1. Bootloader infinite loop +2. Waiting for input +3. Hardware initialization issue + +**Debug steps**: +```bash +# Use debug mode +make qemu-debug + +# Use GDB to inspect state +make qemu-gdb +# In another terminal: gdb kernel/metalos.bin +``` + +## CI/CD Integration + +### GitHub Actions + +Example workflow for QEMU testing: + +```yaml +name: QEMU Boot Test + +on: [push, pull_request] + +jobs: + qemu-test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Install dependencies + run: | + sudo apt-get update + sudo apt-get install -y qemu-system-x86 ovmf mtools + + - name: Build and test in QEMU + run: | + make qemu QEMU_DISPLAY=none & + QEMU_PID=$! + sleep 10 + kill $QEMU_PID || true +``` + +### Docker Testing + +Run QEMU tests in Docker: + +```dockerfile +FROM ubuntu:latest + +RUN apt-get update && apt-get install -y \ + build-essential nasm qemu-system-x86 ovmf mtools + +WORKDIR /metalos +COPY . . + +CMD ["make", "qemu", "QEMU_DISPLAY=none"] +``` + +## Performance Tuning + +### Memory Allocation + +Default: 512 MB + +Adjust memory: +```bash +# Edit Makefile QEMU command +qemu-system-x86_64 ... -m 1024M # Use 1GB +``` + +### CPU Configuration + +Use multiple cores: +```bash +qemu-system-x86_64 ... -smp 2 # 2 cores +``` + +### KVM Acceleration + +Enable KVM for faster emulation (Linux only): +```bash +qemu-system-x86_64 ... -enable-kvm +``` + +**Note**: Requires KVM kernel module and permissions. + +## Best Practices + +1. **Always test in headless mode first** - Ensures CI/CD compatibility +2. **Use serial output for debugging** - More reliable than graphics +3. **Keep disk images small** - 64 MB is sufficient for development +4. **Test with different OVMF versions** - Ensures compatibility +5. **Document boot behavior changes** - Track progress in commits +6. **Use GDB for serious debugging** - More powerful than printf +7. **Test on multiple systems** - Catch compatibility issues early + +## Resources + +- [QEMU Documentation](https://www.qemu.org/docs/master/) +- [OVMF Wiki](https://github.com/tianocore/tianocore.github.io/wiki/OVMF) +- [UEFI Specification](https://uefi.org/specifications) +- [OSDev Wiki - UEFI](https://wiki.osdev.org/UEFI) + +## Related Documentation + +- [BUILD.md](BUILD.md) - Build system and toolchain +- [TESTING.md](TESTING.md) - Complete testing guide +- [ROADMAP.md](ROADMAP.md) - Development phases +- [DEVELOPMENT.md](DEVELOPMENT.md) - Development workflow diff --git a/docs/TESTING.md b/docs/TESTING.md index 99b8bd1..c81276b 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -76,14 +76,14 @@ int main(void) { ### 2. QEMU Integration Tests -QEMU tests verify the full system boots correctly in an emulated environment. +QEMU tests verify the full system boots correctly in an emulated UEFI environment. **Location**: `.github/workflows/qemu-test.yml` **What it does**: -1. Builds bootloader and kernel -2. Creates bootable image -3. Launches QEMU with UEFI firmware +1. Builds bootloader and kernel (or uses placeholders during development) +2. Creates bootable UEFI disk image with FAT32 filesystem +3. Launches QEMU with OVMF (UEFI firmware) 4. Waits for boot sequence (configurable timeout) 5. Captures screenshot of boot state 6. Captures serial output @@ -96,17 +96,40 @@ QEMU tests verify the full system boots correctly in an emulated environment. These artifacts are uploaded to GitHub Actions for inspection. **Running locally**: + ```bash -# Build and test in QEMU +# Build and test in QEMU with UEFI make qemu -# With debug output +# Use graphical display (if not in headless environment) +make qemu QEMU_DISPLAY=gtk + +# With debug output (shows CPU interrupts and resets) make qemu-debug -# With GDB server +# With GDB server for debugging make qemu-gdb + +# Test UEFI firmware setup only (no OS image required) +make qemu-uefi-test ``` +**QEMU UEFI Testing Features**: +- ✅ Automatic OVMF firmware detection across different Linux distributions +- ✅ FAT32 disk image creation with proper UEFI boot structure +- ✅ Headless mode support for CI/CD environments +- ✅ Optional graphical display for local development +- ✅ Serial console output for debugging +- ✅ GDB integration for kernel debugging + +**Display Modes**: +- `none` (default): Headless mode, no graphics, works in CI/CD +- `gtk`: GTK-based graphical window +- `sdl`: SDL-based graphical window +- `curses`: Text-based UI in terminal + +Set display mode: `make qemu QEMU_DISPLAY=` + ## Test Framework MetalOS uses a custom minimal test framework (`tests/include/test_framework.h`) with: