diff --git a/README.md b/README.md index 7cea307..b6123b7 100644 --- a/README.md +++ b/README.md @@ -19,72 +19,19 @@ This OS exists solely to run **one QT6 application** on **AMD64 + Radeon RX 6600 ✅ **Creative freedom** - Not bound by POSIX or tradition ✅ **Precise drivers** - Hardware code follows specs exactly +## GPU Implementation Strategy -1) Reality check: where the bloat really lives (RDNA2) +MetalOS leverages Mesa RADV (userspace Vulkan driver) with a minimal kernel-side GPU API to achieve high performance without excessive complexity. The strategy focuses on implementing only the essential kernel interfaces that RADV requires: -On Navi 23, you will not get good performance without: - • GPU firmware blobs (various dimgrey_cavefish_*.bin files; Navi 23’s codename is “dimgrey cavefish”, and Linux systems load firmware files with that prefix).  - • A real memory manager (VRAM/GTT, page tables, buffer objects) - • Command submission (rings/queues) + fences/semaphores - • A Vulkan driver implementation (or reuse one) +- **Firmware loading** and ASIC initialization for Navi 23 +- **Buffer objects** (VRAM/GTT management) +- **Virtual memory** (GPU page tables) +- **Command submission** (rings/queues) and synchronization primitives -So the “least bloat” strategy is: reuse a Vulkan implementation (Mesa RADV is the obvious candidate), but avoid importing a whole Unix stack by giving it a very small kernel/userspace interface tailored to your OS. +This approach keeps the OS non-POSIX while avoiding the complexity of writing a Vulkan driver from scratch. -RADV is explicitly a userspace Vulkan driver for modern AMD GPUs.  +For detailed implementation notes, see [docs/GPU_IMPLEMENTATION.md](docs/GPU_IMPLEMENTATION.md). -⸻ - -2) The best “toy OS but fast” plan: RADV + a tiny amdgpu-shaped shim - -Why this is the sweet spot - • You keep your OS non-POSIX. - • You avoid writing a Vulkan driver from scratch (the truly hard part). - • You implement only the kernel-facing parts RADV needs: a buffer object + VM + submit + sync API. - -Shape of the stack - -MetalOS kernel - • PCIe enumeration, BAR mapping - • interrupts (MSI/MSI-X) - • DMA mapping (or identity-map if you’re being reckless) - • a GPU kernel driver that exposes a small ioctl-like API - -Userspace - • gpu-service (optional but recommended for structure) - • libradv-metal (a minimal libdrm-like bridge) - • Mesa RADV compiled against your bridge (not Linux libdrm) - -This is “Unix-like internally” only in the sense of interfaces, not user experience. - -⸻ - -3) Minimal kernel GPU API (the smallest set that still performs) - -Think in terms of four pillars: - -A) Firmware load + ASIC init - • gpu_load_firmware(name, blob) - • gpu_init() → returns chip info (gfx1032, VRAM size, doorbells, etc.) - -You will need those Navi23 firmware blobs (again: dimgrey_cavefish_*.bin family is the practical breadcrumb).  - -B) Buffer objects (BOs) - • bo_create(size, domain=VRAM|GTT, flags) - • bo_map(bo) / bo_unmap(bo) (CPU mapping) - • bo_export_handle(bo) (so Vulkan can bind memory) - -C) Virtual memory (GPU page tables) - • vm_create() - • vm_map(vm, bo, gpu_va, size, perms) - • vm_unmap(vm, gpu_va, size) - -D) Submission + synchronization - • queue_create(type=GFX|COMPUTE|DMA) - • queue_submit(queue, cs_buffer, fence_out) - • fence_wait(fence, timeout) - • timeline_semaphore_* (optional, but hugely useful) - -If you implement these correctly, you get real GPU throughput. ## What We Cut @@ -138,9 +85,21 @@ make qemu-uefi-test # Test UEFI firmware setup See [docs/BUILD.md](docs/BUILD.md) for detailed build instructions and [docs/TESTING.md](docs/TESTING.md) for testing guide. +## Dependencies + +MetalOS manages third-party dependencies in-house for reproducibility and offline development: + +- **GPU Firmware** - AMD Navi 23 firmware blobs (dimgrey_cavefish_*.bin) +- **Mesa RADV** - Vulkan driver for AMD GPUs +- **QT6** - Application framework (minimal static build) +- **OVMF** - UEFI firmware for QEMU testing + +See [deps/README.md](deps/README.md) for detailed dependency management instructions. + ## Documentation - [ARCHITECTURE.md](docs/ARCHITECTURE.md) - System architecture and design +- [GPU_IMPLEMENTATION.md](docs/GPU_IMPLEMENTATION.md) - GPU driver strategy and implementation - [MINIMALISM.md](docs/MINIMALISM.md) - Extreme minimalism philosophy - [ROADMAP.md](docs/ROADMAP.md) - Development phases and milestones - [BUILD.md](docs/BUILD.md) - Build system and toolchain diff --git a/deps/README.md b/deps/README.md new file mode 100644 index 0000000..072c569 --- /dev/null +++ b/deps/README.md @@ -0,0 +1,156 @@ +# MetalOS Dependencies + +This directory contains third-party dependencies managed in-house for MetalOS development. + +## Directory Structure + +``` +deps/ +├── firmware/ # AMD GPU firmware blobs +├── mesa-radv/ # Mesa RADV Vulkan driver source +├── qt6/ # QT6 framework (minimal build) +└── ovmf/ # UEFI firmware for testing +``` + +## Firmware Blobs + +**Location**: `deps/firmware/` + +AMD Radeon RX 6600 (Navi 23) requires several firmware files: +- `dimgrey_cavefish_ce.bin` - Command Engine firmware +- `dimgrey_cavefish_me.bin` - MicroEngine firmware +- `dimgrey_cavefish_mec.bin` - MEC (Compute) firmware +- `dimgrey_cavefish_pfp.bin` - Pre-Fetch Parser firmware +- `dimgrey_cavefish_rlc.bin` - Run List Controller firmware +- `dimgrey_cavefish_sdma.bin` - SDMA engine firmware +- `dimgrey_cavefish_vcn.bin` - Video Core Next firmware + +### Obtaining Firmware + +These firmware files are available from the Linux firmware repository: +```bash +# Clone linux-firmware repository +git clone https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git /tmp/linux-firmware + +# Copy Navi 23 firmware files +cp /tmp/linux-firmware/amdgpu/dimgrey_cavefish_*.bin deps/firmware/ +``` + +**License**: These files are redistributable under the AMD GPU firmware license (see firmware files for details). + +## Mesa RADV + +**Location**: `deps/mesa-radv/` + +Mesa RADV is the userspace Vulkan driver for AMD GPUs. MetalOS uses RADV to avoid implementing a Vulkan driver from scratch. + +### Setup + +```bash +# Clone Mesa repository (or download specific release) +git clone https://gitlab.freedesktop.org/mesa/mesa.git /tmp/mesa +cd /tmp/mesa + +# Checkout stable version (e.g., 24.0) +git checkout mesa-24.0.0 + +# Copy RADV driver sources to deps +cp -r src/amd/vulkan ../MetalOS/deps/mesa-radv/ +``` + +We will compile RADV against our custom `libradv-metal` bridge instead of Linux's libdrm. + +**License**: MIT License (see Mesa source for full license text) + +## QT6 Framework + +**Location**: `deps/qt6/` + +QT6 is the application framework used by the single MetalOS application. + +### Setup + +```bash +# Download QT6 source (minimal modules only) +# We only need QtCore, QtGui, and QtWidgets +wget https://download.qt.io/official_releases/qt/6.5/6.5.3/single/qt-everywhere-src-6.5.3.tar.xz + +# Extract and configure minimal build +tar xf qt-everywhere-src-6.5.3.tar.xz +cd qt-everywhere-src-6.5.3 + +# Configure for MetalOS (minimal, static linking) +./configure -static -no-dbus -no-ssl -no-cups -no-fontconfig \ + -prefix $PWD/../deps/qt6 +``` + +**Note**: Adjust the path as needed based on your directory structure. The example above assumes you extracted QT source alongside the MetalOS repository. + +**License**: LGPL v3 / GPL v2 / Commercial (see QT documentation) + +## OVMF (EDK II) + +**Location**: `deps/ovmf/` + +OVMF provides UEFI firmware for QEMU testing. + +### Setup + +```bash +# Download pre-built OVMF binaries +mkdir -p deps/ovmf +wget https://www.kraxel.org/repos/jenkins/edk2/edk2.git-ovmf-x64-0-20230524.209.gf0064ac3af.EOL.no.nore.updates.noarch.rpm + +# Extract OVMF files +rpm2cpio edk2.git-ovmf-*.rpm | cpio -idmv +cp usr/share/edk2/ovmf/OVMF_CODE.fd deps/ovmf/ +cp usr/share/edk2/ovmf/OVMF_VARS.fd deps/ovmf/ +``` + +Alternatively, build from source: +```bash +git clone https://github.com/tianocore/edk2.git +cd edk2 +git submodule update --init +make -C BaseTools +source edksetup.sh +build -a X64 -t GCC5 -p OvmfPkg/OvmfPkgX64.dsc +``` + +**License**: BSD-2-Clause (see EDK II license) + +## Version Management + +Each dependency should be tracked with a specific version/commit: + +- **Firmware**: Linux firmware commit hash +- **Mesa RADV**: Mesa version tag (e.g., `mesa-24.0.0`) +- **QT6**: QT version (e.g., `6.5.3`) +- **OVMF**: EDK II commit hash or release tag + +Create a `VERSION` file in each subdirectory documenting the exact version in use. + +## Build Integration + +The main Makefile will be updated to: +1. Check for dependencies in `deps/` +2. Use in-house versions when available +3. Fall back to system versions if needed +4. Provide targets to download/build dependencies + +## Rationale + +Managing dependencies in-house provides: +- **Reproducibility**: Exact versions are locked and available +- **Offline development**: No external downloads required +- **Control**: We can patch or customize dependencies as needed +- **Simplicity**: Single repository contains everything needed to build + +## Size Considerations + +The deps directory may become large (~500MB-1GB). Consider: +- Using `.gitattributes` for Git LFS on large binary files +- Documenting exact download/build steps instead of committing binaries +- Selective inclusion based on development phase + +For now, we document the structure and process. Actual population of dependencies will occur as needed during implementation phases. diff --git a/deps/firmware/README.md b/deps/firmware/README.md new file mode 100644 index 0000000..21428a7 --- /dev/null +++ b/deps/firmware/README.md @@ -0,0 +1,40 @@ +# AMD GPU Firmware Blobs + +This directory contains firmware files for the AMD Radeon RX 6600 (Navi 23 / "dimgrey cavefish"). + +## Required Files + +- `dimgrey_cavefish_ce.bin` - Command Engine firmware +- `dimgrey_cavefish_me.bin` - MicroEngine firmware +- `dimgrey_cavefish_mec.bin` - MEC (Compute) firmware +- `dimgrey_cavefish_pfp.bin` - Pre-Fetch Parser firmware +- `dimgrey_cavefish_rlc.bin` - Run List Controller firmware +- `dimgrey_cavefish_sdma.bin` - SDMA engine firmware +- `dimgrey_cavefish_vcn.bin` - Video Core Next firmware + +## Source + +These files are available from the [linux-firmware repository](https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git). + +## Obtaining Firmware + +```bash +# From repository root +git clone --depth 1 https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git /tmp/linux-firmware +cp /tmp/linux-firmware/amdgpu/dimgrey_cavefish_*.bin deps/firmware/ +``` + +## License + +These firmware files are redistributable under the AMD GPU firmware license included with each file. + +## Version Tracking + +Create a `VERSION` file documenting: +- Linux firmware repository commit hash +- Date obtained +- File checksums (SHA256) + +## Status + +⚠️ **Not yet populated** - Firmware files will be added during Phase 4 (Hardware Support) diff --git a/deps/mesa-radv/README.md b/deps/mesa-radv/README.md new file mode 100644 index 0000000..fc057cc --- /dev/null +++ b/deps/mesa-radv/README.md @@ -0,0 +1,52 @@ +# Mesa RADV Driver + +This directory will contain the Mesa RADV (AMD Vulkan driver) source code. + +## Overview + +RADV is a userspace Vulkan driver for AMD GPUs. MetalOS uses RADV to provide Vulkan support without implementing a driver from scratch. + +## Source + +Mesa RADV is part of the [Mesa 3D Graphics Library](https://gitlab.freedesktop.org/mesa/mesa). + +## Setup Instructions + +```bash +# Clone Mesa repository +git clone https://gitlab.freedesktop.org/mesa/mesa.git /tmp/mesa +cd /tmp/mesa + +# Checkout stable version +git checkout mesa-24.0.0 + +# Copy RADV driver sources to deps +# Assuming you're in the mesa source directory and MetalOS repo is in parent +cp -r src/amd/vulkan/* ../MetalOS/deps/mesa-radv/ +``` + +## Integration with MetalOS + +MetalOS will compile RADV against a custom `libradv-metal` bridge that provides: +- Buffer object management +- Virtual memory (GPU page tables) +- Command submission +- Synchronization primitives + +This replaces the standard Linux libdrm dependency. + +## License + +MIT License - See Mesa source for full license text. + +## Version Tracking + +Create a `VERSION` file documenting: +- Mesa version/tag (e.g., `mesa-24.0.0`) +- Git commit hash +- Date obtained +- Any local modifications + +## Status + +⚠️ **Not yet populated** - RADV sources will be added during Phase 7 (QT6 Port) diff --git a/deps/ovmf/README.md b/deps/ovmf/README.md new file mode 100644 index 0000000..bdb4bfa --- /dev/null +++ b/deps/ovmf/README.md @@ -0,0 +1,87 @@ +# OVMF UEFI Firmware + +This directory contains OVMF (Open Virtual Machine Firmware) for QEMU testing. + +## Overview + +OVMF provides UEFI firmware implementation for QEMU, allowing us to test MetalOS in a UEFI environment. + +## Required Files + +- `OVMF_CODE.fd` - UEFI firmware code +- `OVMF_VARS.fd` - UEFI variables template + +## Source + +OVMF is part of the [EDK II project](https://github.com/tianocore/edk2). + +## Obtaining OVMF + +### Option 1: Use System-Installed OVMF (Recommended) + +Many distributions provide OVMF packages: +```bash +# Ubuntu/Debian +sudo apt-get install ovmf + +# Copy to deps folder +cp /usr/share/OVMF/OVMF_CODE.fd deps/ovmf/ +cp /usr/share/OVMF/OVMF_VARS.fd deps/ovmf/ +``` + +### Option 2: Download Pre-built Binaries + +Pre-built OVMF binaries are available from various sources: + +```bash +# Visit https://www.kraxel.org/repos/jenkins/edk2/ for current builds +# Or use direct links from distributions' mirror sites +# Example (adjust URL to current available version): +wget https://www.kraxel.org/repos/jenkins/edk2/edk2.git-ovmf-x64-.rpm + +rpm2cpio edk2.git-ovmf-*.rpm | cpio -idmv +cp usr/share/edk2/ovmf/OVMF_CODE.fd deps/ovmf/ +cp usr/share/edk2/ovmf/OVMF_VARS.fd deps/ovmf/ +``` + +**Note**: Pre-built binaries may become unavailable. System packages (Option 1) or building from source (Option 3) are more reliable long-term. + +### Option 3: Build from Source + +```bash +git clone https://github.com/tianocore/edk2.git +cd edk2 +git submodule update --init +make -C BaseTools +source edksetup.sh +build -a X64 -t GCC5 -p OvmfPkg/OvmfPkgX64.dsc + +# Files will be in Build/OvmfX64/RELEASE_GCC5/FV/ +cp Build/OvmfX64/RELEASE_GCC5/FV/OVMF_CODE.fd ../MetalOS/deps/ovmf/ +cp Build/OvmfX64/RELEASE_GCC5/FV/OVMF_VARS.fd ../MetalOS/deps/ovmf/ +``` + +## Usage with QEMU + +```bash +qemu-system-x86_64 \ + -bios deps/ovmf/OVMF_CODE.fd \ + -drive file=metalos.img,format=raw \ + -m 512M +``` + +## License + +BSD-2-Clause - See EDK II license for full details. + +## Version Tracking + +Create a `VERSION` file documenting: +- EDK II commit hash or version +- Build date +- Source (pre-built package, jenkins, or self-built) +- File checksums (SHA256) + +## Status + +⚠️ **Optional** - Can use system-installed OVMF or this in-house copy. The Makefile already detects system OVMF paths. diff --git a/deps/qt6/README.md b/deps/qt6/README.md new file mode 100644 index 0000000..86995d0 --- /dev/null +++ b/deps/qt6/README.md @@ -0,0 +1,66 @@ +# QT6 Framework + +This directory will contain a minimal QT6 build for MetalOS. + +## Overview + +QT6 provides the application framework for the single MetalOS application. We use a minimal, statically-linked build with only essential modules. + +## Required Modules + +- QtCore - Core non-GUI functionality +- QtGui - GUI components +- QtWidgets - Widget toolkit + +## Source + +QT6 is available from the [Qt Project](https://download.qt.io/official_releases/qt/). + +## Setup Instructions + +```bash +# Download QT6 source +wget https://download.qt.io/official_releases/qt/6.5/6.5.3/single/qt-everywhere-src-6.5.3.tar.xz +tar xf qt-everywhere-src-6.5.3.tar.xz +cd qt-everywhere-src-6.5.3 + +# Configure minimal build for MetalOS +./configure -static -release \ + -no-dbus -no-ssl -no-cups -no-fontconfig \ + -no-feature-network -no-feature-sql \ + -prefix $PWD/../deps/qt6 \ + -opensource -confirm-license + +# Build (this will take a while) +make -j$(nproc) +make install +``` + +## MetalOS-Specific Configuration + +We configure QT6 with: +- Static linking (no shared libraries) +- Minimal feature set +- No networking, database, or printing support +- Custom platform plugin for MetalOS framebuffer + +## License + +QT6 is available under: +- LGPL v3 +- GPL v2 +- Commercial License + +For MetalOS, we use LGPL v3. See QT documentation for license details. + +## Version Tracking + +Create a `VERSION` file documenting: +- QT version (e.g., `6.5.3`) +- Build configuration options +- Date built +- Any local patches + +## Status + +⚠️ **Not yet populated** - QT6 will be added during Phase 7 (QT6 Port) diff --git a/docs/GPU_IMPLEMENTATION.md b/docs/GPU_IMPLEMENTATION.md new file mode 100644 index 0000000..481282c --- /dev/null +++ b/docs/GPU_IMPLEMENTATION.md @@ -0,0 +1,100 @@ +# GPU Implementation Strategy + +## Overview + +This document outlines the GPU implementation strategy for MetalOS targeting the AMD Radeon RX 6600 (RDNA2 / Navi 23 architecture). + +## Reality Check: Where the Bloat Really Lives (RDNA2) + +On Navi 23, you will not get good performance without: +- GPU firmware blobs (various `dimgrey_cavefish_*.bin` files; Navi 23's codename is "dimgrey cavefish", and Linux systems load firmware files with that prefix) +- A real memory manager (VRAM/GTT, page tables, buffer objects) +- Command submission (rings/queues) + fences/semaphores +- A Vulkan driver implementation (or reuse one) + +So the "least bloat" strategy is: reuse a Vulkan implementation (Mesa RADV is the obvious candidate), but avoid importing a whole Unix stack by giving it a very small kernel/userspace interface tailored to your OS. + +RADV is explicitly a userspace Vulkan driver for modern AMD GPUs. + +--- + +## The Best "Toy OS but Fast" Plan: RADV + a Tiny amdgpu-shaped Shim + +### Why This is the Sweet Spot + +- You keep your OS non-POSIX +- You avoid writing a Vulkan driver from scratch (the truly hard part) +- You implement only the kernel-facing parts RADV needs: a buffer object + VM + submit + sync API + +### Shape of the Stack + +**MetalOS Kernel:** +- PCIe enumeration, BAR mapping +- Interrupts (MSI/MSI-X) +- DMA mapping (or identity-map if you're being reckless) +- A GPU kernel driver that exposes a small ioctl-like API + +**Userspace:** +- `gpu-service` (optional but recommended for structure) +- `libradv-metal` (a minimal libdrm-like bridge) +- Mesa RADV compiled against your bridge (not Linux libdrm) + +This is "Unix-like internally" only in the sense of interfaces, not user experience. + +--- + +## Minimal Kernel GPU API (The Smallest Set That Still Performs) + +Think in terms of four pillars: + +### A) Firmware Load + ASIC Init + +```c +gpu_load_firmware(name, blob) +gpu_init() → returns chip info (gfx1032, VRAM size, doorbells, etc.) +``` + +You will need those Navi23 firmware blobs (again: `dimgrey_cavefish_*.bin` family is the practical breadcrumb). + +### B) Buffer Objects (BOs) + +```c +bo_create(size, domain=VRAM|GTT, flags) +bo_map(bo) / bo_unmap(bo) // CPU mapping +bo_export_handle(bo) // so Vulkan can bind memory +``` + +### C) Virtual Memory (GPU Page Tables) + +```c +vm_create() +vm_map(vm, bo, gpu_va, size, perms) +vm_unmap(vm, gpu_va, size) +``` + +### D) Submission + Synchronization + +```c +queue_create(type=GFX|COMPUTE|DMA) +queue_submit(queue, cs_buffer, fence_out) +fence_wait(fence, timeout) +timeline_semaphore_* // optional, but hugely useful +``` + +If you implement these correctly, you get real GPU throughput. + +--- + +## Implementation Notes + +- Focus on the minimal API surface that RADV requires +- Firmware blobs are non-negotiable for Navi 23 performance +- Memory management (VRAM/GTT) is critical for proper GPU operation +- Command submission infrastructure must be solid for reliability +- Synchronization primitives (fences/semaphores) enable proper GPU-CPU coordination + +## References + +- Mesa RADV driver source code +- AMD GPU specifications for RDNA2 architecture +- Linux amdgpu kernel driver for reference implementation patterns