From 110d37c3bce07cb068c510bbed2c42d1ccba1b42 Mon Sep 17 00:00:00 2001 From: johndoe6345789 Date: Sun, 1 Feb 2026 18:02:03 +0000 Subject: [PATCH] Add files via upload --- .../docs/UE5_Lighting_System_Documentation.md | 1353 +++++++++++++ .../docs/UE5_Material_System_Documentation.md | 1654 ++++++++++++++++ .../UE5_Reflections_And_GI_Documentation.md | 1728 +++++++++++++++++ .../docs/UE5_Rendering_Documentation.md | 652 +++++++ .../UE5_Shadow_Rendering_Documentation.md | 1317 +++++++++++++ .../UE5_Sky_And_Atmosphere_Documentation.md | 1253 ++++++++++++ 6 files changed, 7957 insertions(+) create mode 100644 gameengine/docs/UE5_Lighting_System_Documentation.md create mode 100644 gameengine/docs/UE5_Material_System_Documentation.md create mode 100644 gameengine/docs/UE5_Reflections_And_GI_Documentation.md create mode 100644 gameengine/docs/UE5_Rendering_Documentation.md create mode 100644 gameengine/docs/UE5_Shadow_Rendering_Documentation.md create mode 100644 gameengine/docs/UE5_Sky_And_Atmosphere_Documentation.md diff --git a/gameengine/docs/UE5_Lighting_System_Documentation.md b/gameengine/docs/UE5_Lighting_System_Documentation.md new file mode 100644 index 000000000..7962d2d35 --- /dev/null +++ b/gameengine/docs/UE5_Lighting_System_Documentation.md @@ -0,0 +1,1353 @@ +# Unreal Engine 5 Lighting System Documentation +## Complete Guide for Game Engine Developers + +This documentation explains how UE5's lighting system works - from light types to deferred rendering and light culling. + +--- + +## Table of Contents +1. [Lighting System Overview](#lighting-system-overview) +2. [Light Types](#light-types) +3. [Light Parameters](#light-parameters) +4. [Light Mobility](#light-mobility) +5. [Deferred Lighting Pipeline](#deferred-lighting-pipeline) +6. [Light Culling (Clustered Deferred)](#light-culling-clustered-deferred) +7. [Forward Lighting](#forward-lighting) +8. [bgfx Implementation Guide](#bgfx-implementation-guide) +9. [Key Files Reference](#key-files-reference) + +--- + +## Lighting System Overview + +UE5 uses a **hybrid lighting system** that combines: + +``` +Lighting Architecture: +┌────────────────────────────────────────────┐ +│ 1. Direct Lighting │ +│ - Directional (Sun) │ +│ - Point (Light bulbs, street lights) │ +│ - Spot (Flashlights, car headlights) │ +│ - Rect (Area lights, windows) │ +└────────────────────────────────────────────┘ + ↓ +┌────────────────────────────────────────────┐ +│ 2. Indirect Lighting │ +│ - Sky Light (Environment) │ +│ - Reflection Probes │ +│ - Lumen GI (Dynamic) │ +│ - Lightmaps (Baked) │ +└────────────────────────────────────────────┘ + ↓ +┌────────────────────────────────────────────┐ +│ 3. Rendering Path │ +│ - Deferred (Most lights) │ +│ - Forward (Translucent, mobile) │ +│ - Clustered Deferred (Many lights) │ +└────────────────────────────────────────────┘ +``` + +**Key File:** `Engine/Source/Runtime/Engine/Public/SceneTypes.h` + +```cpp +enum ELightComponentType +{ + LightType_Directional, // Parallel rays (sun, moon) + LightType_Point, // Omnidirectional (bulb) + LightType_Spot, // Cone-shaped (flashlight) + LightType_Rect, // Area light (window) + LightType_MAX +}; +``` + +--- + +## Light Types + +### 1. Directional Light (Sun, Moon) + +**Key File:** `Engine/Source/Runtime/Engine/Classes/Components/DirectionalLightComponent.h` + +**Characteristics:** +- **Infinite distance**: All rays are parallel +- **No attenuation**: Same intensity everywhere +- **Covers entire scene**: Always evaluated for every pixel +- **Typical use**: Sun, moon, ambient global lighting + +```cpp +class UDirectionalLightComponent : public ULightComponent +{ + // Angular diameter of light source (affects shadow softness) + float LightSourceAngle; // Default: 0.5357° (matches sun) + float LightSourceSoftAngle; // Additional soft shadow angle + + // Shadow parameters + int32 DynamicShadowCascades; // Number of CSM cascades (1-4) + float CascadeDistributionExponent; // 0.0-1.0, controls cascade spacing + float CascadeTransitionFraction; // 0.0-1.0, smooth blending + + // Advanced features + bool bAtmosphereSunLight; // Integrate with sky atmosphere + int32 AtmosphereSunLightIndex; // Which sun (0 or 1) + bool bCastCloudShadows; // Cast shadows from volumetric clouds + float CloudShadowStrength; // Cloud shadow opacity + float CloudShadowOnAtmosphereStrength; + float CloudShadowOnSurfaceStrength; +}; +``` + +**Shadow System:** +- Uses Cascaded Shadow Maps (CSM) +- Typically 2-4 cascades +- Each cascade covers progressively larger area with lower resolution + +**Console Variables:** +```cpp +r.Shadow.CSM.MaxCascades = 4 +r.Shadow.DistanceScale = 1.0 +r.Shadow.CSMSplitPenumbraScale = 0.5 +r.Shadow.MaxResolution = 2048 +``` + +### 2. Point Light (Light Bulbs, Street Lights, Lamps) + +**Key File:** `Engine/Source/Runtime/Engine/Classes/Components/PointLightComponent.h` + +**Characteristics:** +- **Omnidirectional**: Light radiates in all directions +- **Inverse-square falloff**: Physically accurate attenuation +- **Finite radius**: `AttenuationRadius` defines hard cutoff +- **Typical use**: Indoor lights, street lamps, fire, explosions + +```cpp +class UPointLightComponent : public ULocalLightComponent +{ + // Attenuation + bool bUseInverseSquaredFalloff; // Physically accurate vs artistic + float LightFalloffExponent; // When not inverse-squared (2-16) + + // Soft shadows (source size) + float SourceRadius; // Physical light source size (cm) + float SoftSourceRadius; // Additional softening + float SourceLength; // For capsule-shaped lights +}; +``` + +**Attenuation Formula:** + +**Inverse-Squared (Physical):** +```cpp +float GetDistanceAttenuation(float Distance, float AttenuationRadius) +{ + float NormalizedDistance = saturate(Distance / AttenuationRadius); + float Attenuation = 1.0 / (Distance * Distance + 1.0); + + // Smooth cutoff at radius + float WindowFunction = Square(1.0 - Square(NormalizedDistance)); + + return Attenuation * WindowFunction; +} +``` + +**Exponential (Artistic):** +```cpp +float GetDistanceAttenuation(float Distance, float AttenuationRadius, float Exponent) +{ + float NormalizedDistance = saturate(Distance / AttenuationRadius); + float BaseFalloff = saturate(1.0 - pow(NormalizedDistance, 4.0)); + + return pow(BaseFalloff, Exponent); +} +``` + +**Shadow System:** +- Cubemap (6 faces) for omnidirectional shadows +- Can use single-pass geometry shader for efficiency +- Virtual Shadow Maps (VSM) support with 6 pages + +### 3. Spot Light (Flashlights, Car Headlights, Stage Lights) + +**Key File:** `Engine/Source/Runtime/Engine/Classes/Components/SpotLightComponent.h` + +**Characteristics:** +- **Cone-shaped**: Directional with angular falloff +- **Inherits from Point**: All point light features + cone +- **Two angles**: Inner cone (full brightness) and outer cone (falloff) +- **Typical use**: Flashlights, headlights, street lamps, spotlights + +```cpp +class USpotLightComponent : public UPointLightComponent +{ + float InnerConeAngle; // Full brightness angle (degrees) + float OuterConeAngle; // Cutoff angle (degrees) +}; +``` + +**Cone Attenuation:** +```cpp +float GetSpotAngleAttenuation(float3 LightDirection, float3 ToPixel, + float InnerCos, float OuterCos) +{ + float CosAngle = dot(normalize(ToPixel), LightDirection); + + // Smooth transition from outer to inner cone + float AngleAttenuation = saturate((CosAngle - OuterCos) / (InnerCos - OuterCos)); + + return Square(AngleAttenuation); // Squared for smoother falloff +} +``` + +**Shadow System:** +- Single shadow map (perspective projection) +- Matches light cone frustum +- VSM support with single page + +### 4. Rect Light (Area Lights, Windows, Panels) + +**Key File:** `Engine/Source/Runtime/Engine/Classes/Components/RectLightComponent.h` + +**Characteristics:** +- **True area light**: Rectangular emissive surface +- **Accurate soft shadows**: Based on actual light size +- **Barn doors**: Light shaping attachments +- **Texture projection**: Can project images +- **Typical use**: Softboxes, windows, LED panels, TV screens + +```cpp +class URectLightComponent : public ULocalLightComponent +{ + float SourceWidth; // Width in cm + float SourceHeight; // Height in cm + + // Barn doors (light shaping) + float BarnDoorAngle; // Barn door angle (0-90°) + float BarnDoorLength; // Barn door length (cm) + + // Texture projection + UTexture* SourceTexture; // Emissive texture +}; +``` + +**Area Light Integration:** +```hlsl +// Simplified rect light evaluation +float3 RectLightBRDF(float3 N, float3 V, float3 Points[4], float roughness) +{ + // Representative point on rect (approximation) + float3 L = ClosestPointOnRect(Points, V); + float3 H = normalize(V + L); + + // Solid angle calculation + float solidAngle = RectSolidAngle(Points, worldPos); + + // Standard BRDF with area normalization + float D = D_GGX(roughness, saturate(dot(N, H))); + float Vis = Vis_SmithJointApprox(roughness, NoV, NoL); + float3 F = F_Schlick(specularColor, VoH); + + return (D * Vis * F) * solidAngle; +} +``` + +**Shadow System:** +- Single shadow map (perspective or orthographic) +- Soft shadows from accurate area calculation +- Contact shadows for detail + +### 5. Sky Light (Environment Lighting) + +**Key File:** `Engine/Source/Runtime/Engine/Classes/Components/SkyLightComponent.h` + +**Characteristics:** +- **Hemispherical**: Surrounds entire scene +- **Image-based**: Uses cubemap (HDRI or captured) +- **Distant lighting**: Treated as infinitely far +- **Typical use**: Sky, ambient environment, IBL + +```cpp +class USkyLightComponent : public ULightComponentBase +{ + // Source configuration + ESkyLightSourceType SourceType; // Captured scene or specified cubemap + bool bRealTimeCapture; // Update each frame? + UTextureCube* Cubemap; // Source cubemap (if specified) + + // Capture settings + int32 CubemapResolution; // Power of 2 (128-2048) + float SkyDistanceThreshold; // Capture distance cutoff (cm) + float SourceCubemapAngle; // Rotation (0-360°) + + // Hemisphere control + bool bLowerHemisphereIsBlack; // Solid ground below? + FLinearColor LowerHemisphereColor; + + // Advanced + float OcclusionMaxDistance; // AO distance (200-1500 cm) + float Contrast; // AO contrast (0-1) + float MinOcclusion; // Min AO value (0-1) +}; +``` + +**Storage:** +```cpp +// Spherical harmonics for diffuse +FSHVectorRGB3 IrradianceEnvironmentMap; + +// Prefiltered cubemap for specular +FTextureCubeRHIRef ProcessedSkyTexture; +``` + +**Sampling in Shader:** +```hlsl +// Diffuse contribution +float3 DiffuseSkyLight(float3 normal) +{ + // Sample SH for fast diffuse lookup + return SampleSH(IrradianceSH, normal); +} + +// Specular contribution +float3 SpecularSkyLight(float3 reflectDir, float roughness) +{ + // Sample prefiltered cubemap + float mipLevel = RoughnessToMip(roughness); + return SkyLightCubemap.SampleLevel(samplerLinear, reflectDir, mipLevel).rgb; +} +``` + +--- + +## Light Parameters + +### Intensity and Units + +**Key File:** `Engine/Source/Runtime/Engine/Classes/Engine/Scene.h` + +```cpp +enum class ELightUnits : uint8 +{ + Unitless, // Legacy, arbitrary scale + Candelas, // Luminous intensity (normalized) + Lumens, // Luminous power (normalized) + EV, // Exposure value at ISO 100 (normalized) + Nits // Luminance (non-normalized, depends on source size) +}; +``` + +**Conversion to Luminous Power:** + +```cpp +// From LightComponent.cpp +float GetLuminousIntensity() const +{ + switch (IntensityUnits) + { + case ELightUnits::Candelas: + return Intensity; + + case ELightUnits::Lumens: + // Point light: Lumens to candelas + return Intensity / (4.0 * PI); + + case ELightUnits::Unitless: + return Intensity * 16.0; // Legacy conversion + + case ELightUnits::EV: + return pow(2.0, Intensity - 3.0); + + case ELightUnits::Nits: + // Depends on source area + return Intensity * SourceArea * PI; + } +} +``` + +**Typical Values:** +``` +Candle: 1 candela +60W Incandescent Bulb: ~850 lumens (68 candelas for point) +100W Incandescent Bulb: ~1700 lumens (135 candelas) +Flashlight: 100-1000 lumens +Car Headlight: 1000-2000 lumens +Sunlight: ~120,000 lux (EV ~15) +``` + +### Color Temperature + +**Blackbody Radiation:** + +```cpp +// From LightComponent.h +float Temperature; // Kelvin (1700-12000) +bool bUseTemperature; // Enable/disable + +// Preset temperatures: +// - Candle flame: 1850K (orange) +// - Tungsten bulb: 2700K (warm yellow) +// - Halogen: 3200K (yellow-white) +// - Noon sunlight: 5500K (white) +// - D65 white point: 6500K (standard white) +// - Overcast sky: 7000K (cool white) +// - Blue sky: 10000K (blue) +``` + +**Temperature to RGB Conversion:** +```cpp +FLinearColor GetColorTemperature(float Temp) +{ + // Approximation of blackbody radiation + float u = (0.860117757f + 1.54118254e-4f * Temp + 1.28641212e-7f * Temp * Temp) / + (1.0f + 8.42420235e-4f * Temp + 7.08145163e-7f * Temp * Temp); + + float v = (0.317398726f + 4.22806245e-5f * Temp + 4.20481691e-8f * Temp * Temp) / + (1.0f - 2.89741816e-5f * Temp + 1.61456053e-7f * Temp * Temp); + + // Convert CIE uv to RGB (simplified) + return UVToLinearRGB(u, v); +} +``` + +### IES Light Profiles + +**Real-world photometric data:** + +**Key File:** `Engine/Source/Runtime/Engine/Classes/Engine/TextureLightProfile.h` + +```cpp +class UTextureLightProfile : public UTexture2D +{ + float Brightness; // Multiplier + float TextureMultiplier; // Additional scale +}; +``` + +**Usage in Shader:** +```hlsl +// IES profile stored as 1D or 2D texture +Texture2D IESTexture; +SamplerState IESSampler; + +float GetIESAttenuation(float3 worldPos, float3 lightPos, float3 lightDir) +{ + float3 toLightDir = normalize(worldPos - lightPos); + + // Horizontal angle (azimuth) + float phi = atan2(toLightDir.y, toLightDir.x); + + // Vertical angle (elevation) + float theta = acos(dot(toLightDir, lightDir)); + + // Sample IES texture + float2 uv = float2(theta / PI, (phi + PI) / (2.0 * PI)); + float iesValue = IESTexture.SampleLevel(IESSampler, uv, 0).r; + + return iesValue * IESBrightnessScale; +} +``` + +**Common uses:** +- Street lights (specific beam patterns) +- Stage/theatrical lighting +- Automotive headlights +- Architectural lighting + +### Attenuation Radius and Falloff + +```cpp +// From LocalLightComponent.h +float AttenuationRadius; // Hard cutoff distance (cm) +float LightFalloffExponent; // Falloff curve (when not inverse-squared) +``` + +**Radius Estimation:** +```cpp +// Auto-calculate radius from intensity (when radius is 0) +float GetDefaultLightRadius() const +{ + const float MinLightAttenuation = 0.01; // 1% threshold + + if (bUseInverseSquaredFalloff) + { + // Inverse-squared: I / (d^2) = threshold + return sqrt(Intensity / MinLightAttenuation); + } + else + { + // Exponential: estimate based on exponent + return Intensity * SomeArtisticScale; + } +} +``` + +### Source Size (Soft Shadows) + +**Point/Spot Lights:** +```cpp +float SourceRadius; // Physical light source size (cm) +float SoftSourceRadius; // Additional artistic softening (cm) +float SourceLength; // For capsule-shaped lights (cm) +``` + +**Directional Lights:** +```cpp +float LightSourceAngle; // Angular diameter (degrees) +float LightSourceSoftAngle; // Additional soft angle (degrees) + +// Sun: 0.5357° (0.93° with atmosphere) +// Moon: 0.52° +``` + +**Effect on Shadows:** +- Larger source → softer shadows (wider penumbra) +- Used in ray-traced shadows for cone angle +- Used in PCSS for kernel size +- VSM uses this for shadow softness + +--- + +## Light Mobility + +**Key File:** `Engine/Source/Runtime/Engine/Classes/Components/SceneComponent.h` + +```cpp +enum class EComponentMobility : uint8 +{ + Static, // Baked at build time + Stationary, // Hybrid (baked indirect + dynamic direct) + Movable // Fully dynamic +}; +``` + +### Static Lights + +**Characteristics:** +- **Completely baked** into lightmaps +- **Zero runtime cost** (texture reads only) +- **Best quality**: Unlimited bounces, full GI +- **Cannot change**: Position, color, intensity locked +- **Build required**: Must rebuild lighting + +**Use cases:** +- Architectural lighting in static buildings +- Environmental lighting (sun in static scenes) +- Any light that never changes + +**Lightmap Resolution:** +```cpp +// Per-static-mesh setting +int32 OverriddenLightMapRes; // Lightmap texels per unit + +// Project-wide +r.LightMap.DefaultLightMapRes = 64 +``` + +### Stationary Lights + +**Hybrid approach** - most complex: + +**What's Baked:** +- **Indirect lighting**: All bounced light in lightmaps +- **Static shadows**: Shadows from static geometry + +**What's Dynamic:** +- **Direct lighting**: Calculated per-frame (allows intensity/color changes) +- **Dynamic shadows**: Shadows from movable objects +- **Specular**: Real-time specular highlights + +**Key Limitation:** +```cpp +// Maximum 4 overlapping stationary lights per pixel +// Uses 4 shadow map channels +int32 ShadowMapChannel; // 0-3, or -1 if no static shadows +``` + +**Channel Assignment:** +```cpp +// From LightComponent.h +uint32 bHasStaticLighting : 1; +uint32 bHasStaticShadowing : 1; +int32 PreviewShadowMapChannel; // Editor visualization +``` + +**Console Variables:** +```cpp +r.AllowStaticLighting = 1 +r.Shadow.Virtual.Cache = 1 // Cache static shadows in VSM +``` + +**Use cases:** +- Sun in mostly-static scenes with dynamic characters +- Indoor lights where only direct light changes +- Most common for outdoor environments + +### Movable Lights + +**Fully dynamic:** +- **Everything runtime**: Lighting and shadows computed per-frame +- **Can move/change**: Position, color, intensity, radius +- **Higher cost**: Full shadow rendering every frame +- **No baking**: Instant iteration + +**Shadow Options:** +```cpp +bool bCastDynamicShadow; // Enable shadow rendering +bool bCastStaticShadow; // Ignored for movable (always false) +bool bAffectTranslucentLighting; +``` + +**Use cases:** +- Flashlights, muzzle flashes +- Vehicle headlights +- Destructible environments +- Any light that moves or changes + +**Performance:** +```cpp +// Rough cost estimate (depends on scene complexity): +// - Directional: 2-5 ms (CSM rendering) +// - Spot: 0.5-2 ms (single shadow map) +// - Point: 1-4 ms (cubemap) +// - Rect: 1-3 ms (area light) +``` + +--- + +## Deferred Lighting Pipeline + +UE5's primary rendering path for most lights. + +**Key Files:** +- `Engine/Source/Runtime/Renderer/Private/DeferredShadingRenderer.cpp` +- `Engine/Source/Runtime/Renderer/Private/LightRendering.cpp` +- `Engine/Shaders/Private/DeferredLightPixelShaders.usf` + +### Pipeline Overview + +``` +1. GBuffer Pass + ↓ +[GBuffer Textures: Normal, BaseColor, Roughness, Metallic, etc.] + ↓ +2. Lighting Accumulation Pass (for each light) + ↓ +[Scene Color += Light Contribution] + ↓ +3. Post Processing +``` + +### GBuffer Layout + +**From:** `Engine/Shaders/Private/DeferredShadingCommon.ush` + +```hlsl +struct FGBufferData +{ + // Geometric + float3 WorldNormal; + float3 WorldTangent; + float Depth; + + // Material PBR properties + float3 BaseColor; + float3 DiffuseColor; // Derived: BaseColor * (1 - Metallic) + float3 SpecularColor; // Derived: lerp(0.04, BaseColor, Metallic) + float Metallic; + float Specular; + float Roughness; + float Anisotropy; + + // Lighting modifiers + float GBufferAO; + + // Shading model + uint ShadingModelID; + + // Custom data (shading model specific) + float4 CustomData; + + // Baked lighting + float4 PrecomputedShadowFactors; // 4 shadow channels +}; +``` + +**Render Targets:** +``` +RT0: SceneColor (accumulation target) +RT1: GBufferA - WorldNormal (RGB), PerObjectData (A) +RT2: GBufferB - Metallic (R), Specular (G), Roughness (B), ShadingModelID (A) +RT3: GBufferC - BaseColor (RGB), AO (A) +RT4: GBufferD - CustomData (shading model specific) +RT5: GBufferE - PrecomputedShadowFactors (RGBA) +RT6: GBufferF - WorldTangent (RGB), Anisotropy (A) +``` + +### Deferred Light Shader + +**From:** `Engine/Shaders/Private/DeferredLightPixelShaders.usf` + +```hlsl +void DeferredLightPixelMain( + float4 SVPos : SV_POSITION, + out float4 OutColor : SV_Target0) +{ + // 1. Calculate screen UV + float2 ScreenUV = SvPositionToScreenUV(SVPos); + + // 2. Read GBuffer + FGBufferData GBuffer = GetGBufferData(ScreenUV); + + // 3. Reconstruct world position from depth + float SceneDepth = CalcSceneDepth(ScreenUV); + float3 WorldPosition = ScreenToWorld(ScreenUV, SceneDepth); + + // 4. Get light data (from uniform buffer) + FDeferredLightData LightData = InitDeferredLightFromUniforms(); + + // 5. Calculate light attenuation (distance + angle) + float LightMask = GetLocalLightAttenuation( + WorldPosition, + LightData, + LightData.Normal, // For rect lights + LightData.Direction // For spot lights + ); + + if (LightMask <= 0.0) + { + OutColor = 0; + return; // Early out if outside light range + } + + // 6. Get shadow attenuation + float4 LightAttenuation = GetLightAttenuation(ScreenUV); + + // 7. Calculate camera vector + float3 CameraVector = normalize(View.WorldCameraOrigin - WorldPosition); + + // 8. Evaluate BRDF lighting + FDeferredLightingSplit Lighting = GetDynamicLighting( + WorldPosition, + CameraVector, + GBuffer, + 1.0, // AO + GBuffer.ShadingModelID, + LightData, + LightAttenuation, + uint2(SVPos.xy) + ); + + // 9. Output (additive blend) + OutColor = float4(Lighting.SpecularLighting + Lighting.DiffuseLighting, 0); +} +``` + +### BRDF Evaluation + +**From:** `Engine/Shaders/Private/DeferredLightingCommon.ush` + +```hlsl +FDeferredLightingSplit GetDynamicLighting(...) +{ + float3 L = LightData.Direction; // Light direction + float3 V = CameraVector; // View direction + float3 N = GBuffer.WorldNormal; + float3 H = normalize(V + L); + + float NoL = saturate(dot(N, L)); + float NoV = saturate(dot(N, V)); + float VoH = saturate(dot(V, H)); + float NoH = saturate(dot(N, H)); + + // Shadow term + float Shadow = LightAttenuation.r; + + // Diffuse + float3 Diffuse = Diffuse_Burley(GBuffer.DiffuseColor, GBuffer.Roughness, NoV, NoL, VoH); + + // Specular (GGX) + float D = D_GGX(GBuffer.Roughness, NoH); + float Vis = Vis_SmithJointApprox(GBuffer.Roughness, NoV, NoL); + float3 F = F_Schlick(GBuffer.SpecularColor, VoH); + + float3 Specular = (D * Vis) * F; + + // Apply light color and shadow + Lighting.DiffuseLighting = Diffuse * LightData.Color * NoL * Shadow; + Lighting.SpecularLighting = Specular * LightData.Color * NoL * Shadow; + + return Lighting; +} +``` + +### Blend State (Additive) + +Lights accumulate using additive blending: + +```cpp +// From LightRendering.cpp +TStaticBlendState< + CW_RGBA, // Write all channels + BO_Add, // RGB: Add + BF_One, // Src factor: 1 + BF_One, // Dst factor: 1 + BO_Add, // Alpha: Add + BF_One, // Src alpha factor: 1 + BF_One // Dst alpha factor: 1 +>::GetRHI() + +// Result: SceneColor += LightContribution +``` + +### Light Volume Rendering + +For point and spot lights, render light bounds geometry: + +```cpp +// Point light: Render sphere +// Spot light: Render cone + +// Depth test configuration: +// - If camera inside light volume: Disable depth test +// - If camera outside: Use depth test (cull pixels behind geometry) + +uint64 DepthState = bCameraInsideLightGeometry ? + BGFX_STATE_DEPTH_TEST_ALWAYS : BGFX_STATE_DEPTH_TEST_GREATER; +``` + +--- + +## Light Culling (Clustered Deferred) + +For scenes with many lights (100+), use clustered deferred shading. + +**Key Files:** +- `Engine/Source/Runtime/Renderer/Private/ClusteredDeferredShadingPass.cpp` +- `Engine/Source/Runtime/Renderer/Private/LightGridInjection.cpp` +- `Engine/Shaders/Private/ClusteredDeferredShadingPixelShader.usf` + +### 3D Grid Structure + +```cpp +// Grid parameters +int32 GridPixelSize = 64; // Tile size (64×64 pixels) +int32 GridSizeZ = 32; // Depth slices + +// Total cells +int32 GridSizeX = (ScreenWidth + GridPixelSize - 1) / GridPixelSize; +int32 GridSizeY = (ScreenHeight + GridPixelSize - 1) / GridPixelSize; +int32 TotalCells = GridSizeX * GridSizeY * GridSizeZ; +``` + +### Light Grid Injection + +**Build light grid (compute shader):** + +```hlsl +// cs_light_grid_injection.hlsl +[numthreads(4, 4, 4)] // Process 4x4x4 cells per thread group +void BuildLightGridCS(uint3 GroupId : SV_GroupID, uint3 ThreadId : SV_GroupThreadID) +{ + uint3 GridCoord = GroupId * 4 + ThreadId; + + if (any(GridCoord >= GridDimensions)) + return; + + // Calculate cell bounds in world space + float3 MinBounds, MaxBounds; + GetCellBounds(GridCoord, MinBounds, MaxBounds); + + // Test each light + uint NumLightsInCell = 0; + uint LightIndices[MAX_LIGHTS_PER_CELL]; + + for (uint lightIndex = 0; lightIndex < NumLights; ++lightIndex) + { + FLightData light = Lights[lightIndex]; + + // Test light bounds vs cell bounds + if (IntersectLightWithCell(light, MinBounds, MaxBounds)) + { + LightIndices[NumLightsInCell++] = lightIndex; + + if (NumLightsInCell >= MAX_LIGHTS_PER_CELL) + break; // Cell full + } + } + + // Write to light grid + uint cellIndex = GridCoordToIndex(GridCoord); + LightGrid[cellIndex].NumLights = NumLightsInCell; + LightGrid[cellIndex].LightIndexStart = AtomicAdd(GlobalLightIndexCounter, NumLightsInCell); + + // Write light indices + for (uint i = 0; i < NumLightsInCell; ++i) + { + uint writeIndex = LightGrid[cellIndex].LightIndexStart + i; + GlobalLightIndexList[writeIndex] = LightIndices[i]; + } +} +``` + +### Clustered Shading Pass + +**Single fullscreen pass evaluates all lights:** + +```hlsl +// fs_clustered_deferred.hlsl +void ClusteredDeferredPS( + float4 SvPosition : SV_Position, + out float4 OutColor : SV_Target0) +{ + float2 ScreenUV = SvPositionToScreenUV(SvPosition); + + // Read GBuffer + FGBufferData GBuffer = GetGBufferData(ScreenUV); + float3 WorldPos = ReconstructWorldPosition(ScreenUV, GBuffer.Depth); + float3 V = normalize(CameraPos - WorldPos); + + // Find grid cell + uint3 GridCoord = WorldToGridCoord(WorldPos, SvPosition.xy); + uint CellIndex = GridCoordToIndex(GridCoord); + + // Get lights in this cell + uint NumLights = LightGrid[CellIndex].NumLights; + uint LightIndexStart = LightGrid[CellIndex].LightIndexStart; + + // Accumulate lighting + float3 AccumulatedLighting = 0; + + for (uint i = 0; i < NumLights; ++i) + { + uint lightIndex = GlobalLightIndexList[LightIndexStart + i]; + FLightData light = Lights[lightIndex]; + + // Evaluate light + float3 L = normalize(light.Position - WorldPos); + float attenuation = CalculateAttenuation(WorldPos, light); + + if (attenuation > 0.0) + { + // BRDF evaluation + float3 lighting = EvaluateBRDF(GBuffer, V, L) * attenuation * light.Color; + AccumulatedLighting += lighting; + } + } + + OutColor = float4(AccumulatedLighting, 1.0); +} +``` + +### Performance Benefits + +**Traditional deferred:** +- O(Lights × Pixels) - Each light = separate pass +- Example: 100 lights × 1920×1080 = ~200M pixel shades + +**Clustered deferred:** +- O(Pixels + Lights × Cells) - Single pass +- Example: 1920×1080 + 100 lights × (30×17×32) = ~2M + 1.6M = 3.6M + +**Savings:** ~98% reduction in overdraw + +**Console Variables:** +```cpp +r.LightCulling.Quality = 1 // 0=off, 1=on +r.LightGridPixelSize = 64 // Tile size +r.LightGridSizeZ = 32 // Z slices +r.LightGridMaxCulledLights = 256 // Max lights per cell +``` + +--- + +## Forward Lighting + +Alternative rendering path, used for translucency and mobile. + +**Key File:** `Engine/Shaders/Private/ForwardLightingCommon.ush` + +### When Forward is Used + +1. **Translucent materials** (always forward) +2. **Mobile rendering** (primary path) +3. **VR forward renderer** (optional) +4. **Hair/fur** (forward+ with visibility buffer) + +### Forward Lighting Evaluation + +```hlsl +// From ForwardLightingCommon.ush +FDeferredLightingSplit GetForwardDirectLighting( + uint GridIndex, + float3 WorldPosition, + FGBufferData GBuffer, + ...) +{ + FDeferredLightingSplit DirectLighting = (FDeferredLightingSplit)0; + + // 1. Directional light (sun) + const FDirectionalLightData DirectionalLight = GetDirectionalLightData(0); + { + float3 L = DirectionalLight.Direction; + float Shadow = GetDirectionalLightShadow(ScreenUV); + + DirectLighting += EvaluateLight(GBuffer, V, L, DirectionalLight.Color, Shadow); + } + + // 2. Local lights (from light grid) + const FCulledLightsGridData GridData = GetCulledLightsGrid(GridIndex); + + for (uint LocalLightIndex = 0; LocalLightIndex < GridData.NumLocalLights; ++LocalLightIndex) + { + uint LightIndex = ForwardLightData.CulledLightDataGrid[GridData.DataStartIndex + LocalLightIndex]; + FLocalLightData LocalLight = GetLocalLightData(LightIndex); + + float3 L = normalize(LocalLight.Position - WorldPosition); + float Attenuation = CalculateAttenuation(WorldPosition, LocalLight); + float Shadow = GetLocalLightShadow(ScreenUV, LightIndex); + + DirectLighting += EvaluateLight(GBuffer, V, L, LocalLight.Color * Attenuation, Shadow); + } + + return DirectLighting; +} +``` + +### Forward vs Deferred Comparison + +| Feature | Deferred | Forward | +|---------|----------|---------| +| **MSAA Support** | No (GBuffer incompatible) | Yes | +| **Translucency** | Separate pass | Native | +| **Memory** | High (GBuffer) | Lower | +| **Light Count** | Excellent (many lights) | Good (moderate lights) | +| **Material Variations** | Single shader | Many variants | +| **Bandwidth** | High (GBuffer reads/writes) | Lower | + +--- + +## bgfx Implementation Guide + +### Basic Deferred Lighting + +```cpp +class DeferredLightRenderer +{ + // Light uniform buffer + struct LightUniform + { + vec4 position; // xyz = position, w = radius + vec4 color; // rgb = color, a = intensity + vec4 direction; // xyz = direction (spot/directional) + vec4 params; // x = inner cone, y = outer cone, z = falloff exp + }; + + std::vector lights; + bgfx::UniformHandle u_lightData; + bgfx::ProgramHandle directionalLightShader; + bgfx::ProgramHandle pointLightShader; + bgfx::ProgramHandle spotLightShader; + + void RenderLights(const GBuffer& gbuffer) + { + // Bind GBuffer textures + bgfx::setTexture(0, s_gbufferNormal, gbuffer.normalTexture); + bgfx::setTexture(1, s_gbufferBaseColor, gbuffer.baseColorTexture); + bgfx::setTexture(2, s_gbufferMaterial, gbuffer.materialTexture); + bgfx::setTexture(3, s_gbufferDepth, gbuffer.depthTexture); + + // Additive blend state + uint64_t state = BGFX_STATE_WRITE_RGB + | BGFX_STATE_WRITE_A + | BGFX_STATE_BLEND_ADD + | BGFX_STATE_DEPTH_TEST_EQUAL; // Don't write depth + + bgfx::setState(state); + + // Render each light + for (const auto& light : lights) + { + bgfx::setUniform(u_lightData, &light); + + if (light.type == LIGHT_DIRECTIONAL) + { + // Fullscreen quad + DrawFullscreenQuad(); + bgfx::submit(VIEW_LIGHTING, directionalLightShader); + } + else if (light.type == LIGHT_POINT) + { + // Render light sphere volume + DrawLightSphere(light.position, light.radius); + bgfx::submit(VIEW_LIGHTING, pointLightShader); + } + else if (light.type == LIGHT_SPOT) + { + // Render light cone volume + DrawLightCone(light); + bgfx::submit(VIEW_LIGHTING, spotLightShader); + } + } + } +}; +``` + +### Deferred Light Shader (GLSL) + +```glsl +// fs_deferred_point_light.sc +$input v_texcoord0 + +#include + +SAMPLER2D(s_gbufferNormal, 0); +SAMPLER2D(s_gbufferBaseColor, 1); +SAMPLER2D(s_gbufferMaterial, 2); // r=roughness, g=metallic, b=specular +SAMPLER2D(s_gbufferDepth, 3); + +uniform vec4 u_lightPosRadius; // xyz = position, w = radius +uniform vec4 u_lightColor; // rgb = color, a = intensity +uniform vec4 u_lightParams; // x = falloff exponent + +vec3 ReconstructWorldPosition(vec2 uv, float depth) +{ + // Reconstruct from depth + vec4 clipPos = vec4(uv * 2.0 - 1.0, depth, 1.0); + vec4 viewPos = mul(u_invProj, clipPos); + viewPos /= viewPos.w; + vec4 worldPos = mul(u_invView, viewPos); + return worldPos.xyz; +} + +float GetAttenuation(float distance, float radius, float exponent) +{ + float normalizedDist = saturate(distance / radius); + float baseFalloff = saturate(1.0 - pow(normalizedDist, 4.0)); + return pow(baseFalloff, exponent); +} + +void main() +{ + vec2 uv = v_texcoord0; + + // Read GBuffer + vec3 normal = texture2D(s_gbufferNormal, uv).rgb * 2.0 - 1.0; + vec4 baseColorAO = texture2D(s_gbufferBaseColor, uv); + vec3 baseColor = baseColorAO.rgb; + float ao = baseColorAO.a; + + vec3 material = texture2D(s_gbufferMaterial, uv).rgb; + float roughness = material.r; + float metallic = material.g; + float specular = material.b; + + float depth = texture2D(s_gbufferDepth, uv).r; + + // Reconstruct position + vec3 worldPos = ReconstructWorldPosition(uv, depth); + + // Light calculation + vec3 lightPos = u_lightPosRadius.xyz; + float lightRadius = u_lightPosRadius.w; + + vec3 L = lightPos - worldPos; + float distance = length(L); + L /= distance; // Normalize + + // Attenuation + float attenuation = GetAttenuation(distance, lightRadius, u_lightParams.x); + + if (attenuation <= 0.0) + { + discard; // Outside light range + } + + // View vector + vec3 V = normalize(u_cameraPos - worldPos); + + // BRDF (simplified) + vec3 H = normalize(V + L); + float NoL = max(dot(normal, L), 0.0); + float NoV = max(dot(normal, V), 0.0); + float NoH = max(dot(normal, H), 0.0); + float VoH = max(dot(V, H), 0.0); + + // Derived colors + vec3 diffuseColor = baseColor * (1.0 - metallic); + vec3 specularColor = mix(vec3_splat(0.04), baseColor, metallic); + + // Diffuse (Burley) + float FD90 = 0.5 + 2.0 * VoH * VoH * roughness; + float FdV = 1.0 + (FD90 - 1.0) * pow(1.0 - NoV, 5.0); + float FdL = 1.0 + (FD90 - 1.0) * pow(1.0 - NoL, 5.0); + vec3 diffuse = diffuseColor * (1.0 / 3.14159) * FdV * FdL; + + // Specular (GGX) + float a = roughness * roughness; + float a2 = a * a; + + // D (GGX) + float denom = (NoH * a2 - NoH) * NoH + 1.0; + float D = a2 / (3.14159 * denom * denom); + + // Vis (Smith) + float k = a * 0.5; + float vis = 0.5 / ((NoL * (NoV * (1.0 - k) + k) + NoV * (NoL * (1.0 - k) + k))); + + // F (Schlick) + float Fc = pow(1.0 - VoH, 5.0); + vec3 F = specularColor + (vec3_splat(1.0) - specularColor) * Fc; + + vec3 spec = D * vis * F; + + // Combine + vec3 lighting = (diffuse + spec) * u_lightColor.rgb * u_lightColor.a * NoL * attenuation * ao; + + gl_FragColor = vec4(lighting, 1.0); +} +``` + +### Clustered Lighting (Simplified) + +```cpp +// Build light grid +class ClusteredLightCuller +{ + static const int GRID_SIZE_X = 30; // 1920 / 64 + static const int GRID_SIZE_Y = 17; // 1080 / 64 + static const int GRID_SIZE_Z = 32; + + struct GridCell + { + uint32_t lightIndexStart; + uint32_t numLights; + }; + + GridCell cells[GRID_SIZE_X * GRID_SIZE_Y * GRID_SIZE_Z]; + std::vector lightIndices; + + bgfx::DynamicIndexBufferHandle lightIndexBuffer; + bgfx::DynamicVertexBufferHandle cellDataBuffer; + + void BuildLightGrid(const std::vector& lights) + { + lightIndices.clear(); + + for (int z = 0; z < GRID_SIZE_Z; ++z) + { + for (int y = 0; y < GRID_SIZE_Y; ++y) + { + for (int x = 0; x < GRID_SIZE_X; ++x) + { + int cellIdx = x + y * GRID_SIZE_X + z * GRID_SIZE_X * GRID_SIZE_Y; + + // Calculate cell bounds + AABB cellBounds = GetCellBounds(x, y, z); + + // Test each light + uint32_t startIdx = lightIndices.size(); + + for (uint32_t i = 0; i < lights.size(); ++i) + { + if (LightIntersectsCell(lights[i], cellBounds)) + { + lightIndices.push_back(i); + } + } + + cells[cellIdx].lightIndexStart = startIdx; + cells[cellIdx].numLights = lightIndices.size() - startIdx; + } + } + } + + // Upload to GPU + bgfx::update(lightIndexBuffer, 0, bgfx::copy(lightIndices.data(), + lightIndices.size() * sizeof(uint32_t))); + bgfx::update(cellDataBuffer, 0, bgfx::copy(cells, sizeof(cells))); + } +}; +``` + +--- + +## Key Files Reference + +### C++ Source Files + +**Light Components:** +- `Engine/Source/Runtime/Engine/Classes/Components/LightComponent.h` +- `Engine/Source/Runtime/Engine/Classes/Components/DirectionalLightComponent.h` +- `Engine/Source/Runtime/Engine/Classes/Components/PointLightComponent.h` +- `Engine/Source/Runtime/Engine/Classes/Components/SpotLightComponent.h` +- `Engine/Source/Runtime/Engine/Classes/Components/RectLightComponent.h` +- `Engine/Source/Runtime/Engine/Classes/Components/SkyLightComponent.h` + +**Rendering:** +- `Engine/Source/Runtime/Renderer/Private/DeferredShadingRenderer.cpp` + - Main render loop +- `Engine/Source/Runtime/Renderer/Private/LightRendering.cpp` + - Deferred light rendering +- `Engine/Source/Runtime/Renderer/Private/ClusteredDeferredShadingPass.cpp` + - Clustered deferred shading +- `Engine/Source/Runtime/Renderer/Private/LightGridInjection.cpp` + - Light culling and grid building + +**Data Structures:** +- `Engine/Source/Runtime/Engine/Public/SceneTypes.h` + - Light type enums +- `Engine/Source/Runtime/Engine/Classes/Engine/Scene.h` + - Light units enum + +### Shader Files + +**Deferred Lighting:** +- `Engine/Shaders/Private/DeferredLightingCommon.ush` + - BRDF evaluation +- `Engine/Shaders/Private/DeferredLightPixelShaders.usf` + - Per-light pixel shaders +- `Engine/Shaders/Private/DeferredShadingCommon.ush` + - GBuffer utilities + +**Forward Lighting:** +- `Engine/Shaders/Private/ForwardLightingCommon.ush` + - Forward lighting evaluation + +**Clustered:** +- `Engine/Shaders/Private/ClusteredDeferredShadingPixelShader.usf` + - Clustered shading pass +- `Engine/Shaders/Private/LightGridInjection.usf` + - Light grid building compute shader + +--- + +## Summary + +**For Your Custom Engine:** + +1. **Start with basic deferred lighting** + - Directional, point, spot lights + - Simple attenuation and BRDF + - Additive accumulation + +2. **Add light parameters** + - Intensity units (lumens/candelas) + - Color temperature + - Attenuation curves + +3. **Implement light culling** + - Tiled deferred (8×8 tiles) + - Or clustered deferred (3D grid) + - Dramatically improves multi-light performance + +4. **Optimize** + - Light volumes (sphere/cone rendering) + - Depth bounds test + - Stencil masking + +**Reference Values:** +- Indoor bulb: 800-1700 lumens +- Car headlight: 1000-2000 lumens +- Sunlight: EV 15, ~6500K +- Attenuation radius: Auto-calculate from intensity + +UE5's lighting system is battle-tested across hundreds of shipped titles. Following this architecture gives you physically-based, scalable lighting that looks great and performs well. diff --git a/gameengine/docs/UE5_Material_System_Documentation.md b/gameengine/docs/UE5_Material_System_Documentation.md new file mode 100644 index 000000000..05feea5cf --- /dev/null +++ b/gameengine/docs/UE5_Material_System_Documentation.md @@ -0,0 +1,1654 @@ +# Unreal Engine 5 Material System Documentation +## For MaterialX and bgfx Integration + +This documentation explains how UE5's material system works, with specific focus on helping you integrate MaterialX shader graphs with bgfx rendering. + +--- + +## Table of Contents +1. [Material System Architecture](#material-system-architecture) +2. [Material Definitions and Representation](#material-definitions-and-representation) +3. [Shader Generation Pipeline](#shader-generation-pipeline) +4. [GBuffer Encoding](#gbuffer-encoding) +5. [Shader Permutations](#shader-permutations) +6. [BRDF and Lighting Integration](#brdf-and-lighting-integration) +7. [Shading Models](#shading-models) +8. [MaterialX Integration Strategy](#materialx-integration-strategy) +9. [bgfx Implementation Guide](#bgfx-implementation-guide) +10. [Key Files Reference](#key-files-reference) + +--- + +## Material System Architecture + +UE5 uses a node-based material editor that generates optimized HLSL shaders. The pipeline is: + +``` +Material Graph (Artist) → HLSL Code Generation → Shader Compilation → Runtime Rendering +``` + +For your MaterialX + bgfx engine: +``` +MaterialX Graph → Custom Translator → GLSL/HLSL/Metal → bgfx Shader Compilation → Runtime +``` + +### Key Concepts + +**Three-Layer Material System:** + +1. **Material Asset** (`UMaterial`) - The high-level definition + - Material properties (blend mode, shading model, etc.) + - Expression graph (node network) + - Default parameter values + +2. **Material Instance** (`UMaterialInstance`) - Runtime variations + - Overrides parameters (colors, scalars, textures) + - No recompilation needed + - Lightweight for creating variations + +3. **Material Render Proxy** (`FMaterialRenderProxy`) - Per-draw state + - Binds to specific mesh instance + - Provides per-instance parameters + - Used during actual rendering + +**Key File:** `Engine/Source/Runtime/Engine/Public/Materials/Material.h` + +--- + +## Material Definitions and Representation + +### Core Material Inputs + +UE5 materials define these standard inputs (maps directly to MaterialX concept): + +```cpp +// From Engine/Source/Runtime/Engine/Public/Materials/Material.h + +struct UMaterial +{ + // PBR Material Inputs + FColorMaterialInput BaseColor; // Albedo/diffuse color + FScalarMaterialInput Metallic; // 0 = dielectric, 1 = metal + FScalarMaterialInput Specular; // Specular reflectance (usually 0.5) + FScalarMaterialInput Roughness; // 0 = smooth, 1 = rough + FScalarMaterialInput Anisotropy; // -1 to 1, anisotropic reflection + + // Normals and Displacement + FVectorMaterialInput Normal; // Tangent-space normal map + FVectorMaterialInput Tangent; // For anisotropic materials + FVectorMaterialInput WorldPositionOffset; // Vertex displacement + + // Emission and Special + FColorMaterialInput EmissiveColor; // Self-illumination + FScalarMaterialInput Opacity; // For translucent materials + FScalarMaterialInput OpacityMask; // For masked materials + + // Subsurface (for skin, wax, etc.) + FColorMaterialInput SubsurfaceColor; + + // Clear Coat (for car paint, etc.) + FScalarMaterialInput ClearCoat; + FScalarMaterialInput ClearCoatRoughness; + + // Ambient Occlusion + FScalarMaterialInput AmbientOcclusion; + + // Refraction (for glass, water) + FScalarMaterialInput Refraction; + + // Custom data (shading model specific) + FVectorMaterialInput CustomData0; + FVectorMaterialInput CustomData1; +}; +``` + +### Material Properties + +```cpp +// Material Domain - What type of material is this? +enum EMaterialDomain +{ + MD_Surface, // Standard surface material (99% of materials) + MD_DeferredDecal, // Projected decal + MD_LightFunction, // Light cookie/gobo + MD_Volume, // Volumetric material (fog, clouds) + MD_PostProcess, // Post-process effect + MD_UI, // User interface + MD_RuntimeVirtualTexture // Runtime virtual texture output +}; + +// Blend Mode - How does this material composite? +enum EBlendMode +{ + BLEND_Opaque, // Solid, no transparency + BLEND_Masked, // Binary transparency (alpha test) + BLEND_Translucent, // Alpha blending + BLEND_Additive, // Additive blending (particles, effects) + BLEND_Modulate, // Multiplicative blending + BLEND_AlphaComposite, // Pre-multiplied alpha + BLEND_AlphaHoldout // Compositing holdout +}; + +// Shading Model - Which BRDF to use? +enum EMaterialShadingModel +{ + MSM_DefaultLit, // Standard PBR (GGX) + MSM_Subsurface, // Subsurface scattering + MSM_PreintegratedSkin,// Optimized skin shading + MSM_ClearCoat, // Dual-layer (coat + base) + MSM_SubsurfaceProfile,// Profile-based SSS + MSM_TwoSidedFoliage, // Leaf/plant shading + MSM_Hair, // Anisotropic hair + MSM_Cloth, // Fabric with fuzz + MSM_Eye, // Eye with iris/cornea + MSM_SingleLayerWater, // Water surface + MSM_ThinTranslucent // Thin translucent surfaces +}; +``` + +**Key File:** `Engine/Source/Runtime/Engine/Public/MaterialShared.h` + +### MaterialX Mapping + +MaterialX nodes map to UE5 material inputs as follows: + +| MaterialX Node | UE5 Material Input | Notes | +|----------------|-------------------|-------| +| `` | Base material | Core PBR node | +| `base_color` | BaseColor | RGB color | +| `metalness` | Metallic | 0-1 scalar | +| `specular_roughness` | Roughness | 0-1 scalar | +| `normal` | Normal | Tangent-space XYZ | +| `emission` | EmissiveColor | RGB color | +| `coat` | ClearCoat | 0-1 scalar | +| `coat_roughness` | ClearCoatRoughness | 0-1 scalar | +| `transmission` | Opacity (inverted) | For translucent | +| `subsurface` | SubsurfaceColor | RGB color | + +**Your Workflow:** +1. Parse MaterialX document (XML or JSON metadata) +2. Build expression graph +3. Generate shader code for bgfx +4. Extract parameters for runtime editing + +--- + +## Shader Generation Pipeline + +UE5 converts the material graph to HLSL code through `FHLSLMaterialTranslator`. + +**Key File:** `Engine/Source/Runtime/Engine/Private/Materials/HLSLMaterialTranslator.cpp` (695KB!) + +### Translation Process + +``` +Material Expression Graph + ↓ +[1] Traverse nodes starting from outputs (BaseColor, Roughness, etc.) + ↓ +[2] Generate HLSL code for each node + ↓ +[3] Optimize (constant folding, dead code elimination) + ↓ +[4] Fill MaterialTemplate.ush with generated code + ↓ +[5] Compile with shader compiler (DXC/FXC) + ↓ +Final Shader Bytecode +``` + +### Material Template + +UE5 uses a template file that gets filled in during compilation: + +**File:** `Engine/Shaders/Private/MaterialTemplate.ush` + +```hlsl +// Simplified example of MaterialTemplate.ush + +// Platform includes +#include "/Engine/Private/Common.ush" +#include "/Engine/Private/BRDF.ush" +#include "/Engine/Private/ShadingModels.ush" + +// Generated material parameter declarations +// %MATERIAL_PARAMETERS% +// Example output: +// Texture2D Material_Texture2D_0; +// SamplerState Material_Texture2D_0Sampler; +// float4 Material_ScalarParameter_0; + +// Material attributes structure +struct FMaterialPixelParameters +{ + float3 WorldPosition; + float3 WorldNormal; + float3 WorldTangent; + float2 TexCoords[NUM_MATERIAL_TEXCOORDS]; + float4 VertexColor; + // ... more +}; + +// Generated pixel shader code +void CalcPixelMaterialInputs(inout FMaterialPixelParameters Parameters) +{ + // %PIXEL_SHADER_CODE% + // This is where your material graph gets inserted + // Example generated code: + + // BaseColor = Texture2DSample(Material_Texture2D_0, UV) + float3 Local0 = Texture2DSample(Material_Texture2D_0, + Material_Texture2D_0Sampler, + Parameters.TexCoords[0]).rgb; + + // Roughness = ScalarParameter * 0.5 + float Local1 = Material_ScalarParameter_0.x * 0.5; + + // Write to output structure + PixelMaterialInputs.BaseColor = Local0; + PixelMaterialInputs.Roughness = Local1; + PixelMaterialInputs.Metallic = 0.0; + PixelMaterialInputs.Specular = 0.5; + // ... more outputs +} +``` + +### Code Generation Example + +For a simple material graph: +``` +Texture2D (BaseColorTexture) → Multiply (by Color Parameter) → BaseColor Output +``` + +Generated HLSL: +```hlsl +// Parameter declarations +Texture2D Material_Texture2D_0; // BaseColorTexture +SamplerState Material_Texture2D_0Sampler; +float4 Material_VectorParameter_0; // Color Parameter + +// In CalcPixelMaterialInputs(): +float3 Local0 = Texture2DSample(Material_Texture2D_0, + Material_Texture2D_0Sampler, + Parameters.TexCoords[0]).rgb; + +float3 Local1 = Local0 * Material_VectorParameter_0.rgb; + +PixelMaterialInputs.BaseColor = Local1; +``` + +### For Your MaterialX Engine + +You'll need similar translation: + +```python +# Pseudo-code for MaterialX to GLSL/HLSL +class MaterialXTranslator: + def translate_graph(self, materialx_doc): + # 1. Parse MaterialX document + graph = parse_materialx(materialx_doc) + + # 2. Topological sort (dependencies first) + sorted_nodes = topological_sort(graph) + + # 3. Generate shader code + shader_code = ShaderCode() + + for node in sorted_nodes: + if node.type == 'image': + shader_code.add_texture(node.name, node.file) + shader_code.add_sample(f"{node.output} = texture2D({node.name}, uv);") + + elif node.type == 'multiply': + shader_code.add_operation(f"{node.output} = {node.input1} * {node.input2};") + + elif node.type == 'standard_surface': + # This is the output node + shader_code.set_output('baseColor', node.inputs['base_color']) + shader_code.set_output('roughness', node.inputs['specular_roughness']) + # ... etc + + # 4. Generate final shader with bgfx conventions + return generate_bgfx_shader(shader_code) +``` + +**Key File for Reference:** `Engine/Source/Runtime/Engine/Private/Materials/MaterialExpressions.cpp` +- Contains code generation for every material node type +- ~16,000 lines showing how each node generates HLSL + +--- + +## GBuffer Encoding + +UE5 uses deferred rendering, storing material properties in the GBuffer (multiple render targets). + +**Key File:** `Engine/Shaders/Private/DeferredShadingCommon.ush` + +### GBuffer Data Structure + +```cpp +struct FGBufferData +{ + // Geometric properties + half3 WorldNormal; // Unit vector in world space + half3 WorldTangent; // For anisotropic materials + + // PBR properties (from material) + half3 BaseColor; // Albedo (RGB) + half3 DiffuseColor; // Derived: BaseColor * (1 - Metallic) + half3 SpecularColor; // Derived: lerp(0.04, BaseColor, Metallic) + half Metallic; // 0 = dielectric, 1 = metal + half Specular; // Specular intensity (0.5 default) + half Roughness; // Surface roughness (0-1) + half Anisotropy; // Anisotropic highlight (-1 to 1) + + // Lighting modifiers + half GBufferAO; // Ambient occlusion from material + + // Shading model + uint ShadingModelID; // Which BRDF to use + + // Custom data (shading model specific) + float4 CustomData; // Meaning depends on ShadingModelID + + // Advanced + uint SelectiveOutputMask; // Which outputs are written + half PerObjectGBufferData;// Per-object data + half4 PrecomputedShadowFactors; // Static lighting + + // Depth + float Depth; // Linear depth + + // Indirect lighting + half IndirectIrradiance; // Baked lighting +}; +``` + +### GBuffer Layout (Multiple Render Targets) + +UE5 writes to 5-6 render targets simultaneously: + +```hlsl +// Pixel shader outputs +struct FGBufferOutput +{ + // RT0: SceneColor (RGB = lit color, A = unused or AO) + float4 OutTarget0 : SV_Target0; + + // RT1: GBufferA (RGB = Normal, A = PerObjectData) + float4 OutTarget1 : SV_Target1; + + // RT2: GBufferB (R = Metallic, G = Specular, B = Roughness, A = ShadingModelID) + float4 OutTarget2 : SV_Target2; + + // RT3: GBufferC (RGB = BaseColor, A = AO) + float4 OutTarget3 : SV_Target3; + + // RT4: GBufferD (RGBA = CustomData, shading model specific) + float4 OutTarget4 : SV_Target4; + + // RT5: GBufferE (RGBA = PrecomputedShadowFactors) + float4 OutTarget5 : SV_Target5; + + // RT6: GBufferF (RGB = Tangent, A = Anisotropy) - Optional + float4 OutTarget6 : SV_Target6; +}; +``` + +### Encoding Functions + +**Normal Encoding (Octahedron):** +```hlsl +// From Common.ush +// Encodes unit vector to 2 components (saves space) +float2 EncodeNormal(float3 N) +{ + N /= (abs(N.x) + abs(N.y) + abs(N.z)); + float2 p = N.z >= 0.0 ? N.xy : (1.0 - abs(N.yx)) * (N.xy >= 0.0 ? 1.0 : -1.0); + return p * 0.5 + 0.5; // Map to [0,1] +} + +float3 DecodeNormal(float2 p) +{ + p = p * 2.0 - 1.0; + float3 N = float3(p.x, p.y, 1.0 - abs(p.x) - abs(p.y)); + float t = max(-N.z, 0.0); + N.x += N.x >= 0.0 ? -t : t; + N.y += N.y >= 0.0 ? -t : t; + return normalize(N); +} +``` + +**GBuffer Encoding:** +```hlsl +// From DeferredShadingCommon.ush +void EncodeGBuffer(FGBufferData GBuffer, out FGBufferOutput Output) +{ + // GBufferA: Normal + per-object data + Output.OutTarget1.rgb = EncodeNormal(GBuffer.WorldNormal); + Output.OutTarget1.a = GBuffer.PerObjectGBufferData; + + // GBufferB: Material properties + Output.OutTarget2.r = GBuffer.Metallic; + Output.OutTarget2.g = GBuffer.Specular; + Output.OutTarget2.b = GBuffer.Roughness; + Output.OutTarget2.a = EncodeShadingModelIdAndSelectiveOutputMask( + GBuffer.ShadingModelID, + GBuffer.SelectiveOutputMask + ); + + // GBufferC: Base color + AO + Output.OutTarget3.rgb = EncodeBaseColor(GBuffer.BaseColor); + Output.OutTarget3.a = GBuffer.GBufferAO; + + // GBufferD: Custom data + Output.OutTarget4 = GBuffer.CustomData; + + // GBufferE: Precomputed shadows + Output.OutTarget5 = GBuffer.PrecomputedShadowFactors; + + // GBufferF: Tangent + Anisotropy (if enabled) + #if MATERIAL_ANISOTROPY + Output.OutTarget6.rgb = EncodeNormal(GBuffer.WorldTangent); + Output.OutTarget6.a = GBuffer.Anisotropy * 0.5 + 0.5; + #endif +} +``` + +### Custom Data Encoding (Shading Model Specific) + +Different shading models use CustomData differently: + +```hlsl +// From ShadingModelsMaterial.ush + +// Default Lit: CustomData is unused +CustomData = float4(0, 0, 0, 0); + +// Subsurface: RGB = Subsurface Color, A = Opacity +CustomData.rgb = EncodeSubsurfaceColor(SubsurfaceColor); +CustomData.a = Opacity; + +// Clear Coat: R = Coat amount, G = Coat roughness +CustomData.x = ClearCoat; +CustomData.y = ClearCoatRoughness; + +// Cloth: RGB = Fuzz color, A = Cloth amount +CustomData.rgb = EncodeSubsurfaceColor(FuzzColor); +CustomData.a = Cloth; + +// Hair: Complex encoding for hair-specific parameters +CustomData.x = Backlit; +CustomData.y = 0; // Unused +CustomData.z = 0; // Unused +CustomData.w = 0; // Unused + +// Eye: Iris data +CustomData.x = SubsurfaceProfile; +CustomData.w = 1.0 - IrisMask; +CustomData.yz = IrisNormal; // Compressed +``` + +### For Your bgfx Engine + +Design your GBuffer based on what you need: + +**Minimal GBuffer (3 RTs):** +``` +RT0: SceneColor (RGB) + AO (A) +RT1: Normal (RGB, octahedron-encoded in RG) + Roughness (B) + Metallic (A) +RT2: BaseColor (RGB) + Specular (A) +``` + +**Standard GBuffer (4 RTs):** +``` +RT0: SceneColor (RGB) + unused (A) +RT1: Normal (RG, octahedron) + unused (B) + PerObjectData (A) +RT2: Metallic (R) + Specular (G) + Roughness (B) + ShadingModelID (A) +RT3: BaseColor (RGB) + AO (A) +``` + +**Full Featured (5-6 RTs):** +- Add RT4 for CustomData +- Add RT5 for baked lighting +- Add RT6 for anisotropic tangents + +**bgfx GBuffer Setup:** +```cpp +// Create MRT framebuffer +bgfx::TextureHandle gbuffer[4]; +gbuffer[0] = bgfx::createTexture2D(width, height, false, 1, bgfx::TextureFormat::RGBA16F); // SceneColor +gbuffer[1] = bgfx::createTexture2D(width, height, false, 1, bgfx::TextureFormat::RGBA8); // Normal+Data +gbuffer[2] = bgfx::createTexture2D(width, height, false, 1, bgfx::TextureFormat::RGBA8); // Material +gbuffer[3] = bgfx::createTexture2D(width, height, false, 1, bgfx::TextureFormat::RGBA8); // BaseColor+AO + +bgfx::FrameBufferHandle gbufferFB = bgfx::createFrameBuffer(4, gbuffer, true); +``` + +--- + +## Shader Permutations + +UE5 pre-compiles thousands of shader variants to handle different material configurations. + +### Permutation Dimensions + +Shaders vary based on: + +1. **Material Properties:** + - Blend Mode (Opaque, Masked, Translucent, etc.) + - Shading Model (DefaultLit, Subsurface, ClearCoat, etc.) + - Material Domain (Surface, Decal, PostProcess, etc.) + +2. **Material Features:** + - Uses Normal Map? (yes/no) + - Uses Vertex Color? (yes/no) + - Uses World Position Offset? (yes/no) + - Num Texture Coordinates (0-8) + - Num Custom Vertex Interpolators (0-8) + +3. **Platform/Quality:** + - Feature Level (ES3.1, SM5, SM6) + - Quality Level (Low, Medium, High, Epic) + - Platform (PC, Console, Mobile) + +4. **Vertex Factory:** + - Static Mesh + - Skeletal Mesh + - Instanced Static Mesh + - Niagara (particles) + - etc. + +### Permutation Defines + +**File:** `Engine/Shaders/Private/MaterialTemplate.ush` + +```hlsl +// Material blend mode +#define MATERIALBLENDING_OPAQUE 0 +#define MATERIALBLENDING_MASKED 1 +#define MATERIALBLENDING_TRANSLUCENT 2 +#define MATERIALBLENDING_ADDITIVE 3 +#define MATERIALBLENDING_MODULATE 4 + +// Current material uses: +#define MATERIAL_BLENDING_MODE MATERIALBLENDING_OPAQUE + +// Shading model +#define MATERIAL_SHADINGMODEL_DEFAULT_LIT 1 +#define MATERIAL_SHADINGMODEL_SUBSURFACE 1 +#define MATERIAL_SHADINGMODEL_CLEAR_COAT 1 +// ... etc (one is set to 1, rest to 0) + +// Material features +#define NUM_MATERIAL_TEXCOORDS 2 +#define NUM_CUSTOM_VERTEX_INTERPOLATORS 0 +#define USES_WORLD_POSITION_OFFSET 1 +#define MATERIAL_USES_ANISOTROPY 0 +#define MATERIAL_TANGENTSPACENORMAL 1 + +// Platform +#define FEATURE_LEVEL FEATURE_LEVEL_SM5 +#define PLATFORM_SUPPORTS_SRV_UB 1 +``` + +### Uber Shader vs. Permutations + +**UE5 Approach (Static Permutations):** +```hlsl +// Compile-time branching +#if MATERIAL_SHADINGMODEL_SUBSURFACE + // Subsurface-specific code + float3 SubsurfaceLighting = CalculateSubsurface(...); +#elif MATERIAL_SHADINGMODEL_CLEAR_COAT + // Clear coat-specific code + float3 CoatLighting = CalculateClearCoat(...); +#else + // Default lit + float3 Lighting = CalculateDefaultLit(...); +#endif +``` + +Pros: Optimal code for each variant, no runtime branching +Cons: Thousands of shaders, long compile times, large disk space + +**Alternative Approach (Uber Shaders):** +```hlsl +// Runtime branching +uniform int u_shadingModel; + +if (u_shadingModel == SHADINGMODEL_SUBSURFACE) { + return CalculateSubsurface(...); +} else if (u_shadingModel == SHADINGMODEL_CLEARCOAT) { + return CalculateClearCoat(...); +} else { + return CalculateDefaultLit(...); +} +``` + +Pros: Fewer shaders, fast iteration +Cons: Runtime branching overhead, less optimal code + +### For Your Engine + +**Recommended Hybrid Approach:** + +Static permutations for major variants: +- Blend mode (Opaque vs. Translucent) +- Vertex deformation (yes/no) +- Major shading models + +Dynamic branching for minor features: +- Texture usage +- Parameter variations +- Quality levels + +```cpp +// Example: Generate permutations +std::vector GenerateMaterialPermutations(const MaterialX& mtlx) +{ + std::vector permutations; + + // Static: Blend mode + for (auto blend : {Opaque, Masked, Translucent}) { + // Static: Shading model + for (auto model : {DefaultLit, Subsurface, ClearCoat}) { + // Only generate valid combinations + if (IsValidCombination(blend, model, mtlx)) { + ShaderPermutation perm; + perm.defines["BLEND_MODE"] = blend; + perm.defines["SHADING_MODEL"] = model; + perm.source = GenerateShaderSource(mtlx, perm.defines); + permutations.push_back(perm); + } + } + } + + return permutations; +} +``` + +**bgfx Shader Variants:** +```cpp +// bgfx supports shader variants via defines +bgfx::ShaderHandle compileBgfxShader(const char* source, const char* defines) +{ + const bgfx::Memory* mem = bgfx::copy(source, strlen(source) + 1); + + bgfx::ShaderHandle shader = bgfx::createShader(mem); + bgfx::setName(shader, "MaterialShader"); + + return shader; +} + +// Compile permutations +std::map shaderCache; + +uint32_t key = (blendMode << 16) | (shadingModel << 8) | features; +if (shaderCache.find(key) == shaderCache.end()) { + std::string defines = GenerateDefines(blendMode, shadingModel, features); + bgfx::ShaderHandle vs = compileBgfxShader(vertexSource, defines.c_str()); + bgfx::ShaderHandle fs = compileBgfxShader(fragmentSource, defines.c_str()); + shaderCache[key] = bgfx::createProgram(vs, fs, true); +} +``` + +--- + +## BRDF and Lighting Integration + +UE5 uses physically-based rendering with the GGX microfacet BRDF. + +**Key File:** `Engine/Shaders/Private/BRDF.ush` + +### BRDF Context + +```hlsl +struct BxDFContext +{ + float NoV; // Normal · View + float NoL; // Normal · Light + float VoL; // View · Light + float NoH; // Normal · Half + float VoH; // View · Half + + // For anisotropic materials + float XoV, XoL, XoH; // Tangent dot products + float YoV, YoL, YoH; // Bitangent dot products +}; + +BxDFContext Init(float3 N, float3 V, float3 L) +{ + BxDFContext Context; + Context.NoV = saturate(dot(N, V)); + Context.NoL = saturate(dot(N, L)); + Context.VoL = dot(V, L); + + float3 H = normalize(V + L); + Context.NoH = saturate(dot(N, H)); + Context.VoH = saturate(dot(V, H)); + + return Context; +} +``` + +### Diffuse BRDF + +```hlsl +// Lambert (simplest, cheapest) +float3 Diffuse_Lambert(float3 DiffuseColor) +{ + return DiffuseColor * (1.0 / PI); +} + +// Disney's Burley (more realistic, retro-reflection) +float3 Diffuse_Burley(float3 DiffuseColor, float Roughness, float NoV, float NoL, float VoH) +{ + float FD90 = 0.5 + 2.0 * VoH * VoH * Roughness; + float FdV = 1.0 + (FD90 - 1.0) * Pow5(1.0 - NoV); + float FdL = 1.0 + (FD90 - 1.0) * Pow5(1.0 - NoL); + return DiffuseColor * ((1.0 / PI) * FdV * FdL); +} + +// UE5 default for rough surfaces +float3 Diffuse_GGX_Rough(float3 DiffuseColor, float Roughness, float NoV, float NoL, float VoH, float NoH) +{ + // More complex, accounts for microfacet multiple scattering + // See BRDF.ush line 128 +} +``` + +### Specular BRDF (GGX Microfacet) + +Complete GGX BRDF: +``` +f_specular = (D * G * F) / (4 * NoV * NoL) + +Where: + D = Normal Distribution Function (GGX/Trowbridge-Reitz) + G = Geometric Shadowing/Masking (Smith) + F = Fresnel (Schlick approximation) +``` + +**Normal Distribution (D):** +```hlsl +// GGX / Trowbridge-Reitz +float D_GGX(float Roughness, float NoH) +{ + float a = Roughness * Roughness; + float a2 = a * a; + float d = (NoH * a2 - NoH) * NoH + 1.0; + return a2 / (PI * d * d); +} + +// Anisotropic GGX +float D_GGXaniso(float RoughnessX, float RoughnessY, float NoH, float XoH, float YoH) +{ + float ax = RoughnessX * RoughnessX; + float ay = RoughnessY * RoughnessY; + float a2 = ax * ay; + float3 V = float3(ay * XoH, ax * YoH, a2 * NoH); + float S = dot(V, V); + return (1.0 / PI) * a2 * Square(a2 / S); +} +``` + +**Visibility/Geometry (G, split into V):** +```hlsl +// Smith Joint Approximation (Height-Correlated) +float Vis_SmithJointApprox(float Roughness, float NoV, float NoL) +{ + float a = Roughness * Roughness; + float Vis_SmithV = NoL * (NoV * (1.0 - a) + a); + float Vis_SmithL = NoV * (NoL * (1.0 - a) + a); + return 0.5 * rcp(Vis_SmithV + Vis_SmithL); + // Note: Includes 1/(4*NoV*NoL) from BRDF denominator +} + +// Anisotropic Visibility +float Vis_SmithJointAniso(float ax, float ay, float NoV, float NoL, + float XoV, float XoL, float YoV, float YoL) +{ + // More complex, see BRDF.ush line 258 +} +``` + +**Fresnel (F):** +```hlsl +// Schlick approximation +float3 F_Schlick(float3 SpecularColor, float VoH) +{ + float Fc = Pow5(1.0 - VoH); + return Fc + (1.0 - Fc) * SpecularColor; +} + +// With roughness (for environment reflections) +float3 EnvBRDFApprox(float3 SpecularColor, float Roughness, float NoV) +{ + // Pre-integrated environment BRDF + // Uses lookup texture or analytical approximation + float4 c0 = float4(-1.0, -0.0275, -0.572, 0.022); + float4 c1 = float4(1.0, 0.0425, 1.04, -0.04); + float4 r = Roughness * c0 + c1; + float a004 = min(r.x * r.x, exp2(-9.28 * NoV)) * r.x + r.y; + float2 AB = float2(-1.04, 1.04) * a004 + r.zw; + return SpecularColor * AB.x + AB.y; +} +``` + +### Complete Default Lit Shading + +```hlsl +float3 StandardShading(FGBufferData GBuffer, float3 L, float3 V, float3 N, float3 LightColor) +{ + // Setup context + BxDFContext Context = Init(N, V, L); + + // Derived properties + float3 DiffuseColor = GBuffer.BaseColor * (1.0 - GBuffer.Metallic); + float3 SpecularColor = lerp(0.04, GBuffer.BaseColor, GBuffer.Metallic); + float Roughness = max(GBuffer.Roughness, 0.02); // Clamp min + + // Diffuse + float3 Diffuse = Diffuse_Burley(DiffuseColor, Roughness, + Context.NoV, Context.NoL, Context.VoH); + + // Specular + float D = D_GGX(Roughness, Context.NoH); + float Vis = Vis_SmithJointApprox(Roughness, Context.NoV, Context.NoL); + float3 F = F_Schlick(SpecularColor, Context.VoH); + float3 Specular = (D * Vis) * F; + + // Combine + float3 Lighting = (Diffuse + Specular) * LightColor * Context.NoL; + + return Lighting; +} +``` + +### For Your Engine + +Implement this exact BRDF for physically-accurate results: + +```glsl +// GLSL version for bgfx +vec3 StandardBRDF(vec3 N, vec3 V, vec3 L, + vec3 baseColor, float metallic, float roughness) +{ + vec3 H = normalize(V + L); + + float NoV = max(dot(N, V), 0.0); + float NoL = max(dot(N, L), 0.0); + float NoH = max(dot(N, H), 0.0); + float VoH = max(dot(V, H), 0.0); + + // Derive colors + vec3 diffuseColor = baseColor * (1.0 - metallic); + vec3 specularColor = mix(vec3(0.04), baseColor, metallic); + + // Diffuse: Burley + float FD90 = 0.5 + 2.0 * VoH * VoH * roughness; + float FdV = 1.0 + (FD90 - 1.0) * pow(1.0 - NoV, 5.0); + float FdL = 1.0 + (FD90 - 1.0) * pow(1.0 - NoL, 5.0); + vec3 diffuse = diffuseColor * (1.0 / 3.14159) * FdV * FdL; + + // Specular: GGX + float a = roughness * roughness; + float a2 = a * a; + + // D + float denom = (NoH * a2 - NoH) * NoH + 1.0; + float D = a2 / (3.14159 * denom * denom); + + // Vis (includes G/4*NoV*NoL) + float k = a * 0.5; + float vis = 0.5 / ((NoL * (NoV * (1.0 - k) + k) + + NoV * (NoL * (1.0 - k) + k))); + + // F + float Fc = pow(1.0 - VoH, 5.0); + vec3 F = specularColor + (1.0 - specularColor) * Fc; + + vec3 specular = D * vis * F; + + return (diffuse + specular) * NoL; +} +``` + +--- + +## Shading Models + +UE5 supports multiple shading models beyond standard PBR. + +**Key File:** `Engine/Shaders/Private/ShadingModels.ush` + +### 1. Default Lit (Standard PBR) + +Already covered in BRDF section. This is your baseline. + +### 2. Subsurface Scattering + +For skin, wax, marble, etc. + +```hlsl +FDirectLighting SubsurfaceBxDF(FGBufferData GBuffer, float3 N, float3 V, float3 L) +{ + // Front lighting (standard) + BxDFContext Context = Init(N, V, L); + + // Dual specular lobes for skin + float Lobe0Roughness = max(GBuffer.Roughness, 0.02); + float Lobe1Roughness = max(GBuffer.Roughness * 0.5, 0.02); + + float D0 = D_GGX(Lobe0Roughness, Context.NoH); + float D1 = D_GGX(Lobe1Roughness, Context.NoH); + float D = lerp(D0, D1, 0.5); + + // Rest is same as DefaultLit but with dual lobe + + // Transmission (back lighting) + float BackNoL = max(0, -dot(N, L)); + float3 Transmission = GBuffer.SubsurfaceColor * BackNoL; + + Lighting.Diffuse = FrontDiffuse; + Lighting.Specular = FrontSpecular; + Lighting.Transmission = Transmission; + + return Lighting; +} +``` + +**CustomData for Subsurface:** +```hlsl +// In pixel shader: +GBuffer.CustomData.rgb = EncodeSubsurfaceColor(SubsurfaceColor); +GBuffer.CustomData.a = Opacity; + +// In lighting shader: +float3 SubsurfaceColor = DecodeSubsurfaceColor(GBuffer.CustomData.rgb); +float Opacity = GBuffer.CustomData.a; +``` + +### 3. Clear Coat + +For car paint, plastic coatings, lacquered wood. + +```hlsl +FDirectLighting ClearCoatBxDF(FGBufferData GBuffer, float3 N, float3 V, float3 L) +{ + // Two layers: coat (on top) + base (underneath) + + float ClearCoat = GBuffer.CustomData.x; + float ClearCoatRoughness = GBuffer.CustomData.y; + + // Top coat layer (fixed IOR = 1.5, F0 = 0.04) + BxDFContext CoatContext = Init(N, V, L); + + float D_Coat = D_GGX(ClearCoatRoughness, CoatContext.NoH); + float Vis_Coat = Vis_SmithJointApprox(ClearCoatRoughness, + CoatContext.NoV, CoatContext.NoL); + float F_Coat = F_Schlick(0.04, CoatContext.VoH); + + float3 CoatSpec = ClearCoat * (D_Coat * Vis_Coat * F_Coat); + + // Base layer (attenuated by coat transmission) + float3 FresnelTransmission = (1.0 - F_Coat) * (1.0 - F_Coat); + + // Refract V and L through coat (simplified: use original) + float3 BaseDiffuse = Diffuse_Lambert(GBuffer.DiffuseColor); + + float D_Base = D_GGX(GBuffer.Roughness, CoatContext.NoH); + float Vis_Base = Vis_SmithJointApprox(GBuffer.Roughness, + CoatContext.NoV, CoatContext.NoL); + float3 F_Base = F_Schlick(GBuffer.SpecularColor, CoatContext.VoH); + float3 BaseSpec = (D_Base * Vis_Base) * F_Base; + + Lighting.Diffuse = FresnelTransmission * BaseDiffuse; + Lighting.Specular = CoatSpec + FresnelTransmission * BaseSpec; + + return Lighting; +} +``` + +### 4. Two-Sided Foliage + +For leaves, plants. + +```hlsl +FDirectLighting TwoSidedFoliageBxDF(FGBufferData GBuffer, float3 N, float3 V, float3 L) +{ + // Front side: standard lighting + float NoL = saturate(dot(N, L)); + float3 FrontLighting = StandardShading(...) * NoL; + + // Back side: wrapped diffuse + subsurface + float Wrap = 0.5; // Adjustable + float BackNoL = saturate((-dot(N, L) + Wrap) / Square(1.0 + Wrap)); + + float3 BackLighting = GBuffer.SubsurfaceColor * BackNoL; + + Lighting.Diffuse = GBuffer.DiffuseColor * (FrontLighting + BackLighting); + Lighting.Specular = FrontSpecular; + + return Lighting; +} +``` + +### 5. Cloth + +For fabric, velvet. + +```hlsl +FDirectLighting ClothBxDF(FGBufferData GBuffer, float3 N, float3 V, float3 L) +{ + // Two specular lobes: + // 1. Standard GGX for base + // 2. Inverted GGX for fuzz/fabric sheen + + BxDFContext Context = Init(N, V, L); + + // Base specular + float D1 = D_GGX(GBuffer.Roughness, Context.NoH); + float Vis1 = Vis_Cloth(Context.NoV, Context.NoL); // Modified visibility + float3 F1 = F_Schlick(GBuffer.SpecularColor, Context.VoH); + float3 Spec1 = (D1 * Vis1) * F1; + + // Fuzz specular (inverted GGX) + float D2 = D_InvGGX(Pow4(GBuffer.Roughness), Context.NoH); + float Vis2 = Vis_Cloth(Context.NoV, Context.NoL); + float3 FuzzColor = GBuffer.CustomData.rgb; + float3 F2 = F_Schlick(FuzzColor, Context.VoH); + float3 Spec2 = (D2 * Vis2) * F2; + + float Cloth = GBuffer.CustomData.a; + + Lighting.Specular = lerp(Spec1, Spec2, Cloth); + Lighting.Diffuse = Diffuse_Lambert(GBuffer.DiffuseColor); + + return Lighting; +} +``` + +### Shading Model Selection + +In your lighting pass: + +```hlsl +// Read shading model from GBuffer +uint ShadingModel = GBuffer.ShadingModelID; + +// Dispatch to appropriate function +float3 Lighting; +switch (ShadingModel) +{ + case SHADINGMODELID_DEFAULT_LIT: + Lighting = DefaultLitBxDF(GBuffer, N, V, L); + break; + + case SHADINGMODELID_SUBSURFACE: + Lighting = SubsurfaceBxDF(GBuffer, N, V, L); + break; + + case SHADINGMODELID_CLEAR_COAT: + Lighting = ClearCoatBxDF(GBuffer, N, V, L); + break; + + case SHADINGMODELID_TWOSIDED_FOLIAGE: + Lighting = TwoSidedFoliageBxDF(GBuffer, N, V, L); + break; + + case SHADINGMODELID_CLOTH: + Lighting = ClothBxDF(GBuffer, N, V, L); + break; + + default: + Lighting = DefaultLitBxDF(GBuffer, N, V, L); + break; +} +``` + +--- + +## MaterialX Integration Strategy + +MaterialX is an open standard for material description, perfect for your engine. + +### MaterialX to Engine Pipeline + +``` +MaterialX Document (.mtlx) + ↓ +[Parse] MaterialXCore library + ↓ +[Analyze] Build node graph, determine outputs + ↓ +[Generate] Translate to GLSL/HLSL for bgfx + ↓ +[Compile] bgfx shader compilation + ↓ +[Runtime] Bind parameters, render +``` + +### MaterialX Standard Surface Mapping + +MaterialX `` node maps to UE5 materials: + +```xml + + + + + + + + + + + + + +``` + +Translation to your engine: +```cpp +struct Material +{ + vec3 baseColor; // base * base_color + float metallic; // metalness + float roughness; // specular_roughness + float specular; // specular + vec3 normal; // normal (tangent space) + float clearCoat; // coat + float clearCoatRough; // coat_roughness + vec3 emissive; // emission * emission_color + float opacity; // opacity (for translucent) +}; +``` + +### Code Generation from MaterialX + +```cpp +class MaterialXCodeGen +{ +public: + std::string GenerateShader(MaterialX::DocumentPtr doc) + { + // 1. Find shader nodes + auto shaderNodes = MaterialX::findRenderableElements(doc); + + for (auto node : shaderNodes) + { + if (node->getCategory() == "standard_surface") + { + return GenerateStandardSurface(node); + } + } + } + +private: + std::string GenerateStandardSurface(MaterialX::NodePtr node) + { + std::stringstream code; + + // Header + code << "// Generated from MaterialX\n"; + code << "#include \"common.sh\"\n\n"; + + // Parameters + code << "// Material Parameters\n"; + for (auto input : node->getInputs()) + { + code << GenerateParameter(input); + } + + // Main function + code << "void main()\n{\n"; + code << " // Sample inputs\n"; + + // For each connected input, generate sampling code + auto baseColorInput = node->getInput("base_color"); + if (baseColorInput->getConnectedNode()) + { + // It's connected to a texture or node + code << " vec3 baseColor = " + << GenerateNodeCode(baseColorInput->getConnectedNode()) + << ";\n"; + } + else + { + // It's a constant + auto value = baseColorInput->getValue(); + code << " vec3 baseColor = vec3" << ValueToString(value) << ";\n"; + } + + // ... repeat for all inputs + + // Write to GBuffer + code << " // Write GBuffer\n"; + code << " gl_FragData[0] = vec4(baseColor, 1.0);\n"; + code << " gl_FragData[1] = vec4(encodeNormal(normal), roughness, metallic);\n"; + code << "}\n"; + + return code.str(); + } +}; +``` + +### Parameter Binding with LUA + +You mentioned using LUA - perfect for dynamic parameter control: + +```lua +-- material_config.lua +material = { + name = "MyPBRMaterial", + materialx = "materials/pbr_metal.mtlx", + + parameters = { + baseColor = {1.0, 0.8, 0.6}, + metallic = 1.0, + roughness = 0.3, + + textures = { + baseColorMap = "textures/metal_albedo.png", + normalMap = "textures/metal_normal.png", + roughnessMap = "textures/metal_roughness.png" + } + }, + + shadingModel = "DefaultLit", + blendMode = "Opaque" +} +``` + +C++ integration: +```cpp +class MaterialInstance +{ + bgfx::ProgramHandle shader; + bgfx::UniformHandle u_baseColor; + bgfx::UniformHandle u_roughness; + bgfx::UniformHandle u_metallic; + + std::map textures; + + void LoadFromLua(lua_State* L, const char* scriptPath) + { + // Load LUA script + luaL_dofile(L, scriptPath); + + // Get material table + lua_getglobal(L, "material"); + + // Read parameters + lua_getfield(L, -1, "parameters"); + { + // baseColor + lua_getfield(L, -1, "baseColor"); + params.baseColor = ReadVec3(L); + lua_pop(L, 1); + + // metallic + lua_getfield(L, -1, "metallic"); + params.metallic = lua_tonumber(L, -1); + lua_pop(L, 1); + + // Textures + lua_getfield(L, -1, "textures"); + { + lua_getfield(L, -1, "baseColorMap"); + const char* path = lua_tostring(L, -1); + textures["baseColorMap"] = LoadTexture(path); + lua_pop(L, 1); + } + lua_pop(L, 1); + } + lua_pop(L, 1); + } + + void Bind() + { + bgfx::setUniform(u_baseColor, ¶ms.baseColor); + bgfx::setUniform(u_roughness, ¶ms.roughness); + bgfx::setUniform(u_metallic, ¶ms.metallic); + + for (auto& [name, texture] : textures) + { + uint8_t stage = GetTextureStage(name); + bgfx::setTexture(stage, GetSamplerHandle(name), texture); + } + } +}; +``` + +### JSON Metadata for Workflow + +```json +{ + "material": { + "name": "PBR_Metal", + "version": "1.0", + "materialx_source": "materials/pbr_metal.mtlx", + + "properties": { + "shadingModel": "DefaultLit", + "blendMode": "Opaque", + "twoSided": false, + "castsShadows": true + }, + + "parameters": { + "baseColor": { + "type": "color", + "default": [0.8, 0.8, 0.8], + "ui": { + "displayName": "Base Color", + "group": "Surface" + } + }, + "roughness": { + "type": "scalar", + "default": 0.5, + "range": [0.0, 1.0], + "ui": { + "displayName": "Roughness", + "group": "Surface" + } + } + }, + + "textures": { + "baseColorMap": { + "type": "texture2d", + "default": "textures/default_white.png", + "srgb": true + }, + "normalMap": { + "type": "texture2d", + "default": "textures/default_normal.png", + "srgb": false + } + }, + + "workflow": { + "editor": "MaterialX Graph Editor", + "exportFormats": ["mtlx", "glsl", "hlsl"], + "preprocessSteps": ["validate", "optimize", "compile"] + } + } +} +``` + +--- + +## bgfx Implementation Guide + +bgfx is an excellent choice for cross-platform rendering. + +### GBuffer Pass with bgfx + +```cpp +// Setup GBuffer framebuffer +class GBufferRenderer +{ + bgfx::FrameBufferHandle gbufferFB; + bgfx::TextureHandle gbufferTextures[4]; + + void Init(uint16_t width, uint16_t height) + { + // RT0: Base Color + AO + gbufferTextures[0] = bgfx::createTexture2D( + width, height, false, 1, + bgfx::TextureFormat::RGBA8, + BGFX_TEXTURE_RT | BGFX_SAMPLER_U_CLAMP | BGFX_SAMPLER_V_CLAMP + ); + + // RT1: Normal (octahedron) + Roughness + Metallic + gbufferTextures[1] = bgfx::createTexture2D( + width, height, false, 1, + bgfx::TextureFormat::RGBA8, + BGFX_TEXTURE_RT | BGFX_SAMPLER_U_CLAMP | BGFX_SAMPLER_V_CLAMP + ); + + // RT2: Emissive + Specular + gbufferTextures[2] = bgfx::createTexture2D( + width, height, false, 1, + bgfx::TextureFormat::RGBA16F, + BGFX_TEXTURE_RT | BGFX_SAMPLER_U_CLAMP | BGFX_SAMPLER_V_CLAMP + ); + + // RT3: Custom Data (shading model specific) + gbufferTextures[3] = bgfx::createTexture2D( + width, height, false, 1, + bgfx::TextureFormat::RGBA8, + BGFX_TEXTURE_RT | BGFX_SAMPLER_U_CLAMP | BGFX_SAMPLER_V_CLAMP + ); + + // Depth buffer + bgfx::TextureHandle depthTexture = bgfx::createTexture2D( + width, height, false, 1, + bgfx::TextureFormat::D24S8, + BGFX_TEXTURE_RT + ); + + // Create framebuffer with all attachments + bgfx::Attachment attachments[5]; + for (int i = 0; i < 4; ++i) { + attachments[i].init(gbufferTextures[i]); + } + attachments[4].init(depthTexture); + + gbufferFB = bgfx::createFrameBuffer(5, attachments, false); + } + + void RenderGBuffer(const std::vector& objects) + { + // Set GBuffer as render target + bgfx::setViewFrameBuffer(VIEW_GBUFFER, gbufferFB); + bgfx::setViewClear(VIEW_GBUFFER, + BGFX_CLEAR_COLOR | BGFX_CLEAR_DEPTH, + 0x000000ff, 1.0f, 0); + + // Set viewport + bgfx::setViewRect(VIEW_GBUFFER, 0, 0, width, height); + + // Render each object + for (const auto& obj : objects) + { + // Set transform + bgfx::setTransform(obj.transform); + + // Set vertex/index buffers + bgfx::setVertexBuffer(0, obj.vertexBuffer); + bgfx::setIndexBuffer(obj.indexBuffer); + + // Bind material parameters + obj.material->Bind(); + + // Set render state + uint64_t state = BGFX_STATE_WRITE_RGB + | BGFX_STATE_WRITE_A + | BGFX_STATE_WRITE_Z + | BGFX_STATE_DEPTH_TEST_LESS + | BGFX_STATE_MSAA; + + bgfx::setState(state); + + // Submit draw call + bgfx::submit(VIEW_GBUFFER, obj.material->GetShader()); + } + } +}; +``` + +### Lighting Pass with bgfx + +```cpp +class DeferredLightingRenderer +{ + bgfx::ProgramHandle lightingShader; + bgfx::UniformHandle s_gbuffer0; + bgfx::UniformHandle s_gbuffer1; + bgfx::UniformHandle s_gbuffer2; + bgfx::UniformHandle s_gbuffer3; + bgfx::UniformHandle s_depth; + + void RenderLighting(GBufferRenderer& gbuffer) + { + // Bind GBuffer textures + bgfx::setTexture(0, s_gbuffer0, gbuffer.GetTexture(0)); + bgfx::setTexture(1, s_gbuffer1, gbuffer.GetTexture(1)); + bgfx::setTexture(2, s_gbuffer2, gbuffer.GetTexture(2)); + bgfx::setTexture(3, s_gbuffer3, gbuffer.GetTexture(3)); + bgfx::setTexture(4, s_depth, gbuffer.GetDepthTexture()); + + // Bind lighting parameters + bgfx::setUniform(u_lightDirection, &lightDir); + bgfx::setUniform(u_lightColor, &lightColor); + bgfx::setUniform(u_cameraPosition, &cameraPos); + + // Fullscreen quad + screenSpaceQuad(width, height); + + // Render state + uint64_t state = BGFX_STATE_WRITE_RGB + | BGFX_STATE_WRITE_A + | BGFX_STATE_DEPTH_TEST_EQUAL; + + bgfx::setState(state); + + // Submit + bgfx::submit(VIEW_LIGHTING, lightingShader); + } +}; +``` + +### Material Shader (GLSL for bgfx) + +```glsl +// vs_material.sc (Vertex Shader) +$input a_position, a_normal, a_tangent, a_texcoord0 +$output v_worldPos, v_normal, v_tangent, v_bitangent, v_texcoord0 + +#include + +uniform mat4 u_model; + +void main() +{ + vec4 worldPos = mul(u_model, vec4(a_position, 1.0)); + v_worldPos = worldPos.xyz; + + v_normal = mul(u_model, vec4(a_normal, 0.0)).xyz; + v_tangent = mul(u_model, vec4(a_tangent.xyz, 0.0)).xyz; + v_bitangent = cross(v_normal, v_tangent) * a_tangent.w; + + v_texcoord0 = a_texcoord0; + + gl_Position = mul(u_viewProj, worldPos); +} +``` + +```glsl +// fs_material.sc (Fragment Shader - GBuffer output) +$input v_worldPos, v_normal, v_tangent, v_bitangent, v_texcoord0 + +#include + +// Material parameters +uniform vec4 u_baseColor; +uniform vec4 u_materialParams; // x=roughness, y=metallic, z=specular, w=ao + +// Textures +SAMPLER2D(s_baseColorMap, 0); +SAMPLER2D(s_normalMap, 1); +SAMPLER2D(s_roughnessMap, 2); + +// GBuffer outputs +layout(location = 0) out vec4 gbuffer0; // BaseColor + AO +layout(location = 1) out vec4 gbuffer1; // Normal + Roughness + Metallic +layout(location = 2) out vec4 gbuffer2; // Emissive + Specular +layout(location = 3) out vec4 gbuffer3; // CustomData + +// Helper: Encode normal (octahedron) +vec2 encodeNormal(vec3 n) +{ + n /= (abs(n.x) + abs(n.y) + abs(n.z)); + vec2 p = n.z >= 0.0 ? n.xy : (1.0 - abs(n.yx)) * sign(n.xy); + return p * 0.5 + 0.5; +} + +void main() +{ + // Sample textures + vec3 baseColor = texture2D(s_baseColorMap, v_texcoord0).rgb * u_baseColor.rgb; + float roughness = texture2D(s_roughnessMap, v_texcoord0).r * u_materialParams.x; + float metallic = u_materialParams.y; + float specular = u_materialParams.z; + float ao = u_materialParams.w; + + // Sample and transform normal + vec3 tangentNormal = texture2D(s_normalMap, v_texcoord0).xyz * 2.0 - 1.0; + mat3 TBN = mat3(normalize(v_tangent), + normalize(v_bitangent), + normalize(v_normal)); + vec3 worldNormal = normalize(TBN * tangentNormal); + + // Encode to GBuffer + gbuffer0 = vec4(baseColor, ao); + gbuffer1 = vec4(encodeNormal(worldNormal), roughness, metallic); + gbuffer2 = vec4(0.0, 0.0, 0.0, specular); // No emissive + gbuffer3 = vec4(0.0); // No custom data for DefaultLit +} +``` + +--- + +## Key Files Reference + +### C++ Source Files + +**Material System:** +- `Engine/Source/Runtime/Engine/Public/Materials/Material.h` - UMaterial class +- `Engine/Source/Runtime/Engine/Public/MaterialShared.h` - Runtime material types +- `Engine/Source/Runtime/Engine/Private/Materials/HLSLMaterialTranslator.cpp` - Code generation (695KB) +- `Engine/Source/Runtime/Engine/Private/Materials/MaterialExpressions.cpp` - Node implementations (16,000 lines) +- `Engine/Source/Runtime/Renderer/Public/MaterialShader.h` - Material shader base class +- `Engine/Source/Runtime/Engine/Public/MaterialShaderType.h` - Shader type system + +**Shading and BRDF:** +- `Engine/Source/Runtime/Renderer/Private/BasePassRendering.cpp` - GBuffer generation +- `Engine/Source/Runtime/Renderer/Private/LightRendering.cpp` - Deferred lighting + +### Shader Files + +**Material Shaders:** +- `Engine/Shaders/Private/MaterialTemplate.ush` - Main template +- `Engine/Shaders/Private/BRDF.ush` - BRDF functions (D, G, F) +- `Engine/Shaders/Private/ShadingModels.ush` - Shading model implementations +- `Engine/Shaders/Private/ShadingModelsMaterial.ush` - Material-side shading model code + +**GBuffer and Deferred:** +- `Engine/Shaders/Private/DeferredShadingCommon.ush` - GBuffer structure and encoding +- `Engine/Shaders/Private/DeferredLightingCommon.ush` - Lighting calculations + +**Utilities:** +- `Engine/Shaders/Private/Common.ush` - Common utilities (normal encoding, etc.) + +--- + +## Summary + +**For Your MaterialX + bgfx Engine:** + +1. **Material Definition**: Use MaterialX for artist-friendly material authoring +2. **Code Generation**: Translate MaterialX graphs to GLSL/HLSL for bgfx +3. **GBuffer Design**: Start with 3-4 render targets (BaseColor, Normal/Material, Emissive, CustomData) +4. **BRDF**: Implement GGX microfacet BRDF exactly as UE5 does +5. **Shading Models**: Start with DefaultLit, add Subsurface/ClearCoat as needed +6. **Permutations**: Use hybrid approach (static for major features, dynamic for minor) +7. **Parameters**: Use LUA for material instances, JSON for metadata/workflow +8. **bgfx Integration**: Use MRT framebuffers for GBuffer, compute shaders for post-processing + +The UE5 material system is battle-tested on thousands of shipped games. Following its architecture will give you a solid, extensible foundation. diff --git a/gameengine/docs/UE5_Reflections_And_GI_Documentation.md b/gameengine/docs/UE5_Reflections_And_GI_Documentation.md new file mode 100644 index 000000000..28262dd68 --- /dev/null +++ b/gameengine/docs/UE5_Reflections_And_GI_Documentation.md @@ -0,0 +1,1728 @@ +# Unreal Engine 5 Reflections and Global Illumination Documentation +## Modern Techniques for Real-Time Game Engines + +This documentation explains UE5's reflection and global illumination systems, from traditional techniques to cutting-edge Lumen. + +--- + +## Table of Contents +1. [System Overview](#system-overview) +2. [Screen Space Reflections (SSR)](#screen-space-reflections-ssr) +3. [Reflection Capture Probes](#reflection-capture-probes) +4. [Planar Reflections](#planar-reflections) +5. [Ambient Occlusion (SSAO/GTAO)](#ambient-occlusion-ssaogtao) +6. [Lumen Global Illumination](#lumen-global-illumination) +7. [Lumen Reflections](#lumen-reflections) +8. [Implementation Strategy for Custom Engines](#implementation-strategy-for-custom-engines) +9. [bgfx Implementation Examples](#bgfx-implementation-examples) +10. [Key Files Reference](#key-files-reference) + +--- + +## System Overview + +UE5 uses a **hybrid approach** combining multiple reflection and GI techniques: + +``` +Reflection and GI Stack (from cheapest to most expensive): +┌────────────────────────────────────────────────────┐ +│ 1. Ambient Occlusion (SSAO/GTAO) │ +│ - Cost: ~0.5ms │ +│ - Purpose: Local shadowing, depth perception │ +└────────────────────────────────────────────────────┘ + ↓ +┌────────────────────────────────────────────────────┐ +│ 2. Reflection Capture Probes │ +│ - Cost: Free (pre-computed) │ +│ - Purpose: Baseline environment reflections │ +└────────────────────────────────────────────────────┘ + ↓ +┌────────────────────────────────────────────────────┐ +│ 3. Screen Space Reflections (SSR) │ +│ - Cost: ~1-2ms │ +│ - Purpose: Sharp local reflections │ +└────────────────────────────────────────────────────┘ + ↓ +┌────────────────────────────────────────────────────┐ +│ 4. Lumen GI + Reflections │ +│ - Cost: ~4-8ms (scalable) │ +│ - Purpose: Dynamic global illumination │ +└────────────────────────────────────────────────────┘ + ↓ (optional) +┌────────────────────────────────────────────────────┐ +│ 5. Planar Reflections │ +│ - Cost: ~Full scene render │ +│ - Purpose: Perfect mirrors/water │ +└────────────────────────────────────────────────────┘ +``` + +**Key Principle:** Each technique fills gaps in the others: +- **Probes**: Baseline, always available, static environment +- **SSR**: Sharp reflections of visible geometry +- **Lumen**: Dynamic GI, rough reflections, off-screen content +- **Planar**: Perfect mirrors (expensive, used sparingly) +- **AO**: Enhances all the above with contact shadowing + +--- + +## Screen Space Reflections (SSR) + +SSR traces rays in screen space to find reflections. Fast but limited to visible geometry. + +**Key Files:** +- `Engine/Source/Runtime/Renderer/Private/ScreenSpaceReflectionTiles.cpp` +- `Engine/Shaders/Private/ScreenSpaceReflectionTileCommons.ush` + +### Tiled Optimization + +UE5 uses tile-based optimization to skip pixels that don't need SSR: + +```cpp +// Tile categorization (8×8 pixel tiles) +struct FSSRTileClassification +{ + bool bNeedsSSR; // Any pixel in tile needs SSR? + float MaxRoughness; // Roughest pixel in tile + float MinSpecular; // Least specular pixel +}; + +// Mark tiles that need processing +void MarkSSRTiles(FRDGBuilder& GraphBuilder) +{ + // For each 8×8 tile: + // Sample GBuffer (roughness, specular, shading model) + // If roughness < threshold AND specular > threshold: + // Mark tile for SSR processing + // Else: + // Skip tile (use probes instead) + + AddPass_MarkSSRTiles<<>>(); +} + +// Build coherent lists for better cache performance +void BuildSSRTileLists(FRDGBuilder& GraphBuilder) +{ + // Z-order curve for spatial coherency + // Creates indirect dispatch arguments + // Enables efficient skipping of non-SSR tiles +} +``` + +**Console Variables:** +```cpp +r.SSR.TiledComposite = 1 // Enable tiled optimization +r.SSR.TiledComposite.OverrideMaxRoughness = -1 // Max roughness (-1 = use post-process) +r.SSR.TiledComposite.MinSpecular = 0.0 // Ignore low-specular pixels +``` + +### Ray Marching Algorithm + +```hlsl +// Basic SSR ray marching +struct SSRHit +{ + bool bHit; + float2 HitUV; + float Confidence; // How reliable is this hit? +}; + +SSRHit TraceScreenSpaceRay(float3 startPos, float3 direction, + Texture2D depthBuffer, float maxDistance) +{ + const int numSteps = 32; // Linear steps + const int numBinarySearchSteps = 4; // Refinement + + float3 rayPos = startPos; + float stepSize = maxDistance / numSteps; + + // Linear ray march + for (int i = 0; i < numSteps; ++i) + { + rayPos += direction * stepSize; + + // Project to screen space + float4 screenPos = mul(ViewProjection, float4(rayPos, 1.0)); + screenPos.xyz /= screenPos.w; + float2 screenUV = screenPos.xy * 0.5 + 0.5; + screenUV.y = 1.0 - screenUV.y; + + // Out of screen bounds? + if (any(screenUV < 0.0) || any(screenUV > 1.0)) + return (SSRHit)0; + + // Sample depth buffer + float sceneDepth = depthBuffer.SampleLevel(PointSampler, screenUV, 0).r; + float rayDepth = LinearizeDepth(screenPos.z); + + // Check if ray crossed surface + if (rayDepth > sceneDepth) + { + // Binary search refinement + float3 hitPos = BinarySearchIntersection(rayPos - direction * stepSize, + rayPos, + depthBuffer, + numBinarySearchSteps); + + // Calculate confidence (fade at edges, based on angle, etc.) + float edgeFade = EdgeFade(screenUV); + float angleFade = AngleFade(direction, sceneNormal); + + SSRHit hit; + hit.bHit = true; + hit.HitUV = ProjectToScreen(hitPos); + hit.Confidence = edgeFade * angleFade; + return hit; + } + } + + return (SSRHit)0; // Miss +} +``` + +### Hierarchical Ray Marching (HZB) + +More advanced technique using Hi-Z buffer: + +```hlsl +// Use mip hierarchy to accelerate ray marching +SSRHit TraceScreenSpaceRayHZB(float3 startPos, float3 direction, + Texture2D hzbBuffer, int maxLevel) +{ + float3 rayPos = startPos; + int currentMipLevel = 0; + + while (currentMipLevel >= 0) + { + // Sample HZB at current mip + float2 screenUV = ProjectToScreen(rayPos); + float hzbDepth = hzbBuffer.SampleLevel(PointSampler, screenUV, currentMipLevel).r; + float rayDepth = LinearizeDepth(rayPos.z); + + if (rayDepth > hzbDepth) + { + // Hit something - refine with lower mip + currentMipLevel--; + } + else + { + // Empty space - advance ray + // Step size based on mip level (coarser mips = larger steps) + float stepSize = CalculateStepSize(currentMipLevel); + rayPos += direction * stepSize; + + // Increase mip level if we can (larger steps) + if (CanIncreaseMip(rayPos, currentMipLevel, maxLevel)) + currentMipLevel++; + } + + // Reached finest mip and still hitting? Found intersection + if (currentMipLevel < 0) + { + return CreateHit(rayPos); + } + } + + return (SSRHit)0; +} +``` + +### Temporal Reprojection + +SSR benefits greatly from temporal accumulation: + +```hlsl +// Current frame SSR (noisy, few samples) +float3 currentSSR = TraceSSR(worldPos, viewDir, roughness); + +// Reproject previous frame +float2 velocity = VelocityBuffer.Sample(screenUV).xy; +float2 prevUV = screenUV - velocity; + +float3 previousSSR = PreviousSSRBuffer.Sample(prevUV).rgb; + +// Temporal blend +float temporalWeight = 0.9; // High persistence + +// Reject history if scene changed too much +if (DepthDifference(prevUV) > threshold || NormalDifference(prevUV) > threshold) + temporalWeight = 0.0; // Use current frame only + +float3 finalSSR = lerp(currentSSR, previousSSR, temporalWeight); +``` + +### When to Use SSR + +**Good for:** +- Smooth surfaces (roughness < 0.4) +- Reflections of nearby visible geometry +- Indoor scenes with lots of screen-space detail + +**Bad for:** +- Rough surfaces (use probes or Lumen instead) +- Reflections of off-screen objects +- Screen edges (fadeout needed) + +--- + +## Reflection Capture Probes + +Pre-computed cubemaps placed in the world. Free at runtime, static lighting. + +**Key Files:** +- `Engine/Source/Runtime/Renderer/Private/ReflectionEnvironmentCapture.cpp` +- `Engine/Source/Runtime/Renderer/Private/ReflectionEnvironment.cpp` +- `Engine/Shaders/Private/ReflectionEnvironmentShared.ush` + +### Capture Process + +```cpp +// Capture reflection cubemap +void CaptureReflectionProbe(FReflectionCaptureComponent* Probe) +{ + const int32 CubemapSize = 128; // Typical: 128, 256, 512 + const int32 NumMips = 7; // For roughness filtering + + // Create temp cubemap + FTextureCubeRHIRef TempCubemap = CreateTextureCube(CubemapSize, PF_FloatRGBA, NumMips); + + // Render 6 faces + for (int face = 0; face < 6; ++face) + { + // Setup camera facing this direction + FVector CaptureDir = GetCubeFaceDirection(face); + FVector CaptureUp = GetCubeFaceUp(face); + + FViewInfo CaptureView; + CaptureView.Location = Probe->GetComponentLocation(); + CaptureView.Rotation = LookAt(CaptureDir, CaptureUp); + CaptureView.FOV = 90.0f; // Cube face + + // Render scene from this view + RenderSceneToTexture(CaptureView, TempCubemap, face, 0 /* mip 0 */); + } + + // Generate mip chain (importance-sampled roughness filtering) + for (int mip = 1; mip < NumMips; ++mip) + { + float roughness = MipToRoughness(mip); + FilterCubemapForRoughness(TempCubemap, mip, roughness); + } + + // Store in reflection capture array + CopyToReflectionCaptureArray(Probe->CaptureIndex, TempCubemap); +} +``` + +### Roughness-Based Mip Selection + +```hlsl +// From ReflectionEnvironmentShared.ush line 26 +float ComputeReflectionCaptureMipFromRoughness(float Roughness, float CubemapMaxMip) +{ + // Constants for mip mapping + const float REFLECTION_CAPTURE_ROUGHEST_MIP = 1.0; + const float REFLECTION_CAPTURE_ROUGHNESS_MIP_SCALE = 1.2; + + // Logarithmic mapping: rougher surfaces use blurrier mips + float LevelFrom1x1 = REFLECTION_CAPTURE_ROUGHEST_MIP + - REFLECTION_CAPTURE_ROUGHNESS_MIP_SCALE * log2(max(Roughness, 0.001)); + + return CubemapMaxMip - 1.0 - LevelFrom1x1; +} + +// Example: Roughness 0.0 → Mip 0 (sharpest) +// Roughness 0.5 → Mip 3 (medium blur) +// Roughness 1.0 → Mip 6 (blurriest) +``` + +### Cubemap Array Structure + +```cpp +// All reflection captures stored in single TextureCubeArray +class FReflectionEnvironmentCubemapArray +{ + FTextureCubeArrayRHIRef CubemapArray; + + // Typically: + // - Format: PF_FloatRGBA (HDR) + // - Resolution: 128×128 per face + // - Mips: 7 levels + // - Max captures: ~5 per pixel (blended) + + int32 MaxCubemaps = 341; // Configurable +}; +``` + +### Box Projection Correction + +Standard cubemap sampling assumes infinite distance. Box projection corrects for room interiors: + +```hlsl +// From ReflectionEnvironmentShared.ush line 136 +float3 GetLookupVectorForBoxCapture(float3 reflectionVector, + float3 worldPosition, + float4 boxCapturePositionAndRadius, + float4x4 boxTransform, + float3 boxMinimum, + float3 boxMaximum) +{ + // Transform ray to box local space + float3 localPosition = mul(boxTransform, float4(worldPosition, 1.0)).xyz; + float3 localReflection = mul((float3x3)boxTransform, reflectionVector); + + // Intersect ray with box bounds + float3 invDir = 1.0 / localReflection; + float3 firstPlane = (boxMinimum - localPosition) * invDir; + float3 secondPlane = (boxMaximum - localPosition) * invDir; + float3 furthestPlane = max(firstPlane, secondPlane); + + // Find nearest intersection + float distance = min(min(furthestPlane.x, furthestPlane.y), furthestPlane.z); + + // Intersection point in local space + float3 intersectPosition = localPosition + localReflection * distance; + + // Direction from capture center to intersection + float3 captureVector = intersectPosition - boxCapturePositionAndRadius.xyz; + + return captureVector; +} +``` + +### Probe Blending + +Multiple probes can influence a single pixel: + +```hlsl +// Blend up to 3 reflection captures +float3 BlendReflectionCaptures(float3 worldPos, float3 normal, float3 reflectionVec, + float roughness, float3 specularColor) +{ + float3 totalReflection = 0; + float totalWeight = 0; + + // Find nearest probes (sorted by distance) + const int maxProbes = 3; + ReflectionProbe probes[maxProbes]; + FindNearestProbes(worldPos, probes, maxProbes); + + for (int i = 0; i < maxProbes; ++i) + { + // Weight based on distance to probe + float distance = length(worldPos - probes[i].position); + float weight = 1.0 - saturate(distance / probes[i].influenceRadius); + + if (weight > 0.0) + { + // Box projection for parallax correction + float3 lookupVec = BoxProjection(reflectionVec, worldPos, probes[i]); + + // Sample cubemap with roughness-based mip + float mipLevel = RoughnessToMip(roughness, probes[i].numMips); + float3 reflection = ReflectionCubemapArray.SampleLevel(samplerLinear, + float4(lookupVec, probes[i].index), + mipLevel).rgb; + + totalReflection += reflection * weight; + totalWeight += weight; + } + } + + if (totalWeight > 0.0) + totalReflection /= totalWeight; + + // Apply Fresnel + float3 fresnel = EnvBRDFApprox(specularColor, roughness, saturate(dot(normal, -reflectionVec))); + + return totalReflection * fresnel; +} +``` + +### Best Practices + +- **Placement**: Put probes at lighting transitions, room corners +- **Resolution**: 128×128 for most, 256×256 for important areas +- **Influence Radius**: Just large enough to cover area +- **Box vs Sphere**: Box for rooms (better parallax), Sphere for outdoor +- **Count**: Limit to 2-3 overlapping per pixel + +--- + +## Planar Reflections + +Perfect reflections for flat surfaces (mirrors, water, polished floors). + +**Key Files:** +- `Engine/Source/Runtime/Renderer/Private/PlanarReflectionRendering.cpp` +- `Engine/Shaders/Private/PlanarReflectionShared.ush` + +### Rendering Pipeline + +```cpp +void RenderPlanarReflection(FPlanarReflectionSceneProxy* Proxy) +{ + // 1. Setup mirror view + FVector PlaneOrigin = Proxy->ReflectionPlane.Origin; + FVector PlaneNormal = Proxy->ReflectionPlane.Normal; + + // Mirror camera position across plane + FVector MirroredCameraPos = MirrorPoint(CameraPosition, PlaneOrigin, PlaneNormal); + + // Mirror camera direction + FVector MirroredCameraDir = MirrorVector(CameraDirection, PlaneNormal); + + // Flipped view matrix + FMatrix MirrorViewMatrix = CreateViewMatrix(MirroredCameraPos, MirroredCameraDir); + + // Oblique projection (clip to plane) + FMatrix ObliqueProjection = CreateObliqueProjection(ProjectionMatrix, + PlaneOrigin, + PlaneNormal, + MirrorViewMatrix); + + // 2. Render scene from mirror view + FSceneRenderer::RenderReflectionView(MirrorViewMatrix, + ObliqueProjection, + PlanarReflectionRT); + + // 3. Optional: Prefilter for roughness + if (bPrefilterRoughness) + { + FilterPlanarReflectionForRoughness(PlanarReflectionRT, Roughness); + } +} +``` + +### Prefiltering for Roughness + +```cpp +// Apply depth-aware blur based on material roughness +template +class TPrefilterPlanarReflectionPS : public FGlobalShader +{ + void Execute(FRHICommandList& RHICmdList) + { + // Parameters + float KernelRadiusY; // Blur kernel size + float InvPrefilterRoughnessDistance; // Distance-based blur + + // For each pixel: + for (int i = 0; i < numSamples; ++i) + { + float2 offset = GetSampleOffset(i); + + // Sample reflection + float3 reflection = PlanarRT.Sample(uv + offset); + + // Sample depth + float depth = DepthBuffer.Sample(uv + offset); + + // Depth weight (don't blur across depth discontinuities) + float depthWeight = exp(-abs(depth - centerDepth) * depthSharpness); + + sum += reflection * depthWeight * sampleWeight; + totalWeight += depthWeight * sampleWeight; + } + + OutColor = sum / totalWeight; + } +}; +``` + +### Projection in Material + +```hlsl +// Sample planar reflection in pixel shader +float3 SamplePlanarReflection(float3 worldPos, float3 normal, float roughness) +{ + // Transform world pos to planar reflection space + float4 reflectionPos = mul(PlanarReflectionMatrix, float4(worldPos, 1.0)); + reflectionPos.xyz /= reflectionPos.w; + + // Convert to UV + float2 reflectionUV; + reflectionUV.x = reflectionPos.x * 0.5 + 0.5; + reflectionUV.y = -reflectionPos.y * 0.5 + 0.5; // Flip Y + + // Fade at edges + float2 edgeFade = smoothstep(0.0, 0.1, reflectionUV) * + smoothstep(1.0, 0.9, reflectionUV); + float fade = edgeFade.x * edgeFade.y; + + // Sample with roughness-based mip + float mipLevel = roughness * MaxMipLevel; + float3 reflection = PlanarReflectionTexture.SampleLevel(LinearSampler, + reflectionUV, + mipLevel).rgb; + + return reflection * fade; +} +``` + +### Performance Considerations + +Planar reflections are **expensive** (full scene render): + +```cpp +// Optimization: Limit what's rendered +r.PlanarReflection.MaxDistanceFromPlane = 500.0 // Cull distant objects +r.PlanarReflection.ScreenPercentage = 50.0 // Render at half resolution +r.PlanarReflection.Quality = 1 // 0=low, 1=medium, 2=high + +// Typical use cases: +// - Water surfaces (hero feature) +// - Bathroom mirrors (if player can see face) +// - Polished floors in key areas +``` + +**Best Practice:** Use sparingly (1-2 per scene max), prefer SSR or Lumen for most reflections. + +--- + +## Ambient Occlusion (SSAO/GTAO) + +Adds depth perception and contact shadows to indirect lighting. + +**Key Files:** +- `Engine/Source/Runtime/Renderer/Private/CompositionLighting/PostProcessAmbientOcclusion.cpp` +- `Engine/Shaders/Private/PostProcessAmbientOcclusion.usf` + +### SSAO (Screen Space Ambient Occlusion) + +Classic technique using random samples in hemisphere: + +```hlsl +float SSAO(float2 screenUV, Texture2D depthBuffer, Texture2D normalBuffer) +{ + // Reconstruct position and normal + float depth = depthBuffer.Sample(screenUV).r; + float3 worldPos = ReconstructWorldPosition(screenUV, depth); + float3 normal = normalBuffer.Sample(screenUV).rgb * 2.0 - 1.0; + + // Sample kernel (random points in hemisphere) + const int numSamples = 16; + float3 randomVec = RandomTexture.Sample(screenUV * NoiseScale).rgb; + float3 tangent = normalize(randomVec - normal * dot(randomVec, normal)); + float3 bitangent = cross(normal, tangent); + mat3 TBN = mat3(tangent, bitangent, normal); + + float occlusion = 0.0; + + for (int i = 0; i < numSamples; ++i) + { + // Sample point in hemisphere + float3 sampleOffset = TBN * HemisphereSamples[i]; + float3 samplePos = worldPos + sampleOffset * SampleRadius; + + // Project to screen + float4 screenSamplePos = mul(ViewProjection, float4(samplePos, 1.0)); + screenSamplePos.xy /= screenSamplePos.w; + float2 sampleUV = screenSamplePos.xy * 0.5 + 0.5; + + // Sample depth at this point + float sampleDepth = depthBuffer.Sample(sampleUV).r; + float sampleWorldDepth = LinearizeDepth(sampleDepth); + + // Check if sample is occluded + float rangeCheck = smoothstep(0.0, 1.0, SampleRadius / abs(sampleWorldDepth - depth)); + occlusion += (sampleWorldDepth >= depth + Bias ? 1.0 : 0.0) * rangeCheck; + } + + occlusion = 1.0 - (occlusion / numSamples); + + return pow(occlusion, Power); // Enhance contrast +} +``` + +**Multi-level SSAO** (UE5 improvement): + +```cpp +// Sample at multiple scales using HZB mip chain +int32 NumLevels = 3; // Up to 3 passes + +float totalAO = 0.0; +for (int level = 0; level < NumLevels; ++level) +{ + float radius = BaseRadius * pow(2.0, level); + int mipLevel = level; + + // Sample HZB at this mip (faster for large radii) + float ao = SSAO_Level(screenUV, radius, mipLevel); + + totalAO += ao; +} + +totalAO /= NumLevels; +``` + +### GTAO (Ground Truth Ambient Occlusion) + +More accurate method based on horizon angles: + +```hlsl +float GTAO(float2 screenUV, Texture2D depthBuffer, Texture2D normalBuffer) +{ + float depth = depthBuffer.Sample(screenUV).r; + float3 viewPos = ReconstructViewPosition(screenUV, depth); + float3 viewNormal = normalBuffer.Sample(screenUV).rgb; + + // GTAO parameters + const int numDirections = 2; // Angular slices + const int numSteps = 4; // Samples per direction + + float totalAO = 0.0; + + for (int dir = 0; dir < numDirections; ++dir) + { + // Direction in screen space + float angle = (PI / numDirections) * dir; + float2 direction = float2(cos(angle), sin(angle)); + + // Search for horizon angles + float horizonAnglePos = -PI / 2.0; // Start at -90° + float horizonAngleNeg = -PI / 2.0; + + for (int step = 1; step <= numSteps; ++step) + { + float stepSize = (FalloffEnd / numSteps) * step; + float2 sampleUV = screenUV + direction * stepSize; + + // Sample depth + float sampleDepth = depthBuffer.Sample(sampleUV).r; + float3 samplePos = ReconstructViewPosition(sampleUV, sampleDepth); + + // Calculate angle to sample + float3 horizonVec = samplePos - viewPos; + float horizonAngle = atan2(horizonVec.z, length(horizonVec.xy)); + + // Update maximum horizon angle + horizonAnglePos = max(horizonAnglePos, horizonAngle); + + // Sample in negative direction too + sampleUV = screenUV - direction * stepSize; + sampleDepth = depthBuffer.Sample(sampleUV).r; + samplePos = ReconstructViewPosition(sampleUV, sampleDepth); + horizonVec = samplePos - viewPos; + horizonAngle = atan2(horizonVec.z, length(horizonVec.xy)); + horizonAngleNeg = max(horizonAngleNeg, horizonAngle); + } + + // Integrate AO from horizon angles + // (involves integration over the hemisphere, see paper for details) + float ao = IntegrateAO(horizonAnglePos, horizonAngleNeg, viewNormal, direction); + totalAO += ao; + } + + totalAO /= numDirections; + + return totalAO; +} +``` + +**GTAO Async Compute** (UE5 optimization): + +```cpp +// Run GTAO on async compute queue +enum class EGTAOType +{ + EOff, + EAsyncHorizonSearch, // GTAO with async compute + EAsyncCombinedSpatial, // GTAO + spatial filter async + ENonAsync // Graphics pipeline (fallback) +}; + +// Pass 1 (Async): Horizon search +// [GPU fence] +// Pass 2 (Graphics): Integration, spatial filter, temporal filter +// Pass 3 (Graphics): Upsample (if rendered at half-res) +``` + +**Console Variables:** +```cpp +r.AO.Method = 1 // 0=SSAO, 1=GTAO +r.AO.Quality = 2 // 0-4 +r.AO.Radius = 100.0 // World-space radius +r.AO.Power = 1.5 // Contrast enhancement +r.GTAO.NumAngles = 2 // Angular samples (1-16) +r.GTAO.Upsample = 1 // Enable upsampling +``` + +### Bilateral Filtering + +Both SSAO and GTAO benefit from edge-preserving blur: + +```hlsl +float BilateralFilter(float2 uv, Texture2D aoBuffer, + Texture2D depthBuffer, Texture2D normalBuffer) +{ + float centerDepth = depthBuffer.Sample(uv).r; + float3 centerNormal = normalBuffer.Sample(uv).rgb; + float centerAO = aoBuffer.Sample(uv).r; + + float totalAO = centerAO; + float totalWeight = 1.0; + + // 5×5 kernel + for (int y = -2; y <= 2; ++y) + { + for (int x = -2; x <= 2; ++x) + { + if (x == 0 && y == 0) continue; + + float2 offset = float2(x, y) * TexelSize; + float2 sampleUV = uv + offset; + + float sampleDepth = depthBuffer.Sample(sampleUV).r; + float3 sampleNormal = normalBuffer.Sample(sampleUV).rgb; + float sampleAO = aoBuffer.Sample(sampleUV).r; + + // Depth weight (don't blur across edges) + float depthDiff = abs(sampleDepth - centerDepth); + float depthWeight = exp(-depthDiff * DepthSigma); + + // Normal weight + float normalWeight = pow(max(0.0, dot(centerNormal, sampleNormal)), NormalPower); + + // Spatial weight (Gaussian) + float spatialWeight = exp(-dot(offset, offset) * SpatialSigma); + + float weight = depthWeight * normalWeight * spatialWeight; + + totalAO += sampleAO * weight; + totalWeight += weight; + } + } + + return totalAO / totalWeight; +} +``` + +--- + +## Lumen Global Illumination + +UE5's flagship dynamic GI system. Complex but transformative. + +**Key Files:** +- `Engine/Source/Runtime/Renderer/Private/Lumen/LumenScreenProbeGather.cpp` (4,100+ lines) +- `Engine/Source/Runtime/Renderer/Private/Lumen/LumenScene.cpp` +- `Engine/Source/Runtime/Renderer/Private/Lumen/LumenRadianceCache.cpp` + +### Three Core Components + +#### 1. Surface Cache (Scene Representation) + +Lumen represents the scene as "cards" (simplified surfaces): + +```cpp +// Scene stored as oriented cards +struct FLumenCard +{ + FVector Origin; + FVector Orientation; // Local axes + FVector2D Size; // Dimensions + uint32 MeshIndex; // Which mesh + uint32 AtlasAllocation; // Where in atlas +}; + +// Cards rasterized into atlases +struct FLumenSurfaceCache +{ + // Virtual texture atlases + FRDGTextureRef AlbedoAtlas; // Base color + FRDGTextureRef NormalAtlas; // World-space normals + FRDGTextureRef EmissiveAtlas; // Emissive lighting + FRDGTextureRef DepthAtlas; // Depth for parallax + + // Each mesh gets 1-6 cards (like a simplified bounding box) + // Cards are streamed based on importance +}; +``` + +**Card generation:** +```cpp +void GenerateLumenCards(FStaticMeshSceneProxy* Mesh) +{ + // For each axis-aligned direction (±X, ±Y, ±Z): + // Create a card facing that direction + // Size = mesh projection onto that plane + // Store in card array + + // Typically 6 cards per mesh (box-like) + // Complex meshes may use multiple cards per face +} +``` + +#### 2. Screen Probe Gather (Final Gather) + +Distributes probes across screen, each tracing rays: + +```cpp +// Probe placement +struct FScreenProbeGather +{ + // Uniform grid: One probe per 16×16 pixel tile + int32 ProbeDownsampleFactor = 16; + + // Adaptive probes: Additional probes at detail areas + int32 NumAdaptiveProbes = 8; + float AdaptiveProbeFraction = 0.5; + + // Each probe traces ~64 rays (octahedral distribution) + int32 OctahedronResolution = 8; // 8×8 = 64 rays +}; + +void PlaceScreenProbes(FRDGBuilder& GraphBuilder) +{ + // Uniform grid + int probesX = ScreenWidth / DownsampleFactor; + int probesY = ScreenHeight / DownsampleFactor; + + // Place adaptive probes at: + // - Geometric discontinuities (depth edges) + // - Shadow boundaries + // - High variance areas (from previous frame) + + AddPass_PlaceAdaptiveProbes<<<...>>>(); +} +``` + +**Probe tracing:** +```hlsl +// For each screen probe: +void TraceScreenProbe(uint2 probeCoord) +{ + // Get world position for this probe + float depth = DepthBuffer[probeCoord * DownsampleFactor]; + float3 worldPos = ReconstructWorldPosition(probeCoord, depth); + float3 normal = NormalBuffer[probeCoord * DownsampleFactor].rgb; + + // Octahedral ray distribution (uniform hemispherical coverage) + const int resolution = 8; + float3 radiance[resolution * resolution]; + + for (int y = 0; y < resolution; ++y) + { + for (int x = 0; x < resolution; ++x) + { + // Octahedral UV to direction + float2 octUV = float2(x, y) / float2(resolution); + float3 rayDir = OctahedralToDirection(octUV, normal); + + // Trace ray against surface cache + FLumenRayHit hit = TraceLumenRay(worldPos, rayDir, MaxTraceDistance); + + if (hit.bHit) + { + // Sample surface cache at hit point + radiance[y * resolution + x] = SampleSurfaceCache(hit.CardCoord); + } + else + { + // Hit sky + radiance[y * resolution + x] = SampleSkyLight(rayDir); + } + } + } + + // Store in octahedral atlas + WriteOctahedralProbe(probeCoord, radiance); +} +``` + +**Interpolation to pixels:** +```hlsl +// For each pixel: +float3 GatherIndirectLighting(float2 screenUV) +{ + // Find surrounding 4 probes + float2 probeCoord = screenUV / ProbeDownsampleFactor; + int2 probe00 = floor(probeCoord); + int2 probe01 = probe00 + int2(0, 1); + int2 probe10 = probe00 + int2(1, 0); + int2 probe11 = probe00 + int2(1, 1); + + // Bilinear weights + float2 frac = fract(probeCoord); + + // Get world position and normal + float3 worldPos = ...; + float3 normal = ...; + + // Sample each probe (with depth-aware weighting) + float3 lighting = 0; + float totalWeight = 0; + + // Probe 00 + float3 probe00Dir = normalize(GetProbePosition(probe00) - worldPos); + float probe00Weight = max(0, dot(normal, probe00Dir)) * + DepthWeight(worldPos, probe00) * + (1 - frac.x) * (1 - frac.y); + lighting += SampleProbe(probe00, normal) * probe00Weight; + totalWeight += probe00Weight; + + // ... repeat for probe01, probe10, probe11 + + return lighting / totalWeight; +} +``` + +#### 3. Radiance Cache (Far-Field GI) + +World-space probe grid for far-field lighting: + +```cpp +struct FLumenRadianceCache +{ + // 3D clipmap structure (like VSM but in 3D) + int32 NumClipmapLevels = 4; + + struct FClipmapLevel + { + FVector CenterPosition; // Centered on camera + float ProbeSpacing; // Distance between probes + FVector3i GridResolution; // e.g., 64×64×64 + + // Each probe stores octahedral radiance + FRDGTextureRef ProbeRadiance; // R16G16B16A16F, octahedral + FRDGTextureRef ProbeDepth; // For parallax + }; + + FClipmapLevel Levels[NumClipmapLevels]; + + // Level 0: High detail, small area (2m spacing) + // Level 1: Medium detail, medium area (4m spacing) + // Level 2: Low detail, large area (8m spacing) + // Level 3: Very low detail, very large area (16m spacing) +}; +``` + +**Usage:** +```hlsl +// Sample radiance cache for rough reflections or far-field GI +float3 SampleRadianceCache(float3 worldPos, float3 direction) +{ + // Determine clipmap level based on distance from camera + float distanceFromCamera = length(worldPos - CameraPosition); + int level = log2(distanceFromCamera / BaseSpacing); + level = clamp(level, 0, NumLevels - 1); + + // Find surrounding 8 probes (trilinear interpolation) + float3 localPos = (worldPos - Levels[level].CenterPosition) / Levels[level].ProbeSpacing; + int3 probe000 = floor(localPos); + float3 frac = fract(localPos); + + // Sample all 8 probes + float3 radiance = 0; + for (int z = 0; z < 2; ++z) + { + for (int y = 0; y < 2; ++y) + { + for (int x = 0; x < 2; ++x) + { + int3 probeCoord = probe000 + int3(x, y, z); + float weight = lerp(1 - frac.x, frac.x, x) * + lerp(1 - frac.y, frac.y, y) * + lerp(1 - frac.z, frac.z, z); + + // Sample octahedral radiance in direction + float3 probeRadiance = SampleProbeOctahedral(level, probeCoord, direction); + + radiance += probeRadiance * weight; + } + } + } + + return radiance; +} +``` + +### Ray Tracing Methods + +**Software Ray Tracing (Default):** +```cpp +// Uses Signed Distance Fields (SDF) +FLumenRayHit TraceLumenRay(float3 origin, float3 direction, float maxDistance) +{ + // Two-phase tracing: + // 1. Mesh SDF (accurate, short range) + // 2. Global SDF (fast, long range) + + float t = 0; + const int maxSteps = 64; + + // Phase 1: Mesh SDF (up to ~180 units) + while (t < min(maxDistance, MeshSDFTraceDistance)) + { + float3 pos = origin + direction * t; + + // Sample mesh SDF + float sdf = SampleMeshSDF(pos); + + if (sdf < SurfaceThreshold) + { + // Hit! Find which card + FLumenCard card = FindLumenCard(pos); + return CreateHit(card, pos, t); + } + + // Step by SDF distance (sphere tracing) + t += max(sdf, MinStepSize); + } + + // Phase 2: Global SDF (remaining distance) + while (t < maxDistance) + { + float3 pos = origin + direction * t; + + float sdf = SampleGlobalSDF(pos); + + if (sdf < SurfaceThreshold) + { + // Hit coarse geometry + return CreateApproximateHit(pos, t); + } + + t += sdf; + } + + return CreateMiss(); +} +``` + +**Hardware Ray Tracing (Optional):** +```cpp +// If r.Lumen.HardwareRayTracing = 1 +FLumenRayHit TraceLumenRay_HWRT(float3 origin, float3 direction, float maxDistance) +{ + RayDesc ray; + ray.Origin = origin; + ray.Direction = direction; + ray.TMin = 0.01; + ray.TMax = maxDistance; + + FLumenRayPayload payload; + + TraceRay(SceneAccelerationStructure, + RAY_FLAG_NONE, + 0xFF, + 0, 0, 0, + ray, + payload); + + if (payload.HitT > 0) + { + // Hit lighting: Evaluate materials at hit point + // OR Surface cache: Sample pre-cached lighting + return CreateHit(payload); + } + + return CreateMiss(); +} +``` + +### Temporal Accumulation + +Critical for noise reduction: + +```hlsl +// Current frame (noisy) +float3 currentGI = TraceAndGatherGI(worldPos, normal); + +// Reproject previous frame +float2 velocity = VelocityBuffer.Sample(screenUV).xy; +float2 prevUV = screenUV - velocity; + +// Sample history +float3 previousGI = PreviousGIBuffer.Sample(prevUV).rgb; + +// History validation +bool validHistory = true; +validHistory &= (abs(DepthBuffer.Sample(screenUV) - DepthHistory.Sample(prevUV)) < DepthThreshold); +validHistory &= (dot(NormalBuffer.Sample(screenUV), NormalHistory.Sample(prevUV)) > NormalThreshold); +validHistory &= all(prevUV >= 0.0 && prevUV <= 1.0); + +// Temporal blend +float historyWeight = validHistory ? 0.9 : 0.0; + +// Clamp history to reduce ghosting +float3 currentNeighborMin, currentNeighborMax; +ComputeNeighborMinMax(screenUV, currentNeighborMin, currentNeighborMax); +previousGI = clamp(previousGI, currentNeighborMin, currentNeighborMax); + +float3 finalGI = lerp(currentGI, previousGI, historyWeight); +``` + +### Performance Scalability + +```cpp +// Screen probe settings +r.Lumen.ScreenProbeGather.DownsampleFactor = 16 // 8, 16, 32 +r.Lumen.ScreenProbeGather.TracingOctahedronResolution = 8 // 6, 8, 12 + +// Trace distance +r.Lumen.MaxTraceDistance = 20000 // cm +r.Lumen.TraceMeshSDFs.TraceDistance = 180.0 + +// Quality +r.Lumen.ScreenProbeGather.ScreenTraceMaxIterations = 64 +r.Lumen.ScreenProbeGather.SpatialFilterHalfKernelSize = 1 + +// Hardware RT (expensive) +r.Lumen.HardwareRayTracing = 0 // 0=software, 1=hardware + +// Async compute +r.Lumen.DiffuseIndirect.AsyncCompute = 1 +``` + +--- + +## Lumen Reflections + +Lumen also handles reflections (separate from GI, different tracing). + +**Key File:** `Engine/Source/Runtime/Renderer/Private/Lumen/LumenReflections.cpp` (3,600+ lines) + +### Reflection vs. Diffuse GI Differences + +```cpp +// Diffuse GI: +// - Hemispherical gathering (wide cone) +// - Cosine-weighted importance +// - Lower resolution acceptable + +// Reflections: +// - Narrow cone (based on roughness) +// - Mirror direction +// - Needs higher resolution for sharp reflections +``` + +### Roughness-Based Dispatch + +```cpp +// Smooth surfaces: Full ray-traced reflections +// Rough surfaces: Use radiance cache (cheaper) + +float MaxRoughnessForTracing = 0.35; // Configurable + +if (Roughness < MaxRoughnessForTracing) +{ + // Full reflection ray tracing + float3 reflection = TraceLumenReflection(worldPos, reflectDir, Roughness); +} +else +{ + // Sample radiance cache instead + float3 reflection = SampleRadianceCache(worldPos, reflectDir); +} +``` + +### GGX Importance Sampling + +```hlsl +// Sample reflection direction based on material roughness +float3 SampleReflectionDirection(float3 normal, float3 view, float roughness, float2 random) +{ + // GGX distribution sampling + float a = roughness * roughness; + float a2 = a * a; + + float phi = 2.0 * PI * random.x; + float cosTheta = sqrt((1.0 - random.y) / (1.0 + (a2 - 1.0) * random.y)); + float sinTheta = sqrt(1.0 - cosTheta * cosTheta); + + // Spherical to Cartesian + float3 H; + H.x = sinTheta * cos(phi); + H.y = sinTheta * sin(phi); + H.z = cosTheta; + + // Transform to world space + float3 up = abs(normal.z) < 0.999 ? float3(0, 0, 1) : float3(1, 0, 0); + float3 tangent = normalize(cross(up, normal)); + float3 bitangent = cross(normal, tangent); + + H = tangent * H.x + bitangent * H.y + normal * H.z; + + // Reflect view around H + float3 reflectDir = reflect(-view, H); + + return reflectDir; +} +``` + +### Reflection Denoiser + +```cpp +// Four-stage denoiser for Lumen reflections: + +// 1. Screen Space Reconstruction (spatial filter, BRDF-aware) +float3 ScreenSpaceReconstruction(float2 uv, float3 currentReflection) +{ + // 5-tap spatial filter + // Weights based on BRDF similarity (roughness, normal) + // Bilateral filtering +} + +// 2. Temporal Filter +float3 TemporalFilter(float2 uv, float3 currentReflection, float3 previousReflection) +{ + // High temporal weight (up to 12 frames accumulation) + // History rejection based on depth/normal/reflection direction changes +} + +// 3. Bilateral Filter +float3 BilateralFilter(float2 uv, float3 reflection) +{ + // Edge-preserving blur + // Kernel radius: ~8 pixels + // Depth and normal weights +} + +// 4. Firefly Suppression +float3 FireflySuppression(float3 reflection) +{ + // Clamp extreme values (bright reflections) + // Tonemap-based range compression + float MaxIntensity = 40.0; // Configurable + return min(reflection, MaxIntensity); +} +``` + +--- + +## Implementation Strategy for Custom Engines + +Building a full Lumen system takes years. Here's a practical roadmap: + +### Phase 1: Foundation (Months 1-3) + +**Start with the basics:** + +1. **Reflection Probes** (Week 1-2) + - Cubemap capture at editor time + - Roughness-based mip selection + - Simple blending (nearest probe or 2-3 blended) + +2. **SSAO** (Week 3) + - Basic hemisphere sampling + - Bilateral blur + - Good enough for most games + +3. **SSR** (Week 4-6) + - Linear ray marching + - Basic PCF + - Temporal accumulation + - Tile-based optimization + +**Result:** 80% of commercial games stop here. This gives you: +- Indoor reflections (SSR) +- Outdoor reflections (probes) +- Depth perception (SSAO) + +### Phase 2: Intermediate (Months 4-6) + +**Improve quality:** + +4. **GTAO** (Week 7-8) + - Better AO than SSAO + - Async compute implementation + - Upsampling for performance + +5. **Better SSR** (Week 9-10) + - HZB ray marching + - Better filtering (BRDF-aware) + - Improved temporal + +6. **Probe Volumes** (Week 11-12) + - 3D grid of probes (simplified radiance cache) + - Manual placement or auto-generation + - Trilinear interpolation + - Good enough GI for many games + +**Result:** AA/AAA quality reflections and basic GI. + +### Phase 3: Advanced (Months 7-12) + +**If you need dynamic GI:** + +7. **Signed Distance Fields** (Month 7-8) + - Mesh SDF generation + - Global SDF building + - Ray marching against SDFs + - This is hard but essential for Lumen-like GI + +8. **Screen Probe GI** (Month 9-10) + - Probe placement (uniform + adaptive) + - Ray tracing against SDFs + - Octahedral storage + - Interpolation to pixels + +9. **Temporal Stability** (Month 11-12) + - Robust history validation + - Variance estimation + - Adaptive sample counts + - This makes or breaks the system + +**Result:** Lumen-like dynamic GI. + +### Phase 4: Optional (Year 2+) + +10. **Hardware Ray Tracing** + - DXR/Vulkan RT integration + - BVH building + - Hybrid software/hardware + +11. **Surface Cache** + - Card generation + - Virtual texture atlases + - Progressive streaming + +12. **Radiance Cache** + - Clipmap structure + - Probe update heuristics + - Parallax-corrected lookup + +--- + +## bgfx Implementation Examples + +### Reflection Probe System + +```cpp +class ReflectionProbeSystem +{ + struct Probe + { + vec3 position; + float influenceRadius; + bgfx::TextureHandle cubemap; // PF_RGBA16F with mips + mat4 boxTransform; // For box projection + vec3 boxMin, boxMax; + }; + + std::vector probes; + + bgfx::TextureHandle CaptureCubemap(vec3 position) + { + const int size = 128; + const int numMips = 7; + + // Create cubemap + bgfx::TextureHandle cubemap = bgfx::createTextureCube( + size, false, numMips, + bgfx::TextureFormat::RGBA16F, + BGFX_TEXTURE_RT + ); + + // Render 6 faces + vec3 dirs[6] = { {1,0,0}, {-1,0,0}, {0,1,0}, {0,-1,0}, {0,0,1}, {0,0,-1} }; + vec3 ups[6] = { {0,1,0}, {0,1,0}, {0,0,1}, {0,0,1}, {0,1,0}, {0,1,0} }; + + for (int face = 0; face < 6; ++face) + { + // Setup view + mat4 view = lookAt(position, position + dirs[face], ups[face]); + mat4 proj = perspective(90.0f, 1.0f, 0.1f, 1000.0f); + + // Create FBO for this face + bgfx::Attachment attachment; + attachment.init(cubemap, bgfx::Access::Write, 0, 0, face); + bgfx::FrameBufferHandle fbo = bgfx::createFrameBuffer(1, &attachment); + + // Render scene + bgfx::setViewFrameBuffer(VIEW_PROBE_CAPTURE, fbo); + bgfx::setViewTransform(VIEW_PROBE_CAPTURE, &view, &proj); + RenderScene(VIEW_PROBE_CAPTURE); + + bgfx::destroy(fbo); + } + + // Generate mips (importance-sampled for roughness) + GenerateRoughnessMips(cubemap); + + return cubemap; + } + + void Bind(uint8_t stage, bgfx::UniformHandle sampler) + { + // For now, bind single nearest probe + // (Full implementation would blend multiple) + bgfx::setTexture(stage, sampler, probes[0].cubemap); + } +}; +``` + +### Screen Space AO (Compute) + +```cpp +// SSAO compute shader +class SSAORenderer +{ + bgfx::ProgramHandle ssaoCS; + bgfx::TextureHandle aoTexture; + bgfx::TextureHandle randomTexture; + + void Init(uint16_t width, uint16_t height) + { + // AO output texture + aoTexture = bgfx::createTexture2D( + width, height, false, 1, + bgfx::TextureFormat::R8, + BGFX_TEXTURE_COMPUTE_WRITE + ); + + // 4×4 random vectors + randomTexture = bgfx::createTexture2D( + 4, 4, false, 1, + bgfx::TextureFormat::RGBA8, + BGFX_TEXTURE_NONE, + GenerateRandomNoise() + ); + + // Load compute shader + ssaoCS = bgfx::createProgram( + bgfx::createShader(loadMemory("cs_ssao.bin")), + true + ); + } + + void Render(bgfx::TextureHandle depthBuffer, bgfx::TextureHandle normalBuffer) + { + // Bind inputs + bgfx::setImage(0, depthBuffer, 0, bgfx::Access::Read); + bgfx::setImage(1, normalBuffer, 0, bgfx::Access::Read); + bgfx::setImage(2, randomTexture, 0, bgfx::Access::Read); + bgfx::setImage(3, aoTexture, 0, bgfx::Access::Write); + + // Dispatch (8×8 thread groups) + bgfx::dispatch(VIEW_SSAO, + ssaoCS, + (width + 7) / 8, + (height + 7) / 8, + 1); + } +}; +``` + +**GLSL Compute Shader:** +```glsl +// cs_ssao.sc +layout(local_size_x = 8, local_size_y = 8) in; + +IMAGE2D_RO(s_depth, r32f, 0); +IMAGE2D_RO(s_normal, rgba8, 1); +SAMPLER2D(s_random, 2); +IMAGE2D_WR(s_aoOutput, r8, 3); + +uniform vec4 u_ssaoParams; // radius, bias, power, intensity + +const int numSamples = 16; +const vec3 sampleKernel[16] = ...; // Precomputed hemisphere samples + +void main() +{ + ivec2 coord = ivec2(gl_GlobalInvocationID.xy); + + float depth = imageLoad(s_depth, coord).r; + vec3 normal = imageLoad(s_normal, coord).rgb * 2.0 - 1.0; + + // Reconstruct position + vec3 worldPos = reconstructPosition(coord, depth); + + // Random rotation + vec2 noiseUV = vec2(coord) / 4.0; // 4×4 tile + vec3 randomVec = texture2D(s_random, noiseUV).rgb * 2.0 - 1.0; + + // TBN matrix + vec3 tangent = normalize(randomVec - normal * dot(randomVec, normal)); + vec3 bitangent = cross(normal, tangent); + mat3 TBN = mat3(tangent, bitangent, normal); + + // Sample AO + float occlusion = 0.0; + for (int i = 0; i < numSamples; ++i) + { + vec3 samplePos = worldPos + TBN * sampleKernel[i] * u_ssaoParams.x; + + // Project and sample depth + vec4 offset = mul(u_viewProj, vec4(samplePos, 1.0)); + offset.xy /= offset.w; + offset.xy = offset.xy * 0.5 + 0.5; + + float sampleDepth = imageLoad(s_depth, ivec2(offset.xy * imageSize(s_depth))).r; + + // Range check + float rangeCheck = smoothstep(0.0, 1.0, u_ssaoParams.x / abs(worldPos.z - sampleDepth)); + + occlusion += (sampleDepth >= depth + u_ssaoParams.y ? 1.0 : 0.0) * rangeCheck; + } + + occlusion = 1.0 - (occlusion / float(numSamples)); + occlusion = pow(occlusion, u_ssaoParams.z); // Power + + imageStore(s_aoOutput, coord, vec4(occlusion)); +} +``` + +### Simple SSR + +```glsl +// fs_ssr.sc (Screen Space Reflections) +$input v_texcoord0 + +#include + +SAMPLER2D(s_sceneColor, 0); +SAMPLER2D(s_depth, 1); +SAMPLER2D(s_normal, 2); +SAMPLER2D(s_roughness, 3); + +uniform vec4 u_ssrParams; // maxDistance, stepSize, numSteps, falloff + +vec3 TraceSSR(vec3 worldPos, vec3 reflectDir) +{ + const int numSteps = int(u_ssrParams.z); + + vec3 rayPos = worldPos; + float stepSize = u_ssrParams.y; + + for (int i = 0; i < numSteps; ++i) + { + rayPos += reflectDir * stepSize; + + // Project to screen + vec4 screenPos = mul(u_viewProj, vec4(rayPos, 1.0)); + screenPos.xyz /= screenPos.w; + vec2 screenUV = screenPos.xy * 0.5 + 0.5; + + // Out of bounds? + if (any(lessThan(screenUV, vec2_splat(0.0))) || + any(greaterThan(screenUV, vec2_splat(1.0)))) + return vec3_splat(0.0); + + // Sample depth + float sceneDepth = texture2D(s_depth, screenUV).r; + float rayDepth = LinearizeDepth(screenPos.z); + + // Hit? + if (rayDepth > sceneDepth) + { + // Fade at edges + vec2 edgeFade = smoothstep(0.0, 0.1, screenUV) * + smoothstep(1.0, 0.9, screenUV); + float fade = edgeFade.x * edgeFade.y; + + // Sample scene color + vec3 reflection = texture2D(s_sceneColor, screenUV).rgb; + + return reflection * fade; + } + } + + return vec3_splat(0.0); +} + +void main() +{ + vec2 uv = v_texcoord0; + + float depth = texture2D(s_depth, uv).r; + vec3 worldPos = reconstructWorldPosition(uv, depth); + + vec3 normal = texture2D(s_normal, uv).rgb * 2.0 - 1.0; + float roughness = texture2D(s_roughness, uv).r; + + // Skip rough surfaces + if (roughness > 0.5) + { + gl_FragColor = vec4(0.0, 0.0, 0.0, 1.0); + return; + } + + // Reflection vector + vec3 viewDir = normalize(worldPos - u_cameraPos); + vec3 reflectDir = reflect(viewDir, normal); + + // Trace + vec3 reflection = TraceSSR(worldPos, reflectDir); + + gl_FragColor = vec4(reflection, 1.0); +} +``` + +--- + +## Key Files Reference + +### C++ Source Files + +**Screen Space Reflections:** +- `Engine/Source/Runtime/Renderer/Private/ScreenSpaceReflectionTiles.cpp` + - Tiled SSR optimization + +**Reflection Probes:** +- `Engine/Source/Runtime/Renderer/Private/ReflectionEnvironmentCapture.cpp` + - Probe capture and filtering +- `Engine/Source/Runtime/Renderer/Private/ReflectionEnvironment.cpp` + - Runtime probe sampling and blending + +**Planar Reflections:** +- `Engine/Source/Runtime/Renderer/Private/PlanarReflectionRendering.cpp` + - Mirror view rendering + +**Ambient Occlusion:** +- `Engine/Source/Runtime/Renderer/Private/CompositionLighting/PostProcessAmbientOcclusion.cpp` + - SSAO and GTAO implementation + +**Lumen GI:** +- `Engine/Source/Runtime/Renderer/Private/Lumen/LumenScreenProbeGather.cpp` (4,100+ lines) + - Screen probe tracing and gathering +- `Engine/Source/Runtime/Renderer/Private/Lumen/LumenScene.cpp` + - Surface cache (card system) +- `Engine/Source/Runtime/Renderer/Private/Lumen/LumenRadianceCache.cpp` + - Radiance cache (clipmap probes) +- `Engine/Source/Runtime/Renderer/Private/Lumen/LumenDiffuseIndirect.cpp` + - Diffuse GI integration + +**Lumen Reflections:** +- `Engine/Source/Runtime/Renderer/Private/Lumen/LumenReflections.cpp` (3,600+ lines) + - Reflection ray tracing +- `Engine/Source/Runtime/Renderer/Private/Lumen/LumenReflectionTracing.cpp` + - Tracing backend + +### Shader Files + +**Reflections:** +- `Engine/Shaders/Private/ScreenSpaceReflectionTileCommons.ush` + - SSR utilities +- `Engine/Shaders/Private/ReflectionEnvironmentShared.ush` + - Probe sampling + +**AO:** +- `Engine/Shaders/Private/PostProcessAmbientOcclusion.usf` + - SSAO and GTAO shaders + +**Lumen:** +- `Engine/Shaders/Private/Lumen/LumenReflections.usf` + - Reflection tracing +- `Engine/Shaders/Private/Lumen/LumenScreenProbeGather.usf` + - Probe gathering +- `Engine/Shaders/Private/Lumen/LumenRadianceCache.usf` + - Radiance cache sampling + +--- + +## Summary + +**Recommended Implementation Order for Your Custom Engine:** + +1. **Month 1:** Reflection probes + basic SSAO +2. **Month 2:** SSR with temporal accumulation +3. **Month 3:** GTAO (better AO) +4. **Month 4-6:** Probe volumes (basic GI) +5. **Month 7+:** Full Lumen (if you need dynamic GI) + +**80/20 Rule:** Probes + SSR + GTAO gives you 80% of the visual quality for 20% of the implementation effort. Only pursue Lumen if your game absolutely needs dynamic GI (destructible environments, dynamic time of day, etc.). + +UE5's reflection and GI systems represent the cutting edge of real-time rendering. Even implementing a subset of these techniques will put your engine in AAA territory. diff --git a/gameengine/docs/UE5_Rendering_Documentation.md b/gameengine/docs/UE5_Rendering_Documentation.md new file mode 100644 index 000000000..6e00b7a9a --- /dev/null +++ b/gameengine/docs/UE5_Rendering_Documentation.md @@ -0,0 +1,652 @@ +# Unreal Engine 5 Rendering System Documentation +## Understanding How Professional Game Engines Prevent "Full Bright or Completely Dark" Scenes + +This documentation explains how Unreal Engine 5's rendering pipeline works, with specific focus on the systems that prevent scenes from appearing too bright or too dark. + +--- + +## Table of Contents +1. [The Core Problem: Why Scenes Appear Too Bright or Dark](#the-core-problem) +2. [Overall Rendering Pipeline](#overall-rendering-pipeline) +3. [Deferred Shading Architecture](#deferred-shading-architecture) +4. [The Critical Solution: Exposure Control](#the-critical-solution-exposure-control) +5. [Tone Mapping](#tone-mapping) +6. [Implementation Checklist for Your Engine](#implementation-checklist) +7. [Key Files Reference](#key-files-reference) + +--- + +## The Core Problem + +When rendering in **HDR (High Dynamic Range)**, scene luminance values can range from 0.001 to 100,000+. However, displays can only show values from 0 to 1 (or 0 to 255 in 8-bit). + +**Without proper exposure and tone mapping:** +- Bright scenes: Values > 1.0 get clamped to white → "Full Bright" +- Dark scenes: Values < 0.1 appear black on screen → "Completely Dark" + +**The solution requires TWO separate systems:** +1. **Exposure** - Scale HDR values to a reasonable range +2. **Tone Mapping** - Map HDR to LDR with a curve that preserves detail + +--- + +## Overall Rendering Pipeline + +UE5 uses a deferred rendering pipeline with distinct stages: + +``` +┌─────────────────────────────────────────────────────────┐ +│ 1. GEOMETRY PASS (Write to GBuffer) │ +│ - Depth PrePass │ +│ - Base Pass: Albedo, Normals, Roughness, Metallic │ +│ Output: GBuffer (multiple render targets in HDR) │ +└─────────────────────────────────────────────────────────┘ + ↓ +┌─────────────────────────────────────────────────────────┐ +│ 2. LIGHTING PASS (Deferred Lighting) │ +│ - Read GBuffer │ +│ - Accumulate all lights │ +│ - Shadows, reflections, ambient occlusion │ +│ Output: Scene Color (HDR linear, can be 0-10000+) │ +└─────────────────────────────────────────────────────────┘ + ↓ +┌─────────────────────────────────────────────────────────┐ +│ 3. POST-PROCESSING (THIS IS WHERE THE MAGIC HAPPENS) │ +│ a. Calculate scene luminance (histogram/average) │ +│ b. Eye Adaptation (auto exposure calculation) │ +│ c. Bloom │ +│ d. Tone Mapping (exposure + tone curve) │ +│ e. Color grading │ +│ f. Gamma correction │ +│ Output: Final LDR image (0-1 range for display) │ +└─────────────────────────────────────────────────────────┘ +``` + +**Key File:** `Engine/Source/Runtime/Renderer/Private/DeferredShadingRenderer.cpp:1699` +- Function: `FDeferredShadingSceneRenderer::Render()` +- This is the main rendering loop that coordinates all stages + +--- + +## Deferred Shading Architecture + +### What is Deferred Shading? + +Traditional forward rendering: +``` +For each object: + Apply all lights to this object + Output final color +``` + +Deferred rendering: +``` +Pass 1 - Geometry Pass: + For each object: + Write material properties to GBuffer (albedo, normal, roughness, etc.) + +Pass 2 - Lighting Pass: + For each pixel: + Read material properties from GBuffer + For each light: + Calculate lighting contribution + Sum all lighting + Output HDR scene color +``` + +### GBuffer Layout + +UE5's GBuffer contains: +- **GBufferA** - World Normal (RGB), Per-object data (A) +- **GBufferB** - Metallic (R), Specular (G), Roughness (B), Shading Model (A) +- **GBufferC** - Base Color (RGB), Ambient Occlusion (A) +- **GBufferD** - Custom data / Subsurface color +- **GBufferE** - Precomputed shadow factors + +**Key Files:** +- `Engine/Source/Runtime/Renderer/Private/BasePassRendering.cpp` - GBuffer generation +- `Engine/Shaders/Private/DeferredShadingCommon.ush` - GBuffer encoding/decoding +- `Engine/Source/Runtime/Renderer/Private/LightRendering.cpp` - Deferred lighting + +### Why This Matters for Your Engine + +The lighting pass outputs **HDR linear light values** that can be much brighter than 1.0. For example: +- A sunlit white surface might have luminance of 10.0 +- A bright light source might contribute 50.0 +- A dark shadow might be 0.01 + +These HDR values MUST be processed by exposure and tone mapping to look correct on screen. + +--- + +## The Critical Solution: Exposure Control + +This is the #1 system you're missing that causes "full bright or completely dark" scenes. + +### What is Exposure? + +In photography and rendering, **exposure** is how much light reaches the sensor/film. It's controlled by: +- **Aperture (f-stop)** - How wide the lens opens +- **Shutter Speed** - How long the sensor is exposed +- **ISO** - Sensor sensitivity + +UE5 uses the **EV100** system (Exposure Value at ISO 100): +``` +EV100 = log2((f-stop² × shutter_speed × 100) / ISO) +``` + +**Key Insight:** Different scenes need different exposure values: +- Bright outdoor scene: EV100 = 15 (low exposure, dim it down) +- Indoor scene: EV100 = 8 (high exposure, brighten it up) +- Night scene: EV100 = 2 (very high exposure) + +### Eye Adaptation (Auto Exposure) + +UE5 implements automatic exposure similar to a camera or human eye adapting to brightness. + +**File:** `Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessEyeAdaptation.cpp` + +**Three Methods:** + +1. **Histogram-based (AEM_Histogram)** - Most accurate + - Analyzes luminance histogram of the scene + - Finds median luminance + - Adjusts exposure to make median map to "middle grey" (18% or EV 0) + +2. **Basic (AEM_Basic)** - Faster + - Calculates average scene luminance + - Adjusts exposure based on average + +3. **Manual (AEM_Manual)** - Artist controlled + - Fixed exposure value set by designer + +### How Eye Adaptation Works + +**Step 1: Calculate Scene Luminance** +```cpp +// Shader: Engine/Shaders/Private/PostProcessHistogram.usf +// For each pixel in downsampled scene: +float Luminance = dot(SceneColor.rgb, float3(0.299, 0.587, 0.114)); +// Build histogram of luminance values +``` + +**Step 2: Determine Target Exposure** +```cpp +// From PostProcessEyeAdaptation.cpp:395 +float BasePhysicalCameraEV100 = + log2((DepthOfFieldFstop² × CameraShutterSpeed × 100) / max(1, CameraISO)); + +// Apply exposure compensation (artist adjustment) +float FinalEV100 = BasePhysicalCameraEV100 + ExposureCompensation; + +// Convert to linear exposure multiplier +float Exposure = EV100ToLuminance(FinalEV100); +``` + +**Step 3: Temporal Smoothing** +```cpp +// Smooth exposure changes over time to avoid flickering +float NewExposure = lerp(OldExposure, TargetExposure, AdaptationSpeed × DeltaTime); +``` + +**Critical Function:** `FViewInfo::UpdatePreExposure()` at line 1383 + +### Pre-Exposure: A Critical Optimization + +UE5 applies exposure **during rendering** (not just in post-processing) to maintain precision in HDR buffers. + +```cpp +// During lighting pass: +SceneColor = BaseColor × LightColor × LightIntensity × PreExposure; + +// PreExposure is calculated as: +PreExposure = CalculatedExposure × GlobalExposure × VignetteMask; +``` + +**Why Pre-Exposure Matters:** +- Prevents HDR values from getting too large (overflow) +- Maintains precision in 16-bit float buffers +- Reduces banding in bright and dark areas + +**Key File:** `Engine/Shaders/Private/EyeAdaptationCommon.ush` + +### EV100 Conversion Functions + +```cpp +// Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessEyeAdaptation.cpp + +// Convert EV100 to linear luminance multiplier +float EV100ToLuminance(float EV100) +{ + return exp2(EV100 - 3.0f); +} + +// Convert luminance to EV100 +float LuminanceToEV100(float Luminance) +{ + return log2(Luminance) + 3.0f; +} + +// The -3.0 offset accounts for the ISO 100 baseline and lens attenuation +``` + +### Console Variables for Exposure + +```cpp +// Override exposure calculation +r.EyeAdaptation.PreExposureOverride = -1.0 // -1 = auto, > 0 = manual override +r.EyeAdaptation.MethodOverride = -1 // -1 = auto, 0 = manual, 1 = basic, 2 = histogram +r.EyeAdaptation.LensAttenuation = 0.78 // Lens q factor (calibration) +r.EyeAdaptation.ExposureCompensation = 0.0 // Artist bias in EV stops +r.EyeAdaptation.MinBrightness = 0.0 // Minimum scene luminance +r.EyeAdaptation.MaxBrightness = 2.0 // Maximum scene luminance +``` + +--- + +## Tone Mapping + +After exposure scales the HDR values, **tone mapping** converts HDR to LDR for display. + +**File:** `Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessTonemap.cpp` + +### Why Tone Mapping is Necessary + +Even with perfect exposure, HDR still has values outside [0, 1]: +- Bright highlights might be 2.0 +- Very bright lights might be 5.0 +- Specular reflections might be 10.0 + +**Linear clamp** (your current approach?) would make these all white, losing detail. + +**Tone mapping** uses a **curve** that: +- Preserves midtones (values near 0.18) +- Compresses highlights smoothly (values > 1.0) +- Darkens shadows gently (values < 0.18) + +### Tone Mapping Curve Types in UE5 + +**1. Filmic Tone Curve (Default)** + +A custom curve with artistic control: +```cpp +// Parameters: +- Toe: Controls shadow compression +- Shoulder: Controls highlight compression +- Slope: Controls midtone contrast +- WhitePoint: Brightest value that maps to white +``` + +The curve looks like an "S" shape that smoothly compresses both bright and dark values. + +**2. ACES (Academy Color Encoding System)** + +Industry-standard tone mapper used in film: +- ACES 1.3 (previous standard) +- ACES 2.0 (current standard) + +More accurate color preservation, especially for bright lights and HDR workflows. + +**Shader:** `Engine/Shaders/Private/TonemapCommon.ush:415` + +### Filmic Tone Curve Math + +```hlsl +// Simplified version from TonemapCommon.ush +float3 FilmicTonemap(float3 LinearColor) +{ + // Apply exposure first + float3 ExposedColor = LinearColor * Exposure; + + // Filmic curve (simplified) + float3 x = max(0, ExposedColor - 0.004); + float3 ToneMappedColor = (x * (6.2 * x + 0.5)) / (x * (6.2 * x + 1.7) + 0.06); + + return ToneMappedColor; +} +``` + +**Full implementation:** `Engine/Shaders/Private/TonemapCommon.ush:97` (FilmToneMap function) + +### ACES Tone Mapping + +```hlsl +// From TonemapCommon.ush +float3 ACESTonemap(float3 LinearColor) +{ + // Convert to ACES color space + float3 ACESColor = LinearToACES(LinearColor); + + // Apply RRT (Reference Rendering Transform) + ODT (Output Device Transform) + float3 ODTColor = ACESOutputTransforms(ACESColor); + + return ODTColor; +} +``` + +**Full ACES implementation:** `Engine/Shaders/Private/ACES.ush` + +### Gamma Correction + +After tone mapping, the final step is **gamma correction** for display: + +```hlsl +// sRGB gamma curve +float3 LinearToSRGB(float3 LinearColor) +{ + // Simplified: FinalColor = pow(LinearColor, 1/2.2) + // Actual sRGB is piecewise with linear segment near black +} +``` + +Most displays expect sRGB gamma, so this is applied at the very end. + +### Tone Mapping Console Variables + +```cpp +r.Tonemapper.Sharpen = 0.5 // Sharpening amount +r.TonemapperGamma = 2.2 // Gamma curve +r.Tonemapper.GrainQuantization = 1 // Film grain +r.Tonemapper.Quality = 5 // Quality level +``` + +--- + +## Implementation Checklist for Your Engine + +Based on UE5's architecture, here's what you need to fix the "full bright or completely dark" problem: + +### Phase 1: Basic Exposure (Minimum Viable) + +- [ ] **Calculate scene luminance** + - Downsample scene to 64×64 or smaller + - Calculate average luminance: `Lum = 0.299×R + 0.587×G + 0.114×B` + +- [ ] **Calculate exposure value** + ```cpp + float targetLuminance = 0.18; // Middle grey + float avgLuminance = CalculateSceneLuminance(); + float exposure = targetLuminance / (avgLuminance + 0.001); + exposure = clamp(exposure, 0.1, 10.0); // Prevent extremes + ``` + +- [ ] **Apply basic tone mapping** + ```cpp + // Reinhard tone mapping (simplest) + float3 ToneMapped = ExposedColor / (ExposedColor + 1.0); + + // Or Uncharted 2 (better highlights) + float3 ToneMapped = FilmicCurve(ExposedColor); + ``` + +- [ ] **Add gamma correction** + ```cpp + float3 Final = pow(ToneMapped, 1.0/2.2); // sRGB gamma + ``` + +### Phase 2: Smooth Adaptation + +- [ ] **Temporal smoothing of exposure** + ```cpp + float adaptationSpeed = 2.0; // EV per second + newExposure = lerp(oldExposure, targetExposure, + adaptationSpeed × deltaTime); + ``` + +- [ ] **Store exposure in texture/buffer** + - Use 1×1 texture to store previous frame's exposure + - Read in next frame for smooth interpolation + +### Phase 3: Advanced (Match UE5) + +- [ ] **Histogram-based exposure** + - Build 64-bin luminance histogram + - Find median or 80th percentile luminance + - More robust than average + +- [ ] **EV100 system** + ```cpp + float CalculateEV100(float aperture, float shutterSpeed, float ISO) + { + return log2((aperture * aperture * shutterSpeed * 100.0) / ISO); + } + + float EV100ToExposure(float EV100) + { + return exp2(EV100 - 3.0); // -3 for calibration + } + ``` + +- [ ] **Pre-exposure application** + - Apply exposure during lighting pass + - Modify light intensity by pre-exposure value + +- [ ] **ACES tone mapping** + - Use ACES color space transforms + - Industry-standard color accuracy + +### Phase 4: Artist Control + +- [ ] **Exposure compensation** + - Allow artists to bias exposure up/down + - Additive offset to EV100 + +- [ ] **Min/max luminance clamps** + - Prevent adaptation from going too bright/dark + - Useful for specific game scenarios + +- [ ] **Zone-based exposure** + - Weight center of screen more than edges + - Prevents sky from overexposing character + +--- + +## Key Files Reference + +### C++ Source Files + +**Main Rendering Loop:** +- `Engine/Source/Runtime/Renderer/Private/DeferredShadingRenderer.cpp:1699` + - `FDeferredShadingSceneRenderer::Render()` - Main render loop + +**Post-Processing Coordination:** +- `Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessing.cpp` + - `AddPostProcessingPasses()` - Builds post-process chain + - `AddHistogramEyeAdaptationPass()` - Histogram exposure calculation + - `AddBasicEyeAdaptationPass()` - Simple exposure calculation + +**Eye Adaptation (Auto Exposure):** +- `Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessEyeAdaptation.cpp` + - Line 395: Physical camera EV100 calculation + - Line 1383: `FViewInfo::UpdatePreExposure()` - Pre-exposure calculation + - EV100 conversion functions + +**Tone Mapping:** +- `Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessTonemap.cpp` + - Tone mapping pass setup + - Color grading integration + - LUT application + +**Lighting:** +- `Engine/Source/Runtime/Renderer/Private/LightRendering.cpp` + - Deferred light rendering + - Shadow application + - Light accumulation in HDR + +**GBuffer Generation:** +- `Engine/Source/Runtime/Renderer/Private/BasePassRendering.cpp` + - Base pass rendering + - Material property encoding + +### Shader Files (HLSL/USH) + +**Exposure/Eye Adaptation:** +- `Engine/Shaders/Private/EyeAdaptationCommon.ush` + - Exposure buffer access functions + - `EyeAdaptationLookup()` - Read current exposure value + +**Tone Mapping:** +- `Engine/Shaders/Private/TonemapCommon.ush` + - Line 97: `FilmToneMap()` - Filmic tone curve + - Line 415: ACES tone mapping + - Color grading functions + - Gamma correction + +**ACES:** +- `Engine/Shaders/Private/ACES.ush` + - Complete ACES color space implementation + - RRT and ODT transforms + +**Histogram Generation:** +- `Engine/Shaders/Private/PostProcessHistogram.usf` + - Luminance calculation + - Histogram bin accumulation + +**Deferred Shading:** +- `Engine/Shaders/Private/DeferredShadingCommon.ush` + - GBuffer encoding/decoding + - Material property access + +**Lighting Calculations:** +- `Engine/Shaders/Private/DeferredLightingCommon.ush` + - BRDF functions + - Light attenuation + - Shadow sampling + +### Key Data Structures + +**View Information:** +- `Engine/Source/Runtime/Engine/Public/SceneView.h` + - `FSceneView` - Camera and view settings + - Contains exposure values, FOV, projection matrices + +**Scene Representation:** +- `Engine/Source/Runtime/Renderer/Private/ScenePrivate.h` + - `FScene` - Scene graph and objects + - Light arrays, primitive arrays + +--- + +## Quick Reference: The Exposure Pipeline + +``` +Frame N: +┌─────────────────────────────────────────────────────┐ +│ 1. Render scene with PreExposure from Frame N-1 │ +│ SceneColor = Lighting × PreExposure │ +└─────────────────────────────────────────────────────┘ + ↓ +┌─────────────────────────────────────────────────────┐ +│ 2. Calculate luminance of rendered scene │ +│ - Downsample to 64×64 │ +│ - Build histogram (or calculate average) │ +└─────────────────────────────────────────────────────┘ + ↓ +┌─────────────────────────────────────────────────────┐ +│ 3. Calculate target exposure for next frame │ +│ - Find median/average luminance │ +│ - Convert to EV100 │ +│ - Apply compensation and limits │ +│ - Smooth with previous exposure │ +└─────────────────────────────────────────────────────┘ + ↓ +┌─────────────────────────────────────────────────────┐ +│ 4. Apply tone mapping to current frame │ +│ - Apply additional exposure if needed │ +│ - Run tone mapping curve (Filmic/ACES) │ +│ - Color grading │ +│ - Gamma correction │ +└─────────────────────────────────────────────────────┘ + ↓ + Display Frame N +``` + +--- + +## Console Commands for Testing + +When studying UE5, try these console commands to see how exposure affects the scene: + +```cpp +// Disable auto exposure to see raw HDR +r.EyeAdaptation.MethodOverride 0 +r.EyeAdaptation.PreExposureOverride 1.0 + +// Now manually adjust exposure +r.EyeAdaptation.PreExposureOverride 0.5 // Darker +r.EyeAdaptation.PreExposureOverride 2.0 // Brighter +r.EyeAdaptation.PreExposureOverride 10.0 // Very bright + +// Re-enable auto exposure +r.EyeAdaptation.MethodOverride -1 + +// Test different tone mappers +r.Tonemapper.Quality 0 // Legacy +r.Tonemapper.Quality 5 // Full quality with ACES + +// Show debug histogram +r.EyeAdaptation.VisualizeDebugType 1 +``` + +--- + +## Summary: Why Your Engine Shows "Full Bright or Completely Dark" + +Your game engine likely has one or more of these issues: + +1. **Missing Auto Exposure** + - Exposure value is fixed (probably 1.0) + - Bright scenes overflow to white + - Dark scenes underflow to black + +2. **Missing Tone Mapping** + - HDR values > 1.0 get clamped to white + - No smooth rolloff for highlights + - All bright lights look the same + +3. **Missing Gamma Correction** + - Linear light values appear darker on screen + - Should apply gamma 2.2 for sRGB displays + +4. **Incorrect Light Units** + - Light intensities might be in wrong units + - Too bright or too dark by default + +**The minimal fix:** +```cpp +// Pseudocode for post-processing +float avgLuminance = CalculateAverageLuminance(sceneColor); +float exposure = 0.18 / (avgLuminance + 0.001); +exposure = clamp(exposure, 0.1, 10.0); + +float3 exposedColor = sceneColor.rgb * exposure; +float3 toneMapped = exposedColor / (exposedColor + 1.0); // Reinhard +float3 gammaCorrected = pow(toneMapped, 1.0/2.2); + +outputColor = gammaCorrected; +``` + +This simple change will prevent "full bright or completely dark" and give you a starting point to build a more sophisticated system like UE5's. + +--- + +## Further Study + +**Recommended reading order:** + +1. Start with: `PostProcessEyeAdaptation.cpp` - Understand exposure calculation +2. Then read: `TonemapCommon.ush` - See actual tone mapping shaders +3. Study: `PostProcessing.cpp` - See how passes connect +4. Finally: `DeferredShadingRenderer.cpp:1699` - Full render loop + +**Key concepts to understand:** + +- HDR vs LDR rendering +- EV100 exposure system +- Histogram-based exposure +- Temporal smoothing +- Pre-exposure optimization +- Tone mapping curves (Filmic, ACES) +- Gamma correction and color spaces + +Good luck with your game engine! The difference between "full bright or completely dark" and "properly exposed" is entirely in these post-processing steps. diff --git a/gameengine/docs/UE5_Shadow_Rendering_Documentation.md b/gameengine/docs/UE5_Shadow_Rendering_Documentation.md new file mode 100644 index 000000000..bd35c75f8 --- /dev/null +++ b/gameengine/docs/UE5_Shadow_Rendering_Documentation.md @@ -0,0 +1,1317 @@ +# Unreal Engine 5 Shadow Rendering Documentation +## For bgfx Implementation + +This documentation explains how UE5's shadow rendering system works, from basic shadow mapping to advanced Virtual Shadow Maps. + +--- + +## Table of Contents +1. [Shadow System Overview](#shadow-system-overview) +2. [Traditional Shadow Maps](#traditional-shadow-maps) +3. [Virtual Shadow Maps (VSM)](#virtual-shadow-maps-vsm) +4. [Shadow Map Filtering](#shadow-map-filtering) +5. [Contact Shadows](#contact-shadows) +6. [Ray-Traced Shadows](#ray-traced-shadows) +7. [Shadow Bias and Artifacts](#shadow-bias-and-artifacts) +8. [Performance Optimization](#performance-optimization) +9. [bgfx Implementation Guide](#bgfx-implementation-guide) +10. [Key Files Reference](#key-files-reference) + +--- + +## Shadow System Overview + +UE5 uses multiple shadowing techniques simultaneously: + +``` +Shadow System Hierarchy: +┌────────────────────────────────────────────────┐ +│ Primary: Virtual Shadow Maps (UE5 flagship) │ +│ - Page-based virtual texturing │ +│ - 16k virtual resolution per light │ +│ - Dynamic caching for static geometry │ +└────────────────────────────────────────────────┘ + ↓ (fallback) +┌────────────────────────────────────────────────┐ +│ Traditional: Cascaded Shadow Maps (CSM) │ +│ - Multiple cascades for directional lights │ +│ - Cube maps for point lights │ +│ - Single frustum for spot lights │ +└────────────────────────────────────────────────┘ + ↓ (detail layer) +┌────────────────────────────────────────────────┐ +│ Contact Shadows (Screen Space) │ +│ - Fine detail near surfaces │ +│ - Ray marching in screen space │ +└────────────────────────────────────────────────┘ + ↓ (optional, high-end) +┌────────────────────────────────────────────────┐ +│ Ray-Traced Shadows (Hardware RT) │ +│ - Accurate soft shadows │ +│ - Translucent shadows │ +└────────────────────────────────────────────────┘ +``` + +### Light Type Support + +| Light Type | Traditional | VSM | Contact | Ray-Traced | +|------------|-------------|-----|---------|------------| +| Directional | CSM (2-4 cascades) | Clipmap | ✓ | ✓ | +| Spot | Single shadow map | Single page | ✓ | ✓ | +| Point | Cube map | 6 pages | ✓ | ✓ | +| Rect | Single shadow map | Single page | ✓ | ✓ | + +**Key Files:** +- `Engine/Source/Runtime/Renderer/Private/ShadowRendering.cpp` (2,613 lines) +- `Engine/Source/Runtime/Renderer/Private/ShadowDepthRendering.cpp` (2,400 lines) + +--- + +## Traditional Shadow Maps + +Traditional shadow mapping is the baseline technique supported by all hardware. + +### Cascaded Shadow Maps (CSM) for Directional Lights + +**Key File:** `Engine/Source/Runtime/Renderer/Private/ShadowSetup.cpp` + +#### Cascade Setup + +```cpp +// UE5 CSM configuration +int32 NumCascades = 4; // Typically 2-4 cascades +float CascadeDistribution = 0.8; // Exponential distribution + +// Cascade distances are calculated based on view frustum +float CascadeSplits[NumCascades + 1]; +CascadeSplits[0] = NearPlane; +CascadeSplits[NumCascades] = FarPlane; + +// Calculate intermediate splits (exponential + linear blend) +for (int i = 1; i < NumCascades; ++i) +{ + float fraction = (float)i / NumCascades; + + // Logarithmic split + float logSplit = NearPlane * pow(FarPlane / NearPlane, fraction); + + // Linear split + float linearSplit = NearPlane + (FarPlane - NearPlane) * fraction; + + // Blend between log and linear + CascadeSplits[i] = lerp(linearSplit, logSplit, CascadeDistribution); +} +``` + +**Console Variables:** +```cpp +r.Shadow.CSM.MaxCascades = 4 // Number of cascades (1-10) +r.Shadow.DistanceScale = 1.0 // Scale all shadow distances +r.Shadow.CSMSplitPenumbraScale = 0.5 // Softness between cascades +r.Shadow.MaxResolution = 2048 // Per-cascade resolution +``` + +#### Shadow Map Rendering + +```cpp +// For each cascade: +struct FShadowCascade +{ + FMatrix ViewMatrix; // Look from light direction + FMatrix ProjectionMatrix; // Orthographic projection + FBox CasterBounds; // AABB of shadow casters + float SplitNear, SplitFar; // Depth range + FVector2D Resolution; // Shadow map size +}; + +// Rendering process: +void RenderCascadeShadowMap(FShadowCascade& Cascade) +{ + // 1. Setup view from light + SetViewMatrix(Cascade.ViewMatrix); + SetProjectionMatrix(Cascade.ProjectionMatrix); + + // 2. Cull shadow casters + TArray Casters; + CullShadowCasters(Cascade.CasterBounds, Casters); + + // 3. Render depth only + for (auto Caster : Casters) + { + RenderDepthOnly(Caster); + } +} +``` + +### Point Light Shadows (Cube Maps) + +**Traditional Approach:** +```cpp +// Render 6 faces of cube map +enum ECubeFace +{ + Face_PosX, Face_NegX, + Face_PosY, Face_NegY, + Face_PosZ, Face_NegZ +}; + +// For each face: +FMatrix ViewMatrices[6]; +ViewMatrices[Face_PosX] = LookAt(LightPos, LightPos + FVector(1,0,0), FVector(0,1,0)); +ViewMatrices[Face_NegX] = LookAt(LightPos, LightPos + FVector(-1,0,0), FVector(0,1,0)); +// ... etc + +// Perspective projection (90° FOV) +FMatrix ProjectionMatrix = PerspectiveFov(PI/2, 1.0, NearPlane, FarPlane); + +// Render all 6 faces to cube map +for (int face = 0; face < 6; ++face) +{ + SetRenderTarget(CubeMap, face); + SetViewMatrix(ViewMatrices[face]); + RenderDepthOnly(ShadowCasters); +} +``` + +**One-Pass Point Light Shadows (Optimized):** +```cpp +// Use geometry shader to render all 6 faces in one pass +// Geometry shader outputs to different cube faces + +// Vertex shader +struct VS_Output +{ + float4 Position : SV_Position; + float3 WorldPos : WORLDPOS; +}; + +// Geometry shader +[maxvertexcount(18)] // 6 faces × 3 vertices +void GS_PointShadow(triangle VS_Output input[3], + inout TriangleStream outStream) +{ + for (int face = 0; face < 6; ++face) + { + GS_Output output; + output.RTIndex = face; // Which cube face + + for (int v = 0; v < 3; ++v) + { + output.Position = mul(FaceViewProj[face], input[v].WorldPos); + output.Depth = length(input[v].WorldPos - LightPosition); + outStream.Append(output); + } + outStream.RestartStrip(); + } +} + +// Pixel shader +float PS_PointShadow(GS_Output input) : SV_Depth +{ + // Write linear depth + return input.Depth / FarPlane; +} +``` + +### Spot Light Shadows + +Simplest case - single shadow map with perspective projection: + +```cpp +// Setup view/projection +FVector LightPos = SpotLight.Position; +FVector LightDir = SpotLight.Direction; +FVector Up = abs(LightDir.Z) < 0.999 ? FVector(0,0,1) : FVector(1,0,0); + +FMatrix ViewMatrix = LookAt(LightPos, LightPos + LightDir, Up); + +float FOV = SpotLight.OuterConeAngle * 2.0; +FMatrix ProjectionMatrix = PerspectiveFov(FOV, 1.0, NearPlane, FarPlane); + +// Render single shadow map +SetRenderTarget(ShadowMap); +RenderDepthOnly(ShadowCasters); +``` + +### Shadow Projection + +**In lighting pixel shader:** +```hlsl +// Sample shadow map +float SampleShadowMap(float3 worldPos, Texture2D shadowMap, SamplerState shadowSampler, + float4x4 shadowViewProj) +{ + // Transform to light clip space + float4 shadowPos = mul(shadowViewProj, float4(worldPos, 1.0)); + + // Perspective divide + shadowPos.xyz /= shadowPos.w; + + // Transform to [0, 1] texture space + float2 shadowUV = shadowPos.xy * 0.5 + 0.5; + shadowUV.y = 1.0 - shadowUV.y; // Flip Y + + // Sample depth + float shadowDepth = shadowMap.Sample(shadowSampler, shadowUV).r; + + // Compare + float currentDepth = shadowPos.z; + float shadow = currentDepth > shadowDepth + Bias ? 0.0 : 1.0; + + return shadow; +} +``` + +**For CSM (multiple cascades):** +```hlsl +float SampleCSM(float3 worldPos, float viewDepth) +{ + // Determine which cascade + int cascadeIndex = 0; + for (int i = 0; i < NumCascades; ++i) + { + if (viewDepth < CascadeSplits[i + 1]) + { + cascadeIndex = i; + break; + } + } + + // Sample that cascade's shadow map + return SampleShadowMap(worldPos, + CascadeShadowMaps[cascadeIndex], + ShadowSampler, + CascadeViewProj[cascadeIndex]); +} +``` + +**Key File:** `Engine/Shaders/Private/ShadowProjectionCommon.ush` + +--- + +## Virtual Shadow Maps (VSM) + +UE5's flagship shadow technology - dramatically improves quality and performance for complex scenes. + +**Key Files:** +- `Engine/Source/Runtime/Renderer/Private/VirtualShadowMaps/VirtualShadowMapArray.cpp` (5,342 lines) +- `Engine/Shaders/Shared/VirtualShadowMapDefinitions.h` + +### Core Concept + +Instead of fixed-resolution shadow maps, VSM uses: +- **Virtual Address Space**: 16k × 16k per light (128×128 pages of 128×128 pixels each) +- **Physical Page Pool**: Dynamically allocated based on visibility +- **Caching**: Static geometry cached between frames +- **Mip Levels**: 8 levels for LOD + +``` +Virtual Shadow Map (16,384 × 16,384) +┌─────────────────────────────────────┐ +│ ┌─┬─┬─┬─┐ │ +│ ├─┼─┼─┼─┤ Each square = 128×128 px │ +│ ├─┼─┼─┼─┤ (a "page") │ +│ └─┴─┴─┴─┘ │ +│ │ +│ Only visible pages are allocated │ +│ and rendered │ +└─────────────────────────────────────┘ + ↓ +Physical Page Pool (e.g., 2048 pages) +┌────┬────┬────┬────┬────┐ +│Page│Page│Page│Page│... │ Only allocated pages +├────┼────┼────┼────┼────┤ stored in memory +│ 0 │ 1 │ 2 │ 3 │ │ +└────┴────┴────┴────┴────┘ +``` + +### VSM Constants + +```cpp +// From VirtualShadowMapDefinitions.h + +#define VSM_PAGE_SIZE 128 // Each page is 128x128 pixels +#define VSM_LEVEL0_DIM_PAGES_XY 128 // 128x128 pages at level 0 +#define VSM_VIRTUAL_MAX_RESOLUTION_XY (VSM_PAGE_SIZE * VSM_LEVEL0_DIM_PAGES_XY) // 16,384 +#define VSM_MAX_MIP_LEVELS 8 // 8 mip levels + +// Physical pool +r.Shadow.Virtual.MaxPhysicalPages = 2048 // Max pages in memory +``` + +### Page Table + +```cpp +// Virtual → Physical mapping +struct FVirtualShadowMapPageTable +{ + // For each virtual page (128×128 array): + // 0xFFFFFFFF = not allocated + // otherwise = physical page index + TArray PageTable; // 128*128 = 16,384 entries per mip + + // Hierarchical mip levels (for faster lookups) + TArray MipTables[VSM_MAX_MIP_LEVELS]; +}; +``` + +### VSM Rendering Pipeline + +```cpp +// 1. Mark pages based on screen pixels +void MarkRequiredPages(FRDGBuilder& GraphBuilder) +{ + // For each pixel on screen: + // Project to virtual shadow map + // Mark the page as needed + // Mark coarser mip pages for smooth transitions + + // Uses compute shader + AddPass_MarkPixelPages<<<...>>>(); +} + +// 2. Allocate physical pages +void AllocatePages(FRDGBuilder& GraphBuilder) +{ + // For each marked page: + // Check cache (if static geometry) + // If not cached, allocate new physical page + // Update page table + + // LRU eviction if pool is full +} + +// 3. Render shadow depth into allocated pages +void RenderVirtualShadowDepth(FRDGBuilder& GraphBuilder) +{ + // For each allocated page: + // Setup viewport (128×128 sub-region) + // Render only geometry visible to that page + // Write to physical page location in atlas + + // Nanite integration: GPU-driven culling per page + RenderNaniteShadowDepth(AllocatedPages); + RenderNonNaniteShadowDepth(AllocatedPages); +} + +// 4. Build HZB (Hierarchical Z-Buffer) for occlusion culling +void BuildHZB(FRDGBuilder& GraphBuilder) +{ + // Create mip chain of depth buffer + // Used for next frame's culling +} + +// 5. Project and sample during lighting +float SampleVirtualShadowMap(float3 worldPos) +{ + // Project to virtual shadow space + float2 virtualUV = ProjectToVirtualShadow(worldPos); + + // Determine mip level (based on screen coverage) + float mipLevel = CalculateMipLevel(virtualUV); + + // Look up page table + uint pageIndex = PageTable.Sample(virtualUV, mipLevel); + + if (pageIndex == 0xFFFFFFFF) + return 1.0; // Not shadowed (page not allocated) + + // Convert to physical UV + float2 physicalUV = VirtualToPhysicalUV(virtualUV, pageIndex); + + // Sample physical shadow map + return ShadowAtlas.Sample(physicalUV); +} +``` + +### VSM Caching + +```cpp +// Static geometry caching +struct FVirtualShadowMapCacheEntry +{ + uint32 PhysicalPageIndex; + uint32 LastRequestFrame; + bool bStatic; +}; + +// Cache invalidation +void InvalidateMovedPrimitives(TArray MovedPrimitives) +{ + for (auto Primitive : MovedPrimitives) + { + // Mark all pages overlapping this primitive as invalid + TArray AffectedPages = FindAffectedPages(Primitive->Bounds); + + for (uint32 pageIndex : AffectedPages) + { + Cache[pageIndex].bStatic = false; + Cache[pageIndex].LastRequestFrame = 0; + } + } +} + +// Benefits: +// - Static geometry rendered once, cached indefinitely +// - Only dynamic geometry re-rendered each frame +// - 10-100x performance improvement for static scenes +``` + +### VSM for Different Light Types + +**Directional Light (Sun):** +```cpp +// Uses clipmap structure (centered on camera) +struct FVirtualShadowMapClipmap +{ + int32 NumLevels = 8; // 8 clipmap levels + + // Each level is a 128×128 page grid centered on camera + // Level 0: Highest detail, small area + // Level 7: Lowest detail, large area + + float LevelRadii[NumLevels]; // Computed based on view distance +}; +``` + +**Spot/Point Lights:** +```cpp +// Single virtual shadow map (or 6 for point lights) +// Uses full 128×128 page grid +// Pages allocated based on screen coverage +``` + +**Console Variables:** +```cpp +r.Shadow.Virtual = 1 // Enable VSM +r.Shadow.Virtual.Cache = 1 // Enable caching +r.Shadow.Virtual.MaxPhysicalPages = 2048 // Memory budget +r.Shadow.Virtual.ResolutionLodBiasDirectional = 0.0 // Quality bias +r.Shadow.Virtual.ResolutionLodBiasLocal = 0.0 // For local lights +``` + +--- + +## Shadow Map Filtering + +Shadow map filtering reduces aliasing and creates soft shadows. + +**Key File:** `Engine/Shaders/Private/ShadowFilteringCommon.ush` + +### PCF (Percentage Closer Filtering) + +Basic technique - sample shadow map multiple times and average results. + +```hlsl +// 1×1 PCF (4 samples with bilinear) +float PCF_1x1(Texture2D shadowMap, SamplerState shadowSampler, + float2 uv, float compareDepth) +{ + // Hardware PCF (built into sampler) + return shadowMap.SampleCmpLevelZero(shadowSampler, uv, compareDepth); +} + +// 3×3 PCF (16 samples using Gather4) +float PCF_3x3(Texture2D shadowMap, SamplerState shadowSampler, + float2 uv, float compareDepth, float2 texelSize) +{ + float shadow = 0.0; + + // Use Gather4 for 2×2 texel samples + for (int y = -1; y <= 1; ++y) + { + for (int x = -1; x <= 1; ++x) + { + float2 offset = float2(x, y) * texelSize; + float4 depths = shadowMap.Gather(shadowSampler, uv + offset); + + // Compare each of the 4 samples + shadow += (depths.x > compareDepth) ? 0.25 : 0.0; + shadow += (depths.y > compareDepth) ? 0.25 : 0.0; + shadow += (depths.z > compareDepth) ? 0.25 : 0.0; + shadow += (depths.w > compareDepth) ? 0.25 : 0.0; + } + } + + return shadow / 9.0; +} + +// 5×5 PCF (36 samples) +float PCF_5x5(Texture2D shadowMap, SamplerState shadowSampler, + float2 uv, float compareDepth, float2 texelSize) +{ + // Similar but with -2 to +2 range + // Uses Gather4 for efficiency +} +``` + +**Quality Levels:** +```cpp +r.ShadowQuality = 5 // 0-5 (0=off, 5=max) + +// Maps to: +// 0: No filtering (hard shadows) +// 1: 1×1 PCF (bilinear) +// 2: 3×3 PCF (16 samples) +// 3: 5×5 PCF (36 samples) +// 4+: 5×5 PCF + optimizations +``` + +### PCSS (Percentage Closer Soft Shadows) + +Physically-based soft shadows that vary based on occluder distance. + +**Key File:** `Engine/Shaders/Private/ShadowPercentageCloserFiltering.ush` + +```hlsl +// Two-phase algorithm: + +// Phase 1: Blocker search +float FindBlockerDistance(Texture2D shadowMap, float2 uv, + float receiverDepth, float searchRadius) +{ + const int numSamples = 16; // Poisson disk samples + + float blockerSum = 0.0; + float blockerCount = 0.0; + + for (int i = 0; i < numSamples; ++i) + { + float2 offset = PoissonDisk[i] * searchRadius; + float shadowDepth = shadowMap.Sample(shadowSampler, uv + offset).r; + + if (shadowDepth < receiverDepth) + { + blockerSum += shadowDepth; + blockerCount += 1.0; + } + } + + if (blockerCount == 0.0) + return -1.0; // No blockers, fully lit + + return blockerSum / blockerCount; // Average blocker depth +} + +// Phase 2: Penumbra estimation and PCF +float PCSS(Texture2D shadowMap, float2 uv, float receiverDepth) +{ + // Find average blocker distance + float blockerDistance = FindBlockerDistance(shadowMap, uv, receiverDepth, searchWidth); + + if (blockerDistance < 0.0) + return 1.0; // Fully lit + + // Estimate penumbra size based on geometry + // penumbra = (receiver - blocker) / blocker * lightSize + float penumbra = (receiverDepth - blockerDistance) / blockerDistance * lightSize; + + // PCF with variable kernel size + float shadow = PCF_Variable(shadowMap, uv, receiverDepth, penumbra); + + return shadow; +} +``` + +**Parameters:** +```cpp +r.Shadow.FilterMethod = 1 // 0=PCF, 1=PCSS +r.Shadow.MaxSoftKernelSize = 40 // Max kernel radius (pixels) +``` + +### SMRT (Shadow Map Ray Tracing) - VSM Only + +Novel filtering technique for Virtual Shadow Maps. + +**Key File:** `Engine/Shaders/Private/VirtualShadowMaps/VirtualShadowMapSMRTCommon.ush` + +```hlsl +// Traces rays through virtual shadow map hierarchy +struct FSMRTSettings +{ + int RayCount; // Rays per pixel (4-16) + int SamplesPerRay; // Samples along each ray (8-16) + float TexelDitherScale; // Noise reduction + float ExtrapolateMaxSlope; // Early termination +}; + +float SMRT_TraceShadow(float3 worldPos, float3 normal, float3 lightDir) +{ + float shadow = 0.0; + + // Cast multiple rays for soft shadows + for (int ray = 0; ray < RayCount; ++ray) + { + // Jitter ray direction (for soft shadows) + float3 rayDir = JitterRayDirection(lightDir, ray); + + // March ray through VSM + float rayOcclusion = 0.0; + float3 rayPos = worldPos + normal * SurfaceBias; + + for (int step = 0; step < SamplesPerRay; ++step) + { + // Sample VSM at this position + float vsmDepth = SampleVirtualShadowMap(rayPos); + + if (vsmDepth < GetDepth(rayPos)) + { + rayOcclusion = 1.0; + break; // Hit occluder + } + + rayPos += rayDir * StepSize; + } + + shadow += rayOcclusion; + } + + return 1.0 - (shadow / RayCount); +} +``` + +**Benefits:** +- Adaptive sample count (extrapolation in fully lit/shadowed regions) +- Better soft shadows than PCF +- Leverages VSM's mip hierarchy + +**Console Variables:** +```cpp +r.Shadow.Virtual.SMRT.RayCount = 8 // Rays per pixel +r.Shadow.Virtual.SMRT.SamplesPerRay = 12 // Samples per ray +r.Shadow.Virtual.SMRT.TexelDitherScale = 2.0 // Dithering +``` + +--- + +## Contact Shadows + +Screen-space shadows for fine detail near surfaces. + +**Key File:** `Engine/Source/Runtime/Renderer/Private/Shadows/ScreenSpaceShadows.cpp` + +### Why Contact Shadows? + +Shadow maps miss fine detail (e.g., character feet on ground). Contact shadows add this detail cheaply. + +### Ray Marching Approach + +```hlsl +// Compute shader (one thread per pixel) +float ContactShadow(float3 worldPos, float3 lightDir, float maxDistance) +{ + const int numSteps = 8; + const float stepScale = 2.0; // Exponential steps + + float3 rayPos = worldPos; + float stepSize = maxDistance / numSteps; + + for (int i = 0; i < numSteps; ++i) + { + rayPos += lightDir * stepSize; + + // Project to screen space + float4 screenPos = mul(ViewProjection, float4(rayPos, 1.0)); + screenPos.xyz /= screenPos.w; + float2 screenUV = screenPos.xy * 0.5 + 0.5; + + // Sample depth buffer + float sceneDepth = DepthBuffer.SampleLevel(PointSampler, screenUV, 0).r; + float rayDepth = screenPos.z; + + // Check if ray is behind surface + if (rayDepth > sceneDepth) + { + return 0.0; // In shadow + } + + // Exponential step size increase + stepSize *= stepScale; + } + + return 1.0; // Not in shadow +} +``` + +### Stochastic Jittering (UE5 Default) + +```cpp +// Parameters: +GLumenScreenProbeTracingOctahedronResolution = 8; // Ray directions +float Dither = InterleavedGradientNoise(PixelPos); // Noise per pixel + +// Jitter start position to reduce banding +float3 startPos = worldPos + lightDir * (Dither * ContactShadowLength / numSteps); +``` + +### Bend Screen-Space Shadows (Wave Algorithm) + +More advanced, better performance on modern GPUs: + +```hlsl +// Uses GPU waves (64 threads) +// Threads collaborate to share ray marching work + +[numthreads(8, 8, 1)] +void ContactShadowCS_Wave(uint3 dispatchThreadId : SV_DispatchThreadID) +{ + // 60 samples per pixel using wave intrinsics + // Threads in wave share intermediate results + + float shadow = 0.0; + const int samplesPerThread = 60 / WaveGetLaneCount(); + + for (int i = 0; i < samplesPerThread; ++i) + { + float3 samplePos = ComputeSamplePosition(i); + + // Sample and share via wave intrinsics + float depth = SampleDepth(samplePos); + bool occluded = WaveActiveAnyTrue(depth > samplePos.z); + + if (occluded) + { + shadow = 0.0; + break; + } + } + + OutShadow[dispatchThreadId.xy] = shadow; +} +``` + +**Console Variables:** +```cpp +r.ContactShadows = 1 // Enable contact shadows +r.ContactShadows.OverrideLength = -1 // Override length (-1 = per-light) +r.ContactShadows.Standalone.Method = 0 // 0=Jitter, 1=Bend +``` + +--- + +## Ray-Traced Shadows + +Hardware ray tracing for maximum quality (DXR/Vulkan RT). + +**Key File:** `Engine/Source/Runtime/Renderer/Private/RayTracing/RayTracingShadows.cpp` + +### Ray Tracing Shadow Pipeline + +```hlsl +// Ray generation shader +[shader("raygeneration")] +void RayTracedShadowsRGS() +{ + uint2 pixelPos = DispatchRaysIndex().xy; + + // Reconstruct world position from GBuffer + float depth = DepthTexture[pixelPos].r; + float3 worldPos = ReconstructWorldPosition(pixelPos, depth); + + // Get normal from GBuffer + float3 normal = NormalTexture[pixelPos].xyz; + + // Ray direction (to light) + float3 rayDir = normalize(LightPosition - worldPos); + + // Offset start position to avoid self-intersection + float3 rayOrigin = worldPos + normal * NormalBias; + + // Trace shadow ray + RayDesc ray; + ray.Origin = rayOrigin; + ray.Direction = rayDir; + ray.TMin = 0.001; + ray.TMax = LightDistance; + + ShadowPayload payload; + payload.Shadowed = false; + + TraceRay(SceneAccelerationStructure, + RAY_FLAG_ACCEPT_FIRST_HIT_AND_END_SEARCH, // Optimize for shadows + 0xFF, // Instance mask + 0, // Ray contribution to hit group index + 0, // Multiplier for geometry contribution + 0, // Miss shader index + ray, + payload); + + // Write result + float shadow = payload.Shadowed ? 0.0 : 1.0; + OutShadowMask[pixelPos] = shadow; +} + +// Miss shader (ray didn't hit anything = lit) +[shader("miss")] +void RayTracedShadowsMS(inout ShadowPayload payload) +{ + payload.Shadowed = false; +} + +// Any-hit shader (for alpha-tested materials) +[shader("anyhit")] +void RayTracedShadowsAHS(inout ShadowPayload payload, BuiltInTriangleIntersectionAttributes attr) +{ + // Sample material opacity + float opacity = SampleMaterialOpacity(attr); + + if (opacity < 0.5) + IgnoreHit(); // Continue ray + else + AcceptHitAndEndSearch(); // Shadow ray hit +} + +// Closest-hit shader +[shader("closesthit")] +void RayTracedShadowsCHS(inout ShadowPayload payload, BuiltInTriangleIntersectionAttributes attr) +{ + payload.Shadowed = true; +} +``` + +### Soft Shadows with Ray Tracing + +```hlsl +// Multiple rays for area light soft shadows +const int NumShadowRays = 16; + +float shadow = 0.0; +for (int i = 0; i < NumShadowRays; ++i) +{ + // Sample random point on light area + float2 lightUV = Hammersley(i, NumShadowRays); // Low-discrepancy sequence + float3 lightPoint = SampleAreaLight(lightUV); + + // Trace ray to that point + float3 rayDir = normalize(lightPoint - worldPos); + float rayDist = length(lightPoint - worldPos); + + // Trace... + bool hit = TraceRay(...); + + shadow += hit ? 0.0 : 1.0; +} + +shadow /= NumShadowRays; +``` + +**Console Variables:** +```cpp +r.RayTracing.Shadows = 1 // Enable RT shadows +r.RayTracing.Shadows.SamplesPerPixel = 1 // Rays per pixel +r.RayTracing.NormalBias = 0.1 // Surface offset +``` + +--- + +## Shadow Bias and Artifacts + +Shadow artifacts are caused by precision issues. Bias mitigates them. + +### Types of Bias + +**1. Depth Bias (Constant):** +```cpp +// Pushes shadow depth away from surface +float BiasedDepth = ShadowDepth + DepthBias; +``` + +**2. Slope-Scale Depth Bias:** +```cpp +// Scales with surface slope +float slope = length(ddx(depth), ddy(depth)); +float BiasedDepth = ShadowDepth + DepthBias + slope * SlopeScaleBias; +``` + +**3. Normal Bias:** +```cpp +// Offset ray origin along normal +float3 BiasedPos = WorldPos + Normal * NormalBias; +``` + +**4. Receiver Bias:** +```cpp +// Applied during shadow projection (UE5 specific) +float CompareDepth = ShadowSpaceDepth - ReceiverBias; +``` + +### Bias Configuration per Light Type + +```cpp +// Directional/CSM +r.Shadow.CSMDepthBias = 10.0 // Constant bias +r.Shadow.CSMSlopeScaleDepthBias = 3.0 // Slope bias +r.Shadow.CSMReceiverBias = 0.9 // Receiver bias (0-1) + +// Spot lights +r.Shadow.SpotLightDepthBias = 3.0 +r.Shadow.SpotLightSlopeScaleDepthBias = 3.0 +r.Shadow.SpotLightReceiverBias = 0.5 + +// Point lights +r.Shadow.PointLightDepthBias = 0.02 // Much smaller (in [0,1] space) +r.Shadow.PointLightSlopeScaleDepthBias = 3.0 + +// Rect lights +r.Shadow.RectLightDepthBias = 0.025 +r.Shadow.RectLightReceiverBias = 0.3 +``` + +### Common Shadow Artifacts + +| Artifact | Cause | Solution | +|----------|-------|----------| +| **Shadow Acne** | Self-shadowing from precision | Increase depth bias | +| **Peter Panning** | Object appears detached from shadow | Decrease bias, use normal offset | +| **Aliasing** | Insufficient shadow map resolution | Increase resolution, use filtering | +| **Cascade Transition** | Visible seams between CSM cascades | Blend cascades, match bias | +| **Light Bleeding** | VSM/ESM artifact | Use traditional shadow maps | + +### Transition Scale + +```cpp +r.Shadow.TransitionScale = 60.0 // Higher = sharper, more artifacts + +// Controls soft transition at shadow edges +float shadowFade = saturate((CompareDepth - BiasedDepth) * TransitionScale); +``` + +--- + +## Performance Optimization + +### Culling Optimizations + +**1. CPU Culling:** +```cpp +r.Shadow.UseOctreeForCulling = 1 // Spatial acceleration structure +r.Shadow.RadiusThreshold = 0.01 // Cull small shadow casters + +// Frustum culling per cascade/light +TArray CullShadowCasters(FConvexVolume ShadowFrustum) +{ + TArray Casters; + + // Test each primitive against shadow frustum + for (auto Primitive : Scene->Primitives) + { + if (ShadowFrustum.IntersectBox(Primitive->Bounds)) + { + Casters.Add(Primitive); + } + } + + return Casters; +} +``` + +**2. GPU Culling (VSM):** +```cpp +// HZB (Hierarchical Z-Buffer) occlusion culling +// Cull objects occluded in previous frame's depth + +r.Shadow.Virtual.UseHZB = 1 // Enable HZB culling + +// Test object bounds against HZB +bool IsOccluded = TestBoundsAgainstHZB(ObjectBounds, PreviousFrameHZB); +``` + +### Caching Strategies + +**VSM Caching:** +```cpp +r.Shadow.Virtual.Cache = 1 // Enable caching +r.Shadow.Virtual.Cache.StaticSeparate = 1 // Separate static/dynamic pools + +// Cache invalidation strategies: +// 1. Per-primitive tracking (precise, expensive) +// 2. Spatial hashing (fast, conservative) +// 3. Dirty regions (balance) +``` + +**Traditional Shadow Caching:** +```cpp +r.Shadow.CacheWholeSceneShadows = 1 // Cache static directional shadows +r.Shadow.WholeSceneShadowCacheMb = 150 // Memory budget +``` + +### Resolution and LOD + +**Adaptive Resolution (VSM):** +```cpp +r.Shadow.Virtual.ResolutionLodBiasDirectional = 0.0 // Quality bias (-2 to +2) +r.Shadow.Virtual.ResolutionLodBiasLocal = 0.0 // For spot/point lights + +// Mip selection based on screen coverage +// Distant objects get coarser shadow pages +``` + +**Traditional Resolution:** +```cpp +r.Shadow.TexelsPerPixel = 1.27324 // Shadow texels per screen pixel +r.Shadow.MinResolution = 32 // Minimum shadow map size +r.Shadow.MaxResolution = 2048 // Maximum size +r.Shadow.FadeResolution = 64 // Start fading out +``` + +### Parallel Execution + +```cpp +r.ParallelGatherShadowPrimitives = 1 // Multi-threaded culling +r.ParallelInitDynamicShadows = 1 // Parallel shadow setup +``` + +### Stencil and Depth Bounds Optimization + +```cpp +r.Shadow.StencilOptimization = 1 // Avoid stencil buffer clears +r.Shadow.CSMDepthBoundsTest = 1 // Use depth bounds test (faster than stencil) +``` + +### One-Pass Projection (VSM) + +```cpp +r.Shadow.Virtual.OnePassProjection.MaxLightsPerPixel = 16 + +// Projects multiple lights in single pass +// Reduces draw calls for scenes with many lights +``` + +--- + +## bgfx Implementation Guide + +### Basic Shadow Map Setup + +```cpp +class ShadowMapRenderer +{ + bgfx::FrameBufferHandle shadowMapFB; + bgfx::TextureHandle shadowMapDepth; + bgfx::ProgramHandle shadowDepthProgram; + + void Init(uint16_t resolution) + { + // Create depth texture + shadowMapDepth = bgfx::createTexture2D( + resolution, resolution, + false, 1, + bgfx::TextureFormat::D24S8, + BGFX_TEXTURE_RT | BGFX_SAMPLER_COMPARE_LEQUAL + ); + + // Create framebuffer + bgfx::Attachment attachment; + attachment.init(shadowMapDepth); + shadowMapFB = bgfx::createFrameBuffer(1, &attachment, false); + + // Load shadow depth shader + shadowDepthProgram = loadProgram("vs_shadow_depth", "fs_shadow_depth"); + } + + void RenderShadowMap(const Light& light, const std::vector& objects) + { + // Calculate light view/projection + float viewMtx[16], projMtx[16]; + CalculateLightMatrices(light, viewMtx, projMtx); + + // Set as render target + bgfx::setViewFrameBuffer(VIEW_SHADOW, shadowMapFB); + bgfx::setViewRect(VIEW_SHADOW, 0, 0, resolution, resolution); + bgfx::setViewTransform(VIEW_SHADOW, viewMtx, projMtx); + bgfx::setViewClear(VIEW_SHADOW, BGFX_CLEAR_DEPTH, 0, 1.0f, 0); + + // Render depth only + for (const auto& mesh : objects) + { + bgfx::setTransform(mesh.worldMatrix); + bgfx::setVertexBuffer(0, mesh.vb); + bgfx::setIndexBuffer(mesh.ib); + + uint64_t state = BGFX_STATE_WRITE_Z + | BGFX_STATE_DEPTH_TEST_LESS + | BGFX_STATE_CULL_CCW; + + bgfx::setState(state); + bgfx::submit(VIEW_SHADOW, shadowDepthProgram); + } + } +}; +``` + +### Shadow Sampling in Lighting Pass + +```cpp +// Create sampler with comparison +bgfx::UniformHandle s_shadowMap = bgfx::createUniform( + "s_shadowMap", + bgfx::UniformType::Sampler, + 1 +); + +// In lighting pass +bgfx::setTexture(4, s_shadowMap, shadowMapDepth, + BGFX_SAMPLER_COMPARE_LEQUAL); // Hardware PCF +``` + +**GLSL Shader:** +```glsl +// fs_lighting.sc +SAMPLER2DSHADOW(s_shadowMap, 4); // Comparison sampler + +uniform mat4 u_lightViewProj; + +void main() +{ + // Reconstruct world position + vec3 worldPos = ...; + + // Transform to light clip space + vec4 shadowPos = mul(u_lightViewProj, vec4(worldPos, 1.0)); + shadowPos.xyz /= shadowPos.w; + + // Transform to [0,1] + vec3 shadowUV; + shadowUV.xy = shadowPos.xy * 0.5 + 0.5; + shadowUV.y = 1.0 - shadowUV.y; + shadowUV.z = shadowPos.z; + + // Sample with hardware PCF + float shadow = shadow2D(s_shadowMap, shadowUV); + + // Apply shadow to lighting + vec3 lighting = directLighting * shadow; + + gl_FragColor = vec4(lighting, 1.0); +} +``` + +### Cascaded Shadow Maps with bgfx + +```cpp +class CSMRenderer +{ + static const int NUM_CASCADES = 4; + + bgfx::FrameBufferHandle cascadeFB[NUM_CASCADES]; + bgfx::TextureHandle cascadeTextures[NUM_CASCADES]; + + struct Cascade + { + float viewMtx[16]; + float projMtx[16]; + float splitDistance; + }; + Cascade cascades[NUM_CASCADES]; + + void RenderCSM(const Camera& camera, const Light& light, const Scene& scene) + { + // Calculate cascade splits + CalculateCascadeSplits(camera, cascades); + + for (int i = 0; i < NUM_CASCADES; ++i) + { + // Calculate light matrices for this cascade + CalculateCascadeMatrices(camera, light, cascades[i]); + + // Render to cascade shadow map + bgfx::setViewFrameBuffer(VIEW_SHADOW + i, cascadeFB[i]); + bgfx::setViewTransform(VIEW_SHADOW + i, + cascades[i].viewMtx, + cascades[i].projMtx); + + // Cull and render + RenderDepthOnly(scene, cascades[i].frustum); + } + } +}; +``` + +### Contact Shadows with Compute + +```cpp +// bgfx compute shader for contact shadows +bgfx::ProgramHandle contactShadowCS = bgfx::createProgram( + bgfx::createShader(loadMemory("cs_contact_shadow.bin")), + true +); + +// Dispatch compute +bgfx::setImage(0, s_depthBuffer, 0, bgfx::Access::Read); +bgfx::setImage(1, s_shadowOutput, 0, bgfx::Access::Write); +bgfx::setUniform(u_lightDirection, &lightDir); + +bgfx::dispatch(VIEW_COMPUTE, + contactShadowCS, + (width + 7) / 8, + (height + 7) / 8, + 1); +``` + +--- + +## Key Files Reference + +### C++ Source Files + +**Core Shadow Rendering:** +- `Engine/Source/Runtime/Renderer/Private/ShadowRendering.cpp` (2,613 lines) + - Main shadow projection and rendering +- `Engine/Source/Runtime/Renderer/Private/ShadowDepthRendering.cpp` (2,400 lines) + - Shadow depth pass +- `Engine/Source/Runtime/Renderer/Private/ShadowSetup.cpp` + - Shadow frustum calculation and cascade setup + +**Virtual Shadow Maps:** +- `Engine/Source/Runtime/Renderer/Private/VirtualShadowMaps/VirtualShadowMapArray.cpp` (5,342 lines) + - VSM system implementation +- `Engine/Source/Runtime/Renderer/Private/VirtualShadowMaps/VirtualShadowMapCache.cpp` + - Caching system + +**Contact and Ray-Traced Shadows:** +- `Engine/Source/Runtime/Renderer/Private/Shadows/ScreenSpaceShadows.cpp` + - Contact shadows +- `Engine/Source/Runtime/Renderer/Private/RayTracing/RayTracingShadows.cpp` + - Hardware ray-traced shadows + +### Shader Files + +**Shadow Filtering:** +- `Engine/Shaders/Private/ShadowFilteringCommon.ush` + - PCF implementations +- `Engine/Shaders/Private/ShadowPercentageCloserFiltering.ush` + - PCSS implementation + +**Virtual Shadow Maps:** +- `Engine/Shaders/Shared/VirtualShadowMapDefinitions.h` + - VSM constants +- `Engine/Shaders/Private/VirtualShadowMaps/VirtualShadowMapSMRTCommon.ush` + - SMRT filtering + +**Shadow Projection:** +- `Engine/Shaders/Private/ShadowProjectionCommon.ush` + - Shadow sampling utilities +- `Engine/Shaders/Private/ShadowProjectionPixelShader.usf` + - Shadow projection shaders + +--- + +## Summary + +**For Your bgfx Engine:** + +1. **Start Simple**: Implement basic shadow maps (directional CSM, spot single map) +2. **Add Filtering**: PCF is essential, PCSS is great for quality +3. **Contact Shadows**: Cheap detail boost, implement via compute shader +4. **Optimization**: Culling, caching, resolution management +5. **Advanced (Optional)**: VSM is complex but provides best quality/performance for large scenes + +**Implementation Priority:** +1. Basic shadow maps (week 1-2) +2. PCF filtering (week 2) +3. CSM for directional lights (week 3) +4. Contact shadows (week 4) +5. PCSS (week 5-6) +6. VSM (months, optional) + +UE5's shadow system is production-proven across hundreds of shipped titles. Following this architecture will give you robust, high-quality shadows. diff --git a/gameengine/docs/UE5_Sky_And_Atmosphere_Documentation.md b/gameengine/docs/UE5_Sky_And_Atmosphere_Documentation.md new file mode 100644 index 000000000..c818cd139 --- /dev/null +++ b/gameengine/docs/UE5_Sky_And_Atmosphere_Documentation.md @@ -0,0 +1,1253 @@ +# Unreal Engine 5 Sky and Atmosphere System Documentation +## Physically-Based Sky Rendering for Game Engines + +This documentation explains UE5's sky and atmosphere rendering, from simple skyboxes to physically-accurate atmospheric scattering. + +--- + +## Table of Contents +1. [Sky System Overview](#sky-system-overview) +2. [Skybox Types](#skybox-types) +3. [Sky Atmosphere (Physically-Based)](#sky-atmosphere-physically-based) +4. [Volumetric Clouds](#volumetric-clouds) +5. [Sky Light](#sky-light) +6. [Implementation Guide](#implementation-guide) +7. [bgfx Implementation Examples](#bgfx-implementation-examples) +8. [Key Files Reference](#key-files-reference) + +--- + +## Sky System Overview + +UE5 offers multiple sky rendering approaches: + +``` +Sky Rendering Stack (from simplest to most advanced): +┌──────────────────────────────────────────────┐ +│ 1. Static Skybox (Cubemap) │ +│ - Pre-rendered HDRI │ +│ - Instant, zero cost │ +│ - No time-of-day changes │ +└──────────────────────────────────────────────┘ + ↓ +┌──────────────────────────────────────────────┐ +│ 2. Sky Dome (Procedural Material) │ +│ - Shader-based gradients │ +│ - Simple, customizable │ +│ - Limited realism │ +└──────────────────────────────────────────────┘ + ↓ +┌──────────────────────────────────────────────┐ +│ 3. Sky Atmosphere (Physical) │ +│ - Rayleigh scattering (blue sky) │ +│ - Mie scattering (haze) │ +│ - Ozone absorption │ +│ - Sun disk rendering │ +│ - Aerial perspective │ +└──────────────────────────────────────────────┘ + ↓ (optional) +┌──────────────────────────────────────────────┐ +│ 4. Volumetric Clouds │ +│ - Ray-marched 3D clouds │ +│ - Dynamic lighting │ +│ - Weather systems │ +└──────────────────────────────────────────────┘ +``` + +--- + +## Skybox Types + +### 1. HDRI Cubemap Skybox + +**Key File:** `Engine/Source/Runtime/Engine/Classes/Components/SkyLightComponent.h` + +**Simple and effective:** + +```cpp +class USkyLightComponent : public ULightComponentBase +{ + // Source type + ESkyLightSourceType SourceType; // Captured or Specified + + // If specified cubemap: + UTextureCube* Cubemap; // HDRI texture + float SourceCubemapAngle; // Rotation (0-360°) + + // Hemisphere split + bool bLowerHemisphereIsBlack; // Solid ground below? + FLinearColor LowerHemisphereColor; +}; + +enum ESkyLightSourceType +{ + SLS_CapturedScene, // Capture from scene + SLS_SpecifiedCubemap // Use provided HDRI +}; +``` + +**Typical workflow:** +1. Load HDRI texture (`.hdr` or `.exr` format) +2. Create cubemap texture from equirectangular +3. Assign to Sky Light component +4. Sky Light provides ambient lighting to scene + +**Advantages:** +- Zero runtime cost (static texture) +- High quality (real-world HDRIs) +- Perfect for indoor scenes or fixed outdoor + +**Limitations:** +- Cannot change time of day +- No dynamic sun +- No atmospheric effects + +### 2. Procedural Sky Dome + +**Simple gradient-based sky:** + +```hlsl +// Sky dome vertex shader +float4 SkyDomeVS(float3 position : POSITION) : SV_Position +{ + // Just pass through, no transforms needed + return float4(position, 1.0); +} + +// Sky dome pixel shader +float4 SkyDomePS(float4 position : SV_Position) : SV_Target +{ + float3 viewDir = normalize(position.xyz); + + // Vertical gradient + float t = viewDir.z * 0.5 + 0.5; // 0 at horizon, 1 at zenith + + // Lerp between horizon and zenith colors + float3 horizonColor = float3(0.8, 0.9, 1.0); // Light blue + float3 zenithColor = float3(0.2, 0.4, 0.8); // Dark blue + + float3 skyColor = lerp(horizonColor, zenithColor, t); + + return float4(skyColor, 1.0); +} +``` + +**Common approach for stylized games:** +- Mobile games +- Low-spec targets +- Artistic control + +--- + +## Sky Atmosphere (Physically-Based) + +UE5's flagship sky system - physically accurate atmospheric scattering. + +**Key Files:** +- `Engine/Source/Runtime/Engine/Classes/Components/SkyAtmosphereComponent.h` +- `Engine/Shaders/Private/SkyAtmosphere.usf` +- `Engine/Shaders/Private/SkyAtmosphereCommon.ush` + +### Core Parameters + +```cpp +class USkyAtmosphereComponent : public USceneComponent +{ + // Planet properties + float BottomRadius; // Ground radius (km), default: 6360 + float AtmosphereHeight; // Atmosphere layer (km), default: 60 + FColor GroundAlbedo; // Ground reflectance + float MultiScatteringFactor; // Dual scattering approximation + + // Transform mode + ESkyAtmosphereTransformMode TransformMode; +}; + +enum ESkyAtmosphereTransformMode +{ + PlanetTopAtAbsoluteWorldOrigin, // Planet surface at world 0,0,0 + PlanetTopAtComponentTransform, // Planet surface at component location + PlanetCenterAtComponentTransform // Planet center at component location +}; +``` + +### Physical Model + +The atmosphere is modeled using **participating media** with three components: + +#### 1. Rayleigh Scattering (Blue Sky) + +**Why the sky is blue:** + +```cpp +// Rayleigh scattering parameters +struct FRayleighScattering +{ + float ScatteringScale; // Overall intensity (0-2) + FLinearColor Scattering; // RGB coefficients (1/km) + float ExponentialDistribution; // Altitude falloff (km) +}; + +// Default values (Earth-like): +// Scattering = (0.0331, 0.0697, 0.1649) // Blue scatters most +// ExponentialDistribution = 8.0 km // Scale height +``` + +**Density calculation:** +```hlsl +// Atmosphere density decreases exponentially with altitude +float DensityRay = exp(-SampleHeight / ScaleHeight); + +// Scattering coefficient +float3 ScatteringRay = DensityRay * Atmosphere.RayleighScattering.rgb; +``` + +**Phase function** (angular distribution): +```hlsl +float RayleighPhase(float cosTheta) +{ + // Symmetric scattering + const float Factor = 3.0 / (16.0 * PI); + return Factor * (1.0 + cosTheta * cosTheta); +} +``` + +**Physics:** +- Short wavelengths (blue) scatter more than long (red) +- Factor: λ^-4 (inverse fourth power) +- Results in blue sky during day, red sunset + +#### 2. Mie Scattering (Haze, Fog) + +**Aerosols and particles:** + +```cpp +struct FMieScattering +{ + float ScatteringScale; // Overall intensity (0-5) + FLinearColor Scattering; // RGB coefficients (1/km) + float AbsorptionScale; // Absorption intensity (0-5) + FLinearColor Absorption; // RGB absorption (1/km) + float Anisotropy; // Phase function g (-0.999 to 0.999) + float ExponentialDistribution; // Altitude falloff (km) +}; + +// Default values: +// Scattering = (0.0021, 0.0021, 0.0021) // Grayscale +// Absorption = (0.0045, 0.0045, 0.0045) +// Anisotropy = 0.8 // Forward scattering +// ExponentialDistribution = 1.2 km // Low altitude +``` + +**Density calculation:** +```hlsl +float DensityMie = exp(-SampleHeight / ScaleHeight); + +float3 ScatteringMie = DensityMie * Atmosphere.MieScattering.rgb; +float3 AbsorptionMie = DensityMie * Atmosphere.MieAbsorption.rgb; +float3 ExtinctionMie = ScatteringMie + AbsorptionMie; +``` + +**Henyey-Greenstein phase function:** +```hlsl +float HenyeyGreensteinPhase(float g, float cosTheta) +{ + float numer = 1.0 - g * g; + float denom = 1.0 + g * g + 2.0 * g * cosTheta; + return numer / (4.0 * PI * denom * sqrt(denom)); +} + +// g > 0: Forward scattering (haze around sun) +// g = 0: Isotropic (uniform) +// g < 0: Backward scattering (rare) +``` + +**Physics:** +- Larger particles (dust, water droplets) +- Wavelength-independent (white/gray) +- Strong forward scattering (bright halo around sun) +- Responsible for haze, fog, sunsets + +#### 3. Ozone Absorption + +**Stratospheric ozone layer:** + +```cpp +struct FOzoneAbsorption +{ + float AbsorptionScale; // Overall intensity (0-0.2) + FLinearColor Absorption; // RGB absorption (1/km) + FTentDistribution Distribution; // Altitude distribution +}; + +struct FTentDistribution +{ + float TipAltitude; // Peak altitude (km), default: 25 + float TipValue; // Peak density, default: 1.0 + float Width; // Distribution width (km), default: 15 +}; + +// Default absorption: +// Absorption = (0.065, 0.1881, 0.0085) // Absorbs red/green, not blue +``` + +**Tent distribution:** +```hlsl +float TentDistribution(float altitude, FTentDistribution tent) +{ + float distance = abs(altitude - tent.TipAltitude); + float value = saturate(1.0 - distance / tent.Width); + return value * tent.TipValue; +} +``` + +**Physics:** +- Ozone (O₃) in stratosphere (15-35 km) +- Absorbs UV and red/green light +- Contributes to orange/red sunsets +- Blue sky at horizon vs overhead + +### Rendering Pipeline + +UE5 uses **Look-Up Tables (LUTs)** for performance: + +``` +Precomputed LUTs: +┌────────────────────────────────┐ +│ 1. Transmittance LUT (256×64) │ +│ - View height × zenith angle │ +│ - Transmittance through atm │ +└────────────────────────────────┘ + ↓ +┌────────────────────────────────┐ +│ 2. Multi-Scattering LUT (32×32)│ +│ - Approximates 2+ bounces │ +│ - Energy conservation │ +└────────────────────────────────┘ + ↓ +┌────────────────────────────────┐ +│ 3. Sky View LUT (192×104) │ +│ - Pre-rendered sky directions│ +│ - Fast sky lookup │ +└────────────────────────────────┘ + ↓ +┌────────────────────────────────┐ +│ 4. Aerial Perspective (32×32×16)│ +│ - 3D volume for fog │ +│ - Applied to opaque geometry │ +└────────────────────────────────┘ +``` + +#### 1. Transmittance LUT + +**What it stores:** Transmittance from any height through atmosphere + +```hlsl +// Compute shader: RenderTransmittanceLutCS +[numthreads(8, 8, 1)] +void TransmittanceLutCS(uint2 ThreadId : SV_DispatchThreadID) +{ + // UV to view height and zenith angle + float2 uv = (ThreadId + 0.5) / float2(256, 64); + + float viewHeight, viewZenithCosAngle; + UvToLutTransmittanceParams(uv, viewHeight, viewZenithCosAngle); + + // Ray march from viewHeight to atmosphere top + float3 worldPos = float3(0, 0, viewHeight); + float3 worldDir = float3( + sqrt(1.0 - viewZenithCosAngle * viewZenithCosAngle), + 0, + viewZenithCosAngle + ); + + // Find intersection with atmosphere top + float tMax = RaySphereIntersection(worldPos, worldDir, AtmosphereTopRadius); + + // Ray march and accumulate optical depth + const int numSamples = 10; + float dt = tMax / numSamples; + + float3 opticalDepth = 0; + + for (int i = 0; i < numSamples; ++i) + { + float t = (i + 0.5) * dt; + float3 samplePos = worldPos + worldDir * t; + float sampleHeight = length(samplePos) - BottomRadius; + + // Sample medium + FMediumSample medium = SampleMedium(sampleHeight); + + // Accumulate extinction + opticalDepth += medium.Extinction * dt; + } + + // Transmittance = exp(-optical depth) + float3 transmittance = exp(-opticalDepth); + + TransmittanceLUT[ThreadId] = float4(transmittance, 1.0); +} +``` + +**UV Parameterization:** +```hlsl +void UvToLutTransmittanceParams(float2 uv, out float viewHeight, out float viewZenithCosAngle) +{ + float xMu = uv.x; + float xR = uv.y; + + float H = sqrt(AtmosphereTopRadius^2 - BottomRadius^2); + float rho = xR * H; + + viewHeight = sqrt(rho^2 + BottomRadius^2); + + float dMin = AtmosphereTopRadius - viewHeight; + float dMax = rho + H; + float d = dMin + xMu * (dMax - dMin); + + viewZenithCosAngle = (H^2 - rho^2 - d^2) / (2.0 * viewHeight * d); +} +``` + +#### 2. Multi-Scattering LUT + +**Approximates multiple bounces of light:** + +```hlsl +// Simplified multi-scattering +[numthreads(8, 8, 1)] +void MultiScatteringLutCS(uint2 ThreadId : SV_DispatchThreadID) +{ + float2 uv = (ThreadId + 0.5) / 32.0; + + // Sample in spherical coordinates + float cosSunZenith = uv.x * 2.0 - 1.0; // -1 to 1 + float viewHeight = lerp(BottomRadius, AtmosphereTopRadius, uv.y); + + // Integrate over all view directions (hemisphere) + float3 multiScattering = 0; + + const int numDirs = 64; // Sample directions + + for (int i = 0; i < numDirs; ++i) + { + float3 viewDir = FibonacciHemisphere(i, numDirs); + + // Compute single scattering for this direction + SingleScatteringResult ss = IntegrateSingleScattering( + viewHeight, viewDir, cosSunZenith + ); + + multiScattering += ss.Scattering; + } + + multiScattering /= numDirs; + multiScattering *= MultiScatteringFactor; + + MultiScatteringLUT[ThreadId] = float4(multiScattering, 1.0); +} +``` + +#### 3. Sky View LUT (Fast Sky) + +**Pre-rendered sky for fast lookups:** + +```hlsl +[numthreads(8, 8, 1)] +void SkyViewLutCS(uint2 ThreadId : SV_DispatchThreadID) +{ + float2 uv = (ThreadId + 0.5) / float2(192, 104); + + // UV to spherical direction (with horizon compression) + float3 viewDir = SkViewLutParamsToDirection(uv); + + // Camera position + float3 cameraPos = float3(0, 0, BottomRadius + CameraAltitude); + + // Ray march sky + SingleScatteringResult sky = IntegrateSingleScattering( + cameraPos, + viewDir, + SunDirection, + MaxDistance + ); + + // Store radiance + SkyViewLUT[ThreadId] = float4(sky.L, 1.0); +} +``` + +**Usage:** +```hlsl +// Sample sky instead of ray marching +float3 GetSkyColor(float3 viewDir) +{ + float2 uv = DirectionToSkyViewLutParams(viewDir); + return SkyViewLUT.SampleLevel(LinearSampler, uv, 0).rgb; +} +``` + +#### 4. Aerial Perspective Volume + +**3D fog volume applied to opaque geometry:** + +```hlsl +[numthreads(4, 4, 4)] +void AerialPerspectiveCS(uint3 ThreadId : SV_DispatchThreadID) +{ + float3 uvw = (ThreadId + 0.5) / float3(32, 32, 16); + + // UV to world position + float2 screenUV = uvw.xy; + float depth = DepthFromSlice(uvw.z); // Non-linear distribution + + float3 worldPos = ScreenToWorld(screenUV, depth); + float3 cameraPos = View.WorldCameraOrigin; + + // Ray march from camera to world position + SingleScatteringResult aerial = IntegrateSingleScattering( + cameraPos, + normalize(worldPos - cameraPos), + SunDirection, + length(worldPos - cameraPos) + ); + + // Store scattering and transmittance + AerialPerspectiveVolume[ThreadId] = float4(aerial.L, aerial.Transmittance); +} +``` + +**Application:** +```hlsl +// In deferred lighting or forward pass +float3 sceneColor = ...; // Lit scene color + +// Sample aerial perspective +float3 uvw = float3(screenUV, DepthToSlice(depth)); +float4 aerialPerspective = AerialPerspectiveVolume.SampleLevel(LinearSampler, uvw, 0); + +// Apply fog +float3 finalColor = sceneColor * aerialPerspective.a + aerialPerspective.rgb; +``` + +### Sun Disk Rendering + +**Add bright sun disk:** + +```hlsl +float3 GetSunDisk(float3 viewDir, float3 sunDir, float sunAngularRadius) +{ + float ViewDotSun = dot(viewDir, sunDir); + float CosSunRadius = cos(sunAngularRadius); + + if (ViewDotSun > CosSunRadius) + { + // Inside sun disk + + // Get transmittance to sun + float3 transmittance = GetAtmosphereTransmittance( + ViewHeight, + ViewDotSun + ); + + // Soft edge to prevent bloom artifacts + float EdgeSoftness = saturate(2.0 * (ViewDotSun - CosSunRadius) / (1.0 - CosSunRadius)); + + // Sun luminance (in lux or nits) + float3 sunLuminance = SunIlluminance * EdgeSoftness; + + return transmittance * sunLuminance; + } + + return 0; +} +``` + +### Console Variables + +```cpp +// Quality +r.SkyAtmosphere.SampleCountMin = 2 +r.SkyAtmosphere.SampleCountMax = 32 +r.SkyAtmosphere.DistanceToSampleCountMaxInv = 150 // km + +// Fast sky (LUT-based) +r.SkyAtmosphere.FastSkyLUT = 1 +r.SkyAtmosphere.FastSkyLUT.SampleCountMax = 32 + +// Aerial perspective +r.SkyAtmosphere.AerialPerspectiveLUT.Width = 32 +r.SkyAtmosphere.AerialPerspectiveLUT.DepthResolution = 16 +r.SkyAtmosphere.AerialPerspectiveLUT.Depth = 96 // km + +// Async compute +r.SkyAtmosphereAsyncCompute = 0 +``` + +--- + +## Volumetric Clouds + +3D ray-marched clouds with realistic lighting. + +**Key Files:** +- `Engine/Source/Runtime/Engine/Classes/Components/VolumetricCloudComponent.h` +- `Engine/Shaders/Private/VolumetricCloud.usf` + +### Cloud Parameters + +```cpp +class UVolumetricCloudComponent : public USceneComponent +{ + // Layer configuration + float LayerBottomAltitude; // Cloud base (km) + float LayerHeight; // Cloud thickness (km) + + // Trace distances + float TracingStartMaxDistance; // Max start distance (km) + float TracingMaxDistance; // Max trace distance (km) + + // Quality + float ViewSampleCountScale; // Primary view samples (0.05-8) + float ReflectionViewSampleCountScaleValue; // Reflection samples + float ShadowViewSampleCountScale; // Shadow samples + float ShadowTracingDistance; // Shadow trace distance (km) + + // Lighting + bool bUsePerSampleAtmosphericLightTransmittance; + float SkyLightCloudBottomOcclusion; // Sky light AO (0-1) + + // Material + UMaterialInterface* CloudMaterial; // Volume domain material +}; +``` + +### Cloud Material (Volume Domain) + +```hlsl +// Cloud material outputs +struct FCloudMaterialOutput +{ + float3 Extinction; // How much light is blocked (RGB) + float3 Albedo; // Scattering color (RGB) + float3 Emissive; // Self-illumination + float AmbientOcclusion; // Sky light shadowing +}; +``` + +**Typical cloud material:** +- Sample 3D noise textures (Perlin, Worley) +- Combine at multiple frequencies (detail) +- Remap density based on altitude +- Add weather patterns + +### Ray Marching + +```hlsl +// Simplified cloud ray marching +float4 RayMarchClouds(float3 rayOrigin, float3 rayDir, float maxDistance) +{ + const int maxSteps = 128; // Configurable quality + + float3 accumulatedLight = 0; + float accumulatedTransmittance = 1.0; + + float t = 0; + float dt = maxDistance / maxSteps; + + for (int step = 0; step < maxSteps; ++step) + { + float3 samplePos = rayOrigin + rayDir * t; + + // Sample cloud density from material + FCloudMaterialOutput cloud = EvaluateCloudMaterial(samplePos); + + if (cloud.Extinction.r > 0.0) + { + // In cloud + + // Beer's law transmittance + float sampleTransmittance = exp(-cloud.Extinction.r * dt); + + // Light contribution + float3 lighting = 0; + + // Sun lighting (shadow ray march) + float3 shadowRayDir = SunDirection; + float shadowOpticalDepth = MarchShadowRay(samplePos, shadowRayDir); + float sunTransmittance = exp(-shadowOpticalDepth); + + lighting += SunColor * sunTransmittance * cloud.Albedo; + + // Sky light (ambient) + lighting += SkyLightColor * (1.0 - cloud.AmbientOcclusion) * cloud.Albedo; + + // Emissive + lighting += cloud.Emissive; + + // Accumulate + accumulatedLight += lighting * accumulatedTransmittance * (1.0 - sampleTransmittance); + accumulatedTransmittance *= sampleTransmittance; + + // Early termination + if (accumulatedTransmittance < 0.01) + break; + } + + t += dt; + } + + return float4(accumulatedLight, accumulatedTransmittance); +} +``` + +### Performance Optimization + +**Temporal reprojection:** +```hlsl +// Current frame (noisy, few samples) +float4 currentClouds = RayMarchClouds(rayOrigin, rayDir, maxDistance); + +// Reproject previous frame +float2 velocity = VelocityBuffer.Sample(screenUV).xy; +float2 prevUV = screenUV - velocity; + +float4 previousClouds = PreviousCloudBuffer.Sample(prevUV); + +// Temporal blend +float blendWeight = 0.95; // High persistence + +// Validate history +if (DepthChanged(prevUV) || OutOfBounds(prevUV)) + blendWeight = 0.0; + +float4 finalClouds = lerp(currentClouds, previousClouds, blendWeight); +``` + +**Adaptive sampling:** +```cpp +// Fewer samples in clear sky, more in clouds +int numSamples = baseSamples; + +if (hitCloud) + numSamples *= 4; // Increase detail in clouds +``` + +--- + +## Sky Light + +Provides ambient environment lighting. + +**Key File:** `Engine/Source/Runtime/Engine/Classes/Components/SkyLightComponent.h` + +### Capture Modes + +```cpp +// Static capture +bRealTimeCapture = false; +SourceType = SLS_SpecifiedCubemap; +Cubemap = MyHDRICubemap; + +// Dynamic capture +bRealTimeCapture = true; +SourceType = SLS_CapturedScene; +SkyDistanceThreshold = 10000; // cm, capture beyond this distance +``` + +### Cubemap Processing + +**Convolution for diffuse:** +```hlsl +// Irradiance map (diffuse) +float3 GenerateIrradianceMap(float3 normal, TextureCube envMap) +{ + float3 irradiance = 0; + + // Sample hemisphere around normal + const int numSamples = 1024; + + for (int i = 0; i < numSamples; ++i) + { + float2 xi = Hammersley(i, numSamples); + float3 sampleDir = UniformSampleHemisphere(xi, normal); + + float3 envColor = envMap.SampleLevel(LinearSampler, sampleDir, 0).rgb; + + irradiance += envColor * dot(normal, sampleDir); + } + + irradiance *= PI / numSamples; + + return irradiance; +} +``` + +**Prefiltered specular:** +```hlsl +// Prefilter for roughness (mip level = roughness) +float3 PrefilterEnvMap(float3 reflectDir, float roughness, TextureCube envMap) +{ + float3 prefilteredColor = 0; + float totalWeight = 0; + + const int numSamples = 256; + + for (int i = 0; i < numSamples; ++i) + { + float2 xi = Hammersley(i, numSamples); + float3 H = ImportanceSampleGGX(xi, roughness, reflectDir); + float3 L = 2.0 * dot(reflectDir, H) * H - reflectDir; + + float NoL = max(dot(reflectDir, L), 0.0); + + if (NoL > 0.0) + { + float3 envColor = envMap.SampleLevel(LinearSampler, L, 0).rgb; + + prefilteredColor += envColor * NoL; + totalWeight += NoL; + } + } + + return prefilteredColor / totalWeight; +} +``` + +### Spherical Harmonics (SH) + +**Efficient diffuse representation:** + +```cpp +struct FSHVectorRGB3 +{ + float V[3][9]; // RGB × 9 SH coefficients +}; + +// Project cubemap to SH (L=2, 9 coefficients) +FSHVectorRGB3 ProjectToSH(TextureCube envMap) +{ + FSHVectorRGB3 sh = {}; + + // Sample cubemap + for each face + for each pixel + float3 direction = PixelToDirection(face, pixel); + float3 color = envMap.Load(face, pixel).rgb; + + // Evaluate SH basis functions + float basis[9] = EvaluateSHBasis(direction); + + // Accumulate + for (int i = 0; i < 9; ++i) + sh.V[0][i] += color.r * basis[i]; + sh.V[1][i] += color.g * basis[i]; + sh.V[2][i] += color.b * basis[i]; + + // Normalize + sh /= totalPixels; + + return sh; +} +``` + +**Reconstruction:** +```hlsl +float3 SampleSH(FSHVectorRGB3 sh, float3 normal) +{ + // Evaluate SH basis + float basis[9] = EvaluateSHBasis(normal); + + // Dot product with coefficients + float3 irradiance = 0; + + for (int i = 0; i < 9; ++i) + { + irradiance.r += sh.V[0][i] * basis[i]; + irradiance.g += sh.V[1][i] * basis[i]; + irradiance.b += sh.V[2][i] * basis[i]; + } + + return max(0, irradiance); +} +``` + +--- + +## Implementation Guide + +### Minimum Viable Sky System + +**Week 1: Static Skybox** +1. Load HDRI cubemap +2. Render skybox (inverted sphere or cube) +3. Sky Light with SH for ambient + +**Week 2: Basic Atmosphere** +4. Transmittance LUT (compute shader) +5. Simple ray marching (no multi-scattering) +6. Sun disk rendering + +**Week 3-4: Full Atmosphere** +7. Multi-scattering LUT +8. Mie scattering and ozone +9. Aerial perspective volume +10. Sky View LUT for performance + +**Week 5-8: Clouds (Optional)** +11. 3D noise generation +12. Cloud ray marching +13. Cloud lighting +14. Temporal reprojection + +### Key Algorithms + +**Ray-Sphere Intersection:** +```hlsl +float RaySphereIntersection(float3 rayOrigin, float3 rayDir, float sphereRadius) +{ + float a = dot(rayDir, rayDir); + float b = 2.0 * dot(rayOrigin, rayDir); + float c = dot(rayOrigin, rayOrigin) - sphereRadius * sphereRadius; + + float discriminant = b * b - 4.0 * a * c; + + if (discriminant < 0.0) + return -1.0; // No intersection + + float t = (-b + sqrt(discriminant)) / (2.0 * a); + + return t; +} +``` + +**Analytical Integration (Better than dt):** +```hlsl +// Instead of: L += Throughput * Scattering * dt +// Use: Analytical integration of exponential + +float3 IntegrateScatteringOverSegment(float3 scattering, float3 extinction, float dt) +{ + float3 safeExtinction = max(extinction, 0.0001); + + // ∫ e^(-extinction*t) dt from 0 to dt + float3 transmittance = exp(-extinction * dt); + + // Analytical: (scattering / extinction) * (1 - transmittance) + float3 integrated = (scattering / safeExtinction) * (1.0 - transmittance); + + return integrated; +} +``` + +--- + +## bgfx Implementation Examples + +### Simple Skybox + +```cpp +class SkyboxRenderer +{ + bgfx::TextureHandle skyboxCubemap; + bgfx::ProgramHandle skyboxProgram; + bgfx::VertexBufferHandle cubeVB; + bgfx::IndexBufferHandle cubeIB; + + void RenderSkybox() + { + // Disable depth writes, always pass depth test + uint64_t state = BGFX_STATE_WRITE_RGB + | BGFX_STATE_DEPTH_TEST_LEQUAL; + + bgfx::setState(state); + + // Remove translation from view matrix + mat4 skyboxView = view; + skyboxView[3] = vec4(0, 0, 0, 1); // Clear translation + + bgfx::setTransform(identity); + bgfx::setViewTransform(VIEW_SKY, &skyboxView, &projection); + + // Bind cubemap + bgfx::setTexture(0, s_skybox, skyboxCubemap); + + // Render cube + bgfx::setVertexBuffer(0, cubeVB); + bgfx::setIndexBuffer(cubeIB); + + bgfx::submit(VIEW_SKY, skyboxProgram); + } +}; +``` + +**Skybox shader:** +```glsl +// vs_skybox.sc +$input a_position +$output v_dir + +#include + +void main() +{ + // Output direction (no transformation) + v_dir = a_position; + + // Project position + gl_Position = mul(u_viewProj, vec4(a_position, 1.0)); + + // Force maximum depth + gl_Position.z = gl_Position.w; +} + +// fs_skybox.sc +$input v_dir + +#include + +SAMPLERCUBE(s_skybox, 0); + +void main() +{ + vec3 dir = normalize(v_dir); + gl_FragColor = textureCube(s_skybox, dir); +} +``` + +### Atmosphere Transmittance LUT + +```cpp +// Compute shader to generate transmittance LUT +class AtmosphereRenderer +{ + bgfx::TextureHandle transmittanceLUT; + bgfx::ProgramHandle transmittanceCS; + + void Init() + { + // Create transmittance LUT texture + transmittanceLUT = bgfx::createTexture2D( + 256, 64, false, 1, + bgfx::TextureFormat::RGBA16F, + BGFX_TEXTURE_COMPUTE_WRITE | BGFX_SAMPLER_U_CLAMP | BGFX_SAMPLER_V_CLAMP + ); + + // Load compute shader + transmittanceCS = bgfx::createProgram( + bgfx::createShader(loadMemory("cs_transmittance_lut.bin")), + true + ); + } + + void GenerateTransmittanceLUT() + { + // Bind output texture + bgfx::setImage(0, transmittanceLUT, 0, bgfx::Access::Write); + + // Set atmosphere parameters + bgfx::setUniform(u_atmosphereParams, &atmosphereParams); + + // Dispatch (256/8 = 32, 64/8 = 8) + bgfx::dispatch(VIEW_COMPUTE, transmittanceCS, 32, 8, 1); + } +}; +``` + +**Transmittance compute shader:** +```glsl +// cs_transmittance_lut.sc +#include + +IMAGE2D_WR(s_transmittanceLUT, rgba16f, 0); + +uniform vec4 u_atmosphereParams; // x=bottomRadius, y=topRadius, z=scaleHeight + +float RaySphereIntersection(vec3 ro, vec3 rd, float r) +{ + float a = dot(rd, rd); + float b = 2.0 * dot(ro, rd); + float c = dot(ro, ro) - r * r; + float discriminant = b * b - 4.0 * a * c; + + if (discriminant < 0.0) + return -1.0; + + return (-b + sqrt(discriminant)) / (2.0 * a); +} + +NUM_THREADS(8, 8, 1) +void main() +{ + vec2 uv = (gl_GlobalInvocationID.xy + 0.5) / vec2(256.0, 64.0); + + float bottomRadius = u_atmosphereParams.x; + float topRadius = u_atmosphereParams.y; + float scaleHeight = u_atmosphereParams.z; + + // UV to view parameters + float viewHeight = mix(bottomRadius, topRadius, uv.y); + float viewZenithCos = uv.x * 2.0 - 1.0; + + // Ray march + vec3 worldPos = vec3(0, 0, viewHeight); + vec3 worldDir = vec3(sqrt(1.0 - viewZenithCos * viewZenithCos), 0, viewZenithCos); + + float tMax = RaySphereIntersection(worldPos, worldDir, topRadius); + + const int numSamples = 10; + float dt = tMax / float(numSamples); + + vec3 opticalDepth = vec3_splat(0.0); + + for (int i = 0; i < numSamples; ++i) + { + float t = (float(i) + 0.5) * dt; + vec3 samplePos = worldPos + worldDir * t; + float sampleHeight = length(samplePos) - bottomRadius; + + // Rayleigh density + float density = exp(-sampleHeight / scaleHeight); + + // Scattering coefficient (wavelength dependent) + vec3 scattering = vec3(0.0331, 0.0697, 0.1649) * density; + + opticalDepth += scattering * dt; + } + + vec3 transmittance = exp(-opticalDepth); + + imageStore(s_transmittanceLUT, ivec2(gl_GlobalInvocationID.xy), vec4(transmittance, 1.0)); +} +``` + +### Sky Rendering with Atmosphere + +```glsl +// fs_sky_atmosphere.sc +$input v_viewDir + +#include + +SAMPLER2D(s_transmittanceLUT, 0); + +uniform vec4 u_sunDirection; +uniform vec4 u_atmosphereParams; + +vec3 IntegrateSkyAtmosphere(vec3 viewDir) +{ + float bottomRadius = u_atmosphereParams.x; + float topRadius = u_atmosphereParams.y; + + vec3 cameraPos = vec3(0, 0, bottomRadius + 0.001); // 1m above ground + + // Find intersection with atmosphere + float tMax = RaySphereIntersection(cameraPos, viewDir, topRadius); + + const int numSamples = 16; + float dt = tMax / float(numSamples); + + vec3 L = vec3_splat(0.0); + vec3 throughput = vec3_splat(1.0); + + for (int i = 0; i < numSamples; ++i) + { + float t = (float(i) + 0.5) * dt; + vec3 samplePos = cameraPos + viewDir * t; + float sampleHeight = length(samplePos) - bottomRadius; + + // Medium properties + float density = exp(-sampleHeight / 8.0); + vec3 scattering = vec3(0.0331, 0.0697, 0.1649) * density; + + // Phase function (Rayleigh) + float cosTheta = dot(viewDir, u_sunDirection.xyz); + float phase = 0.75 * (1.0 + cosTheta * cosTheta); + + // Sun transmittance (sample LUT) + float sunHeight = length(samplePos); + float sunZenithCos = dot(normalize(samplePos), u_sunDirection.xyz); + vec2 transmittanceUV = vec2((sunZenithCos + 1.0) * 0.5, (sunHeight - bottomRadius) / (topRadius - bottomRadius)); + vec3 sunTransmittance = texture2D(s_transmittanceLUT, transmittanceUV).rgb; + + // Scattering contribution + vec3 S = scattering * phase * sunTransmittance; + + // Integrate + vec3 sampleTransmittance = exp(-scattering * dt); + vec3 Sint = (S - S * sampleTransmittance) / scattering; + L += throughput * Sint; + throughput *= sampleTransmittance; + } + + return L; +} + +void main() +{ + vec3 skyColor = IntegrateSkyAtmosphere(normalize(v_viewDir)); + gl_FragColor = vec4(skyColor, 1.0); +} +``` + +--- + +## Key Files Reference + +### C++ Source Files + +**Components:** +- `Engine/Source/Runtime/Engine/Classes/Components/SkyAtmosphereComponent.h` +- `Engine/Source/Runtime/Engine/Classes/Components/SkyLightComponent.h` +- `Engine/Source/Runtime/Engine/Classes/Components/VolumetricCloudComponent.h` + +**Rendering:** +- `Engine/Source/Runtime/Renderer/Private/SkyAtmosphereRendering.cpp` + - Main atmosphere rendering logic +- `Engine/Source/Runtime/Renderer/Private/VolumetricCloudRendering.cpp` + - Cloud rendering + +**Common Data:** +- `Engine/Source/Runtime/Engine/Public/Rendering/SkyAtmosphereCommonData.h` + - Shared data structures + +### Shader Files + +**Atmosphere:** +- `Engine/Shaders/Private/SkyAtmosphere.usf` + - Main atmosphere shaders +- `Engine/Shaders/Private/SkyAtmosphereCommon.ush` + - Common utilities and functions +- `Engine/Shaders/Private/ParticipatingMediaCommon.ush` + - Phase functions, scattering math + +**Clouds:** +- `Engine/Shaders/Private/VolumetricCloud.usf` + - Cloud ray marching +- `Engine/Shaders/Private/VolumetricCloudCommon.ush` + - Cloud utilities + +--- + +## Summary + +**For Your Custom Engine:** + +1. **Start simple**: Static HDRI skybox + sky light (Week 1) +2. **Add atmosphere**: Transmittance LUT + basic ray marching (Week 2-3) +3. **Optimize**: Multi-scattering, sky view LUT, aerial perspective (Week 4-6) +4. **Advanced**: Volumetric clouds (Weeks 7-12, optional) + +**Key Insights:** +- **LUTs are essential**: Precompute expensive calculations +- **Analytical integration**: More accurate than simple dt multiplication +- **Temporal reprojection**: Critical for cloud performance +- **Physical units**: Use real-world values (km, wavelengths) + +**Typical Performance Budget:** +- Skybox: <0.1ms (trivial) +- Atmosphere (with LUTs): 0.5-1ms +- Clouds (full quality): 2-8ms (very expensive) + +UE5's sky and atmosphere system provides cinematic-quality outdoor environments. Even implementing basic atmosphere dramatically improves visual realism compared to simple skyboxes.