metabuilder/gameengine/docs/UE5_Rendering_Documentation.md

Unreal Engine 5 Rendering System Documentation

Understanding How Professional Game Engines Prevent "Full Bright or Completely Dark" Scenes

This documentation explains how Unreal Engine 5's rendering pipeline works, with specific focus on the systems that prevent scenes from appearing too bright or too dark.

---

Table of Contents

1. The Core Problem: Why Scenes Appear Too Bright or Dark
2. Overall Rendering Pipeline
3. Deferred Shading Architecture
4. The Critical Solution: Exposure Control
5. Tone Mapping
6. Implementation Checklist for Your Engine
7. Key Files Reference

---

The Core Problem

When rendering in HDR (High Dynamic Range), scene luminance values can range from 0.001 to 100,000+. However, displays can only show values from 0 to 1 (or 0 to 255 in 8-bit).

Without proper exposure and tone mapping:

• Bright scenes: Values > 1.0 get clamped to white → "Full Bright"
• Dark scenes: Values < 0.1 appear black on screen → "Completely Dark"

The solution requires TWO separate systems:

1. Exposure - Scale HDR values to a reasonable range
2. Tone Mapping - Map HDR to LDR with a curve that preserves detail

---

Overall Rendering Pipeline

UE5 uses a deferred rendering pipeline with distinct stages:

┌─────────────────────────────────────────────────────────┐
│ 1. GEOMETRY PASS (Write to GBuffer)                    │
│    - Depth PrePass                                      │
│    - Base Pass: Albedo, Normals, Roughness, Metallic   │
│    Output: GBuffer (multiple render targets in HDR)    │
└─────────────────────────────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────┐
│ 2. LIGHTING PASS (Deferred Lighting)                   │
│    - Read GBuffer                                       │
│    - Accumulate all lights                             │
│    - Shadows, reflections, ambient occlusion           │
│    Output: Scene Color (HDR linear, can be 0-10000+)   │
└─────────────────────────────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────┐
│ 3. POST-PROCESSING (THIS IS WHERE THE MAGIC HAPPENS)   │
│    a. Calculate scene luminance (histogram/average)    │
│    b. Eye Adaptation (auto exposure calculation)       │
│    c. Bloom                                            │
│    d. Tone Mapping (exposure + tone curve)             │
│    e. Color grading                                     │
│    f. Gamma correction                                  │
│    Output: Final LDR image (0-1 range for display)    │
└─────────────────────────────────────────────────────────┘

Key File: Engine/Source/Runtime/Renderer/Private/DeferredShadingRenderer.cpp:1699

• Function: FDeferredShadingSceneRenderer::Render()
• This is the main rendering loop that coordinates all stages

---

Deferred Shading Architecture

What is Deferred Shading?

Traditional forward rendering:

For each object:
    Apply all lights to this object
    Output final color

Deferred rendering:

Pass 1 - Geometry Pass:
    For each object:
        Write material properties to GBuffer (albedo, normal, roughness, etc.)

Pass 2 - Lighting Pass:
    For each pixel:
        Read material properties from GBuffer
        For each light:
            Calculate lighting contribution
        Sum all lighting
        Output HDR scene color

GBuffer Layout

UE5's GBuffer contains:

• GBufferA - World Normal (RGB), Per-object data (A)
• GBufferB - Metallic (R), Specular (G), Roughness (B), Shading Model (A)
• GBufferC - Base Color (RGB), Ambient Occlusion (A)
• GBufferD - Custom data / Subsurface color
• GBufferE - Precomputed shadow factors

Key Files:

• Engine/Source/Runtime/Renderer/Private/BasePassRendering.cpp - GBuffer generation
• Engine/Shaders/Private/DeferredShadingCommon.ush - GBuffer encoding/decoding
• Engine/Source/Runtime/Renderer/Private/LightRendering.cpp - Deferred lighting

Why This Matters for Your Engine

The lighting pass outputs HDR linear light values that can be much brighter than 1.0. For example:

• A sunlit white surface might have luminance of 10.0
• A bright light source might contribute 50.0
• A dark shadow might be 0.01

These HDR values MUST be processed by exposure and tone mapping to look correct on screen.

---

The Critical Solution: Exposure Control

This is the #1 system you're missing that causes "full bright or completely dark" scenes.

What is Exposure?

In photography and rendering, exposure is how much light reaches the sensor/film. It's controlled by:

• Aperture (f-stop) - How wide the lens opens
• Shutter Speed - How long the sensor is exposed
• ISO - Sensor sensitivity

UE5 uses the EV100 system (Exposure Value at ISO 100):

EV100 = log2((f-stop² × shutter_speed × 100) / ISO)

Key Insight: Different scenes need different exposure values:

• Bright outdoor scene: EV100 = 15 (low exposure, dim it down)
• Indoor scene: EV100 = 8 (high exposure, brighten it up)
• Night scene: EV100 = 2 (very high exposure)

Eye Adaptation (Auto Exposure)

UE5 implements automatic exposure similar to a camera or human eye adapting to brightness.

File: Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessEyeAdaptation.cpp

Three Methods:

1. Histogram-based (AEM_Histogram) - Most accurate

   • Analyzes luminance histogram of the scene
   • Finds median luminance
   • Adjusts exposure to make median map to "middle grey" (18% or EV 0)

2. Basic (AEM_Basic) - Faster

   • Calculates average scene luminance
   • Adjusts exposure based on average

3. Manual (AEM_Manual) - Artist controlled

   • Fixed exposure value set by designer

How Eye Adaptation Works

Step 1: Calculate Scene Luminance

// Shader: Engine/Shaders/Private/PostProcessHistogram.usf
// For each pixel in downsampled scene:
float Luminance = dot(SceneColor.rgb, float3(0.299, 0.587, 0.114));
// Build histogram of luminance values

Step 2: Determine Target Exposure

// From PostProcessEyeAdaptation.cpp:395
float BasePhysicalCameraEV100 =
    log2((DepthOfFieldFstop² × CameraShutterSpeed × 100) / max(1, CameraISO));

// Apply exposure compensation (artist adjustment)
float FinalEV100 = BasePhysicalCameraEV100 + ExposureCompensation;

// Convert to linear exposure multiplier
float Exposure = EV100ToLuminance(FinalEV100);

Step 3: Temporal Smoothing

// Smooth exposure changes over time to avoid flickering
float NewExposure = lerp(OldExposure, TargetExposure, AdaptationSpeed × DeltaTime);

Critical Function: FViewInfo::UpdatePreExposure() at line 1383

Pre-Exposure: A Critical Optimization

UE5 applies exposure during rendering (not just in post-processing) to maintain precision in HDR buffers.

// During lighting pass:
SceneColor = BaseColor × LightColor × LightIntensity × PreExposure;

// PreExposure is calculated as:
PreExposure = CalculatedExposure × GlobalExposure × VignetteMask;

Why Pre-Exposure Matters:

• Prevents HDR values from getting too large (overflow)
• Maintains precision in 16-bit float buffers
• Reduces banding in bright and dark areas

Key File: Engine/Shaders/Private/EyeAdaptationCommon.ush

EV100 Conversion Functions

// Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessEyeAdaptation.cpp

// Convert EV100 to linear luminance multiplier
float EV100ToLuminance(float EV100)
{
    return exp2(EV100 - 3.0f);
}

// Convert luminance to EV100
float LuminanceToEV100(float Luminance)
{
    return log2(Luminance) + 3.0f;
}

// The -3.0 offset accounts for the ISO 100 baseline and lens attenuation

Console Variables for Exposure

// Override exposure calculation
r.EyeAdaptation.PreExposureOverride = -1.0  // -1 = auto, > 0 = manual override
r.EyeAdaptation.MethodOverride = -1         // -1 = auto, 0 = manual, 1 = basic, 2 = histogram
r.EyeAdaptation.LensAttenuation = 0.78      // Lens q factor (calibration)
r.EyeAdaptation.ExposureCompensation = 0.0  // Artist bias in EV stops
r.EyeAdaptation.MinBrightness = 0.0         // Minimum scene luminance
r.EyeAdaptation.MaxBrightness = 2.0         // Maximum scene luminance

---

Tone Mapping

After exposure scales the HDR values, tone mapping converts HDR to LDR for display.

File: Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessTonemap.cpp

Why Tone Mapping is Necessary

Even with perfect exposure, HDR still has values outside [0, 1]:

• Bright highlights might be 2.0
• Very bright lights might be 5.0
• Specular reflections might be 10.0

Linear clamp (your current approach?) would make these all white, losing detail.

Tone mapping uses a curve that:

• Preserves midtones (values near 0.18)
• Compresses highlights smoothly (values > 1.0)
• Darkens shadows gently (values < 0.18)

Tone Mapping Curve Types in UE5

1. Filmic Tone Curve (Default)

A custom curve with artistic control:

// Parameters:
- Toe: Controls shadow compression
- Shoulder: Controls highlight compression
- Slope: Controls midtone contrast
- WhitePoint: Brightest value that maps to white

The curve looks like an "S" shape that smoothly compresses both bright and dark values.

2. ACES (Academy Color Encoding System)

Industry-standard tone mapper used in film:

• ACES 1.3 (previous standard)
• ACES 2.0 (current standard)

More accurate color preservation, especially for bright lights and HDR workflows.

Shader: Engine/Shaders/Private/TonemapCommon.ush:415

Filmic Tone Curve Math

// Simplified version from TonemapCommon.ush
float3 FilmicTonemap(float3 LinearColor)
{
    // Apply exposure first
    float3 ExposedColor = LinearColor * Exposure;

    // Filmic curve (simplified)
    float3 x = max(0, ExposedColor - 0.004);
    float3 ToneMappedColor = (x * (6.2 * x + 0.5)) / (x * (6.2 * x + 1.7) + 0.06);

    return ToneMappedColor;
}

Full implementation: Engine/Shaders/Private/TonemapCommon.ush:97 (FilmToneMap function)

ACES Tone Mapping

// From TonemapCommon.ush
float3 ACESTonemap(float3 LinearColor)
{
    // Convert to ACES color space
    float3 ACESColor = LinearToACES(LinearColor);

    // Apply RRT (Reference Rendering Transform) + ODT (Output Device Transform)
    float3 ODTColor = ACESOutputTransforms(ACESColor);

    return ODTColor;
}

Full ACES implementation: Engine/Shaders/Private/ACES.ush

Gamma Correction

After tone mapping, the final step is gamma correction for display:

// sRGB gamma curve
float3 LinearToSRGB(float3 LinearColor)
{
    // Simplified: FinalColor = pow(LinearColor, 1/2.2)
    // Actual sRGB is piecewise with linear segment near black
}

Most displays expect sRGB gamma, so this is applied at the very end.

Tone Mapping Console Variables

r.Tonemapper.Sharpen = 0.5              // Sharpening amount
r.TonemapperGamma = 2.2                 // Gamma curve
r.Tonemapper.GrainQuantization = 1      // Film grain
r.Tonemapper.Quality = 5                // Quality level

---

Implementation Checklist for Your Engine

Based on UE5's architecture, here's what you need to fix the "full bright or completely dark" problem:

Phase 1: Basic Exposure (Minimum Viable)

• Calculate scene luminance

  • Downsample scene to 64×64 or smaller
  • Calculate average luminance: Lum = 0.299×R + 0.587×G + 0.114×B

• Calculate exposure value

  float targetLuminance = 0.18; // Middle grey
  float avgLuminance = CalculateSceneLuminance();
  float exposure = targetLuminance / (avgLuminance + 0.001);
  exposure = clamp(exposure, 0.1, 10.0); // Prevent extremes
  

• Apply basic tone mapping

  // Reinhard tone mapping (simplest)
  float3 ToneMapped = ExposedColor / (ExposedColor + 1.0);
  
  // Or Uncharted 2 (better highlights)
  float3 ToneMapped = FilmicCurve(ExposedColor);
  

• Add gamma correction

  float3 Final = pow(ToneMapped, 1.0/2.2); // sRGB gamma
  

Phase 2: Smooth Adaptation

• Temporal smoothing of exposure

  float adaptationSpeed = 2.0; // EV per second
  newExposure = lerp(oldExposure, targetExposure,
                     adaptationSpeed × deltaTime);
  

• Store exposure in texture/buffer

  • Use 1×1 texture to store previous frame's exposure
  • Read in next frame for smooth interpolation

Phase 3: Advanced (Match UE5)

• Histogram-based exposure

  • Build 64-bin luminance histogram
  • Find median or 80th percentile luminance
  • More robust than average

• EV100 system

  float CalculateEV100(float aperture, float shutterSpeed, float ISO)
  {
      return log2((aperture * aperture * shutterSpeed * 100.0) / ISO);
  }
  
  float EV100ToExposure(float EV100)
  {
      return exp2(EV100 - 3.0); // -3 for calibration
  }
  

• Pre-exposure application

  • Apply exposure during lighting pass
  • Modify light intensity by pre-exposure value

• ACES tone mapping

  • Use ACES color space transforms
  • Industry-standard color accuracy

Phase 4: Artist Control

• Exposure compensation

  • Allow artists to bias exposure up/down
  • Additive offset to EV100

• Min/max luminance clamps

  • Prevent adaptation from going too bright/dark
  • Useful for specific game scenarios

• Zone-based exposure

  • Weight center of screen more than edges
  • Prevents sky from overexposing character

---

Key Files Reference

C++ Source Files

Main Rendering Loop:

• Engine/Source/Runtime/Renderer/Private/DeferredShadingRenderer.cpp:1699
  • FDeferredShadingSceneRenderer::Render() - Main render loop

Post-Processing Coordination:

• Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessing.cpp
  • AddPostProcessingPasses() - Builds post-process chain
  • AddHistogramEyeAdaptationPass() - Histogram exposure calculation
  • AddBasicEyeAdaptationPass() - Simple exposure calculation

Eye Adaptation (Auto Exposure):

• Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessEyeAdaptation.cpp
  • Line 395: Physical camera EV100 calculation
  • Line 1383: FViewInfo::UpdatePreExposure() - Pre-exposure calculation
  • EV100 conversion functions

Tone Mapping:

• Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessTonemap.cpp
  • Tone mapping pass setup
  • Color grading integration
  • LUT application

Lighting:

• Engine/Source/Runtime/Renderer/Private/LightRendering.cpp
  • Deferred light rendering
  • Shadow application
  • Light accumulation in HDR

GBuffer Generation:

• Engine/Source/Runtime/Renderer/Private/BasePassRendering.cpp
  • Base pass rendering
  • Material property encoding

Shader Files (HLSL/USH)

Exposure/Eye Adaptation:

• Engine/Shaders/Private/EyeAdaptationCommon.ush
  • Exposure buffer access functions
  • EyeAdaptationLookup() - Read current exposure value

Tone Mapping:

• Engine/Shaders/Private/TonemapCommon.ush
  • Line 97: FilmToneMap() - Filmic tone curve
  • Line 415: ACES tone mapping
  • Color grading functions
  • Gamma correction

ACES:

• Engine/Shaders/Private/ACES.ush
  • Complete ACES color space implementation
  • RRT and ODT transforms

Histogram Generation:

• Engine/Shaders/Private/PostProcessHistogram.usf
  • Luminance calculation
  • Histogram bin accumulation

Deferred Shading:

• Engine/Shaders/Private/DeferredShadingCommon.ush
  • GBuffer encoding/decoding
  • Material property access

Lighting Calculations:

• Engine/Shaders/Private/DeferredLightingCommon.ush
  • BRDF functions
  • Light attenuation
  • Shadow sampling

Key Data Structures

View Information:

• Engine/Source/Runtime/Engine/Public/SceneView.h
  • FSceneView - Camera and view settings
  • Contains exposure values, FOV, projection matrices

Scene Representation:

• Engine/Source/Runtime/Renderer/Private/ScenePrivate.h
  • FScene - Scene graph and objects
  • Light arrays, primitive arrays

---

Quick Reference: The Exposure Pipeline

Frame N:
┌─────────────────────────────────────────────────────┐
│ 1. Render scene with PreExposure from Frame N-1    │
│    SceneColor = Lighting × PreExposure              │
└─────────────────────────────────────────────────────┘
                        ↓
┌─────────────────────────────────────────────────────┐
│ 2. Calculate luminance of rendered scene           │
│    - Downsample to 64×64                           │
│    - Build histogram (or calculate average)        │
└─────────────────────────────────────────────────────┘
                        ↓
┌─────────────────────────────────────────────────────┐
│ 3. Calculate target exposure for next frame        │
│    - Find median/average luminance                 │
│    - Convert to EV100                              │
│    - Apply compensation and limits                 │
│    - Smooth with previous exposure                 │
└─────────────────────────────────────────────────────┘
                        ↓
┌─────────────────────────────────────────────────────┐
│ 4. Apply tone mapping to current frame             │
│    - Apply additional exposure if needed           │
│    - Run tone mapping curve (Filmic/ACES)          │
│    - Color grading                                 │
│    - Gamma correction                              │
└─────────────────────────────────────────────────────┘
                        ↓
                Display Frame N

---

Console Commands for Testing

When studying UE5, try these console commands to see how exposure affects the scene:

// Disable auto exposure to see raw HDR
r.EyeAdaptation.MethodOverride 0
r.EyeAdaptation.PreExposureOverride 1.0

// Now manually adjust exposure
r.EyeAdaptation.PreExposureOverride 0.5  // Darker
r.EyeAdaptation.PreExposureOverride 2.0  // Brighter
r.EyeAdaptation.PreExposureOverride 10.0 // Very bright

// Re-enable auto exposure
r.EyeAdaptation.MethodOverride -1

// Test different tone mappers
r.Tonemapper.Quality 0  // Legacy
r.Tonemapper.Quality 5  // Full quality with ACES

// Show debug histogram
r.EyeAdaptation.VisualizeDebugType 1

---

Summary: Why Your Engine Shows "Full Bright or Completely Dark"

Your game engine likely has one or more of these issues:

1. Missing Auto Exposure

   • Exposure value is fixed (probably 1.0)
   • Bright scenes overflow to white
   • Dark scenes underflow to black

2. Missing Tone Mapping

   • HDR values > 1.0 get clamped to white
   • No smooth rolloff for highlights
   • All bright lights look the same

3. Missing Gamma Correction

   • Linear light values appear darker on screen
   • Should apply gamma 2.2 for sRGB displays

4. Incorrect Light Units

   • Light intensities might be in wrong units
   • Too bright or too dark by default

The minimal fix:

// Pseudocode for post-processing
float avgLuminance = CalculateAverageLuminance(sceneColor);
float exposure = 0.18 / (avgLuminance + 0.001);
exposure = clamp(exposure, 0.1, 10.0);

float3 exposedColor = sceneColor.rgb * exposure;
float3 toneMapped = exposedColor / (exposedColor + 1.0); // Reinhard
float3 gammaCorrected = pow(toneMapped, 1.0/2.2);

outputColor = gammaCorrected;

This simple change will prevent "full bright or completely dark" and give you a starting point to build a more sophisticated system like UE5's.

---

Further Study

Recommended reading order:

1. Start with: PostProcessEyeAdaptation.cpp - Understand exposure calculation
2. Then read: TonemapCommon.ush - See actual tone mapping shaders
3. Study: PostProcessing.cpp - See how passes connect
4. Finally: DeferredShadingRenderer.cpp:1699 - Full render loop

Key concepts to understand:

• HDR vs LDR rendering
• EV100 exposure system
• Histogram-based exposure
• Temporal smoothing
• Pre-exposure optimization
• Tone mapping curves (Filmic, ACES)
• Gamma correction and color spaces

Good luck with your game engine! The difference between "full bright or completely dark" and "properly exposed" is entirely in these post-processing steps.