🧠 Rendering Fundamentals

🎯 The Art of Simulating Light

Realistic rendering simulates how light interacts with surfaces, volumes, and materials to produce photorealistic images. It combines light transport, material models, camera simulation, and post-processing. The goal: make virtual objects indistinguishable from reality.

The Rendering Equation

The Rendering Equation (Kajiya, 1986): L_o(p, ω_o) = L_e(p, ω_o) + ∫ f_r(p, ω_i, ω_o) L_i(p, ω_i) (n · ω_i) dω_i. It describes how light leaves a surface. Solving this equation (approximately) is the core of realistic rendering — via rasterization, ray tracing, or path tracing.

📌 Light Transport📌 BRDF📌 Rasterization📌 Ray Tracing

// The rendering equation (GLSL pseudocode)
vec3 render(vec3 position, vec3 normal, vec3 viewDir) {
    vec3 color = vec3(0.0);
    
    // Direct lighting
    for (int i = 0; i < numLights; i++) {
        vec3 lightDir = normalize(lights[i].position - position);
        vec3 lightColor = lights[i].color * lights[i].intensity;
        
        // BRDF: diffuse + specular
        vec3 brdf = diffuse(normal, lightDir, viewDir) + specular(normal, lightDir, viewDir);
        
        // Cosine falloff
        float cosTheta = max(dot(normal, lightDir), 0.0);
        color += brdf * lightColor * cosTheta;
    }
    
    // Indirect lighting (ambient, IBL, GI)
    color += indirectLighting(position, normal, viewDir);
    
    return color;
}

🎨 Physically Based Rendering (PBR)

🎯 Energy-conserving materials

PBR is a shading model that conserves energy and follows physical laws. It uses parameters like albedo, metallic, roughness, and normal maps to describe materials consistently across lighting conditions.

The PBR Workflow

Albedo: Base color (diffuse). Metallic: 0 = dielectric, 1 = metal. Roughness: 0 = mirror, 1 = matte. Normal: Surface detail. AO: Ambient occlusion. Height/Displacement: Geometric detail. PBR ensures materials look correct under any lighting.

🎨 Albedo🔩 Metallic📏 Roughness🧭 Normal

// PBR shader (Cook-Torrance BRDF)
vec3 pbrShading(vec3 albedo, float metallic, float roughness, vec3 N, vec3 V, vec3 L, vec3 lightColor) {
    vec3 H = normalize(V + L);
    
    // Fresnel (Schlick approximation)
    vec3 F0 = mix(vec3(0.04), albedo, metallic);
    vec3 F = F0 + (1.0 - F0) * pow(1.0 - max(dot(H, V), 0.0), 5.0);
    
    // Normal Distribution (GGX/Trowbridge-Reitz)
    float a = roughness * roughness;
    float a2 = a * a;
    float NdotH = max(dot(N, H), 0.0);
    float NdotH2 = NdotH * NdotH;
    float denom = NdotH2 * (a2 - 1.0) + 1.0;
    float D = a2 / (PI * denom * denom);
    
    // Geometry (Smith)
    float k = (roughness + 1.0) * (roughness + 1.0) / 8.0;
    float G = G_SchlickGGX(max(dot(N, V), 0.0), k) * G_SchlickGGX(max(dot(N, L), 0.0), k);
    
    // Specular
    vec3 specular = (D * F * G) / (4.0 * max(dot(N, V), 0.0) * max(dot(N, L), 0.0) + 0.001);
    
    // Diffuse (Lambert)
    vec3 kD = (1.0 - F) * (1.0 - metallic);
    vec3 diffuse = kD * albedo / PI;
    
    return (diffuse + specular) * lightColor * max(dot(N, L), 0.0);
}

💡 Lighting Models

🎯 Direct and indirect light

Lighting models simulate how light sources interact with surfaces: direct lighting (from sources), indirect lighting (bounced), ambient (global), and image-based lighting (IBL).

Light Types

Directional: Sun (parallel rays). Point: Omnidirectional (bulb). Spot: Cone-shaped (flashlight). Area: Rectangular/disk (soft shadows). IBL: Environment map as light source. GI: Global illumination for realistic bounce light.

☀️ Directional💡 Point🔦 Spot🌐 IBL🌍 GI

// Punctual light attenuation (point light)
float distanceAttenuation(float distance, float range) {
    float d = max(distance, 0.001);
    float att = 1.0 / (d * d);
    // Smooth falloff at range
    float factor = clamp(1.0 - pow(d / range, 4.0), 0.0, 1.0);
    return att * factor * factor;
}

// Spot light
float spotAttenuation(vec3 L, vec3 spotDir, float outerCos, float innerCos) {
    float cosAngle = dot(-L, spotDir);
    return smoothstep(outerCos, innerCos, cosAngle);
}

🌑 Shadows & Shadow Mapping

🎯 Grounding objects in the scene

Shadows are critical for realism — they ground objects and convey depth. Techniques: shadow mapping, shadow volumes, ray-traced shadows, and contact shadows.

Shadow Mapping

Render depth from light's view → store in shadow map. Compare fragment depth with shadow map to determine if in shadow. PCF (Percentage Closer Filtering) softens edges. CSM (Cascaded Shadow Maps) for large scenes. Ray-traced shadows are perfectly accurate.

🗺️ Shadow Map🔍 PCF🏔️ CSM⚡ Ray-Traced

// Shadow mapping (GLSL)
float shadowCalculation(vec4 fragPosLightSpace, vec3 normal, vec3 lightDir) {
    // Perspective divide
    vec3 projCoords = fragPosLightSpace.xyz / fragPosLightSpace.w;
    projCoords = projCoords * 0.5 + 0.5;
    
    // Get closest depth from light's perspective
    float closestDepth = texture(shadowMap, projCoords.xy).r;
    float currentDepth = projCoords.z;
    
    // Bias to prevent shadow acne
    float bias = max(0.05 * (1.0 - dot(normal, lightDir)), 0.005);
    
    // PCF for soft shadows
    float shadow = 0.0;
    vec2 texelSize = 1.0 / textureSize(shadowMap, 0);
    for (int x = -1; x <= 1; x++) {
        for (int y = -1; y <= 1; y++) {
            float pcfDepth = texture(shadowMap, projCoords.xy + vec2(x, y) * texelSize).r;
            shadow += currentDepth - bias > pcfDepth ? 1.0 : 0.0;
        }
    }
    return shadow / 9.0;
}

🪞 Reflections & IBL

🎯 Mirror-like and glossy reflections

Reflections are essential for realistic materials (metals, glass, water). Techniques: reflection probes, planar reflections, screen-space reflections (SSR), and ray-traced reflections.

Reflection Techniques

Reflection probes: Pre-baked cubemaps for static reflections. Planar reflections: Mirror the scene across a plane (floors, water). SSR: Ray-march in screen space — dynamic but limited to visible geometry. Ray-traced: Perfect reflections, expensive. IBL: Environment map for ambient reflections.

🔮 Reflection Probes🪞 Planar📺 SSR⚡ Ray-Traced

// Screen-space reflection (SSR) in GLSL
vec3 screenSpaceReflection(vec3 fragPos, vec3 normal, vec3 viewDir) {
    vec3 reflectDir = reflect(-viewDir, normal);
    vec3 rayPos = fragPos;
    vec3 rayStep = reflectDir * stepSize;
    
    for (int i = 0; i < maxSteps; i++) {
        rayPos += rayStep;
        vec4 projected = projection * view * vec4(rayPos, 1.0);
        vec2 uv = projected.xy / projected.w * 0.5 + 0.5;
        
        if (uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0) break;
        
        float sceneDepth = texture(depthMap, uv).r;
        float rayDepth = projected.z / projected.w * 0.5 + 0.5;
        
        if (rayDepth > sceneDepth) {
            return texture(sceneColor, uv).rgb;
        }
    }
    return vec3(0.0);
}

🧱 Materials & Textures

🎯 Defining surface appearance

Materials define how surfaces interact with light. Textures provide spatial variation: albedo, normal, roughness, metallic, height, AO.

Texture Maps

Albedo/Diffuse: Base color. Normal: Fake geometric detail (RGB = XYZ). Roughness: Microsurface variation. Metallic: Metal vs dielectric. Height/Displacement: Actual geometric detail. AO: Cavity shadows. Emissive: Self-illumination.

🎨 Albedo🧭 Normal📏 Roughness🔩 Metallic

// Sampling PBR textures (GLSL)
uniform sampler2D albedoMap;
uniform sampler2D normalMap;
uniform sampler2D roughnessMap;
uniform sampler2D metallicMap;
uniform sampler2D aoMap;

void main() {
    vec3 albedo = texture(albedoMap, uv).rgb;
    vec3 normal = texture(normalMap, uv).rgb * 2.0 - 1.0;
    normal = normalize(TBN * normal);  // transform to world space
    float roughness = texture(roughnessMap, uv).r;
    float metallic = texture(metallicMap, uv).r;
    float ao = texture(aoMap, uv).r;
    
    vec3 color = pbrShading(albedo, metallic, roughness, normal, viewDir, lightDir, lightColor);
    color *= ao;
}

✨ Post-Processing Effects

🎯 The final polish

Post-processing adds cinematic quality: bloom, depth of field, motion blur, color grading, vignette, chromatic aberration, and film grain.

Post-Processing Stack

Bloom: Glow around bright areas. DOF: Focus on specific depth. Motion Blur: Camera/object movement. Color Grading: LUTs for cinematic look. Tone Mapping: HDR → LDR (ACES, Reinhard). Anti-Aliasing: TAA, FXAA, MSAA.

🌟 Bloom📸 DOF💨 Motion Blur🎨 Color Grading

// Bloom (GLSL)
vec3 bloom(vec2 uv) {
    vec3 color = vec3(0.0);
    float totalWeight = 0.0;
    
    // Gaussian blur samples
    for (int x = -bloomRadius; x <= bloomRadius; x++) {
        for (int y = -bloomRadius; y <= bloomRadius; y++) {
            vec2 offset = vec2(x, y) * texelSize;
            float weight = exp(-(x*x + y*y) / (2.0 * sigma * sigma));
            vec3 sample = texture(brightPass, uv + offset).rgb;
            color += sample * weight;
            totalWeight += weight;
        }
    }
    return color / totalWeight;
}

// Tone mapping (ACES)
vec3 ACESToneMapping(vec3 color) {
    float a = 2.51; float b = 0.03; float c = 2.43; float d = 0.59; float e = 0.14;
    return clamp((color * (a * color + b)) / (color * (c * color + d) + e), 0.0, 1.0);
}

🌍 Global Illumination

🎯 Realistic light bounce

Global Illumination (GI) simulates indirect light bouncing off surfaces. Techniques: lightmaps (baked), irradiance probes, voxel GI, SSGI, and path tracing.

GI Methods

Baked lightmaps: Pre-computed, static only. Irradiance probes: Spherical harmonics for dynamic objects. Voxel GI: Voxelized scene for dynamic GI. SSGI: Screen-space GI (approximate). Path tracing: Ground truth, real-time with RTX.

🗺️ Lightmaps🔮 Probes🧊 Voxel GI⚡ Path Tracing

// Irradiance probe sampling (spherical harmonics)
vec3 sampleIrradiance(vec3 normal) {
    // SH coefficients (L2 band)
    vec4 shAr = texture(shArMap, probeUV);
    vec4 shAg = texture(shAgMap, probeUV);
    vec4 shAb = texture(shAbMap, probeUV);
    
    // Evaluate SH at normal direction
    vec3 irradiance;
    irradiance.r = dot(shAr, shBasis(normal));
    irradiance.g = dot(shAg, shBasis(normal));
    irradiance.b = dot(shAb, shBasis(normal));
    
    return max(irradiance, 0.0);
}

vec4 shBasis(vec3 n) {
    return vec4(0.282095,
                0.488603 * n.y,
                0.488603 * n.z,
                0.488603 * n.x);
}

⚡ Real-Time Ray Tracing

🎯 Photorealistic light transport

Real-time ray tracing (RTX, DXR, Vulkan RT) traces rays from the camera to simulate reflections, shadows, and GI accurately. Used in games (Cyberpunk 2077, Minecraft RTX) and AR/VR.

Ray Tracing Techniques

Ray-traced reflections: Perfect mirror/glossy. Ray-traced shadows: Soft, accurate. Ray-traced GI: One-bounce indirect. Path tracing: Full global illumination (offline or DLSS-accelerated). Denoising: AI (DLSS, OptiX) to clean up noise.

🪞 Reflections🌑 Shadows🌍 GI🧠 DLSS

// Ray tracing with DXR (HLSL)
[shader("raygeneration")]
void RayGen() {
    uint3 launchIndex = DispatchRaysIndex();
    RayDesc ray;
    ray.Origin = cameraPos;
    ray.Direction = computeRayDirection(launchIndex);
    ray.TMin = 0.01;
    ray.TMax = 1000.0;
    
    Payload payload;
    TraceRay(scene, RAY_FLAG_NONE, 0xFF, 0, 0, 0, ray, payload);
    
    // Write result
    output[launchIndex.xy] = payload.color;
}

[shader("closesthit")]
void ClosestHit(inout Payload payload, in BuiltInTriangleIntersectionAttributes attr) {
    // Compute hit position, normal, material
    float3 barycentrics = float3(1.0 - attr.barycentrics.x - attr.barycentrics.y,
                                  attr.barycentrics.x, attr.barycentrics.y);
    float3 hitPos = ObjectRayOrigin() + ObjectRayDirection() * RayTCurrent();
    float3 normal = interpolateNormal(barycentrics);
    
    // Shade with PBR
    payload.color = pbrShading(hitPos, normal, -RayDirection());
}

🌫️ Volumetrics & Atmospherics

🎯 Light in participating media

Volumetrics simulate light scattering in fog, smoke, dust, and atmosphere. Effects: god rays, fog, clouds, atmospheric scattering.

Volumetric Techniques

Ray marching: Sample density along rays. Froxels: Volumetric fog in frustum. Participating media: Absorption, scattering, emission. Atmospheric scattering: Rayleigh + Mie for sky/atmosphere.

🌫️ Fog☁️ Clouds🌟 God Rays🌅 Atmosphere

// Volumetric ray marching (GLSL)
vec3 volumetricFog(vec3 rayOrigin, vec3 rayDir, float maxDist) {
    vec3 color = vec3(0.0);
    float transmittance = 1.0;
    
    for (int i = 0; i < numSteps; i++) {
        float t = (i + 0.5) * stepSize;
        if (t > maxDist) break;
        
        vec3 pos = rayOrigin + rayDir * t;
        float density = sampleFogDensity(pos);
        
        // Light scattering
        vec3 lightDir = normalize(lightPos - pos);
        float lightAttenuation = sampleShadow(pos, lightDir);
        vec3 scattering = density * lightColor * lightAttenuation * phaseFunction(rayDir, lightDir);
        
        // Accumulate
        color += transmittance * scattering * stepSize;
        transmittance *= exp(-density * stepSize * extinction);
    }
    
    return color;
}

🧴 Subsurface Scattering

🎯 Light penetrating translucent materials

Subsurface Scattering (SSS) simulates light entering and scattering inside materials like skin, wax, marble, and leaves. It's key for realistic skin rendering.

SSS Techniques

Dipole approximation: Two-point diffusion. BSSRDF: Bidirectional scattering. Separable SSS: Real-time approximation. Burley diffusion: Disney's model. Pre-integrated skin: Texture-based lookup.

🧴 Skin🕯️ Wax🌿 Leaves💎 Marble

// Subsurface scattering (Burley diffusion)
vec3 subsurfaceScattering(vec3 N, vec3 L, vec3 V, vec3 albedo, float radius) {
    float NdotL = dot(N, L);
    float NdotV = dot(N, V);
    
    // Burley diffusion profile
    float fresnel = pow(1.0 - NdotV, 5.0);
    vec3 diffuse = albedo * (1.0 - fresnel) / PI;
    
    // Translucency (backlight)
    vec3 backLight = -L;
    float backScatter = pow(max(dot(V, backLight), 0.0), 4.0) * radius;
    
    return diffuse * lightColor * (NdotL + backScatter);
}

💇 Hair & Fur Rendering

🎯 Strands, not surfaces

Hair and fur are rendered as strands or shells, not solid surfaces. Techniques: hair strand rendering, shell texturing, card-based fur.

Hair Rendering Approaches

Strand-based: Render individual curves (Filament, UE5 Groom). Shell-based: Layered shells with alpha (fur). Card-based: Textured quads (stylized). Marschner model: Physically-based hair shading (R, TT, TRT lobes).

💇 Strands🐻 Fur Shells🃏 Cards✨ Marschner

// Marschner hair shading (simplified)
vec3 hairShading(vec3 tangent, vec3 viewDir, vec3 lightDir, vec3 hairColor) {
    // Compute angles
    float sinThetaI = dot(viewDir, tangent);
    float sinThetaO = dot(lightDir, tangent);
    float cosPhi = dot(normalize(viewDir - tangent * sinThetaI),
                       normalize(lightDir - tangent * sinThetaO));
    
    // R lobe (specular)
    vec3 R = hairColor * fresnel(sinThetaI, sinThetaO, cosPhi);
    
    // TT lobe (transmission)
    vec3 TT = hairColor * transmission(sinThetaI, sinThetaO, cosPhi);
    
    // TRT lobe (secondary specular)
    vec3 TRT = hairColor * secondarySpecular(sinThetaI, sinThetaO, cosPhi);
    
    return R + TT + TRT;
}

🥽 AR/VR Realistic Rendering

🎯 Realism in immersive contexts

AR/VR rendering must be realistic AND performant (90+ fps). Challenges: stereo rendering, lens distortion, latency, and matching real-world lighting.

AR/VR Rendering Techniques

Single-pass stereo: Render both eyes in one pass. Foveated rendering: Lower resolution in periphery. Light estimation: Match real-world lighting (AR). Occlusion: Depth-based for AR. Reprojection: ASW/ATW for low latency.

👁️ Stereo🎯 Foveated💡 Light Est.🧱 Occlusion

// Single-pass stereo rendering (Unity URP)
// In shader:
#if defined(UNITY_STEREO_INSTANCING_ENABLED)
    UNITY_SETUP_STEREO_EYE_INDEX_POST_VERTEX(input);
    // eyeIndex = 0 (left) or 1 (right)
    float4 stereoEyeOffset = unity_StereoEyeIndex == 0 ? leftEyeOffset : rightEyeOffset;
    positionCS.x += stereoEyeOffset.x;
#endif

// AR light estimation (ARCore → Unity)
void UpdateLighting(LightEstimate lightEstimate) {
    if (lightEstimate.State == LightEstimate.State.Valid) {
        directionalLight.intensity = lightEstimate.PixelIntensity;
        RenderSettings.ambientLight = lightEstimate.ColorCorrection;
    }
}

🚀 Future: Neural Rendering

🎯 AI-powered realism

Neural rendering uses deep learning to achieve photorealistic results at unprecedented speeds: NeRFs, Gaussian Splatting, DLSS, and generative rendering.

Neural Rendering Techniques

NeRF: Neural Radiance Fields — photorealistic novel view synthesis. Gaussian Splatting: Real-time photorealistic rendering. DLSS: AI upscaling and frame generation. Generative: Create 3D content from text/images. Neural BRDF: Learned material models.

🧠 NeRF✨ Gaussian Splat⚡ DLSS🎨 Generative

# NeRF rendering (PyTorch)
import torch
import torch.nn as nn

class NeRF(nn.Module):
    def __init__(self, hidden_dim=256):
        super().__init__()
        self.mlp = nn.Sequential(
            nn.Linear(3 + 3, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, 4)  # RGB + density
        )
    
    def forward(self, ray_origins, ray_directions, near, far, num_samples=64):
        # Sample points along rays
        t_vals = torch.linspace(near, far, num_samples)
        points = ray_origins[..., None, :] + ray_directions[..., None, :] * t_vals[..., :, None]
        
        # Query network
        raw = self.mlp(points)
        rgb = torch.sigmoid(raw[..., :3])
        density = torch.relu(raw[..., 3])
        
        # Volume rendering
        return volume_rendering(rgb, density, t_vals)