Implement opt-in sharp screen-space reflections for the deferred rend…

…erer, with improved raymarching code. (#13418)

This commit, a revamp of #12959, implements screen-space reflections
(SSR), which approximate real-time reflections based on raymarching
through the depth buffer and copying samples from the final rendered
frame. This patch is a relatively minimal implementation of SSR, so as
to provide a flexible base on which to customize and build in the
future. However, it's based on the production-quality [raymarching code
by Tomasz
Stachowiak](https://gist.github.com/h3r2tic/9c8356bdaefbe80b1a22ae0aaee192db).

For a general basic overview of screen-space reflections, see
[1](https://lettier.github.io/3d-game-shaders-for-beginners/screen-space-reflection.html).
The raymarching shader uses the basic algorithm of tracing forward in
large steps, refining that trace in smaller increments via binary
search, and then using the secant method. No temporal filtering or
roughness blurring, is performed at all; for this reason, SSR currently
only operates on very shiny surfaces. No acceleration via the
hierarchical Z-buffer is implemented (though note that
#12899 will add the
infrastructure for this). Reflections are traced at full resolution,
which is often considered slow. All of these improvements and more can
be follow-ups.

SSR is built on top of the deferred renderer and is currently only
supported in that mode. Forward screen-space reflections are possible
albeit uncommon (though e.g. *Doom Eternal* uses them); however, they
require tracing from the previous frame, which would add complexity.
This patch leaves the door open to implementing SSR in the forward
rendering path but doesn't itself have such an implementation.
Screen-space reflections aren't supported in WebGL 2, because they
require sampling from the depth buffer, which Naga can't do because of a
bug (`sampler2DShadow` is incorrectly generated instead of `sampler2D`;
this is the same reason why depth of field is disabled on that
platform).

To add screen-space reflections to a camera, use the
`ScreenSpaceReflectionsBundle` bundle or the
`ScreenSpaceReflectionsSettings` component. In addition to
`ScreenSpaceReflectionsSettings`, `DepthPrepass` and `DeferredPrepass`
must also be present for the reflections to show up. The
`ScreenSpaceReflectionsSettings` component contains several settings
that artists can tweak, and also comes with sensible defaults.

A new example, `ssr`, has been added. It's loosely based on the
[three.js ocean
sample](https://threejs.org/examples/webgl_shaders_ocean.html), but all
the assets are original. Note that the three.js demo has no screen-space
reflections and instead renders a mirror world. In contrast to #12959,
this demo tests not only a cube but also a more complex model (the
flight helmet).

## Changelog

### Added

* Screen-space reflections can be enabled for very smooth surfaces by
adding the `ScreenSpaceReflections` component to a camera. Deferred
rendering must be enabled for the reflections to appear.

![Screenshot 2024-05-18
143555](https://github.com/bevyengine/bevy/assets/157897/b8675b39-8a89-433e-a34e-1b9ee1233267)

![Screenshot 2024-05-18
143606](https://github.com/bevyengine/bevy/assets/157897/cc9e1cd0-9951-464a-9a08-e589210e5606)

Cargo.toml

@@ -3038,6 +3038,17 @@ description = "Demonstrates visibility ranges"
 category = "3D Rendering"
 wasm = true
 
+[[example]]
+name = "ssr"
+path = "examples/3d/ssr.rs"
+doc-scrape-examples = true
+
+[package.metadata.example.ssr]
+name = "Screen Space Reflections"
+description = "Demonstrates screen space reflections with water ripples"
+category = "3D Rendering"
+wasm = false
+
 [[example]]
 name = "color_grading"
 path = "examples/3d/color_grading.rs"

assets/shaders/water_material.wgsl

@@ -0,0 +1,59 @@
+// A shader that creates water ripples by overlaying 4 normal maps on top of one
+// another.
+//
+// This is used in the `ssr` example. It only supports deferred rendering.
+
+#import bevy_pbr::{
+    pbr_deferred_functions::deferred_output,
+    pbr_fragment::pbr_input_from_standard_material,
+    prepass_io::{VertexOutput, FragmentOutput},
+}
+#import bevy_render::globals::Globals
+
+// Parameters to the water shader.
+struct WaterSettings {
+    // How much to displace each octave each frame, in the u and v directions.
+    // Two octaves are packed into each `vec4`.
+    octave_vectors: array<vec4<f32>, 2>,
+    // How wide the waves are in each octave.
+    octave_scales: vec4<f32>,
+    // How high the waves are in each octave.
+    octave_strengths: vec4<f32>,
+}
+
+@group(0) @binding(1) var<uniform> globals: Globals;
+
+@group(2) @binding(100) var water_normals_texture: texture_2d<f32>;
+@group(2) @binding(101) var water_normals_sampler: sampler;
+@group(2) @binding(102) var<uniform> water_settings: WaterSettings;
+
+// Samples a single octave of noise and returns the resulting normal.
+fn sample_noise_octave(uv: vec2<f32>, strength: f32) -> vec3<f32> {
+    let N = textureSample(water_normals_texture, water_normals_sampler, uv).rbg * 2.0 - 1.0;
+    // This isn't slerp, but it's good enough.
+    return normalize(mix(vec3(0.0, 1.0, 0.0), N, strength)); 
+}
+
+// Samples all four octaves of noise and returns the resulting normal.
+fn sample_noise(uv: vec2<f32>, time: f32) -> vec3<f32> {
+    let uv0 = uv * water_settings.octave_scales[0] + water_settings.octave_vectors[0].xy * time;
+    let uv1 = uv * water_settings.octave_scales[1] + water_settings.octave_vectors[0].zw * time;
+    let uv2 = uv * water_settings.octave_scales[2] + water_settings.octave_vectors[1].xy * time;
+    let uv3 = uv * water_settings.octave_scales[3] + water_settings.octave_vectors[1].zw * time;
+    return normalize(
+        sample_noise_octave(uv0, water_settings.octave_strengths[0]) +
+        sample_noise_octave(uv1, water_settings.octave_strengths[1]) +
+        sample_noise_octave(uv2, water_settings.octave_strengths[2]) +
+        sample_noise_octave(uv3, water_settings.octave_strengths[3])
+    );
+}
+
+@fragment
+fn fragment(in: VertexOutput, @builtin(front_facing) is_front: bool) -> FragmentOutput {
+    // Create the PBR input.
+    var pbr_input = pbr_input_from_standard_material(in, is_front);
+    // Bump the normal.
+    pbr_input.N = sample_noise(in.uv, globals.time);
+    // Send the rest to the deferred shader.
+    return deferred_output(in, pbr_input);
+}

crates/bevy_core_pipeline/src/core_3d/mod.rs

@@ -41,6 +41,26 @@ pub mod graph {
 // PERF: vulkan docs recommend using 24 bit depth for better performance
 pub const CORE_3D_DEPTH_FORMAT: TextureFormat = TextureFormat::Depth32Float;
 
+/// True if multisampled depth textures are supported on this platform.
+///
+/// In theory, Naga supports depth textures on WebGL 2. In practice, it doesn't,
+/// because of a silly bug whereby Naga assumes that all depth textures are
+/// `sampler2DShadow` and will cheerfully generate invalid GLSL that tries to
+/// perform non-percentage-closer-filtering with such a sampler. Therefore we
+/// disable depth of field and screen space reflections entirely on WebGL 2.
+#[cfg(all(feature = "webgl", target_arch = "wasm32", not(feature = "webgpu")))]
+pub const DEPTH_TEXTURE_SAMPLING_SUPPORTED: bool = false;
+
+/// True if multisampled depth textures are supported on this platform.
+///
+/// In theory, Naga supports depth textures on WebGL 2. In practice, it doesn't,
+/// because of a silly bug whereby Naga assumes that all depth textures are
+/// `sampler2DShadow` and will cheerfully generate invalid GLSL that tries to
+/// perform non-percentage-closer-filtering with such a sampler. Therefore we
+/// disable depth of field and screen space reflections entirely on WebGL 2.
+#[cfg(any(feature = "webgpu", not(target_arch = "wasm32")))]
+pub const DEPTH_TEXTURE_SAMPLING_SUPPORTED: bool = true;
+
 use std::ops::Range;
 
 use bevy_asset::AssetId;

crates/bevy_core_pipeline/src/dof/mod.rs

@@ -56,7 +56,7 @@ use smallvec::SmallVec;
 use crate::{
     core_3d::{
         graph::{Core3d, Node3d},
-        Camera3d,
+        Camera3d, DEPTH_TEXTURE_SAMPLING_SUPPORTED,
     },
     fullscreen_vertex_shader::fullscreen_shader_vertex_state,
 };
@@ -883,23 +883,3 @@ impl DepthOfFieldPipelines {
         }
     }
 }
-
-/// Returns true if multisampled depth textures are supported on this platform.
-///
-/// In theory, Naga supports depth textures on WebGL 2. In practice, it doesn't,
-/// because of a silly bug whereby Naga assumes that all depth textures are
-/// `sampler2DShadow` and will cheerfully generate invalid GLSL that tries to
-/// perform non-percentage-closer-filtering with such a sampler. Therefore we
-/// disable depth of field entirely on WebGL 2.
-#[cfg(all(feature = "webgl", target_arch = "wasm32", not(feature = "webgpu")))]
-const DEPTH_TEXTURE_SAMPLING_SUPPORTED: bool = false;
-
-/// Returns true if multisampled depth textures are supported on this platform.
-///
-/// In theory, Naga supports depth textures on WebGL 2. In practice, it doesn't,
-/// because of a silly bug whereby Naga assumes that all depth textures are
-/// `sampler2DShadow` and will cheerfully generate invalid GLSL that tries to
-/// perform non-percentage-closer-filtering with such a sampler. Therefore we
-/// disable depth of field entirely on WebGL 2.
-#[cfg(any(feature = "webgpu", not(target_arch = "wasm32")))]
-const DEPTH_TEXTURE_SAMPLING_SUPPORTED: bool = true;

crates/bevy_core_pipeline/src/tonemapping/tonemapping_shared.wgsl

@@ -11,8 +11,8 @@
     @group(0) @binding(3) var dt_lut_texture: texture_3d<f32>;
     @group(0) @binding(4) var dt_lut_sampler: sampler;
 #else
-    @group(0) @binding(19) var dt_lut_texture: texture_3d<f32>;
-    @group(0) @binding(20) var dt_lut_sampler: sampler;
+    @group(0) @binding(20) var dt_lut_texture: texture_3d<f32>;
+    @group(0) @binding(21) var dt_lut_sampler: sampler;
 #endif
 
 // Half the size of the crossfade region between shadows and midtones and

crates/bevy_pbr/src/deferred/mod.rs

@@ -1,7 +1,8 @@
 use crate::{
     graph::NodePbr, irradiance_volume::IrradianceVolume, prelude::EnvironmentMapLight,
     MeshPipeline, MeshViewBindGroup, RenderViewLightProbes, ScreenSpaceAmbientOcclusionSettings,
-    ViewLightProbesUniformOffset,
+    ScreenSpaceReflectionsUniform, ViewLightProbesUniformOffset,
+    ViewScreenSpaceReflectionsUniformOffset,
 };
 use bevy_app::prelude::*;
 use bevy_asset::{load_internal_asset, Handle};
@@ -147,6 +148,7 @@ impl ViewNode for DeferredOpaquePass3dPbrLightingNode {
         &'static ViewLightsUniformOffset,
         &'static ViewFogUniformOffset,
         &'static ViewLightProbesUniformOffset,
+        &'static ViewScreenSpaceReflectionsUniformOffset,
         &'static MeshViewBindGroup,
         &'static ViewTarget,
         &'static DeferredLightingIdDepthTexture,
@@ -162,6 +164,7 @@ impl ViewNode for DeferredOpaquePass3dPbrLightingNode {
             view_lights_offset,
             view_fog_offset,
             view_light_probes_offset,
+            view_ssr_offset,
             mesh_view_bind_group,
             target,
             deferred_lighting_id_depth_texture,
@@ -216,6 +219,7 @@ impl ViewNode for DeferredOpaquePass3dPbrLightingNode {
                 view_lights_offset.offset,
                 view_fog_offset.offset,
                 **view_light_probes_offset,
+                **view_ssr_offset,
             ],
         );
         render_pass.set_bind_group(1, &bind_group_1, &[]);
@@ -260,7 +264,7 @@ impl SpecializedRenderPipeline for DeferredLightingLayout {
             } else if method == MeshPipelineKey::TONEMAP_METHOD_REINHARD_LUMINANCE {
                 shader_defs.push("TONEMAP_METHOD_REINHARD_LUMINANCE".into());
             } else if method == MeshPipelineKey::TONEMAP_METHOD_ACES_FITTED {
-                shader_defs.push("TONEMAP_METHOD_ACES_FITTED ".into());
+                shader_defs.push("TONEMAP_METHOD_ACES_FITTED".into());
             } else if method == MeshPipelineKey::TONEMAP_METHOD_AGX {
                 shader_defs.push("TONEMAP_METHOD_AGX".into());
             } else if method == MeshPipelineKey::TONEMAP_METHOD_SOMEWHAT_BORING_DISPLAY_TRANSFORM {
@@ -301,6 +305,10 @@ impl SpecializedRenderPipeline for DeferredLightingLayout {
             shader_defs.push("MOTION_VECTOR_PREPASS".into());
         }
 
+        if key.contains(MeshPipelineKey::SCREEN_SPACE_REFLECTIONS) {
+            shader_defs.push("SCREEN_SPACE_REFLECTIONS".into());
+        }
+
         // Always true, since we're in the deferred lighting pipeline
         shader_defs.push("DEFERRED_PREPASS".into());
 
@@ -406,7 +414,10 @@ pub fn prepare_deferred_lighting_pipelines(
             Option<&Tonemapping>,
             Option<&DebandDither>,
             Option<&ShadowFilteringMethod>,
-            Has<ScreenSpaceAmbientOcclusionSettings>,
+            (
+                Has<ScreenSpaceAmbientOcclusionSettings>,
+                Has<ScreenSpaceReflectionsUniform>,
+            ),
             (
                 Has<NormalPrepass>,
                 Has<DepthPrepass>,
@@ -424,7 +435,7 @@ pub fn prepare_deferred_lighting_pipelines(
         tonemapping,
         dither,
         shadow_filter_method,
-        ssao,
+        (ssao, ssr),
         (normal_prepass, depth_prepass, motion_vector_prepass),
         has_environment_maps,
         has_irradiance_volumes,
@@ -473,6 +484,9 @@ pub fn prepare_deferred_lighting_pipelines(
         if ssao {
             view_key |= MeshPipelineKey::SCREEN_SPACE_AMBIENT_OCCLUSION;
         }
+        if ssr {
+            view_key |= MeshPipelineKey::SCREEN_SPACE_REFLECTIONS;
+        }
 
         // We don't need to check to see whether the environment map is loaded
         // because [`gather_light_probes`] already checked that for us before

crates/bevy_pbr/src/lib.rs

@@ -34,6 +34,7 @@ mod pbr_material;
 mod prepass;
 mod render;
 mod ssao;
+mod ssr;
 mod volumetric_fog;
 
 use bevy_color::{Color, LinearRgba};
@@ -51,6 +52,7 @@ pub use pbr_material::*;
 pub use prepass::*;
 pub use render::*;
 pub use ssao::*;
+pub use ssr::*;
 pub use volumetric_fog::*;
 
 pub mod prelude {
@@ -87,6 +89,8 @@ pub mod graph {
         VolumetricFog,
         /// Label for the compute shader instance data building pass.
         GpuPreprocess,
+        /// Label for the screen space reflections pass.
+        ScreenSpaceReflections,
     }
 }
 
@@ -319,6 +323,7 @@ impl Plugin for PbrPlugin {
                     use_gpu_instance_buffer_builder: self.use_gpu_instance_buffer_builder,
                 },
                 VolumetricFogPlugin,
+                ScreenSpaceReflectionsPlugin,
             ))
             .configure_sets(
                 PostUpdate,

crates/bevy_pbr/src/meshlet/material_draw_nodes.rs

@@ -8,7 +8,7 @@ use super::{
 };
 use crate::{
     MeshViewBindGroup, PrepassViewBindGroup, PreviousViewUniformOffset, ViewFogUniformOffset,
-    ViewLightProbesUniformOffset, ViewLightsUniformOffset,
+    ViewLightProbesUniformOffset, ViewLightsUniformOffset, ViewScreenSpaceReflectionsUniformOffset,
 };
 use bevy_core_pipeline::prepass::ViewPrepassTextures;
 use bevy_ecs::{query::QueryItem, world::World};
@@ -35,6 +35,7 @@ impl ViewNode for MeshletMainOpaquePass3dNode {
         &'static ViewLightsUniformOffset,
         &'static ViewFogUniformOffset,
         &'static ViewLightProbesUniformOffset,
+        &'static ViewScreenSpaceReflectionsUniformOffset,
         &'static MeshletViewMaterialsMainOpaquePass,
         &'static MeshletViewBindGroups,
         &'static MeshletViewResources,
@@ -52,6 +53,7 @@ impl ViewNode for MeshletMainOpaquePass3dNode {
             view_lights_offset,
             view_fog_offset,
             view_light_probes_offset,
+            view_ssr_offset,
             meshlet_view_materials,
             meshlet_view_bind_groups,
             meshlet_view_resources,
@@ -103,6 +105,7 @@ impl ViewNode for MeshletMainOpaquePass3dNode {
                 view_lights_offset.offset,
                 view_fog_offset.offset,
                 **view_light_probes_offset,
+                **view_ssr_offset,
             ],
         );
         render_pass.set_bind_group(1, meshlet_material_draw_bind_group, &[]);

crates/bevy_pbr/src/render/mesh.rs

@@ -1342,7 +1342,8 @@ bitflags::bitflags! {
         const LIGHTMAPPED                       = 1 << 13;
         const IRRADIANCE_VOLUME                 = 1 << 14;
         const VISIBILITY_RANGE_DITHER           = 1 << 15;
-        const LAST_FLAG                         = Self::VISIBILITY_RANGE_DITHER.bits();
+        const SCREEN_SPACE_REFLECTIONS          = 1 << 16;
+        const LAST_FLAG                         = Self::SCREEN_SPACE_REFLECTIONS.bits();
 
         // Bitfields
         const MSAA_RESERVED_BITS                = Self::MSAA_MASK_BITS << Self::MSAA_SHIFT_BITS;
@@ -1676,7 +1677,7 @@ impl SpecializedMeshPipeline for MeshPipeline {
             } else if method == MeshPipelineKey::TONEMAP_METHOD_REINHARD_LUMINANCE {
                 shader_defs.push("TONEMAP_METHOD_REINHARD_LUMINANCE".into());
             } else if method == MeshPipelineKey::TONEMAP_METHOD_ACES_FITTED {
-                shader_defs.push("TONEMAP_METHOD_ACES_FITTED ".into());
+                shader_defs.push("TONEMAP_METHOD_ACES_FITTED".into());
             } else if method == MeshPipelineKey::TONEMAP_METHOD_AGX {
                 shader_defs.push("TONEMAP_METHOD_AGX".into());
             } else if method == MeshPipelineKey::TONEMAP_METHOD_SOMEWHAT_BORING_DISPLAY_TRANSFORM {
@@ -1923,14 +1924,15 @@ impl<P: PhaseItem, const I: usize> RenderCommand<P> for SetMeshViewBindGroup<I>
         Read<ViewLightsUniformOffset>,
         Read<ViewFogUniformOffset>,
         Read<ViewLightProbesUniformOffset>,
+        Read<ViewScreenSpaceReflectionsUniformOffset>,
         Read<MeshViewBindGroup>,
     );
     type ItemQuery = ();
 
     #[inline]
     fn render<'w>(
         _item: &P,
-        (view_uniform, view_lights, view_fog, view_light_probes, mesh_view_bind_group): ROQueryItem<
+        (view_uniform, view_lights, view_fog, view_light_probes, view_ssr, mesh_view_bind_group): ROQueryItem<
             'w,
             Self::ViewQuery,
         >,
@@ -1946,6 +1948,7 @@ impl<P: PhaseItem, const I: usize> RenderCommand<P> for SetMeshViewBindGroup<I>
                 view_lights.offset,
                 view_fog.offset,
                 **view_light_probes,
+                **view_ssr,
             ],
         );