Implement opt-in sharp screen-space reflections for the deferred renderer. by pcwalton · Pull Request #12959 · bevyengine/bevy

pcwalton · 2024-04-14T00:50:56Z

This commit implements screen-space reflections (SSR), which approximate real-time reflections based on raymarching through the depth buffer and copying samples from the final rendered frame. Numerous variations and refinements to screen-space reflections exist in the literature. This patch foregoes all of them in favor of implementing the bare minimum, so as to provide a flexible base on which to customize and build in the future.

For a general basic overview of screen-space reflections, see 1. The raymarching shader uses the basic algorithm of tracing forward in large steps (what I call a major trace), and then refining that trace in smaller increments via binary search (what I call a minor trace). No filtering, whether temporal or spatial, is performed at all; for this reason, SSR currently only operates on very shiny surfaces. No acceleration via the hierarchical Z-buffer is implemented (though note that #12899 will add the infrastructure for this). Reflections are traced at full resolution, which is often considered slow. All of these improvements and more can be follow-ups.

SSR is built on top of the deferred renderer and is currently only supported in that mode. Forward screen-space reflections are possible albeit uncommon (though e.g. Doom Eternal uses them); however, they require tracing from the previous frame, which would add complexity. This patch leaves the door open to implementing SSR in the forward rendering path but doesn't itself have such an implementation. Screen-space reflections are supported in WebGL 2.

To add screen-space reflections to a camera, use the ScreenSpaceReflectionsBundle bundle or the ScreenSpaceReflectionsSettings component. In addition to ScreenSpaceReflectionsSettings, DepthPrepass and DeferredPrepass must also be present for the reflections to show up. The ScreenSpaceReflectionsSettings component contains several settings that artists can tweak, and also comes with sensible defaults.

A new example, ssr, has been added. It's loosely based on the three.js ocean sample, but all the assets are original. Note that the three.js demo has no screen-space reflections and instead renders a mirror world.

Additionally, this patch fixes a random bug I ran across: that the "TONEMAP_METHOD_ACES_FITTED" #define is incorrectly supplied to the shader as "TONEMAP_METHOD_ACES_FITTED " (with an extra space) in some paths.

Changelog

Added

Screen-space reflections can be enabled for very smooth surfaces by adding the ScreenSpaceReflections component to a camera. Deferred rendering must be enabled for the reflections to appear.

github-actions · 2024-04-14T01:00:57Z

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

pcwalton · 2024-04-14T02:59:12Z

This should be ready for review now.

renderer. This commit implements *screen-space reflections* (SSR), which approximate real-time reflections based on raymarching through the depth buffer and copying samples from the final rendered frame. Numerous variations and refinements to screen-space reflections exist in the literature. This patch foregoes all of them in favor of implementing the bare minimum, so as to provide a flexible base on which to customize and build in the future. For a general basic overview of screen-space reflections, see [1]. The raymarching shader uses the basic algorithm of tracing forward in large steps (what I call a *major* trace), and then refining that trace in smaller increments via binary search (what I call a *minor* trace). No filtering, whether temporal or spatial, is performed at all; for this reason, SSR currently only operates on very shiny surfaces. No acceleration via the hierarchical Z-buffer is implemented (though note that bevyengine#12899 will add the infrastructure for this). Reflections are traced at full resolution, which is often considered slow. All of these improvements and more can be follow-ups. SSR is built on top of the deferred renderer and is currently only supported in that mode. Forward screen-space reflections are possible albeit uncommon (though e.g. *Doom Eternal* uses them); however, they require tracing from the previous frame, which would add complexity. This patch leaves the door open to implementing SSR in the forward rendering path but doesn't itself have such an implementation. Screen-space reflections *are* supported in WebGL 2. To add screen-space reflections to a camera, use the `ScreenSpaceReflections` component. `DepthPrepass` and `DeferredPrepass` must also be present for the reflections to show up. The `ScreenSpaceReflections` component contains several settings that artists can tweak, and also comes with sensible defaults. A new example, `ssr`, has been added. It's loosely based on the [three.js ocean sample], but all the assets are original. Note that the three.js demo has no screen-space reflections and instead renders a mirror world. Additionally, this patch fixes a random bug I ran across: that the `"TONEMAP_METHOD_ACES_FITTED"` `#define` is incorrectly supplied to the shader as `"TONEMAP_METHOD_ACES_FITTED "` (with an extra space) in some paths. [1]: https://lettier.github.io/3d-game-shaders-for-beginners/screen-space-reflection.html [three.js ocean sample]: https://threejs.org/examples/webgl_shaders_ocean.html

crates/bevy_pbr/src/ssr/mod.rs

add a bundle

pcwalton · 2024-04-14T20:18:31Z

I renamed ScreenSpaceReflections to ScreenSpaceReflectionsSettings and added a bundle, ScreenSpaceReflectionsBundle.

Cargo.toml

aevyrie · 2024-04-18T20:51:33Z

Don't have time for a review, but saw the request. Just wanted to note that specular occlusion should use SSR if it is available. I believe I cited some papers in the shader code that talk about how to use SSR with specular occlusion. This might be out of scope, but it popped in my head when I saw this.

pcwalton · 2024-04-19T05:30:47Z

@aevyrie Yes, I agree. I think that should be a followup as it'll be a decent amount of code.

chronicl · 2024-04-22T14:25:08Z

The example crashes if any other prepasses are added to the camera, for example NormalPrepass.

 Incompatible bind group at index 0 in the current render pipeline
      note: Should be compatible an with an explicit bind group layout with label = `mesh_view_layout_depth_deferred`
      note: Assigned explicit bind group layout with label = `mesh_view_layout_depth_normal_deferred`

I believe the MeshPipelineViewLayoutKey of the ssr pipeline should be adjusted for those cases, unless I'm doing something wrong or this is intended.

pcwalton · 2024-04-23T02:13:05Z

@chronicl Fixed, thanks!

crates/bevy_pbr/src/deferred/mod.rs

crates/bevy_pbr/src/render/mesh_view_bindings.rs

JMS55

Works surprisingly well with TAA, I was expecting the reflections to cause issues.

A bit surprised that the water reflection from the envmap is part of SSR, and not just part of the PBR shader, but maybe there's a good reason for that.

Some weird shader defs / things that stand out to me about how SSR is setup on the CPU. Not sure it's entirely right (although, it does work).

This PR is going to conflict with the clearcoat PR, probably want to merge that one first.

JMS55 · 2024-05-03T02:47:21Z

crates/bevy_pbr/src/ssr/mod.rs

+/// [`DeferredPrepass`] to the camera as well.
+///
+/// SSR currently performs no roughness filtering for glossy reflections, so
+/// only very smooth surfaces will reflect objects in screen space. You can


Please link to the StandardMaterial roughness property here.

JMS55 · 2024-05-03T02:50:54Z

crates/bevy_pbr/src/ssr/mod.rs

+
+    for (view, ssr_settings) in views.iter() {
+        let uniform_offset = match ssr_settings {
+            None => 0,


What's going on here? Why not use UniformComponentPlugin or similar?

JMS55 · 2024-05-03T02:51:33Z

crates/bevy_pbr/src/ssr/mod.rs

+        let mut shader_defs = vec![
+            "DEPTH_PREPASS".into(),
+            "DEFERRED_PREPASS".into(),
+            "SSR".into(),


What's the SSR shader def here for?

JMS55 · 2024-05-03T02:52:06Z

crates/bevy_pbr/src/deferred/mod.rs

        }

+        if key.contains(MeshPipelineKey::SCREEN_SPACE_REFLECTIONS) {
+            shader_defs.push("SSR".into());


Suggested change

shader_defs.push("SSR".into());

shader_defs.push("SSR".into());

SSR -> SCREEN_SPACE_REFLECTIONS?

Yeah, good point, that would be more consistent with SSAO

JMS55 · 2024-05-03T02:52:56Z

crates/bevy_pbr/src/deferred/mod.rs

            shader_defs.push("MOTION_VECTOR_PREPASS".into());
        }

+        if key.contains(MeshPipelineKey::SCREEN_SPACE_REFLECTIONS) {


This needs to be added to meshlet/material_draw_prepare.rs too, iirc. Please test the meshlet example with SSR.

JMS55 · 2024-05-03T02:54:23Z

crates/bevy_pbr/src/ssr/mod.rs

+    {
+        // SSR is only supported in the deferred pipeline, which has no MSAA
+        // support. Thus we can assume MSAA is off.
+        let mut mesh_pipeline_view_key = MeshPipelineViewLayoutKey::from(Msaa::Off)


Why use MeshPipelineViewLayoutKey here? Also, why check for prepasses at all? All you need is the deferred prepass?

The depth prepass is also needed, otherwise depth isn't generated by the deferred pass

IceSentry

Like mentioned in discord, there's currently a bug where the image gets brighter when toggling SSR.

There also seems to be an issue with the transmission example. It's probably just a wrong ifdef.

Other than that, the code is high quality and everything looks good.

IceSentry · 2024-05-03T03:14:15Z

examples/3d/ssr.rs

+};
+
+// The speed of camera movement.
+const CAMERA_KEYBOARD_ZOOM_SPEED: f32 = 0.1;


You can do it if you feel like it but it's not required at all to merge. (I already made a note for me to do that in a follow-up PR).

We generally try to put things like camera controllers and debug text in a separate submodule so only the setup part that shows how to enable the feature is at the top.

IceSentry · 2024-05-03T04:01:22Z

@JMS55

A bit surprised that the water reflection from the envmap is part of SSR, and not just part of the PBR shader, but maybe there's a good reason for that.

Unless I'm misunderstanding what you are saying. The envmap reflection is still handled by the PBR shader when SSR is not present and when it's present it's still done when the roughness is under a certain threshold.

`SCREEN_SPACE_REFLECTIONS`

JMS55 · 2024-05-03T04:14:05Z

In the SSR example, disabling SSR turns off both the cube reflection, but also the reflection on the water from the env map, which surprises me.

pcwalton · 2024-05-03T04:14:43Z

The reason why SSR delays evaluation of the environment map until the SSR pass is that the specular part of the environment map is only taken into account if the ray misses (after all, the environment map is "infinitely" far away), and we don't know whether the ray is going to miss until we evaluate SSR.

In other words, SSR effectively doubles as raymarched occlusion for the environment map.

JMS55 · 2024-05-03T04:42:27Z

Oh I guess doesn't have to be this PR, but it's probably better to use a compute shader for SSR.

IceSentry · 2024-05-03T04:56:01Z

Yeah, definitely not this PR. I'd much rather have it as-is and introduce a compute path later. We'll also want to reuse the depth pyramid once we have that.

pcwalton · 2024-05-03T06:57:36Z

Will have to redo this later.

pcwalton · 2024-05-03T07:08:59Z

Closing as I am burned out and have no motivation to get this landed anymore.

…erer, with improved raymarching code. (#13418) This commit, a revamp of #12959, implements screen-space reflections (SSR), which approximate real-time reflections based on raymarching through the depth buffer and copying samples from the final rendered frame. This patch is a relatively minimal implementation of SSR, so as to provide a flexible base on which to customize and build in the future. However, it's based on the production-quality [raymarching code by Tomasz Stachowiak](https://gist.github.com/h3r2tic/9c8356bdaefbe80b1a22ae0aaee192db). For a general basic overview of screen-space reflections, see [1](https://lettier.github.io/3d-game-shaders-for-beginners/screen-space-reflection.html). The raymarching shader uses the basic algorithm of tracing forward in large steps, refining that trace in smaller increments via binary search, and then using the secant method. No temporal filtering or roughness blurring, is performed at all; for this reason, SSR currently only operates on very shiny surfaces. No acceleration via the hierarchical Z-buffer is implemented (though note that #12899 will add the infrastructure for this). Reflections are traced at full resolution, which is often considered slow. All of these improvements and more can be follow-ups. SSR is built on top of the deferred renderer and is currently only supported in that mode. Forward screen-space reflections are possible albeit uncommon (though e.g. *Doom Eternal* uses them); however, they require tracing from the previous frame, which would add complexity. This patch leaves the door open to implementing SSR in the forward rendering path but doesn't itself have such an implementation. Screen-space reflections aren't supported in WebGL 2, because they require sampling from the depth buffer, which Naga can't do because of a bug (`sampler2DShadow` is incorrectly generated instead of `sampler2D`; this is the same reason why depth of field is disabled on that platform). To add screen-space reflections to a camera, use the `ScreenSpaceReflectionsBundle` bundle or the `ScreenSpaceReflectionsSettings` component. In addition to `ScreenSpaceReflectionsSettings`, `DepthPrepass` and `DeferredPrepass` must also be present for the reflections to show up. The `ScreenSpaceReflectionsSettings` component contains several settings that artists can tweak, and also comes with sensible defaults. A new example, `ssr`, has been added. It's loosely based on the [three.js ocean sample](https://threejs.org/examples/webgl_shaders_ocean.html), but all the assets are original. Note that the three.js demo has no screen-space reflections and instead renders a mirror world. In contrast to #12959, this demo tests not only a cube but also a more complex model (the flight helmet). ## Changelog ### Added * Screen-space reflections can be enabled for very smooth surfaces by adding the `ScreenSpaceReflections` component to a camera. Deferred rendering must be enabled for the reflections to appear. ![Screenshot 2024-05-18 143555](https://github.com/bevyengine/bevy/assets/157897/b8675b39-8a89-433e-a34e-1b9ee1233267) ![Screenshot 2024-05-18 143606](https://github.com/bevyengine/bevy/assets/157897/cc9e1cd0-9951-464a-9a08-e589210e5606)

pcwalton marked this pull request as draft April 14, 2024 00:51

pcwalton force-pushed the ssr branch 5 times, most recently from 2cb94e1 to f44dfb3 Compare April 14, 2024 02:58

pcwalton marked this pull request as ready for review April 14, 2024 02:59

pcwalton requested review from IceSentry, aevyrie and superdump April 14, 2024 02:59

pcwalton force-pushed the ssr branch from f44dfb3 to 9358e0f Compare April 14, 2024 03:43

Kurble reviewed Apr 14, 2024

View reviewed changes

crates/bevy_pbr/src/ssr/mod.rs Outdated Show resolved Hide resolved

NthTensor added C-Feature A new feature, making something new possible A-Rendering Drawing game state to the screen labels Apr 14, 2024

Rename ScreenSpaceReflections to ScreenSpaceReflectionsSettings, and

1a248ac

add a bundle

mockersf reviewed Apr 14, 2024

View reviewed changes

Cargo.toml Show resolved Hide resolved

doc-scrape-examples

17c3939

JMS55 self-requested a review April 17, 2024 15:28

pcwalton added this to the 0.14 milestone Apr 22, 2024

pcwalton added 2 commits April 22, 2024 19:05

Merge remote-tracking branch 'origin/main' into ssr

d7423d8

Support normal and motion vector prepasses

7272a92

JMS55 reviewed Apr 25, 2024

View reviewed changes

crates/bevy_pbr/src/deferred/mod.rs Outdated Show resolved Hide resolved

mockersf reviewed Apr 26, 2024

View reviewed changes

crates/bevy_pbr/src/render/mesh_view_bindings.rs Outdated Show resolved Hide resolved

pcwalton added 5 commits April 26, 2024 00:32

Address review comments

4ec4390

Merge remote-tracking branch 'origin/main' into ssr

37db4b9

Fix compile failure in release mode

4f58cf2

Merge remote-tracking branch 'origin/main' into ssr

58348ed

Merge remote-tracking branch 'origin/main' into ssr

581edd2

JMS55 suggested changes May 3, 2024

View reviewed changes

IceSentry reviewed May 3, 2024

View reviewed changes

IceSentry added the M-Release-Note Work that should be called out in the blog due to impact label May 3, 2024

Fix brightness problem. Fully rename the SSR ifdef to

b56da8f

`SCREEN_SPACE_REFLECTIONS`

Merge remote-tracking branch 'origin/main' into ssr

ed153b0

pcwalton closed this May 3, 2024

pcwalton mentioned this pull request May 18, 2024

Implement opt-in sharp screen-space reflections for the deferred renderer, with improved raymarching code. #13418

Merged

	shader_defs.push("SSR".into());
	shader_defs.push("SSR".into());

Uh oh!

Conversation

pcwalton commented Apr 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog

Added

Uh oh!

github-actions bot commented Apr 14, 2024

Uh oh!

pcwalton commented Apr 14, 2024

Uh oh!

Uh oh!

pcwalton commented Apr 14, 2024

Uh oh!

Uh oh!

aevyrie commented Apr 18, 2024

Uh oh!

pcwalton commented Apr 19, 2024

Uh oh!

chronicl commented Apr 22, 2024

Uh oh!

pcwalton commented Apr 23, 2024

Uh oh!

Uh oh!

Uh oh!

JMS55 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

IceSentry left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

IceSentry commented May 3, 2024

Uh oh!

JMS55 commented May 3, 2024

Uh oh!

pcwalton commented May 3, 2024

Uh oh!

JMS55 commented May 3, 2024

Uh oh!

IceSentry commented May 3, 2024

Uh oh!

pcwalton commented May 3, 2024

Uh oh!

pcwalton commented May 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

pcwalton commented Apr 14, 2024 •

edited

Loading