UI render layers #5414

nicopap · 2022-07-21T11:32:53Z

Objective

This enables having different UI per camera. With #5225, this enables
having different interactive UIs per window. Although, to properly
complete this, the focus system will need to account for RenderLayer of
UI nodes.

Solution

Add the render_layers: RenderLayers field to UI bundles
Add the ui_render_layers field to UiCameraConfig
Keep track of UI RenderLayers in the UI render code

Changelog

Add RenderLayers support to UI nodes, you now can display different UIs on different cameras.

This enables having different UI per camera. With bevyengine#5225, this enables having different interactive UIs per window. Although, to properly complete this, the focus system will need to account for RenderLayer of UI nodes.

crates/bevy_ui/src/lib.rs

alice-i-cecile · 2022-07-21T14:48:04Z

crates/bevy_ui/src/render/mod.rs

+/// This component is inserted into the render world in the
+/// [`extract_default_ui_camera_view`] system.
+///
+/// The component is attached to the "actual" viewport's camera.


Ugh, this is making me second guess removing explicitly initialized UI cameras. Seems like the right design if we're not changing that though.

crates/bevy_ui/src/render/mod.rs

alice-i-cecile · 2022-07-21T14:50:22Z

crates/bevy_ui/src/update.rs

+        }
+        for (mut visibility, node_layers) in &mut nodes {
+            if config.ui_render_layers.intersects(node_layers) {
+                visibility.set_visible_in_view();


Will this override all other forms of computed visibility?

I think it does, which will be an issue with inherited visibility. Good catch, need to update the code.

I don't like this, though. The default visibility check is buggy on UI entities since it cannot account for UiCameraConfig, but it does the inherited visibility thing.

In practice, that means the checks in check_visibility needs to be completely discarded and recomputed here :/

Yeah, why not merge this together with those checks in some way?

This would create a circular dependency between bevy_render and bevy_ui (need access to the UiCameraConfig and Node components, and you wouldn't want bevy_render to depend on bevy_ui regardless).

An alternative design where the UI camera is spawned as any other camera in the app world is possible, but the fact that it's a sort of special camera that shares a viewport with another one makes it awkward. I've given motivation in the opening comment of #5252.

Hot take: should just remove Visibility and ComputedVisibility from UI entities. Bonus: avoids confusion with display: None :D

But I feel like this is a discussion that should be had in a different PR or an issue.

Oof, right :(

alice-i-cecile

Definitely having some regrets about moving to implicit UI cameras seeing the hoops this PR (and its sibling) have to jump through. That said, I'm largely happy with the changes here otherwise, and think that this is an important feature.

mockersf · 2022-07-29T07:20:48Z

This is adding a "gotcha" with layout.

Layout is not looking at render layers, and it shouldn't because cameras can use masks as they want, but in the basic use case of one camera => one layer, ui on each camera will impact the layout of each other

nicopap · 2022-07-29T07:31:31Z

This is adding a "gotcha" with layout.

I think it's a major limitation, that I'm not sure how to address. My approach was simply to ignore it, because otherwise I'd have to argue why I chose an approach over another. 😅

nicopap · 2022-07-29T10:46:40Z

I think the best way of handling it would be to support multiple UI roots with an explicit component that also encodes the settings for the whole tree under the root. This would be one place to specify the RenderLayer (or for a better higher level API: specify the camera that renders the UI under the root)

aevyrie · 2022-08-03T07:06:51Z

After reviewing the PR, I don't think this is the right approach.

I don't see why renderlayers are necessary to enable multi camera rendering.
Renderlayers per node add the opportunity for strange bugs with widgets in a tree showing up on different camera.
Ultimately, I think this is adding unnecessary complexity.

Considering we are now doing camera driven rendering with implicit ui cameras, it would make more sense to me if every ui camera was also a ui root node. Layout only really makes sense in the context of a "document", which in this case is the camera's rendertarget.

This maps well to input, where pointers map well to render targets.
A multi window focus system is much simpler if every node in a ui tree is rendered to the same view, That is solved in Bevy Pointer Abstraction #5540

nicopap · 2022-08-03T08:12:47Z

I think I'll close this PR and try a different approach. I agree with most of the objections.

I'm looking forward to see how your mouse picking proposal works with render priority and FocusPolicy (I still need to check your bevy_mod_picking rework 😁).

A few notes though:

it would make more sense to me if every ui camera was also a ui root node.

But if the UI camera is implicit, then there is no such thing as a "UI camera", what do you mean by that? Maybe adding a field to UiCameraConfig that points to a UI root node would work? But then how to pick the default?

I personally think that adding by default a UI to every cameras is a bad idea. I would be in favor of making explicit UI in render targets.

aevyrie · 2022-08-03T19:37:53Z

But if the UI camera is implicit, then there is no such thing as a "UI camera", what do you mean by that? Maybe adding a field to UiCameraConfig that points to a UI root node would work? But then how to pick the default?

Good catch. I think it would make sense to make the ui camera explicit, and add the ui nodes to it.

Weibye · 2022-08-04T06:17:55Z

I've opened #5570 which relates to this conversation

oceantume · 2022-09-06T18:44:05Z

@nicopap @aevyrie

it would make more sense to me if every ui camera was also a ui root node.

But if the UI camera is implicit, then there is no such thing as a "UI camera", what do you mean by that? Maybe adding a field to UiCameraConfig that points to a UI root node would work? But then how to pick the default?

I personally think that adding by default a UI to every cameras is a bad idea. I would be in favor of making explicit UI in render targets.

I wrote it before reading about this one, but in my recent draft #5892 I take the reverse approach of mapping UI root nodes (so only UI nodes that are at the root of a UI tree) to a camera entity instead. This has the benefit of enabling any number of UI trees to exist in a single camera and making sure that a UI tree is always constrained within that camera's target and viewport.

One limitation is that a single UI tree can't be rendered to multiple cameras, but that sounds like a complex use-case to me that you could work around if you really wanted to by having multiple of the same UI tree or rendering the UI to a camera that renders to a texture and presenting that elsewhere.

If you don't add that component to your UI root, the assumption would either be that you only have one camera and that you want the UI to fill it, or that you just want to render a UI on the full main window and don't really care about specific cameras for UI.

This seems to make some sense, but I've yet to write the rendering part so I'm not sure if it'll work in practice.

If you're up for it, please let me know what you think!

nicopap · 2022-09-07T06:57:08Z

@oceantume I'm on board with your implementation. I don't see any realistic blocker, so it's probably good to go. I'd really like to help you get it working, but I'm a bit short on time.

Edit: I have one worry, it's knowing in "which" UI tree a given UI node is. This will always require navigating up to the root. A costly operation if it is a common occurence

oceantume · 2022-09-07T12:27:11Z

@nicopap

Edit: I have one worry, it's knowing in "which" UI tree a given UI node is. This will always require navigating up to the root. A costly operation if it is a common occurence

I have an idea for making this a little better. I plan to alter the UiStack that I started in #5877 in which I recursively go through the tree to generate a sorted Vec of entities. If this came with a Vec of UI trees instead, each containing a struct with tree info with a simple Vec of sorted entities, the UI root information would be available during extraction without having to go through the entire tree again. I think this is a decent idea that will also be useful for the interaction system once we have multi-windows support.

nicopap force-pushed the ui-render-layers branch from 15a248b to 2d73c50 Compare July 21, 2022 11:35

nicopap added 2 commits July 21, 2022 13:46

Add RenderLayer support to UI cameras

d5ee6b4

This enables having different UI per camera. With bevyengine#5225, this enables having different interactive UIs per window. Although, to properly complete this, the focus system will need to account for RenderLayer of UI nodes.

Update Node ComputedVisibility based on UI RenderLayers

17153b6

nicopap force-pushed the ui-render-layers branch from 2d73c50 to 17153b6 Compare July 21, 2022 11:46

nicopap mentioned this pull request Jul 21, 2022

User UI camera control #5252

Closed

alice-i-cecile added C-Feature A new feature, making something new possible A-Rendering Drawing game state to the screen A-UI Graphical user interfaces, styles, layouts, and widgets labels Jul 21, 2022

alice-i-cecile reviewed Jul 21, 2022

View reviewed changes

crates/bevy_ui/src/lib.rs Outdated Show resolved Hide resolved

alice-i-cecile reviewed Jul 21, 2022

View reviewed changes

crates/bevy_ui/src/render/mod.rs Outdated Show resolved Hide resolved

alice-i-cecile reviewed Jul 21, 2022

View reviewed changes

Reword some docs

bcab783

nicopap closed this Aug 3, 2022

Weibye mentioned this pull request Aug 8, 2022

Support multiple UI roots #5622

Closed

nicopap mentioned this pull request Apr 9, 2023

Make UiNode render pass respect render layers. #8332

Closed

bardt mentioned this pull request Dec 2, 2023

Camera-driven UI #10559

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UI render layers #5414

UI render layers #5414

nicopap commented Jul 21, 2022

alice-i-cecile Jul 21, 2022

alice-i-cecile Jul 21, 2022

nicopap Jul 21, 2022

alice-i-cecile Jul 21, 2022

nicopap Jul 21, 2022

alice-i-cecile Jul 21, 2022

alice-i-cecile left a comment

mockersf commented Jul 29, 2022

nicopap commented Jul 29, 2022

nicopap commented Jul 29, 2022

aevyrie commented Aug 3, 2022 •

edited

Loading

nicopap commented Aug 3, 2022

aevyrie commented Aug 3, 2022

Weibye commented Aug 4, 2022

oceantume commented Sep 6, 2022 •

edited

Loading

nicopap commented Sep 7, 2022 •

edited

Loading

oceantume commented Sep 7, 2022 •

edited

Loading

UI render layers #5414

UI render layers #5414

Conversation

nicopap commented Jul 21, 2022

Objective

Solution

Changelog

alice-i-cecile Jul 21, 2022

Choose a reason for hiding this comment

alice-i-cecile Jul 21, 2022

Choose a reason for hiding this comment

nicopap Jul 21, 2022

Choose a reason for hiding this comment

alice-i-cecile Jul 21, 2022

Choose a reason for hiding this comment

nicopap Jul 21, 2022

Choose a reason for hiding this comment

alice-i-cecile Jul 21, 2022

Choose a reason for hiding this comment

alice-i-cecile left a comment

Choose a reason for hiding this comment

mockersf commented Jul 29, 2022

nicopap commented Jul 29, 2022

nicopap commented Jul 29, 2022

aevyrie commented Aug 3, 2022 • edited Loading

nicopap commented Aug 3, 2022

aevyrie commented Aug 3, 2022

Weibye commented Aug 4, 2022

oceantume commented Sep 6, 2022 • edited Loading

nicopap commented Sep 7, 2022 • edited Loading

oceantume commented Sep 7, 2022 • edited Loading

aevyrie commented Aug 3, 2022 •

edited

Loading

oceantume commented Sep 6, 2022 •

edited

Loading

nicopap commented Sep 7, 2022 •

edited

Loading

oceantume commented Sep 7, 2022 •

edited

Loading