Multiple viewports on winit backend

[egasimus]

Hi everybody! I've always been inspired by FVWM2's infinite desktop, but never learned how the hell to configure the damn thing. Much easier to build my own, right? Say hello to Charlie:

Screencast.from.2023-01-11.22-25-15.webm

What you're seeing is two viewports into the same screen space, rendered into separate winit windows. I managed to get multi-head output from winit by implementing my own winit backend which shares the winit event loop between multiple windows.

I posted this the other day which enables it, but when @heavyrain266 mentioned I had left the repo on private, I realized I need to polish that code some more. Well, here it is.

I actually got the single-window version working last week with 0.3.0 stable (complete with glxgears and chromium -- which curiously fail to show up right now), but then I failed miserably at porting what I had to the udev backend. The wrapper code was just so different, especially considering that the winit backend assumes a single output and on udev there is no such limitation. (And also I had no idea what I was doing and was just cutting up and rearranging the Anvil code until I figured out where to start making modifications.)

So there was no clear path from point A (single winit window) to point B (multiple udev displays). Before I would try my hand at establishing such a path (by way of the Engine trait), I decided to upgrade to master -- and boy oh boy was that a wild ride. It took me a couple days just to opt out of the new damage tracking! I plan to try my hand at writing my own damage tracking and component system later on.

Then it took me some more time to figure out why my surfaces/buffers weren't being imported. I love what yall doing with the delegations, typed state, and global dispatch, but there's just no way a newb like me could wrap their noggin around the control flow by just grepping through the codebase, right? Probably if I had read up on the protocol, it would've taken me less time to figure out that I needed this - but I learn better by doing.

It's great to see how Rust enables me to build things I've previously only dreamed of. A lot of the time I'm thinking, "what the hell am I even doing", then it finally compiles, and works out of the box, and I get a shard of my soul back. I've got a couple of pretty damn crazy ideas for how I want this to work, so stay tuned.

[heavyrain266]

Looks pretty interesting but why multiple viewports?

I'm actually too young for FVWM 2 but I'm interested in infinite desktop which behaves as cardboard where you can pin quick notes to windows etc.

[egasimus]

why multiple viewports

Basically to lay the groundwork for multi-monitor support. The winit backend explicitly disallows this (I guess nobody had the time to rewrite it with one event loop shared between multiple windows), while at the same time the udev backend requires it. So I had to find some sort of common ground between those two environments.

And, well, here it is. So now it's time to port to udev, and I've run against the Wrong type parameter passed to Handle::create_global() runtime panic. I need to implement DmabufHandler separately for WinitEngine and UdevEngine, but I don't want the root AppState struct to be generic over Engine. And apparently Smithay doesn't support delegating to more than one struct, which is a bit of a bummer, but nothing that can't be worked around.

E.g. PointerHandle::motion here wants me to pass State - but Pointer has no reference to State, it's the other way around! It's the same with the WinitEngine/UdevEngine dispatch - DmabufState::create_global won't accept the Engine struct as a type parameter since the delegations are all on AppState, but passing AppState loses the dispatch (since the DmabufHandler impls for winit and udev need to be different).

I did ultimately end up with an App struct which contains both AppState and Engine, and is generic over both of their types, so in this case I don't think there's anything stopping me from just moving all the delegations to that. But I do wish the global dispatch stuff was more flexible that way.

[egasimus]

I did ultimately end up with an App struct which contains both AppState and Engine, and is generic over both of their types, so in this case I don't think there's anything stopping me from just moving all the delegations to that.

Turns out I was wrong. As it stands, I see no viable way to decouple platform state from compositor state because of how global dispatch works. See #860

[Drakulix]

So, in principle I welcome work on multi-window support, because we have been trying to allow that for our X11-backend and also plan for it on the Wayland-backend.

BUT, we actually want to deprecate the winit-backend (which is flawed in multiple ways and not really useful outside of prototyping) in the medium-term. So I am hesitant to add any new features to it.

If you want to use it, I would actually suggest copying it into your own source tree (and make any modifications to it you feel like).

Though this might raise another discussion. I think we have talked about some smithay-community or smithay-contrib crate at some point for some wlr-protocols. I could see the winit-backend moving into such a crate, instead of deprecating it. In that case we would want to keep it in shape and possibly even add new features.
I think @i509VCB might have some thoughts about our nested backends as well.

[egasimus]

Sure, I only started out with winit because that was the only one that I managed to get working. I'm not really pushing for my multi-window winit code to be accepted into mainline - it works just fine from downstream, aside from this.

The big thing that I'd like to see would be pluggable backends (linking Engine trait once again as a rough example of the scope) -- so that downstream would only have to implement the actual compositor and pass it into a standard udev/x11/wayland/winit/etc context. Right now, with how the global dispatch works, I just don't see a good way to do that without coupling the backend and compositor (as Anvil does with App<WinitData>/App<UdevData>).

[Drakulix]

You absolutely can build this today, but you would have to do the required dynamic dispatching internally of your App/State whatever type. You could use a Box<Engine>, cosmic-comp for example uses an enum to do that. Anvil just goes for a generic parameter, because it is easy.

Also standard udev/x11/wayland/winit/etc aren't really standard, which is the biggest reason there isn't a unified Backend-trait.

udev in particular has very specific needs with new outputs appearing, while the nested backends could just create any number of outputs, etc. Input also works differently depending on the backend. Instead of trying to hide all these details and breaking them all in subtle ways (as the winit backend does for proper nesting), smithay exposes them and downstream is supposed to handle them how they please.

[Drakulix]

Also btw:

Then it took me some more time to figure out why my surfaces/buffers weren't being imported. I love what yall doing with the delegations, typed state, and global dispatch, but there's just no way a newb like me could wrap their noggin around the control flow by just grepping through the codebase, right? Probably if I had read up on the protocol, it would've taken me less time to figure out that I needed this - but I learn better by doing.

Yes control flow is a little obscure, but commits and configures are a very delicate dance demanded by the xdg-shell protocol. While your solution likely works for most client, it is not really, what the protocol demands and if you want to succeed in writing a usable compositor, I strongly suggest reading up on at least the core-protocol and the xdg-shell protocol. wayland.app is a really nice website to do just that.

[egasimus]

While your solution likely works for most client, it is not really, what the protocol demands

Ohhh so that's why only one of three clients I tried worked. 🤣 Yeah I'm definitely gonna have to read up on the protocol. I'm looking forward to writing to you from a Charlie session on udev :-)