Questions about wayland

[StripedMonkey]

I understand and acknowledge that wayland intentionally makes this difficult, annoying, and far more trouble than it's probably worth. However, I'm trying to investigate what the intended ways of making this work are. Is it through some XDG Portal? Is there a wayland extension? How much of the framework needs to be done on the compositor side for libuiohook to have a chance for it to work?

I'm not sure these are even valid questions to ask, or if I'm misunderstanding the software stack required for this to work.

[thebigG]

I'd be very interested in this as well. I also wonder what it will take for the API to support Wayland.

[kwhat]

It is very possible, at least with the same level of support we currently have with X11. There are some projects like wayland-keylogger that could be a good starting point.

Do any distros currently support Wayland out of the box? If I can get a VM up, I can do some testing.

[thebigG]

I believe that Ubuntu 21.04 ships with Wayland by default. However for Nvidia users like myself Ubuntu21.04 still defaults to X since it looks like there isn't much Wayland support in the Nvidia world.

[kwhat]

Yah, I am stuck with nvidia for the time being as well so I will be testing in a VM.

[thebigG]

Thanks so much!

[khimaros]

debian bookworm also works with wayland out of the box.

[3laws]

Do we have any updates on this? I mainly ask from a inbut-overlay user POV. However, now with Wayland being the default in several major distros, I think it's time for libuiohook to fast track support.

[kwhat]

Do we have any updates on this? I mainly ask from a inbut-overlay user POV. However, now with Wayland being the default in several major distros, I think it's time for libuiohook to fast track support.

There is a pretty major X11 update in the pipeline, Wayland probably after that. I think the biggest issue with this is going to be something that can support both wayland and x11 at runtime from the same binary. I'll start with a build time option first and then branch out from there.

[StripedMonkey]

AFAIK implementation of this is still a pretty open question in general.

The previously linked Wayland Keylogger is dependent on every application you want to use libuiohook in conjunction with having the shared library injected into it. This is mildly unfortunate, and doesn't really work for the same usecases as libuiohook, although it's a possible solution for those like me who use libuiohook for game key logging going onto OBS.

Global keybindings on Gnome/Wayland are (currently) implemented without actual key logging on the application side, so unfortunately this does not help.

Discussions of adding this into the wayland specification have many polarizing thoughts on the issue, and nothing substantial pushing it forward within wayland itself. Discussions of implementing this within the xdg-desktop-portal have primarily focused on providing access to keybindings for specific keys/events, which may entirely conflict with how this library is designed, (I've not actually investigated libuiohook at all).

tl;dr I suspect that this is going to be a grueling if not impossible task to implement in short order, certainly not with feature parity. Discussions need to be had on how this can/should be implemented within Wayland/surrounding ecosystem in order for it to work. This may work if we inject an LD_PRELOAD=libuiohook.so into ~/.profile or similar, but I am hesitant to say that'd work well or be a particularly reasonable solution especially with the existence of things like flatpak which may not work well with that.

I quite frankly have no idea how this can feasibly be implemented with some level of feature parity.

[glyh]

@StripedMonkey There is Kmonad which works for me under Sway(Wayland), so I guess there is at least some workaround.

[thebigG]

@glyh
Yeah...they are using udev. Those require root access, I think.

[kwhat]

Baring any major issues libevdev will be used moving forward to support both x11, wayland and probably everything.

This doesn't require root, but it does require additional privileges. Usually group membership to something like "input." This is more inline with what modern operating systems like OS X have implemented to restrict access to what can potentially be sensitive information. The hardest part will be converting the relative mouse coordinates to window manger value. There will probably be some x11/wayland specific code but it should be small.

On a side note, somewhere on the internet I came across a really nice example of how to use libevdev_new_from_fd will all available input devices and I cant seem to find it anymore.

[kwhat]

Just and update, evdev works pretty well by itself but getting it to work with both X11 and Wayland is going to be a nightmare. The code will need it's own Wayland client (because Wayland is kind of a dumpster fire) and it will also depend on XKB for keyboard layout support. All of the X11 and Wayland dependent code required to make this work will all have to be loaded at runtime via dlsym making this whole thing even more painful. It is going to be a lot of work and its going to take a long time. The current plan is to start with the X11 portion and put it together at compile time, then add Wayland support (also at compile time) then finally refactor it all to use dlsym at runtime.

[kwhat]

So the next problem is that epoll seems to change the order in which events are received and I have no idea why.

This is an example of the problem take from https://gitlab.freedesktop.org/libevdev/libevdev.git. I have hacked epoll into the example: tools/libevdev-events.c. You can build with autotools and run something like tools/libevdev-events /dev/input/by-id/usb-Microsoft_Microsoft_Ergonomic_Keyboard_605519294221-event-kbd to see the problem. TL;DR the key release events happen before the key press events, but only when using epoll. Using epoll with udev doesn't appear to have this problem and I have no idea why. If anyone can figure it out, let me know. I am more than a bit burned out right now and I done feel like working on this anymore.

// SPDX-License-Identifier: MIT
/*
* Copyright © 2013 Red Hat, Inc.
*/

#include "config.h"

#include <assert.h>
#include <errno.h>
#include <fcntl.h>
#include <linux/input.h>
#include <stdio.h>
#include <string.h>
#include <sys/epoll.h>
#include <sys/stat.h>
#include <sys/types.h>

#include "libevdev/libevdev.h"

static void
print_props(struct libevdev *dev)
{
        unsigned int i;
        printf("Properties:\n");

        for (i = 0; i <= INPUT_PROP_MAX; i++) {
                if (libevdev_has_property(dev, i))
                        printf("  Property type %d (%s)\n", i,
                                        libevdev_property_get_name(i));
        }
}

static int
print_event(struct input_event *ev)
{
        if (ev->type == EV_SYN)
                printf("Event: time %ld.%06ld, ++++++++++++++++++++ %s +++++++++++++++\n",
                                ev->input_event_sec,
                                ev->input_event_usec,
                                libevdev_event_type_get_name(ev->type));
        else
                printf("Event: time %ld.%06ld, type %d (%s), code %d (%s), value %d\n",
                        ev->input_event_sec,
                        ev->input_event_usec,
                        ev->type,
                        libevdev_event_type_get_name(ev->type),
                        ev->code,
                        libevdev_event_code_get_name(ev->type, ev->code),
                        ev->value);
        return 0;
}

static int
print_sync_event(struct input_event *ev)
{
        printf("SYNC: ");
        print_event(ev);
        return 0;
}

int
main(int argc, char **argv)
{
        struct libevdev *dev = NULL;
        const char *file;
        int fd;
        int rc = 1;

        if (argc < 2)
                goto out;

        int epoll_fd = epoll_create1(0);
        if (epoll_fd == -1) {
                perror("Failed to create epoll");
                goto out;
        }

        file = argv[1];
        fd = open(file, O_RDONLY);
        if (fd < 0) {
                perror("Failed to open device");
                goto out;
        }

        rc = libevdev_new_from_fd(fd, &dev);
        if (rc < 0) {
                fprintf(stderr, "Failed to init libevdev (%s)\n", strerror(-rc));
                goto out;
        }

        struct epoll_event event_buffer = {};
        event_buffer.events = EPOLLIN;
        event_buffer.data.fd = fd;
        if (epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &event_buffer) < 0) {
                perror("Failed to open device");
                goto out;
        }


        printf("Input device ID: bus %#x vendor %#x product %#x\n",
                        libevdev_get_id_bustype(dev),
                        libevdev_get_id_vendor(dev),
                        libevdev_get_id_product(dev));
        printf("Evdev version: %x\n", libevdev_get_driver_version(dev));
        printf("Input device name: \"%s\"\n", libevdev_get_name(dev));
        printf("Phys location: %s\n", libevdev_get_phys(dev));
        printf("Uniq identifier: %s\n", libevdev_get_uniq(dev));
        print_props(dev);

        struct epoll_event buffer;
        do {
                int event_count = epoll_wait(epoll_fd, &buffer, 1, 1000);

                for (int i = 0; i < event_count; i++) {
                        struct input_event ev;
                        rc = libevdev_next_event(dev, LIBEVDEV_READ_FLAG_NORMAL|LIBEVDEV_READ_FLAG_BLOCKING, &ev);
                        if (rc == LIBEVDEV_READ_STATUS_SYNC) {
                                printf("::::::::::::::::::::: dropped ::::::::::::::::::::::\n");
                                while (rc == LIBEVDEV_READ_STATUS_SYNC) {
                                        print_sync_event(&ev);
                                        rc = libevdev_next_event(dev, LIBEVDEV_READ_FLAG_SYNC, &ev);
                                }
                                printf("::::::::::::::::::::: re-synced ::::::::::::::::::::::\n");
                        } else if (rc == LIBEVDEV_READ_STATUS_SUCCESS) {
                                if (ev.type == EV_KEY) {
                                        print_event(&ev);
                                }
                        }
                }
        } while (rc == LIBEVDEV_READ_STATUS_SYNC || rc == LIBEVDEV_READ_STATUS_SUCCESS || rc == -EAGAIN);

        if (rc != LIBEVDEV_READ_STATUS_SUCCESS && rc != -EAGAIN) {
                fprintf(stderr, "Failed to handle events: %s\n", strerror(-rc));
        }

        rc = 0;
out:
        libevdev_free(dev);

        return rc;
}

[Lassebq]

https://gitlab.gnome.org/GNOME/xdg-desktop-portal-gnome/-/merge_requests/61
If I understand correctly GNOME's XDG desktop portal already implements requesting input capture?
Will there's be a support for it? (So far I've only tested obs input capture plugin, which uses this library and it doesn't request listening for input. It works only in xwayland clients, which don't require user permission)

[kwhat]

Will there's be a support for it?

No, probably not. If I support gnome directly that's going to leave KDE and everyone else out in the cold. This is part of the reason Wayland support is so difficult. Sure I can support Plasma or Gnome but then what about the others. What if someone is using a different WM? Oh the joys of the Linux desktop...

The input capture seems to be working pretty well with libevdev. It requires some udev rules and group permissions, but it's manageable. The big problem now is most of the functionality in system_properties.c. All of this stuff is highly dependent on the compositor used and there is no simple wayland replacement.

[Lassebq]

By "it" I mean XDG desktop portals as a whole and XDG desktop portals are universal. You're not leaving anyone behind if you're using them. KDE will catch up with their portal (they might have already implemented it in fact).

All of this stuff is highly dependent on the compositor

Exactly my reason for implementing support for XDG desktop portals. As it stands XDG is a Cross Desktop Group. Their protocols are expected to be supported everywhere, and that's where Wayland is heading

What if someone is using a different WM?

WMs are X11 and there's already a way for global input capture. A good Wayland compositor has to do much more than just window management

[kwhat]

Well, I looked into XDG and it looks like it might solve some problems but its pretty far from being a complete solution.

The initial suggestion about using InputCapture wont really work. As I understand it, this is an active capture of the input device so no events would be delivered to the desktop, only the app requesting the capture.

From the Input Capture docs:

Once the compositor activates input capturing, events from physical or logical devices are sent directly to the application instead of using those events to update the pointer position on-screen.

We would still need to use something like uinput to deliver the events after we grabbed them with capture making libevdev more suitable for our needs. The fact that you need to be in the input group is basically on par with how macOS does things where input capture requires special access making this library less of a key logger and more of an accessibility tool as it was intended to be.

Where XDG seems to be more promising is in the system_properties.c issues I've been trying to solve. Some of this information seems to be exposed though Settings like org.gnome.settings-daemon.peripherals.mouse -> double-click but that information isn't on the settings panel and the information that is like mouse sensitivity isn't here. You may have also noticed that this settings path is for gnome and I am sure KDE and other desktops will all do their own thing in this department.

I am still at a standstill on further development for Wayland. No one seems to have any good answer to these basic problems and the documentation pretty sparse for what does exist.