Initial linux touch implementation by ap4ss3rby · Pull Request #1706 · LizardByte/Sunshine

ap4ss3rby · 2023-10-05T22:17:51Z

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work.

Needs a properly patched Moonlight client for android testing with this patch reverted

Description

Currently, the code in this PR does the following:

~~Break Windows and macOS builds because I couldn't be bothered to fix their touch() functions for the touch device parameter~~
Report that we infact have touch capability on linux hosts
Probably break absolute mouse mode even more
Report native touch events using uinput. Making stuff like streaming waydroid to a phone for playing Android games at better quality than just playing directly on the phone possible
~~Not do multitouch because uinput is stupid and doesn't like how I'm feeding it multitouch data~~

Screenshot

Issues Fixed or Closed

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Dependency update (updates to dependencies)
Documentation update (changes to documentation)
Repository update (changes to repository files, e.g. .github/...)

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

I want maintainers to keep my branch updated

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work. Needs a properly patched Moonlight client for android testing

CLAassistant · 2023-10-05T22:17:56Z

All committers have signed the CLA.

asdf

…d macOS builds

ap4ss3rby · 2023-10-07T19:53:25Z

After fixing a bunch of mistakes with my understanding of uinput and implementing what I learned I was able to get at least 8 points recognized (couldn't test more because I ran out of space on my phone haha), and it should recognize up to 10

cgutman · 2023-10-07T20:33:22Z

src/platform/linux/input.cpp

  get_capabilities() {
-    return 0;
+    platform_caps::caps_t caps = 0;
+    caps |= platform_caps::pen_touch;


You also have to implement pen() or Moonlight will start sending native pen events that just get ignored.

Will implement but will require external testing since I don't have anything capable of doing pen events.

I can test with my devices

cgutman · 2023-10-07T20:36:43Z

src/platform/linux/input.cpp

+    libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_SLOT, &mtslot);
+    libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_TRACKING_ID, &trkid);
+    libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_X, &absx);
+    libevdev_enable_event_code(dev.get(), EV_ABS, ABS_MT_POSITION_Y, &absy);


You should use a separate uinput device for pen/touch input, create it on demand (on the first pen/touch event), and store it in the client_input_t. You can use the Windows implementation as reference.

cgutman · 2023-10-07T20:53:17Z

src/platform/linux/input.cpp

+    libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_TRACKING_ID, id);
+    libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_X, scaled_x);
+    libevdev_uinput_write_event(touchscreen, EV_ABS, ABS_MT_POSITION_Y, scaled_y);
+    libevdev_uinput_write_event(touchscreen, EV_SYN, SYN_REPORT, 0);


You should also implement ABS_MT_TOOL_TYPE, ABS_MT_PRESSURE, ABS_MT_ORIENTATION, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR.

ABS_MT_TOOL_TYPE should always be MT_TOOL_FINGER in this codepath.

ABS_MT_PRESSURE, ABS_MT_TOUCH_MAJOR, and ABS_MT_TOUCH_MINOR should be pretty simple. Just scale the 0-1 float to some arbitrary integer range (touch major and minor have to be same range as X/Y) and be sure to handle the "unknown" cases. See the API docs here: https://github.com/moonlight-stream/moonlight-common-c/blob/d457fbb48714d4ce79de2594180dfd73bc5f070c/src/Limelight.h#L594-L629

Finally, ABS_MT_ORIENTATION might need some extra conversion logic to get to the evdev format.

You can reuse all this logic for the pen implementation too, in addition to adding ABS_MT_DISTANCE, BTN_STYLUS/BTN_STYLUS2, and ABS_MT_TOOL_TYPE. I think ABS_TILT_X and ABS_TILT_Y is also valid for stylus devices.

So we're going towards a full fat implementation instead of a basic one. Got it. Keep in mind I dropped some tablet related events since they seemed to cause problems with KDE Plasma and the easiest solution was to just drop tablet related events.

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

If I can get around them I'll implement them into one device type. Otherwise I'll split touchscreen and tablet types if we're gonna create a new global context whenever we recieve a touch/tablet packet just to save myself a bit of headache

I assume you mean splitting into separate uinput devices. That's totally fine if it's more convenient to expose separate pen and multitouch touchscreen devices.

src/platform/common.h

src/platform/linux/input.cpp

WesleyVanNeck · 2023-10-27T10:05:42Z

@ap4ss3rby do you know why on wayland i only get about 2 touch inputs

WesleyVanNeck · 2023-10-27T10:27:36Z

on kde it works fine do the mouse cursor shows up

ReenigneArcher · 2023-12-30T18:05:06Z

@ap4ss3rby are you able to implement the remaining changes requested by @cgutman?

Initial linux touch implementation

b6f75e8

I may or may not have missed a lot of details, but I hope someone else can fix my semi-working MT code to properly work. Needs a properly patched Moonlight client for android testing

Merge branch 'nightly' into nightly

eacf0dd

ReenigneArcher marked this pull request as draft October 5, 2023 22:19

ap4ss3rby changed the title ~~Initial linux touch implementation~~ WIP: Initial linux touch implementation Oct 5, 2023

ap4ss3rby added 8 commits October 7, 2023 22:41

properly define mtslot and trkid

9677286

Cleanup variables and comments

aa8164e

correct pointerId and id logic

ec33374

Handle CANCEL events

364232c

Fix for loop syntax

7996f8c

Declare touchscreen in the beginning of the function

9ea3f5c

asdf

Get viewport width from touch_port

9c53843

Properly label the new variable for touch function and fix Windows an…

8f1a7e7

…d macOS builds

ap4ss3rby force-pushed the nightly branch from 8cb1c25 to 8f1a7e7 Compare October 7, 2023 19:48

ap4ss3rby changed the title ~~WIP: Initial linux touch implementation~~ Initial linux touch implementation Oct 7, 2023

Merge branch 'nightly' into nightly

5231c3f

ap4ss3rby marked this pull request as ready for review October 7, 2023 19:52

ReenigneArcher requested a review from cgutman October 7, 2023 19:56

cgutman requested changes Oct 7, 2023

View reviewed changes

ap4ss3rby added 3 commits October 8, 2023 02:05

Drop unnecessary + 4 to the assigned ID

8c95056

Use global input context from client_input_t on linux

dfd6ab4

Drop any unhandled events and handle LI_TOUCH_EVENT_MOVE/DOWN

9837c94

cgutman mentioned this pull request Jan 13, 2024

Implement pen and touch support for Linux #2017

Merged

15 tasks

cgutman closed this in #2017 Jan 14, 2024

Uh oh!

Conversation

ap4ss3rby commented Oct 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Screenshot

Issues Fixed or Closed

Type of Change

Checklist

Branch Updates

Uh oh!

CLAassistant commented Oct 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ap4ss3rby commented Oct 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cgutman Oct 7, 2023

Choose a reason for hiding this comment

Uh oh!

ap4ss3rby Oct 7, 2023

Choose a reason for hiding this comment

Uh oh!

cgutman Oct 8, 2023

Choose a reason for hiding this comment

Uh oh!

cgutman Oct 7, 2023

Choose a reason for hiding this comment

Uh oh!

cgutman Oct 7, 2023

Choose a reason for hiding this comment

Uh oh!

ap4ss3rby Oct 7, 2023

Choose a reason for hiding this comment

Uh oh!

cgutman Oct 8, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

WesleyVanNeck commented Oct 27, 2023

Uh oh!

WesleyVanNeck commented Oct 27, 2023

Uh oh!

ReenigneArcher commented Dec 30, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ap4ss3rby commented Oct 5, 2023 •

edited

Loading

CLAassistant commented Oct 5, 2023 •

edited

Loading

ap4ss3rby commented Oct 7, 2023 •

edited

Loading