First draft of a glutin backend#9
Conversation
|
Woaw, that's quite some work you've done here! I haven't yet looked in detail at your code, but first of all I have some reactions to your PR message:
Yeah, that's because wayland-client and wayland-server depend of wayland-sys, and glutin depends on wayland-client[dlopen]. But I would expect this to just work™ nonetheless. The version mismatch is indeed probably the cause of our trouble.
The core wayland protocol only handles mouses, keyboards and touchscreens. Other input devices (like gamepads) are afaik often handled directly by the clients which need them, using specialized libs. Any way, for now lets just stick to these 3.
My opinion on handling inputs in general is that, as the core goal is to run a compositor on the bare OS, libinput should be the final target, and we could design the input backend api using it as an inspiration. The goal is not to perfectly match it, but it to be the lowest-overhead backend.
I think I cannot give answers to your other points without reading your code, so I'll do that right now. |
|
Regarding the glutin backend and the event loop, actually things are likely to get much simpler with the upcoming new api of winit/glutin: there will be 2 objects, the |
| fn is_current(&self) -> bool; | ||
| unsafe fn make_current(&self); | ||
| fn get_api(&self) -> Api; | ||
| fn get_pixel_format(&self) -> PixelFormat; |
There was a problem hiding this comment.
What is you general feeling about this trait?
I'm not extremely used to OpenGL, but fom what I've seen in glutin/glium, it looks like it pretty much covers the fundamental abstraction of "an OpenGL canvas to draw on".
On a nitpick, we should not directly expose glutin's types, but rather expose our own and do the translation in the impl of the backend (possibly via a From<_> impl).
There was a problem hiding this comment.
It's okay-ish. I am not happy with copying most of the stuff from glutin and glium. It works, but I think we should drop it completely. A backend should just expose it's own rendering backend. This would avoid copying even more types, which is necessary, not just a nitpick, because I would like to make glutin a feature.
There was a problem hiding this comment.
Isn't this part of glutin/glium a direct expression of how OpenGL works?
I mean, for example I expect all graphics backends (glutin/x11/wayland/native) to be directly usable with OpenGL wrappers such as glium and gfx...
So I guess my question here is: what in this trait is glutin/glium-specific?
There was a problem hiding this comment.
gluim does not need get_api or get_pixel_format, actually no real library needs it, glutin just provides both of them. glx does not even need any of these, except get_proc_address. It just depends on the gl context being the current one and also expects that not to change over time and figures out the rest itself via low-level c calls internally. However anybody using raw GL might want functionally such as get_api to ease development.
I am just not happy with how directly linked this backend is to gliums requirements. It does not feel very generic or useful, if you just copy types and traits from glutin.
However you are right, it should be quite easy to put this over any backend type, we are currently aiming at. And I feel like it is already very complete.
There was a problem hiding this comment.
I see you concerns. I didn't realise half of these methods where only required by glium.
I think the scope we should target for this backend is the union of the requirements of the various opengl libraries that may be used with smithay, as long as it does not imply any significant overhead (be it a runtime cost or a development cost).
But overall, I'd rather be in favour of providing too many methods than too few.
There was a problem hiding this comment.
I am also in favour of implementing traits of said opengl libraries. This will ease integration but most importantly always make sure at least the advertised libraries are working on the trait. If the situation gets to complicated at any time, we can still discuss dropping it.
There was a problem hiding this comment.
Yes, we can completely add a glium and a gfx cargo features (or others), each of them activating the boilerplate to crate a glium / gfx context from the OpenGLRenderer trait. 👍
| use std::hash::Hash; | ||
|
|
||
| #[derive(Hash, PartialEq, Eq, Clone, Copy)] | ||
| pub enum InputDevice<T: PartialEq + Eq + Hash + Clone + Copy> |
There was a problem hiding this comment.
I'm not a big fan of this "ID" thing regarding the input devices...
I'd rather follow the "seat" route of libinput (which matches pretty closely the wayland protocol, probably by design): there can be one or more logical Seats, but each of them has at most 1 of each input device type.
In this case, we could rather have a SeatHandler trait, with a type of which an instance is created for each seat, so the trait would probably have to include a constructor method.
In the glutin case, there would be only one seat.
There was a problem hiding this comment.
So we could not distinguish the devices on one seat? How would that work out with different keyboards and layouts. I still try to understand how xkbcommon would handle that stuff.
There was a problem hiding this comment.
I think there would simply be one xkbcommon context per seat. And the idea is that seat is a logical representation, and libinput has a system-wide configuration setup apparently.
So basically, it would be up to the computer user to decide whether their two keyboard should be treated as a single one, or appear as two different logical seats.
There was a problem hiding this comment.
You can find some explanation about libinput's seats here: https://wayland.freedesktop.org/libinput/doc/latest/seats.html
There was a problem hiding this comment.
And the udev config for libinput: https://wayland.freedesktop.org/libinput/doc/latest/udev_config.html
There was a problem hiding this comment.
From a protocol point of view, switching keyboard layout implies sending a new wl_keyboard::keymap event to all clients, and these clients are probably supposed to load a new keymap themselves.
Thats some very interesting info. Can we as a libinput user actually detect, which keyboard is sending a key and just switch the layout if the other is used?
There was a problem hiding this comment.
If the two keyboards has been configured libinput-side to be on different seats, yes.
My take on this would be:
- Have the
InputBackends expose seats like libinput (glutin backend would have a single seat, wayland backend would just forward the seats it receives, x11 backedn I don't know) - Have the
WlSeathelper expect input organized as seats - Allow the user of smithay to map them to each other as they please
This way, a compositor built on smithay could merge two seats if it wants, and juggle with the keymaps if needed.
Then, in a second time, we could provide in smithay a simple helper to match the two sides of this pipeline 1-1, for compositors that just want a simple case. But it's important to still expose the very low APIs. I don't want smithay to be very opinionated as how compositors should behave.
There was a problem hiding this comment.
If the two keyboards has been configured libinput-side to be on different seats, yes.
Again the same problem. I see this is not fixable easily. I don't really get why the protocol supports different keymaps per window, but not between different keyboards (on the same seat).
My take on this would be
Seems reasonable, lets start with that.
There was a problem hiding this comment.
It's not really "different keymap per window", but rather "the compositor advertizes the current keymap to the clients, but the clients are free to use their own". The wl_keyboard::keymap event is only informative. Client are free to use their own keymap config.
There was a problem hiding this comment.
^ It is actually possible to differentiate between devices. And it is necessary for device configuration. I guess we need a way to access devices or more callbacks now.
|
|
||
| //TODO render stuff | ||
|
|
||
| //TODO put input handling on the event loop |
There was a problem hiding this comment.
This will probably be blocked on some work in wayland-server, I need to implement the support of secondary event FDs (needed for libinput support for example) as well as a mechanism to send messages to an event loop.
I am hoping the EventsLoop will be |
|
As per winit's API, the |
|
Any idea how quickly this will be available in glutin? Anyway I know what I need to do for now and will ping you once I got some new stuff to show of. |
I don't know, all I can say is that we can follow rust-windowing/glutin#864 |
|
Okay I have pushed a second version. It is still missing some stuff I want to implement, but at first the changes:
One interesting note here is, that - if I am not mistaken - libinput has no real multi-touch support outside of gestures. The api does not allow to differentiate between different fingers like glutin's api does. The remaining libinput events are not a very high priority, but I also plan to add them later down the line. Documentation is obviously missing and will be added. The glutin translation should mostly be ready, but is of course (like most of the code) untested. glium's Backend trait is also not yet provided, also something on my todo list. |
|
Regarding the API we are converging to for GraphicsBackend and InputBackend traits, I really like the way it's going! 👍 Regarding touch events, and especially multitouch: my understanding is that
I don't know what's best between this and relative events. We want to support "teleporting" the pointer if the compositors wants to (and the backend has this capability, which I believe native and x11 have, and possibly glutin on x11 too). But let's keep that this way for now, we can change it later.
Okay. As for the previous point, this'll do for now, and we can increase the guarantees on that afterwards if needed.
Hmm, not sure what you would mean by that. |
| fn on_pointer_button(&mut self, seat: &Seat, time: u32, button: MouseButton, state: MouseButtonState); | ||
| fn on_pointer_scroll(&mut self, seat: &Seat, time: u32, axis: Axis, source: AxisSource, amount: f64); | ||
| fn on_touch(&mut self, seat: &Seat, time: u32, event: TouchEvent); | ||
| } |
There was a problem hiding this comment.
We need a callback to advertise the capabilities of a seat (keyboard/mouse/touch), the wayland protocol requires this.
There was a problem hiding this comment.
Do we need a callback? I would suggest promoting Seat to a real type and add this via a method. So you can do something like: seat.capabilities() or even seat.capabilities.contain(Capability::Mouse) or something along those lines.
There was a problem hiding this comment.
That could be a way too, but we'll need to properly handle the case where a seat gains or loses a capability during its lifetime. In this context, the callback is the best guarantee I have in mind that the handler will be notified about this change.
There was a problem hiding this comment.
What about doing seat.capabilities and on_seat_changed as a new callback. I feel like capabilities is not something all backends will have and emulation is pretty annoying. This way we could also add other properties to seat's of the other backends.
| }, | ||
| Event::Focused(focused) => if focused { | ||
| handler.on_input_attached(InputDevice::Keyboard(())); | ||
| handler.on_seat_created(&0u32); |
There was a problem hiding this comment.
I'm not sure destroying/recreating the seat each time focus is gained/lost is a proper thing to do.
I'd rather say that the glutin backend has a single everlasting seat, which is advertized a single time in the set_handler method, and never destroyed.
There was a problem hiding this comment.
This was done mostly because I was too lazy to do this outside of process_events and because it helps testing, if you don't need to restart to trigger this event. In theory a compositor should be able to handle this, but I see your point. I will change it.
|
I'm also in favour a creating an pub trait InputBackend<H: InputHandler> {
pub fn set_handler(&mut self, handler: H);
pub fn get_handler(&mut self) -> &mut H;
pub fn clear_handler(&mut self);
}In order to allow writing code generic over input backends, as the |
|
Hmm, github is handling poorly my comments on your second commit, why does it show them as on an "outdated diff"? |
Really no idea. Seems very weird.
I totally looked over that! I have probably mistaken it for
I would suggest adding platform specific behavior to the different
I will document this for later. Supporting this is no issue for libinput at all, just a little more difficult on glutin. Do we want microsecond precision?
Didn't realize this was client only. I have no much knowledge of the wayland protocol, I just thought we are maybe linking libinput's with wayland's seats somehow. I guess an id or a full blown type implementing |
Yes, in most cases absolute coordinates are the simpler. But things like mouse-grabbing protocol extensions (for games mostly) can require to process the relative motion without moving the pointer at all on the screen... Anyway, this is not a priority, but something we'll be able to add/change afterwards, depending on how it appears effectively needed.
For the seats, probably, yes. I'm in favour of a type with |
|
Third draft:
Whats missing:
|
|
Alright the fourth draft fixes all remaining issues, that I wanted to solve before merging. Please give me some final feedback and once everything is good, I will rewrite and cleanup the (ugly) commit history of this branch and open it for merging. |
|
Could you restart the nightly travis job? @vberger |
|
Sure! |
|
|
||
| [dependencies] | ||
| wayland-server = "0.8.4" | ||
| wayland-server = { version = "0.8.4", features = ["dlopen"] } |
There was a problem hiding this comment.
Update to 0.8.6 for fixing the missing symbol error.
Also, we should make the "dlopen" feature conditioned on the glutin feature of smithay
There was a problem hiding this comment.
Any clue on how to do this without renaming the glutin feature?
There was a problem hiding this comment.
Maybe we can accept to rename the feature into something explicit and adopt some convention, I'm thinking something like:
- for the backends:
backend_glutin,backend_x11,backend_wayland,backend_tty - for the opengl apis:
renderer_glium,renderer_gfx, (mayberenderer_vulkanosome time in the future ? ^^)
And in general, a pattern like <function>_<implementation name> for every feature of this kind.
| /// | ||
| /// The linked `GlutinWindowedRenderer` will error with a lost Context and should | ||
| /// not be used anymore as well. | ||
| pub fn process_new_events(&mut self) -> Result<(), GlutinInputError> |
There was a problem hiding this comment.
I've started a trend of calling these kings of methods using the word dispatch, as they are called in the wayland libs. To fit this pattern, I'd name this method dispatch_pending_events.
| { | ||
| if let Some(ref mut handler) = self.handler { | ||
| match event { | ||
| Event::KeyboardInput(state, key_code/*TODO: Is this really the keycode? glutins docs don't tell*/, _) => handler.on_keyboard_key(&self.seat, self.time_counter, key_code as u32, state.into(), 1), |
There was a problem hiding this comment.
For the wayland backend, of glutin, this is the raw keycode (truncated from u16 to u8). It's probably the same for others, afaik.
Also, this 1 means we do not track simultaneous key presses? See my comment on the definition of the trait.
There was a problem hiding this comment.
We don't at least for glutin. I have no idea, how to get that information. Also yes the name and description of the variable is an error, see below.
| ContextLost, | ||
| /// The buffers have already been swapped. | ||
| /// | ||
| /// This error can be returned when `swap_buffers` has been called multiple times |
There was a problem hiding this comment.
"... multiple times without any modification in between.", is that what you mean here?
| pub struct PixelFormat { | ||
| /// Is the format hardware accelerated | ||
| pub hardware_accelerated: bool, | ||
| /// bits used for colors |
There was a problem hiding this comment.
"number of bits...", same for the other fields of this struct
| /// - check if events can arrive out of order. | ||
| /// - Make stronger time guarantees | ||
| fn on_pointer_move(&mut self, seat: &Seat, time: u32, to: (u32, u32)); | ||
| /// Called when a new keyboard event was received. |
There was a problem hiding this comment.
this is a pointer button event
| /// - check if events can arrive out of order. | ||
| /// - Make stronger time guarantees | ||
| fn on_pointer_button(&mut self, seat: &Seat, time: u32, button: MouseButton, state: MouseButtonState); | ||
| /// Called when a new keyboard event was received. |
There was a problem hiding this comment.
this is a pointer scroll event
| /// - check if events can arrive out of order. | ||
| /// - Make stronger time guarantees | ||
| fn on_pointer_scroll(&mut self, seat: &Seat, time: u32, axis: Axis, source: AxisSource, amount: f64); | ||
| /// Called when a new keyboard event was received. |
| } | ||
|
|
||
| fn clear_handler(&mut self) { | ||
| if let Some(ref mut handler) = self.handler { |
There was a problem hiding this comment.
if let Some(handler) = self.handler.take() { ... } and removing the self.handler = None;
|
|
||
| trait NewIdType { | ||
| fn new(id: u32) -> Self; | ||
| } |
There was a problem hiding this comment.
I'm not a huge fan of this trait, but I see the use. I've been using freestanding functions to do the same job in other projects, which is not really better.
Hopefully pub(restricted) will save us somewhere in the future.
There was a problem hiding this comment.
Yes I hate that too, totally unreasonable position, purely dictated by the visibility rules.
|
I like the structure of this overall. Great work! 👍 The only thing in your list that I would like to see added before merging is the advertising of seat capability. Simply something like adding a struct like this: and a callback like this on the fn new_capability(&mut self, seat: Seat, capability: SeatCapability) |
|
Still don't know, if I like this capability thing.
Like this we could also add: The trait is getting quite big already. We (mostly you 😄 ) should decide, if we want to hide some less often used properties/events in the Also about the devices: I expect that device configuration will be very different (or even not available) for all backends. How should we best deal with that? Maybe make the |
|
Hmm, so you suggest we could have the Regarding device configuration, to be honest I don't understand what kind of configuration exists and should be exposed to the user... ? |
|
I just have the fear, that it is too annoying to figure out what exactly has changed. Then again, if you really care, you can most likely just do the whole initialization of the device configurations again anyway without caring about the exact differences. Similar things should be doable for capabilities. Most important configuration flags are e.g. for enabling tab_to_click on a track pad, or to control its sensitivity. |
|
Hmm, I see... As you said, these configuration will very likely be dependant on the backend (actually, I think only the libinput backend will have them). I think this kind of stuff could remain as inherent impls on the backend type, don't you think? |
But you need callbacks for that stuff as well. Would that mean, that you have to register two different handlers on the libinput backend in that case? |
|
Callbacks too here? What are the events that would be callback-ed on libinput? Is it just "a new device has been connected/disconnected", or something more complex? |
|
Also, as I've merged the inclusion of rustfmt & clippy in the travis build, could you rebase this PR on master? |
|
Sure. |
|
I've been looking at weston's internals for #12, and I think we should find a better name for the graphics backend traits: they are not renderers, but rather the renderers will be built upon them (and may be part of smithay at some point too). |
|
What about "graphics backends" then? |
|
So |
e6c8929 to
369c8a9
Compare
|
Okay! Next round @vberger . I hope I have not forgotten anything. |
| } | ||
|
|
||
| impl<T: OpenglRenderer> Backend for T | ||
| pub struct GliumGraphicBackend<T: OpenglGraphicsBackend>(T); |
There was a problem hiding this comment.
Why did you need to introduce this struct and trait?
|
(Ugh, again github showing my review as "outdated".) Anyway, I like it ! 👍 I just wonder why you had to introduce this wrapper for glium, but otherwise I think we can merge this as a good starting point. We'll probably need to make these API change & evolve as we dig into the other functionalities and discover needs & constraints we had not anticipated anyway 😅 |
|
I cannot do |
|
Oh right, I see. Forgot about coherence rules. All is right then. 👍 |
|
Okay, let me fix the formatting and remove the WIP tag an then feel free to merge. |
|
Okay formatting is not fixed, because your travis config still uses rustfmt 0.7.*, which I did format with rustfmt 0.8. I guess we need to force install rustfmt or use something like cargo-update. |
|
Oh. Yes, I guess we could add something like this to the travis.yml Can you add a commit doing it in this PR? |
|
Hmm, I think we can remove the reporting of TODOs and FIXMEs. This is far too early for it... 😅 |
|
Okay done. Lets hope that's it for this PR. |
|
The failure is clippy itself not compiling... welp, I'll merge. |
|
Maybe for the future (because this can happen with any new nightly) allow nightly builds to fail. |
So here is a good amount of code I have been experimenting and playing with and that needs a lot of discussion, which is why I am uploading it (please don't merge yet, it doesn't even compile).
backend/graphicsmodule right now. The OpenglRenderer is currently only coping traits and types fromglutinandglium. I also made sure there is a way to layergfx-rson top of that, but currently there is a lot of code duplication happening right now. I feel there is probably need for a general opengl context library, that just describes how aGlContextshould look like and that every library uses. But that will not happen any time soon and I don't find it troublesome, if every backend just provides it's own rendering api. We can perhaps layer a very simple optional one for drawing windows on top of all of them. But it is fact, that every rendering backend will work totally different and it does not seem reasonable to merge all OpenGL backends together.InputDevicegeneric. But I think this approach avoids much code duplication when dealing with multiple backends at once. If you don't care about the internal identifier of the different backends you should be able to handle all of them with a single handler.