Skip to content

Add comprehensive touchscreen and mouse support with fullscreen mode#91

Closed
jcarr71 wants to merge 1 commit intolibretro:masterfrom
jcarr71:pr-touchscreen-features
Closed

Add comprehensive touchscreen and mouse support with fullscreen mode#91
jcarr71 wants to merge 1 commit intolibretro:masterfrom
jcarr71:pr-touchscreen-features

Conversation

@jcarr71
Copy link
Copy Markdown
Contributor

@jcarr71 jcarr71 commented Nov 12, 2025

Overview

This PR adds comprehensive touchscreen and mouse support for the FreeIntv core across all platforms (Android, Windows, and Linux), along with a flexible fullscreen mode and PNG-based overlay system.

Key Features

  • Dual-Screen Rendering: Game screen (704×448) + Keypad overlay (370×600) side-by-side
  • Utility Buttons:
    • Toggle Layout: Switch game/keypad positions
    • Fullscreen Toggle: Switch between dual-screen and fullscreen modes
    • Overlay Toggle: Show/hide keypad in fullscreen mode
  • Auto-Hide Control Strip: Fullscreen mode includes a 5-second auto-hide strip that can be revealed by swiping from the bottom
  • PNG-Based Overlay System: Support for custom game-specific keypad graphics via PNG files
  • 12-Button Hotspot Detection: Visual feedback on button presses with configurable highlighting
  • Cross-Platform: Full support for Android (all ABIs), Windows (MinGW), and Linux
  • Backwards Compatible: Fully functional with existing RetroPad controllers; touchscreen is optional

Technical Details

  • Modified src/libretro.c: Added ~2200 lines for rendering, input handling, and fullscreen logic
  • Updated build configs: Makefile.common, jni/Android.mk, .gitlab-ci.yml
  • Added comprehensive documentation: New USER_GUIDE.md and updated README.md

Testing

  • ✅ Builds successfully on all platforms
  • ✅ Dual-screen mode works with controller input
  • ✅ Fullscreen mode with auto-hide functions correctly
  • ✅ Backwards compatible with existing code (no breaking changes)
  • ✅ Graceful degradation when PNG assets are unavailable

User-Facing Changes

  • New installation section in README.md for asset files
  • Comprehensive user guide covering all new features
  • Optional PNG assets for enhanced UI (users can still run without them)

Co-authored-by

Jason Carr (jcarr71)

… mode

- Implement dual-screen rendering (game + keypad overlay) for all platforms
- Add three utility buttons: Toggle Layout, Fullscreen Toggle, Overlay Toggle
- Implement fullscreen mode with auto-hide control strip (5 second timeout)
- Add PNG-based overlay system for game-specific keypad graphics
- Support 12-button hotspot detection with visual feedback
- Cross-platform support: Android, Windows (MinGW), and Linux
- Fully backwards compatible with existing RetroPad controllers
- Add comprehensive user guide and documentation
@jcarr71
Copy link
Copy Markdown
Contributor Author

jcarr71 commented Nov 19, 2025

@msheehan79 @markwkidd any chance one of you could take a look at this proposed update when you have a moment? thanks!

@msheehan79
Copy link
Copy Markdown
Contributor

@jcarr71 happy to give it a try!

I only use Windows x64 so can provide feedback on that platform. I just tried to compile your PR but got this error (I use msys2 to compile here). The regular branch of freeintv compiles OK for me.

src/libretro.c:39:10: fatal error: stb_image.h: No such file or directory 39 | #include "stb_image.h" | ^~~~~~~~~~~~~ compilation terminated. make: *** [Makefile:655: src/libretro.o] Error 1

@jcarr71
Copy link
Copy Markdown
Contributor Author

jcarr71 commented Nov 20, 2025

looks like it's not finding the stb_image.h file that is in the src folder. Do you see that in your local copy? I used MingW64 on Windows 11 x64, but I don't think the error is pointing to your compiler - more like it can't find that header file.

Here's a link in case it's missing for some reason :

https://github.com/jcarr71/FreeIntvTSOverlay/blob/2396b05a0e562753555963020794137c0fa06dd7/src/stb_image.h

@msheehan79
Copy link
Copy Markdown
Contributor

Yes, looks like a few files are missing from your PR that are needed for it to build - I grabbed stb_image_impl.c, stb_image.h and your modified versions of controller.c and controller.h from your repo and with those files added to src it builds. I'll drop it in and try it out shortly in my RA setup.

@msheehan79
Copy link
Copy Markdown
Contributor

Overall, this is really nice! Played around with it a bit and I have a few thoughts, this is coming from a Desktop/controller setup point of view:

  • When I first load up a game without putting any of the Assets/Overlays in from your setup guide, I get a screen with blocked out sections for where the Overlay and Hotkey buttons should go. Without the images, its difficult to know how to toggle the overlay on or off etc. This could be jarring if someone just updates the core and isn't aware of this update to download assets etc, it might be best to put this new rendering behind a Core Option that could be toggled on/off if possible.

  • I noticed on my PC when I ran it in the fullscreen layout the performance tanked compared to the overlay on the side view. I didn't test too much but the difference in FPS was very noticeable. On a different RA install with default settings it wasn't as noticeable as my main setup where I have things set for lower latency etc, but it did definitely impact things. If the overlays add a performance hit at least having a core option to turn it off/stick with the original rendering only will be helpful for weaker devices.

  • As far as quality of life features (Not for this PR but just general thoughts on where this could go later), I would love to see more controller-friendly options for the utility buttons you added to toggle the overlays and view modes

  • more layout options where you could shrink the Overlay in favor of a larger screen size (an example would be how the MelonDS cores have 2 different hybrid ratios to show the 2nd screen as smaller or larger with a corresponding increase/decrease in the primary screen size)

image

@jcarr71
Copy link
Copy Markdown
Contributor Author

jcarr71 commented Nov 20, 2025

Thanks for the feedback! That definitely gives me some ideas for better implementation. I'll pull this back for now and will work on some ideas over the next week or so and we'll try again. I didn't notice the slowdown, but will certainly do some more testing. I started this geared towards an alternative core to use, but with it potentially being integrated into the main, I've got some more digging to do to make sure I don't hinder the basic functionality for those not interested in the additions.

@jcarr71
Copy link
Copy Markdown
Contributor Author

jcarr71 commented Nov 20, 2025

Closing this pull request, as I am going to fine tune this a bit more before resubmitting.

@jcarr71 jcarr71 closed this Nov 20, 2025
@jcarr71 jcarr71 deleted the pr-touchscreen-features branch November 20, 2025 22:21
@msheehan79
Copy link
Copy Markdown
Contributor

@jcarr71 No worries - I can give the revised one a test when you are ready. I don't have a lot of time at the moment but I will also try and troubleshoot the fullscreen performance drop if I can, or at least share my settings. I have my main system settings pretty aggressive (runahead enabled, bfi, low latency video settings, 4K resolution etc) so its probably a lot more resource-hungry than the default settings and why it shows the issue.

Ultimately I think your additions will be a big plus, I have struggled with the lack of the overlays a lot when playing but just didn't have any real video coding knowledge for how to incorporate it into the core itself - the best I managed to do was to put the overlays into my frontend (Pegasus) theme so I could try and memorize them before I launched a game :)

@jcarr71
Copy link
Copy Markdown
Contributor Author

jcarr71 commented Nov 25, 2025 via email

@msheehan79
Copy link
Copy Markdown
Contributor

Great, I will give it a shot! I probably won't have too much time to try it until the weekend, but will certainly give feedback when I can. I was able to download the asset files from your repo last time, so unless the structure or files changed with this PR I should be good to go to test!

@markwkidd
Copy link
Copy Markdown
Contributor

markwkidd commented Nov 25, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants