Separate UI and server into two processes & Implement windows installer #170

Dimi20cen · 2025-07-01T17:48:13Z

Summary

This pull request refactors the Kolibri Windows application to use a two-process architecture, separating the UI from the Kolibri server and introduces a Windows installer with optional service support. This is the part of a larger effort to modernize the Windows installer and application behavior.

Explanation of core changes (Windows):

Process Separation (UI vs. Server):
- What: The main application (kolibri_app) now acts as a lightweight UI process, responsible only for the wxPython window and webview. It spawns the Kolibri server as a separate, hidden child subprocess using the same executable but with a new --run-as-server flag.
Robust Lifecycle Management with Job Objects:
- What: On launch, the UI process creates a Windows Job Object and assigns the server subprocess to it.
- Why: The Job Object is configured with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE. That way, if the main UI process is closed or crashes, the operating system will terminate the server subprocess. This prevents orphaned or "zombie" Kolibri processes from being left behind.
Reliable Startup Handshake with Named Pipes:
- What: A pull-based Inter-Process Communication (IPC) mechanism using Named Pipes.
  1. The UI process starts its named pipe client and tries to connect.
  2. It launches the server process.
  3. Inside the server process, the new WindowsIpcPlugin creates the named pipe server and waits for a connection.
  4. Once connected, the UI client sends a request_server_info message.
  5. The server only responds with the port and URL after the Kolibri backend is fully initialized and ready to serve requests.
- Why: This pull-based handshake is important for avoiding race conditions. The UI waits for an explicit signal, ensuring that the first URL loaded is always valid and the server is prepared to handle it.
Windows Installer with Service Support:
- What: Inno Setup installer that provides two installation modes:
  - Desktop application only
  - Windows service + desktop application (runs on system boot)
- Features:
  - Automatic WebView2 runtime installation if needed
  - Version detection with upgrade/downgrade/repair logic
  - NSSM integration that handles service management
  - Service runs as LocalService account with appropriate file permissions

The existing in-process, thread-based behavior for macOS and Linux is preserved and refactored into server_manager_posix.py to ensure no regressions on those platforms.

Code Pointers

src/kolibri_app/application.py: The central integration point. Note the conditional import of WindowsServerManager and PosixServerManager and how start_server() and shutdown() direct to the appropriate manager based on the OS.
src/kolibri_app/server_manager_windows.py: (UI Process Logic) Contains the client-side code for Windows. This file is responsible for launching the subprocess, creating and managing the Job Object, and handling the named pipe client communication.
src/kolibri_app/server_process_windows.py: (Server Process Logic) This is the entry point for the server subprocess (--run-as-server). It initializes Kolibri and subscribes the WindowsIpcPlugin, which is responsible for managing the named pipe server and responding to requests from the UI process.
src/kolibri_app/main.py: Note the new argument parsing at the top of main() that routes execution to ServerProcess when the --run-as-server flag is present.
kolibri.iss: Inno Setup script that defines the Windows installer behavior, including service installation, permissions, and WebView2 deployment.
Makefile: New targets for building the Windows installer (build-installer-windows) and managing dependencies (WebView2, NSSM).

References

Fixes #167, concerning the UI/Server Process seperation for Windows and implementing Windows Service

Reviewer guidance

The most critical area for review is the new Windows-specific logic, including both the process separation architecture and the installer/service functionality.

How to test on Windows:

Install dependencies again, (make dependencies).
Run

make get-whl whl="https://github.com/learningequality/kolibri/releases/download/v0.18.1/kolibri-0.18.1-py2.py3-none-any.whl"

Run make pyinstaller to build the application.
Run the executable located in dist/Kolibri-0.18.1/Kolibri-0.18.1.exeto test the standalone app.
To test the installer: Run make build-installer-windows (requires Inno Setup installed)
Then run the installer located in dist-installer/kolibri-setup-0.18.1.exe

Primary focus for testing:

Standalone Application Tests:

Standard Startup & Shutdown:
1. Launch the app. Does the UI appear and show the loading screen?
2. Does the Kolibri interface load successfully without errors?
3. Close the main application window.
4. Open Task Manager and confirm that the Kolibri-0.18.1.exe and Kolibri processes terminate completely within a few seconds. There should be no lingering processes.
Robustness Test (Process Crash Simulation):
1. With the application running, open Task Manager. You should see two processes one Kolibri-0.18.1.exe process and a Kolibri process. (Kolibri-0.18.1.exe is the server and Kolibri is the ui).
2. Find the server subprocess (Kolibri-0.18.1.exe) and manually terminate it ("End Task").
3. The main application window should remain open, but the webview will likely show a connection error. The UI itself should not crash.
4. Now, close the main application window. It should close cleanly.

Installer

Fresh Installation Tests

Clean Install with Service:
- Run installer on a system with no previous Kolibri installation
- Select "Install Kolibri as a Windows Service" option
- Verify installer completes successfully
- Check that service appears in Windows Services (services.msc) as "Kolibri"
- Verify service is set to "Automatic" startup
- Check Task Manager shows only the service process running
- Launch Kolibri from desktop/start menu
- Verify UI connects to existing service (should see log: "Detected that the 'Kolibri' service is running")
- Verify Kolibri loads correctly
Clean Install without Service:
- Run installer on a clean system
- Do NOT select service option
- Verify desktop app launches after installation
- Check Task Manager shows two processes (UI + server subprocess) running as the user (Check the "User Named" in Task Manager->Details)
- Verify no service is created in Windows Services

Upgrade/Downgrade Tests

Upgrade from Older Version:
- Install an older version first
  - or go to regedit -> Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Kolibri and modify the value of "Version" to and older version e.g., 0.18.0
- Run new installer
- Verify upgrade dialog appears
- Accept upgrade
- Verify files are updated correctly
- Check service configuration is preserved if it existed
Downgrade Prevention:
- go to regedit -> Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Kolibri and modify the value of "Version" to and future version e.g., 0.18.2
- Try to install
- Verify installer prevents downgrade
Same Version Reinstall:
- Install current version
- Run installer again
- Verify repair dialog appears
- Accept repair
Changing Installation Mode
- Install WITHOUT service option
- Run installer again, SELECT service option
- Verify service is installed
- Run installer again, DESELECT service option
- Verify service is successfully removed

Service Management Tests

Service Stop:
- Install with service option
- Stop service via Windows Services
- Verify UI now creates a server subprocess and sucessfully connects to it
System Reboot:
- Install with service
- Reboot system
- Verify service starts automatically
- Launch UI and verify it connects to service

WebView2 Not Installed Test:

Use a system without WebView2 runtime
- OR in regedit, remove this key: Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\EdgeUpdate\Clients\{F3017226-FE2A-4295-8BDF-00C3A9A7E4C5}
- and delete this file: "C:\Program Files (x86)\Microsoft\EdgeWebView\Application\138.0.3351.83\msedgewebview2.exe"
- Then run the installer (the WebView2 installer should run silently) and both the key and the msedgewebview2.exe should be restored and the UI should work

Uninstallation Test:

Complete Uninstall with Service:
- Install with service option
- Run uninstaller ("C:\Program Files\Kolibri\unins000.exe") or uninstall from Windows Setting->apps->Installed Apps
- Verify service is stopped and removed
  - Check sevices.msc - no Kolibri service should exist
- Verify UI process is terminated, if it was open before uninstallation
- Verify C:\Program Files\Kolibri is removed
- Verify C:\ProgramData\kolibri is removed

Development vs Production Path Tests

Production Build (PyInstaller):
- Verify KOLIBRI_HOME is set to C:\ProgramData\kolibri
Development Mode:
- Run from source (not PyInstaller build), make run-dev
- Verify KOLIBRI_HOME is set to C:\Users\<user>\.kolibri_dev

SILENT and VERYSILENT Installation

Test command line silent install: kolibri-setup-0.18.1.exe /SILENT
- Verify installation completes without user interaction
Test command line very silent install: kolibri-setup-0.18.1.exe /VERYSILENT
- Verify installation completes without user interaction and without any windows opening

Testing on macOS/Linux (Regression Check):

Build and run the application on a non-Windows machine to confirm that there are no regressions. The behavior should be identical to the main branch.
Expected Behavior: The application should start, load Kolibri, and shut down exactly as it does on the main branch. This refactoring should have introduced no functional changes on non-Windows platforms.

This refactors the Windows application to use a robust two-process architecture, separating the UI from the Kolibri server. This is a foundational change to improve stability, responsiveness, and enable future Windows-specific features like a background service. The architecture includes: - Process Spawning: The main executable now detects a `--run-as-server` flag to launch in a headless server mode. The UI process spawns this server as a hidden child process. - Job Object Lifecycle Management: The server subprocess is assigned to a Windows Job Object. This ensures the OS automatically and reliably terminates the server if the main UI application closes or crashes, preventing orphaned processes. - Named Pipe IPC: A pull-based handshake using named pipes is implemented for communication. The UI process requests connection information, and the server only responds after Kolibri has fully initialized, preventing startup race conditions. - POSIX Refactor: The existing single-process, threaded model for macOS and Linux has been refactored into `server_manager_posix.py` to preserve existing behavior and prevent regressions on non-Windows platforms. Partially addresses learningequality#167

Dimi20cen · 2025-07-13T14:04:48Z

I think rebasing the branch to main worked (thumbs up emoji)

- Implement Inno Setup installer script (kolibri.iss) for Windows deployment - Add Windows service installation option using NSSM (Non-Sucking Service Manager) - Include WebView2 runtime installer in setup package - Add service detection to UI process to connect to existing service instead of spawning subprocess - Configure KOLIBRI_HOME to use system-wide ProgramData folder for production builds - Add installer build targets to Makefile (build-installer-windows, codesign-installer-windows) - Implement upgrade/downgrade/repair logic in installer with version detection - Configure proper file permissions for service account (LocalService) and users - Add security descriptor for named pipe to allow UI-service communication The installer provides two installation modes: 1. Desktop application only 2. Windows service + desktop application Fixes learningequality#167

ozer550

Iv'e tested the Pr locally. I noticed that if the Kolibri service is stopped via services.msc while the app is still running, the UI repeatedly shows "Disconnected from the server" and doesn’t recover. To get it working again, I had to manually close and reopen the app. Other than that some other code improvements and some other type hint issues.

kolibri.iss

src/kolibri_app/__init__.py

src/kolibri_app/server_manager_windows.py

src/kolibri_app/server_process.py

Makefile

rtibbles

Some questions - mostly just interested in how we can keep the architecture clean and avoid too many nested processes/threads/calls etc.

Makefile

src/kolibri_app/__init__.py

src/kolibri_app/server_manager_windows.py

src/kolibri_app/server_process.py

The `WindowsServerManager` can now handle two failure scenarios: 1. Service Disconnection: If the app is connected to the Kolibri Windows service and the service stops, the app will now attempt to reconnect a few times. If it fails, it will automatically fall back to launching a local server for the duration of the session, preventing the UI from getting stuck. 2. Local Server Crash: If the app is running its own local server process and that process crashes or terminates unexpectedly, it will now be automatically restarted.

…s, improve UX for downgrades - Change LocalService permissions from Full Control to Modify - Add path constants to reduce duplication - Define AppGuid and WebView2RuntimeGuid constants with MS documentation links - Downgrade detection message now shows both installed and installer versions - Fix NSSM path in UninstallRun to match actual installation location

This commit refactors the Windows server subprocess (`server_process.py` renamed to `server_process_windows.py`). - A new `WindowsIpcPlugin` now encapsulates all named pipe communication logic, replacing the previous manually managed `PipeServerThread`. - The plugin uses the `START`, `STOP`, and `SERVING` events from `KolibriProcessBus` to manage its lifecycle and state, ensuring cleaner setup and teardown. - The `ServerProcess` class is simplified to be an initializer for Kolibri and the bus/plugin. - Large methods within the plugin have been broken down into smaller, single-responsibility functions to improve readability. - Error handling is now more specific, replacing blanket `except Exception` clauses.

…anager_windows.py) - Decomposed the monolithic `_launch_server_process` and `_handle_pipe_error` methods into smaller, single-responsibility helper functions. - Replaced blanket `except Exception` clauses with specific exception types (e.g., `OSError`, `subprocess.SubprocessError`, `pywintypes.error`, `JSONDecodeError`).

…Dimi20cen/kolibri-app into feat/windows/separate-processes

ozer550

Ive tested the changes locally and all previously mentioned issues are now fixed. Commented some other suggestions that caught my eye.

src/kolibri_app/server_process_windows.py

kolibri.iss

A race condition could occur in the WindowsIpcPlugin where two threads could attempt to close the named pipe handle simultaneously. This could happen if a client disconnected at the same moment the server was shutting down, leading to an error. This change introduces a threading.Lock, preventing this scenario.

rtibbles · 2025-08-07T14:52:24Z

@pcenov @radinamatic the goal here is just to do a regression test on the Mac App asset - no need to do any testing of the (not yet complete) Windows App.

If you find any regressions with the Mac App, please file as follow up issues, as @Dimi20cen doesn't have a Mac to develop on.

pcenov · 2025-08-11T11:44:38Z

Hi @rtibbles - I didn't observe any regressions caused by the changes made here while testing the Mac app.

ozer550

LGTM! Nice work.

Dimi20cen force-pushed the feat/windows/separate-processes branch from ea07ad9 to bf802e4 Compare July 13, 2025 14:02

Dimi20cen changed the title ~~Separate UI and server into two processes~~ Separate UI and server into two processes & Implement windows installer Jul 14, 2025

ozer550 self-assigned this Jul 14, 2025

ozer550 requested changes Jul 16, 2025

View reviewed changes

rtibbles reviewed Jul 16, 2025

View reviewed changes

Makefile Outdated Show resolved Hide resolved

rtibbles reviewed Jul 16, 2025

View reviewed changes

Dimi20cen and others added 12 commits July 17, 2025 15:04

Add option for deleting user data during app uninstallation

87020d9

Revert back to using .kolibri as the KOLIBRI_HOME for make run-dev

070dc55

Fix pyright error

fe561d1

Remove redundant subscription()

fa7b408

Merge branch 'main' into feat/windows/separate-processes

71b7fda

Merge branch 'feat/windows/separate-processes' of https://github.com/…

f29f1ef

…Dimi20cen/kolibri-app into feat/windows/separate-processes

Define nssm version at the top of Makefile

5e08b2a

Fix bug: Now local server auto-restarts after service fallback

a93f7a8

Dimi20cen marked this pull request as ready for review July 28, 2025 10:12

ozer550 reviewed Aug 1, 2025

View reviewed changes

src/kolibri_app/server_process_windows.py Show resolved Hide resolved

kolibri.iss Show resolved Hide resolved

ozer550 approved these changes Aug 11, 2025

View reviewed changes

ozer550 merged commit 7594364 into learningequality:main Aug 11, 2025
4 checks passed

Dimi20cen mentioned this pull request Aug 17, 2025

Add Localization Support to the Inno Setup Installer #181

Open

Separate UI and server into two processes & Implement windows installer #170

Separate UI and server into two processes & Implement windows installer #170

Uh oh!

Conversation

Dimi20cen commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Explanation of core changes (Windows):

Code Pointers

References

Reviewer guidance

How to test on Windows:

Primary focus for testing:

Standalone Application Tests:

Installer

Uh oh!

Dimi20cen commented Jul 13, 2025

Uh oh!

ozer550 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rtibbles left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ozer550 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rtibbles commented Aug 7, 2025

Uh oh!

pcenov commented Aug 11, 2025

Uh oh!

ozer550 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Dimi20cen commented Jul 1, 2025 •

edited

Loading

ozer550 left a comment •

edited

Loading