Fix WinUI tray menu crash with invisible anchor window pattern

TRAY_MENU_CRASH_FIX.md

@@ -0,0 +1,139 @@
+# WinUI 3 Tray Menu MenuFlyout Crash - Solution Documentation
+
+## Problem Summary
+
+The WinUI 3 tray application was experiencing intermittent crashes when clicking the tray icon to show the menu. The crashes occurred in the native WinUI windowing layer (`Microsoft.UI.Windowing.Core.dll`) and bypassed all .NET exception handlers.
+
+### Crash Characteristics
+- **Faulting module**: `Microsoft.UI.Windowing.Core.dll` (version 10.0.27108.1025)
+- **Exception code**: `0xe0464645` (CLR exception marker)
+- **Platform**: Windows 11 ARM64
+- **Windows App SDK**: 1.8.250906003
+- **Timing**: Crashes became more likely after idle periods or after opening/closing other windows
+
+## Root Cause
+
+WinUI 3's `MenuFlyout.ShowAt()` requires a valid UIElement with proper visual tree context to display correctly. When attempting to show a MenuFlyout directly from a TrayIcon event handler:
+
+1. The TrayIcon doesn't provide a proper WinUI UIElement context
+2. MenuFlyout cannot find a valid visual tree to anchor to
+3. This causes native-level crashes in the windowing layer
+
+### Research References
+
+Based on community research:
+- [WinUIEx TrayIcon issues](https://github.com/dotMorten/WinUIEx/issues/244) - MenuFlyout positioning and stability issues
+- [MenuFlyout crash in Windows App SDK](https://github.com/microsoft/microsoft-ui-xaml/issues/8954) - Known MenuFlyout crash issues
+- [Stack Overflow discussion](https://stackoverflow.com/questions/79008202/) - MenuFlyout without Window or UIElement
+
+## Solution: Invisible Anchor Window
+
+Instead of showing the MenuFlyout directly from the tray icon, we now use a small invisible anchor window that provides the required UIElement context.
+
+### Implementation
+
+1. **TrayMenuAnchorWindow** (`Windows/TrayMenuAnchorWindow.xaml[.cs]`)
+   - A minimal 1x1 pixel window with a transparent Grid
+   - Configured to not appear in task switchers
+   - Positioned at the cursor location when showing the menu
+   - Reused across menu invocations to avoid creation/destruction overhead
+
+2. **App.xaml.cs Updates**
+   - Maintains strong reference to anchor window (`_trayMenuAnchor`)
+   - `ShowTrayMenuFlyoutWithAnchor()` method positions anchor and shows flyout
+   - Anchor window is created once on first use and kept alive
+
+### Key Code Pattern
+
+```csharp
+// In App.xaml.cs
+private TrayMenuAnchorWindow? _trayMenuAnchor; // Keep-alive for GC prevention
+
+private void ShowTrayMenuFlyoutWithAnchor()
+{
+    // Create anchor window once, reuse thereafter
+    if (_trayMenuAnchor == null)
+    {
+        _trayMenuAnchor = new TrayMenuAnchorWindow();
+    }
+
+    // Position at cursor
+    if (GetCursorPos(out POINT cursorPos))
+    {
+        _trayMenuAnchor.PositionAtCursor(cursorPos.X, cursorPos.Y);
+    }
+
+    // Show flyout anchored to window
+    var flyout = BuildTrayMenuFlyout();
+    _trayMenuAnchor.ShowFlyout(flyout);
+}
+```
+
+## Why This Works
+
+1. **Valid Visual Tree**: The anchor window provides a proper WinUI visual tree for MenuFlyout to attach to
+2. **Proper Lifecycle**: Window is kept alive to prevent garbage collection issues
+3. **Correct Positioning**: Window is positioned at cursor, so MenuFlyout appears in the right location
+4. **Reusability**: Single window is reused, avoiding creation/destruction overhead that could trigger crashes
+
+## Alternatives Considered
+
+### 1. Custom Window Popup (Original Approach)
+- **Tried**: Creating/destroying `TrayMenuWindow` on each click
+- **Problem**: Rapid window creation/destruction triggered native crashes
+- **Result**: Abandoned
+
+### 2. Window Reuse Pattern
+- **Tried**: Hide() instead of Close() on deactivation
+- **Problem**: Black square appeared instead of menu content
+- **Result**: Abandoned
+
+### 3. Direct MenuFlyout Assignment
+- **Tried**: `e.Flyout = BuildTrayMenuFlyout()` in tray event handlers
+- **Problem**: No valid UIElement anchor, causing crashes
+- **Result**: This was the problematic approach we replaced
+
+### 4. Native Win32 Popup Menu
+- **Considered**: Using `TrackPopupMenu` Win32 API
+- **Decision**: Rejected - would lose WinUI styling and XAML flexibility
+- **Note**: Could be future fallback if anchor window approach fails
+
+## Testing Recommendations
+
+Since this is a race condition / timing-sensitive crash, testing should include:
+
+1. **Basic functionality**: Click tray icon multiple times in succession
+2. **Idle scenario**: Leave app idle for several minutes, then click tray icon
+3. **Window interaction**: Open/close Settings window, then click tray icon
+4. **Rapid clicking**: Click tray icon rapidly to test window reuse
+5. **Long session**: Run app for extended period with periodic tray clicks
+
+## Future Considerations
+
+### Monitoring
+- Watch for any new crash reports in `%LOCALAPPDATA%\OpenClawTray\crash.log`
+- Monitor Windows Event Viewer for exceptions in `Microsoft.UI.Windowing.Core.dll`
+
+### Potential Improvements
+1. **Alternative Libraries**: Consider switching to H.NotifyIcon if issues persist
+2. **SDK Updates**: Monitor Windows App SDK releases for native fixes
+3. **Telemetry**: Add success/failure tracking for menu display operations
+
+## Related Issues
+
+- GitHub Issue: [link to be added when issue is created]
+- WinUI GitHub: https://github.com/microsoft/microsoft-ui-xaml/issues/8954
+- WinUIEx GitHub: https://github.com/dotMorten/WinUIEx/issues/244
+
+## Credits
+
+Solution based on community research and best practices from:
+- WinUIEx documentation and issue discussions
+- Microsoft Learn WinUI 3 documentation
+- Community blog posts on WinUI 3 tray icon implementations
+- Stack Overflow discussions on MenuFlyout anchoring
+
+---
+
+**Last Updated**: 2026-01-30
+**Author**: GitHub Copilot (with human review)

src/OpenClaw.Tray.WinUI/App.xaml.cs

@@ -13,6 +13,7 @@
 using System.IO;
 using System.IO.Pipes;
 using System.Linq;
+using System.Runtime.InteropServices;
 using System.Threading;
 using System.Threading.Tasks;
 using Updatum;
@@ -59,6 +60,7 @@ public partial class App : Application
     private StatusDetailWindow? _statusDetailWindow;
     private NotificationHistoryWindow? _notificationHistoryWindow;
     private TrayMenuWindow? _trayMenuWindow;
+    private TrayMenuAnchorWindow? _trayMenuAnchor; // Keep-alive anchor for MenuFlyout
 
     private string[]? _startupArgs;
     private static readonly string CrashLogPath = Path.Combine(
@@ -187,14 +189,67 @@ private void InitializeTrayIcon()
 
     private void OnTrayIconSelected(TrayIcon sender, TrayIconEventArgs e)
     {
-        // Left-click: show flyout menu (avoids window creation crash)
-        e.Flyout = BuildTrayMenuFlyout();
+        // Left-click: show flyout menu using anchor window to prevent crash
+        ShowTrayMenuFlyoutWithAnchor();
     }
 
     private void OnTrayContextMenu(TrayIcon sender, TrayIconEventArgs e)
     {
-        // Right-click: show flyout menu
-        e.Flyout = BuildTrayMenuFlyout();
+        // Right-click: show flyout menu using anchor window to prevent crash
+        ShowTrayMenuFlyoutWithAnchor();
+    }
+
+    [DllImport("user32.dll")]
+    [return: MarshalAs(UnmanagedType.Bool)]
+    private static extern bool GetCursorPos(out POINT lpPoint);
+
+    [StructLayout(LayoutKind.Sequential)]
+    private struct POINT
+    {
+        public int X;
+        public int Y;
+    }
+
+    /// <summary>
+    /// Shows the tray menu using an invisible anchor window to prevent crashes.
+    /// 
+    /// WinUI 3 MenuFlyout requires a valid UIElement anchor. TrayIcon doesn't provide
+    /// proper context, causing crashes. This method uses a tiny invisible window positioned
+    /// at the cursor as the anchor point.
+    /// 
+    /// See TRAY_MENU_CRASH_FIX.md for detailed explanation.
+    /// </summary>
+    private void ShowTrayMenuFlyoutWithAnchor()
+    {
+        try
+        {
+            // Ensure anchor window exists (created once, reused)
+            if (_trayMenuAnchor == null)
+            {
+                _trayMenuAnchor = new TrayMenuAnchorWindow();
+            }
+
+            // Get cursor position for positioning the anchor window
+            if (GetCursorPos(out POINT cursorPos))
+            {
+                // Position the tiny anchor window at cursor location
+                _trayMenuAnchor.PositionAtCursor(cursorPos.X, cursorPos.Y);
+            }
+            else
+            {
+                // Fallback: position offscreen if we can't get cursor
+                _trayMenuAnchor.PositionOffscreen();
+            }
+
+            // Build and show the flyout anchored to the window
+            var flyout = BuildTrayMenuFlyout();
+            _trayMenuAnchor.ShowFlyout(flyout);
+        }
+        catch (Exception ex)
+        {
+            LogCrash("ShowTrayMenuFlyoutWithAnchor", ex);
+            Logger.Error($"Failed to show tray menu: {ex.Message}");
+        }
     }
 
     private MenuFlyout BuildTrayMenuFlyout()

src/OpenClaw.Tray.WinUI/Windows/TrayMenuAnchorWindow.xaml

@@ -0,0 +1,9 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Window
+    x:Class="OpenClawTray.Windows.TrayMenuAnchorWindow"
+    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
+    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
+
+    <!-- Invisible 1x1 window that serves as an anchor for tray menu flyouts -->
+    <Grid x:Name="RootGrid" Background="Transparent" />
+</Window>

src/OpenClaw.Tray.WinUI/Windows/TrayMenuAnchorWindow.xaml.cs

@@ -0,0 +1,104 @@
+using Microsoft.UI.Xaml;
+using Microsoft.UI.Xaml.Controls;
+using System;
+using System.Runtime.InteropServices;
+
+namespace OpenClawTray.Windows;
+
+/// <summary>
+/// A minimal, invisible window used as an anchor point for displaying MenuFlyout from tray icon.
+/// 
+/// BACKGROUND:
+/// WinUI 3's MenuFlyout.ShowAt() requires a valid UIElement with proper visual tree context.
+/// TrayIcon doesn't provide this context, causing crashes in Microsoft.UI.Windowing.Core.dll.
+/// 
+/// SOLUTION:
+/// This 1x1 pixel window is positioned at the cursor and provides the required anchor.
+/// It's created once and reused to avoid creation/destruction overhead.
+/// A strong reference is maintained in App to prevent garbage collection.
+/// 
+/// See TRAY_MENU_CRASH_FIX.md for detailed documentation.
+/// </summary>
+public sealed partial class TrayMenuAnchorWindow : Window
+{
+    [DllImport("user32.dll")]
+    private static extern bool SetWindowPos(IntPtr hWnd, IntPtr hWndInsertAfter, int X, int Y, int cx, int cy, uint uFlags);
+
+    [DllImport("user32.dll")]
+    private static extern int GetSystemMetrics(int nIndex);
+
+    private const int SM_CXSCREEN = 0;
+    private const int SM_CYSCREEN = 1;
+    private const uint SWP_NOACTIVATE = 0x0010;
+    private const uint SWP_SHOWWINDOW = 0x0040;
+    private static readonly IntPtr HWND_TOPMOST = new IntPtr(-1);
+
+    public TrayMenuAnchorWindow()
+    {
+        InitializeComponent();
+
+        // Configure window to be invisible but present
+        this.AppWindow.IsShownInSwitchers = false;
+        this.ExtendsContentIntoTitleBar = true;
+        this.SystemBackdrop = null; // No backdrop effect
+
+        // Set to 1x1 size - minimal footprint
+        this.AppWindow.Resize(new Windows.Graphics.SizeInt32(1, 1));
+    }
+
+    /// <summary>
+    /// Positions the anchor window at the cursor location (for tray menu positioning)
+    /// </summary>
+    public void PositionAtCursor(int x, int y)
+    {
+        var hwnd = WinRT.Interop.WindowNative.GetWindowHandle(this);
+
+        // Position window at cursor location, topmost, without activating
+        SetWindowPos(hwnd, HWND_TOPMOST, x, y, 1, 1, SWP_NOACTIVATE | SWP_SHOWWINDOW);
+    }
+
+    /// <summary>
+    /// Positions the anchor window off-screen (hidden but still valid for anchoring)
+    /// </summary>
+    public void PositionOffscreen()
+    {
+        var hwnd = WinRT.Interop.WindowNative.GetWindowHandle(this);
+
+        // Move far off-screen where it won't be visible
+        int screenWidth = GetSystemMetrics(SM_CXSCREEN);
+        int screenHeight = GetSystemMetrics(SM_CYSCREEN);
+
+        SetWindowPos(hwnd, HWND_TOPMOST, screenWidth + 100, screenHeight + 100, 1, 1, SWP_NOACTIVATE | SWP_SHOWWINDOW);
+    }
+
+    /// <summary>
+    /// Shows the flyout anchored to this window's root grid
+    /// </summary>
+    public void ShowFlyout(MenuFlyout flyout)
+    {
+        if (flyout == null)
+            throw new ArgumentNullException(nameof(flyout));
+
+        // Ensure the flyout closes when it loses focus
+        flyout.Closed += (s, e) => this.Hide();
+
+        // Show flyout anchored to the root grid of this window
+        flyout.ShowAt(RootGrid);
+    }
+
+    /// <summary>
+    /// Hides the window (keeps it in memory for reuse)
+    /// </summary>
+    public void Hide()
+    {
+        try
+        {
+            // Move offscreen instead of truly hiding to keep it valid as an anchor
+            PositionOffscreen();
+        }
+        catch
+        {
+            // Ignore errors during hide
+        }
+    }
+}