OGA Memory Leak — Model and Tokenizer Instances Not Disposed

[leestott]

OGA Memory Leak — Model and Tokenizer Instances Not Disposed

Severity: Warning
Component: ONNX GenAI Runtime (OGA)
Reproduction: Run any example to completion — the leak message appears on process exit

OGA Error: 1 instances of struct Generators::Model were leaked.
OGA Error: 1 instances of struct Generators::Tokenizer were leaked.
    Please see the documentation for the API being used to ensure proper cleanup.

In the Whisper scenario, the leak count was higher:

OGA Error: 1 instances of struct Generators::Model were leaked.
OGA Error: 2 instances of struct Generators::Tokenizer were leaked.

Impact: The Foundry Local SDK does not expose a Dispose() or Unload() method for models in the C# API. There is no documented way for application code to release these native resources.

Expected: Either:

• Expose IDisposable/IAsyncDisposable on the model or manager objects, or
• Suppress the leak warnings when the SDK is managing the lifecycle, or
• Document the proper cleanup pattern.

[f2bo]

I'm unable to reproduce the leak with any of the samples, which reference the latest published Microsoft.AI.Foundry.Local.WinML NuGet package (v0.8.2.1), unless I comment out the call to the model's UnloadAsync method. For example:

Foundry-Local/samples/cs/GettingStarted/src/HelloFoundryLocalSdk/Program.cs

Line 67 in 756ed34

await model.UnloadAsync();

When I comment out that line, I do see the same as what you report.

However, I wanted to test the latest changes to the AudioClient in SDK V2 (i.e. language settings), so I replaced the NuGet package reference with a reference to the Microsoft.AI.Foundry.Local project in this repo. When I do this, on my development machine, I get an access violation in dxcompiler.dll when the program exits. With the UnloadAsync method commented out, it doesn't happen and I do see the same OGA handle leaks that you report, so there seems to be a problem with the way resources are disposed here.

Unhandled exception at 0x00007FFC3C8138C0 (dxcompiler.dll) in HelloFoundryLocalSdk.exe: 0xC0000005: Access violation reading location 0x0000000000000008.

Call Stack

dxcompiler.dll!00007ffc3c8138c0()
dxcompiler.dll!00007ffc3c568f3c()
dxcompiler.dll!00007ffc3c568f73()
dxcompiler.dll!00007ffc3c4bd1f0()
[Inline Frame] onnxruntime.dll!Microsoft::WRL::ComPtr<IDxcValidator>::InternalRelease() Line 235
	at C:\Program Files (x86)\Windows Kits\10\Include\10.0.26100.0\winrt\wrl\client.h(235)
[Inline Frame] onnxruntime.dll!Microsoft::WRL::ComPtr<IDxcValidator>::{dtor}() Line 290
	at C:\Program Files (x86)\Windows Kits\10\Include\10.0.26100.0\winrt\wrl\client.h(290)
onnxruntime.dll!dawn::native::d3d12::Backend::`scalar deleting destructor'(unsigned int)
onnxruntime.dll!`eh vector destructor iterator'(void * ptr, unsigned __int64 size, unsigned __int64 count, void(*)(void *) destructor)
onnxruntime.dll!dawn::native::InstanceBase::~InstanceBase() Line 224
	at E:\_work\1\b\RelWithDebInfo\_deps\dawn-src\src\dawn\native\Instance.cpp(224)
onnxruntime.dll!dawn::native::InstanceBase::`scalar deleting destructor'(unsigned int)
onnxruntime.dll!dawn::native::InstanceBase::DeleteThis() Line 245
	at E:\_work\1\b\RelWithDebInfo\_deps\dawn-src\src\dawn\native\Instance.cpp(245)
ucrtbase.dll!<lambda>(void)()
ucrtbase.dll!__crt_seh_guarded_call<int>::operator()<<lambda_7777bce6b2f8c936911f934f8298dc43>,<lambda>(void) &,<lambda_3883c3dff614d5e0c5f61bb1ac94921c>>()
ucrtbase.dll!_execute_onexit_table()
onnxruntime.dll!dllmain_crt_process_detach(const bool is_terminating) Line 182
	at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\dll_dllmain.cpp(182)
onnxruntime.dll!dllmain_dispatch(HINSTANCE__ * const instance, const unsigned long reason, void * const reserved) Line 293
	at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\dll_dllmain.cpp(293)
ntdll.dll!LdrpCallInitRoutineInternal()
ntdll.dll!LdrpCallInitRoutine()
ntdll.dll!LdrShutdownProcess()
ntdll.dll!RtlExitUserProcess()
kernel32.dll!ExitProcessImplementation()
ucrtbase.dll!common_exit()
HelloFoundryLocalSdk.exe!00007ff7aa6325af()
kernel32.dll!BaseThreadInitThunk()
ntdll.dll!RtlUserThreadStart()

[samuel100]

@leestott this is caused if your app doesn't dispose of the model on exit. Does the app do a model.unload() on exit?

[leestott]

Root cause: Application code does not dispose of the model on exit. The OGA runtime tracks allocated Model and Tokenizer objects and reports any that were not explicitly freed before the process terminates.

Workaround: Add cleanup code before exit:

# Python — unload model on exit
manager.unload_model("phi-3.5-mini")

// JavaScript — unload model and stop service
await model.unload();
await manager.stopWebService();

// C# — unload model
await model.UnloadAsync(default);

Expected: Either:

• Expose IDisposable/IAsyncDisposable on the model or manager objects for automatic cleanup with using blocks, or
• Suppress the leak warnings when the SDK is managing the lifecycle, or
• Document the proper cleanup pattern prominently in the SDK reference