Skip to content

Logging interceptor#649

Merged
martindevans merged 8 commits intoSciSharp:masterfrom
martindevans:logging_interceptor
Apr 5, 2024
Merged

Logging interceptor#649
martindevans merged 8 commits intoSciSharp:masterfrom
martindevans:logging_interceptor

Conversation

@martindevans
Copy link
Copy Markdown
Member

Rewritten logging callbacks so it can be configured without touching native methods. This allows it to be used before llama.cpp dll is loaded.

Resolves #645

…allback

 - Delaying binding of this into llama.cpp until after `NativeLibraryConfig` has loaded
@SanftMonster
Copy link
Copy Markdown
Collaborator

Will this work also suppress native logs from llava?

@martindevans
Copy link
Copy Markdown
Member Author

Unsure. It's just using the llama.cpp logging callback (llama_log_set), so if llava messages are not captured by it I think that would be a bug that would be easy to fix upstream.

Comment thread LLama/Native/NativeApi.Load.cs Outdated
Comment thread LLama/Native/NativeLogConfig.cs
Comment thread LLama/Native/NativeLogConfig.cs
…_empty_call`, this is specifically selected to be a method that does nothing and is just there for triggering DLL loading.
… always call whatever log callbacks you have registered.

 - Removed alternative path for `ILogger` in NativeLibraryConfig, instead it redirects to wrapping it in a delegate.
Comment thread LLama/Native/NativeLogConfig.cs Outdated
Comment thread LLama/Native/NativeApi.Load.cs
Comment thread LLama/Native/NativeLogConfig.cs Outdated
 - Using `ThreadLocal<StringBuilder>` to ensure messages from separate threads don't get mixed together.
@dlyz
Copy link
Copy Markdown
Contributor

dlyz commented Apr 5, 2024

Thanks @martindevans for your work! I have no further notes.

@martindevans
Copy link
Copy Markdown
Member Author

Thanks very much for your detailed review!

@martindevans martindevans merged commit 58107bb into SciSharp:master Apr 5, 2024
@martindevans martindevans deleted the logging_interceptor branch April 5, 2024 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Can not intercept all the log messages with llama_log_set

3 participants