Expunge stale loggers in InternalLoggerRegistry on method invocation#3474
Expunge stale loggers in InternalLoggerRegistry on method invocation#3474Suvrat1629 wants to merge 5 commits intoapache:2.xfrom Suvrat1629:expunge-stale-entries
Conversation
…so added InternalLoggerRegistryGCTest
|
@vy I have always had a hard time with test cases, I have opened a pr regardless please take look and provide further insights on how i can improve it. |
vy
left a comment
There was a problem hiding this comment.
Please put back all the removed comment blocks.
|
Ok sorry about that, I have put back all the comment blocks as they were. |
| Map<String, WeakReference<Logger>> loggerRefByName = loggerRefByNameByMessageFactory.get(messageFactory); | ||
| if (loggerRefByName != null) { | ||
| loggerRefByName.remove(name); | ||
| if (loggerRefByName.isEmpty()) { | ||
| loggerRefByNameByMessageFactory.remove(messageFactory); // Cleanup | ||
| } | ||
| } |
There was a problem hiding this comment.
Assume I move this logic (and only this!) to a method called unsafeRemoveLogger(Logger). Note that this method receives a Logger, not a Reference.
| Reference<? extends Logger> loggerRef; | ||
| while ((loggerRef = staleLoggerRefs.poll()) != null) { | ||
| removeLogger(loggerRef); | ||
| } |
There was a problem hiding this comment.
Would you mind changing this loop as follows, please?
boolean locked = false;
Reference<? extends Logger> loggerRef;
while ((loggerRef = staleLoggerRefs.poll()) != null) {
Logger logger = loggerRef.get();
if (logger != null) {
if (!locked) {
writeLock.lock();
}
unsafeRemoveLogger(logger);
}
}
if (locked) {
writeLock.unlock();
}
See my other comment regarding the unsafeRemoveLogger(Logger) method.
There was a problem hiding this comment.
Note that if loggerRef.get() returns null, we still need to remove the entry that holds the WeakReference.
What do you think about such an approach:
- We
poll()the queue until its empty. - If there was any reference in the queue we iterate once over all the entries of the map and we remove all the invalidated
WeakReferences, not just those that were in the queue.
As of my previous comment, this seems a little bit inefficient if loggerRef.get() is not null, but is more efficient than iterating over all entries once-per-reference.
There was a problem hiding this comment.
@ppkarwasz, you're right. @Suvrat1629, please proceed with the suggestion of @ppkarwasz. Though I suggest having double-checking:
if (!refQueue.isEmpty) {
lock();
try {
if (!refQueue.isEmpty) {
// Clear `refQueue`
// Clean up `loggerRefByNameByMessageFactory`
}
} finally {
unlock();
}
}
There was a problem hiding this comment.
Can you replace LoggerContext.getContext() calls as follows, please?
@Test
void someTest(TestInfo testInfo) {
try (final LoggerContext loggerContext = new LoggerContext(testInfo.getDisplayName)) {
// Use `loggerContext`
}
}
That is, don't disrupt the global one, use ephemeral LCs instead.
| Logger logger = | ||
| registry.computeIfAbsent("testLogger", messageFactory, (name, factory) -> LoggerContext.getContext() | ||
| .getLogger(name, factory)); | ||
|
|
||
| WeakReference<Logger> weakRef = new WeakReference<>(logger); | ||
| logger = null; // Dereference to allow GC | ||
|
|
||
| // Retry loop to give GC time to collect | ||
| for (int i = 0; i < 10; i++) { | ||
| System.gc(); | ||
| Thread.sleep(100); | ||
| if (weakRef.get() == null) { | ||
| break; | ||
| } | ||
| } | ||
|
|
||
| // Access the registry to potentially trigger cleanup | ||
| registry.computeIfAbsent("tempLogger", messageFactory, (name, factory) -> LoggerContext.getContext() | ||
| .getLogger(name, factory)); | ||
|
|
||
| assertNull(weakRef.get(), "Logger should have been garbage collected"); | ||
| assertNull( | ||
| registry.getLogger("testLogger", messageFactory), "Stale logger should be removed from the registry"); |
There was a problem hiding this comment.
Can we instead replace this as follows, please?
- Use an ephemeral
LoggerContext(as I described in another comment) - Create 1000
Loggers (should be many enough to create GC pressure) - Log using all
Loggers (so we imitate a real-world setup whereLoggers are used at some point) - Use
Awaitility.waitAtMost(...).until(...)to verify that after aSystem.gc(),InternalLoggerRegistry::loggerRefByNameByMessageFactoryis emptied
You can access to InternalLoggerRegistry::loggerRefByNameByMessageFactory using reflection. That is, first extract the LoggerContext::loggerRegistry field, and then the InternalLoggerRegistry::loggerRefByNameByMessageFactory field.
This PR updates InternalLoggerRegistry to automatically remove stale loggers when its methods are invoked.
Changes Introduced:
Introduced a ReferenceQueue to track loggers that have been reclaimed by the garbage collector.
Implemented expungeStaleEntries(), which removes stale loggers before executing registry operations.
Updated methods like getLogger() and computeIfAbsent() to invoke expungeStaleEntries() before proceeding.
Added tests to verify stale loggers are expunged without relying on private methods.
Fixes #3430