Skip to content

Reduce coordinator logs #17566

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kfaraz merged 2 commits into from
Dec 16, 2024
Merged

Reduce coordinator logs #17566

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
acc6f7f
Reduce coordinator logs
kfaraz Dec 13, 2024
f27e770
Remove extra comment
kfaraz Dec 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions ...rc/main/java/org/apache/druid/server/compaction/DataSourceCompactibleSegmentIterator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -491,10 +491,9 @@ static List<Interval> filterSkipIntervals(Interval totalInterval, List<Interval>
remainingStart = skipInterval.getEnd();
} else {
// Ignore this skipInterval
log.warn(
log.debug(
Copy link
Copy Markdown
Contributor Author

@kfaraz kfaraz Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this log doesn't provide much useful info and can remain a debug

Comment thread
kfaraz marked this conversation as resolved.
"skipInterval[%s] is not contained in remainingInterval[%s]",
skipInterval,
new Interval(remainingStart, remainingEnd)
skipInterval, new Interval(remainingStart, remainingEnd)
);
}
}
Expand Down
15 changes: 6 additions & 9 deletions server/src/main/java/org/apache/druid/server/coordinator/loading/HttpLoadQueuePeon.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,14 +76,10 @@
public class HttpLoadQueuePeon implements LoadQueuePeon
{
public static final TypeReference<List<DataSegmentChangeRequest>> REQUEST_ENTITY_TYPE_REF =
new TypeReference<List<DataSegmentChangeRequest>>()
{
};
new TypeReference<>() {};
Copy link
Copy Markdown
Contributor Author

@kfaraz kfaraz Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Java 9+ supports diamond operator with anonymous classes.
Since we have dropped support for Java 8, we can use this syntax here (and in several other places in the code which can be done in a separate PR).

cc: @Akshat-Jain

Copy link
Copy Markdown
Contributor

@Akshat-Jain Akshat-Jain Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have raised PR: #17567. The PR isn't limited to just anonymous class usages, but rather does it for all usages where explicit type argument was redundant. I don't see an easy way to identify usages only for anonymous classes, and I didn't see any reason why we shouldn't do it for other usages as well, so ended up doing that. Hope that works!

Appreciate your review on the PR, thanks!


public static final TypeReference<List<DataSegmentChangeResponse>> RESPONSE_ENTITY_TYPE_REF =
new TypeReference<List<DataSegmentChangeResponse>>()
{
};
new TypeReference<>() {};

private static final EmittingLogger log = new EmittingLogger(HttpLoadQueuePeon.class);

Expand Down Expand Up @@ -389,9 +385,10 @@ public void stop()
log.info("Stopping load queue peon for server[%s].", serverId);
stopped = true;

// Cancel all queued requests
queuedSegments.forEach(holder -> onRequestCompleted(holder, RequestStatus.CANCELLED));
log.info("Cancelled [%d] requests queued on server [%s].", queuedSegments.size(), serverId);
if (!queuedSegments.isEmpty()) {
queuedSegments.forEach(holder -> onRequestCompleted(holder, RequestStatus.CANCELLED));
log.info("Cancelled [%d] requests queued on server[%s].", queuedSegments.size(), serverId);
Copy link
Copy Markdown
Contributor Author

@kfaraz kfaraz Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

log only when non-empty.

}

segmentsToDrop.clear();
segmentsToLoad.clear();
Expand Down
6 changes: 2 additions & 4 deletions server/src/test/java/org/apache/druid/server/coordinator/loading/HttpLoadQueuePeonTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@

package org.apache.druid.server.coordinator.loading;

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.util.concurrent.Futures;
import com.google.common.util.concurrent.ListenableFuture;
Expand Down Expand Up @@ -350,9 +349,8 @@ public <Intermediate, Final> ListenableFuture<Final> go(
httpResponseHandler.handleResponse(httpResponse, null);
try {
List<DataSegmentChangeRequest> changeRequests = MAPPER.readValue(
request.getContent().array(), new TypeReference<List<DataSegmentChangeRequest>>()
{
}
request.getContent().array(),
HttpLoadQueuePeon.REQUEST_ENTITY_TYPE_REF
);

List<DataSegmentChangeResponse> statuses = new ArrayList<>(changeRequests.size());
Expand Down