[fix][broker] In the trimDeletedEntries method, release the removed entry #18305

lordcheng10 · 2022-11-02T13:24:31Z

Motivation

In the trimDeletedEntries method, the removed entry is not released：

pulsar/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java

Lines 3307 to 3310 in 2e878e8

    
           public void trimDeletedEntries(List<Entry> entries) { 
        
               entries.removeIf(entry -> ((PositionImpl) entry.getPosition()).compareTo(markDeletePosition) <= 0 
        
                       || individualDeletedMessages.contains(entry.getLedgerId(), entry.getEntryId())); 
        
           }

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java

Lines 582 to 585 in 2e878e8

    
           protected synchronized boolean trySendMessagesToConsumers(ReadType readType, List<Entry> entries) { 
        
               if (needTrimAckedMessages()) { 
        
                   cursor.trimDeletedEntries(entries); 
        
               }

Documentation

doc
doc-required
doc-not-needed
doc-complete

Matching PR in forked repository

PR in forked repository: lordcheng10#38

lordcheng10 · 2022-11-03T00:56:18Z

@codelipenghui @Jason918 PTAL,thanks!

codelipenghui

Nice catch!

lordcheng10 · 2022-11-03T02:30:30Z

/pulsarbot run-failure-checks

codecov-commenter · 2022-11-03T02:51:02Z

Codecov Report

Merging #18305 (56d0e6e) into master (0866c3a) will increase coverage by 12.57%.
The diff coverage is 46.41%.

@@              Coverage Diff              @@
##             master   #18305       +/-   ##
=============================================
+ Coverage     38.97%   51.54%   +12.57%     
+ Complexity     8311     7356      -955     
=============================================
  Files           683      400      -283     
  Lines         67325    43678    -23647     
  Branches       7217     4479     -2738     
=============================================
- Hits          26239    22516     -3723     
+ Misses        38079    18735    -19344     
+ Partials       3007     2427      -580

Flag	Coverage Δ
unittests	`51.54% <46.41%> (+12.57%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
.../main/java/org/apache/pulsar/PulsarStandalone.java	`0.00% <ø> (ø)`
...che/pulsar/broker/BookKeeperClientFactoryImpl.java	`0.00% <0.00%> (ø)`
.../org/apache/pulsar/broker/admin/AdminResource.java	`65.42% <0.00%> (-1.09%)`	⬇️
...pulsar/broker/admin/impl/PersistentTopicsBase.java	`58.95% <0.00%> (+6.65%)`	⬆️
...pache/pulsar/broker/admin/v2/PersistentTopics.java	`71.68% <0.00%> (+0.88%)`	⬆️
...pulsar/broker/service/PulsarCommandSenderImpl.java	`73.29% <ø> (-1.07%)`	⬇️
...ersistentStreamingDispatcherMultipleConsumers.java	`0.00% <0.00%> (ø)`
...ransaction/buffer/impl/InMemTransactionBuffer.java	`0.00% <ø> (ø)`
...nsaction/buffer/impl/TransactionBufferDisable.java	`52.63% <ø> (ø)`
.../metadata/v2/TransactionBufferSnapshotIndexes.java	`0.00% <0.00%> (ø)`
... and 360 more

lordcheng10 · 2022-11-03T03:50:00Z

/pulsarbot run-failure-checks

lordcheng10 · 2022-11-03T03:50:45Z

@AnonHxy @Jason918 @HQebupt PTAL,thanks!

lordcheng10 · 2022-11-03T04:03:10Z

@Technoboy- PTAL,thanks!

lordcheng10 · 2022-11-03T05:01:11Z

/pulsarbot run-failure-checks

lordcheng10 · 2022-11-03T07:47:11Z

/pulsarbot run-failure-checks

lordcheng10 · 2022-11-03T12:00:20Z

/pulsarbot run-failure-checks

coderzc · 2022-11-03T12:42:48Z

/pulsarbot run-failure-checks

Pulsar CI Flaky is not necessary.

…ntry (#18305) (cherry picked from commit 79a3f85)

…ntry (apache#18305) (merge request !161) Squash merge branch 'release-2.8.1.4-fix_mem_leak' into 'release-2.8.1.4' Fixes #<xyz> ### Motivation [fix][broker] In the trimDeletedEntries method, release the removed entry (apache#18305)

…ntry (#18305) (cherry picked from commit 79a3f85)

…ntry (apache#18305) (merge request !161) Squash merge branch 'release-2.8.1.4-fix_mem_leak' into 'release-2.8.1.4' Fixes #<xyz> ### Motivation [fix][broker] In the trimDeletedEntries method, release the removed entry (apache#18305)

…ntry (apache#18305) (cherry picked from commit 79a3f85) (cherry picked from commit 4eb93ec)

…ntry (#18305)

lordcheng10 added 2 commits November 2, 2022 21:06

release entry when trimDeletedEntries

bc69655

update test testTrimDeletedEntries

56d0e6e

lordcheng10 mentioned this pull request Nov 2, 2022

[fix][broker] In the trimDeletedEntries method, release the removed entry lordcheng10/pulsar#38

Closed

github-actions bot added the doc-not-needed Your PR changes do not impact docs label Nov 2, 2022

codelipenghui approved these changes Nov 3, 2022

View reviewed changes

codelipenghui assigned lordcheng10 Nov 3, 2022

codelipenghui added type/bug The PR fixed a bug or issue reported a bug area/broker labels Nov 3, 2022

codelipenghui added this to the 2.12.0 milestone Nov 3, 2022

codelipenghui added release/2.9.4 release/2.11.1 release/2.10.3 labels Nov 3, 2022

coderzc approved these changes Nov 3, 2022

View reviewed changes

codelipenghui merged commit 79a3f85 into apache:master Nov 3, 2022

congbobo184 pushed a commit that referenced this pull request Nov 8, 2022

[fix][broker] In the trimDeletedEntries method, release the removed e…

98309d4

…ntry (#18305) (cherry picked from commit 79a3f85)

congbobo184 added the cherry-picked/branch-2.9 Archived: 2.9 is end of life label Nov 8, 2022

congbobo184 pushed a commit that referenced this pull request Nov 26, 2022

[fix][broker] In the trimDeletedEntries method, release the removed e…

6c0fc32

…ntry (#18305) (cherry picked from commit 79a3f85)

liangyepianzhou pushed a commit that referenced this pull request Dec 14, 2022

[fix][broker] In the trimDeletedEntries method, release the removed e…

4eb93ec

…ntry (#18305) (cherry picked from commit 79a3f85)

liangyepianzhou added the cherry-picked/branch-2.10 label Dec 14, 2022

nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Jan 10, 2023

[fix][broker] In the trimDeletedEntries method, release the removed e…

4078003

…ntry (apache#18305) (cherry picked from commit 79a3f85) (cherry picked from commit 4eb93ec)

nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Jan 11, 2023

[fix][broker] In the trimDeletedEntries method, release the removed e…

32450a7

…ntry (apache#18305) (cherry picked from commit 79a3f85) (cherry picked from commit 4eb93ec)

Technoboy- pushed a commit that referenced this pull request Feb 8, 2023

[fix][broker] In the trimDeletedEntries method, release the removed e…

903a719

…ntry (#18305)

Technoboy- added the cherry-picked/branch-2.11 label Feb 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix][broker] In the trimDeletedEntries method, release the removed entry #18305

[fix][broker] In the trimDeletedEntries method, release the removed entry #18305

Uh oh!

lordcheng10 commented Nov 2, 2022 •

edited

Loading

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

codelipenghui left a comment

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

codecov-commenter commented Nov 3, 2022 •

edited

Loading

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

coderzc commented Nov 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

	public void trimDeletedEntries(List<Entry> entries) {
	entries.removeIf(entry -> ((PositionImpl) entry.getPosition()).compareTo(markDeletePosition) <= 0
	\|\| individualDeletedMessages.contains(entry.getLedgerId(), entry.getEntryId()));
	}

	protected synchronized boolean trySendMessagesToConsumers(ReadType readType, List<Entry> entries) {
	if (needTrimAckedMessages()) {
	cursor.trimDeletedEntries(entries);
	}

[fix][broker] In the trimDeletedEntries method, release the removed entry #18305

[fix][broker] In the trimDeletedEntries method, release the removed entry #18305

Uh oh!

Conversation

lordcheng10 commented Nov 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Documentation

Matching PR in forked repository

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

codelipenghui left a comment

Choose a reason for hiding this comment

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

codecov-commenter commented Nov 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

lordcheng10 commented Nov 3, 2022

Uh oh!

coderzc commented Nov 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

lordcheng10 commented Nov 2, 2022 •

edited

Loading

codecov-commenter commented Nov 3, 2022 •

edited

Loading