Skip to content

Minor: Empty list returned by poll() destroys the source task when assert is enabled#4958

Closed
chia7712 wants to merge 1 commit intoapache:trunkfrom
chia7712:assert_in_WorkerSourceTask
Closed

Minor: Empty list returned by poll() destroys the source task when assert is enabled#4958
chia7712 wants to merge 1 commit intoapache:trunkfrom
chia7712:assert_in_WorkerSourceTask

Conversation

@chia7712
Copy link
Copy Markdown
Member

@chia7712 chia7712 commented May 2, 2018

I noticed this issue when running the embedded connectors. There is a assert in SourceRecordWriteCounter.

        public SourceRecordWriteCounter(int batchSize, SourceTaskMetricsGroup metricsGroup) {
            assert batchSize > 0;   // this one
            assert metricsGroup != null;
            this.batchSize = batchSize;
            counter = batchSize;
            this.metricsGroup = metricsGroup;
        }

Hence, the empty list will cease the source task. By contrast, empty list is not a issue in production since the assert is usually disabled in production. It seems to me making the behavior consistent is necessary. A simple fix is to remove the assert. Or empty list should be handled as null list. I prefer later since both of them mean "no data can be processed".

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@chia7712 chia7712 force-pushed the assert_in_WorkerSourceTask branch from 8cf3f89 to 117686f Compare April 14, 2020 07:51
@kkonstantine
Copy link
Copy Markdown
Contributor

Closing this PR in favor of #7491 which seems to have fixed the issue.

@chia7712 chia7712 deleted the assert_in_WorkerSourceTask branch March 25, 2024 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants