Skip to content

MINOR: Wait for embedded clusters to start before using them in Connect OffsetsApiIntegrationTest#16286

Merged
C0urante merged 1 commit intoapache:trunkfrom
C0urante:offsets-api-integration-test-await-cluster-startup
Jun 11, 2024
Merged

MINOR: Wait for embedded clusters to start before using them in Connect OffsetsApiIntegrationTest#16286
C0urante merged 1 commit intoapache:trunkfrom
C0urante:offsets-api-integration-test-await-cluster-startup

Conversation

@C0urante
Copy link
Copy Markdown
Contributor

@C0urante C0urante commented Jun 11, 2024

There are some flaky failures for the OffsetsApiIntegrationTest:: testResetSourceConnectorOffsetsExactlyOnceSupportEnabled case (see Gradle Enterprise). Every failure was either unrelated to Kafka Connect, or was caused by the very first REST request to the cluster timing out.

By adding an assertion that the cluster has started before issuing REST requests to it, we 1) give the cluster more time to get on its feet (five minutes vs. ninety seconds) and 2) help narrow down potential causes of failure in the future if this test becomes or remains flaky.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@C0urante C0urante added connect tests Test fixes (including flaky tests) labels Jun 11, 2024
gharris1727
gharris1727 approved these changes Jun 11, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@gharris1727 gharris1727 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If only start() waited for the cluster to start, so many assertions could be avoided in tests

@C0urante
Copy link
Copy Markdown
Contributor Author

C0urante commented Jun 11, 2024 •
edited
Loading

Great idea! Filed KAFKA-16935 and published a PR for it.

I think we can still merge this as a quick fix (unclear how long the other one will take for review as it's a bit lengthy); just waiting on CI to confirm that everything's good to go.

@C0urante C0urante merged commit 2fa2c72 into apache:trunk Jun 11, 2024
@C0urante C0urante deleted the offsets-api-integration-test-await-cluster-startup branch June 11, 2024 21:15
C0urante added a commit that referenced this pull request Jun 11, 2024
@C0urante
MINOR: Wait for embedded clusters to start before using them in Conne…
520fbb4 
…ct OffsetsApiIntegrationTest (#16286)

Reviewers: Greg Harris <greg.harris@aiven.io>
gongxuanzhang pushed a commit to gongxuanzhang/kafka that referenced this pull request Jun 12, 2024
@C0urante @gongxuanzhang
MINOR: Wait for embedded clusters to start before using them in Conne…
5890a9d 
…ct OffsetsApiIntegrationTest (apache#16286)

Reviewers: Greg Harris <greg.harris@aiven.io>
apourchet added a commit to apourchet/kafka that referenced this pull request Jun 12, 2024
@apourchet
Squashed commit of the following:
4ba836d 
commit 9368ef8
Author: Gantigmaa Selenge <39860586+tinaselenge@users.noreply.github.com>
Date:   Wed Jun 12 16:04:24 2024 +0100

    KAFKA-16865: Add IncludeTopicAuthorizedOperations option for DescribeTopicPartitionsRequest (apache#16136)

    Reviewers: Mickael Maison <mickael.maison@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>, Calvin Liu <caliu@confluent.io>, Andrew Schofield <andrew_schofield@live.com>, Apoorv Mittal <amittal@confluent.io>

commit 46eb081
Author: gongxuanzhang <gongxuanzhang@foxmail.com>
Date:   Wed Jun 12 22:23:39 2024 +0800

    KAFKA-10787 Apply spotless to log4j-appender, trogdor, jmh-benchmarks, examples, shell and generator (apache#16296)

    Reviewers: Chia-Ping Tsai <chia7712@gmail.com>

commit 79b9c44
Author: gongxuanzhang <gongxuanzhang@foxmail.com>
Date:   Wed Jun 12 22:19:47 2024 +0800

    KAFKA-10787 Apply spotless to connect module (apache#16299)

    Reviewers: Chia-Ping Tsai <chia7712@gmail.com>

commit b5fb654
Author: Abhijeet Kumar <abhijeet.cse.kgp@gmail.com>
Date:   Wed Jun 12 19:47:46 2024 +0530

    KAFKA-15265: Dynamic broker configs for remote fetch/copy quotas (apache#16078)

    Reviewers: Kamal Chandraprakash<kamal.chandraprakash@gmail.com>, Satish Duggana <satishd@apache.org>

commit faee6a4
Author: Dmitry Werner <grimekillah@gmail.com>
Date:   Wed Jun 12 15:44:11 2024 +0500

    MINOR: Use predetermined dir IDs in ReplicationQuotasTest

    Use predetermined directory IDs instead of Uuid.randomUuid() in ReplicationQuotasTest.

    Reviewers: Igor Soarez <soarez@apple.com>

commit 638844f
Author: David Jacot <djacot@confluent.io>
Date:   Wed Jun 12 08:29:50 2024 +0200

    KAFKA-16770; [2/2] Coalesce records into bigger batches (apache#16215)

    This patch is the continuation of apache#15964. It introduces the records coalescing to the CoordinatorRuntime. It also introduces a new configuration `group.coordinator.append.linger.ms` which allows administrators to chose the linger time or disable it with zero. The new configuration defaults to 10ms.

    Reviewers: Jeff Kim <jeff.kim@confluent.io>, Justine Olshan <jolshan@confluent.io>

commit 39ffdea
Author: Bruno Cadonna <cadonna@apache.org>
Date:   Wed Jun 12 07:51:38 2024 +0200

    KAFKA-10199: Enable state updater by default (apache#16107)

    We have already enabled the state updater by default once.
    However, we ran into issues that forced us to disable it again.
    We think that we fixed those issues. So we want to enable the
    state updater again by default.

    Reviewers: Lucas Brutschy <lbrutschy@confluent.io>, Matthias J. Sax <matthias@confluent.io>

commit 0782232
Author: Antoine Pourchet <antoine@responsive.dev>
Date:   Tue Jun 11 22:31:43 2024 -0600

    KAFKA-15045: (KIP-924 pt. 22) Add RackAwareOptimizationParams and other minor TaskAssignmentUtils changes (apache#16294)

    We now provide a way to more easily customize the rack aware
    optimizations that we provide by way of a configuration class called
    RackAwareOptimizationParams.

    We also simplified the APIs for the optimizeXYZ utility functions since
    they were mutating the inputs anyway.

    Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>

commit 226ac5e
Author: Murali Basani <muralidhar.basani@aiven.io>
Date:   Wed Jun 12 05:38:50 2024 +0200

    KAFKA-16922 Adding unit tests for NewTopic  (apache#16255)

    Reviewers: Chia-Ping Tsai <chia7712@gmail.com>

commit 23fe71d
Author: Abhijeet Kumar <abhijeet.cse.kgp@gmail.com>
Date:   Wed Jun 12 06:27:02 2024 +0530

    KAFKA-15265: Integrate RLMQuotaManager for throttling copies to remote storage (apache#15820)

    - Added the integration of the quota manager to throttle copy requests to the remote storage. Reference KIP-956
    - Added unit-tests for the copy throttling logic.

    Reviewers: Satish Duggana <satishd@apache.org>, Luke Chen <showuon@gmail.com>, Kamal Chandraprakash<kamal.chandraprakash@gmail.com>

commit 2fa2c72
Author: Chris Egerton <chrise@aiven.io>
Date:   Tue Jun 11 23:15:07 2024 +0200

    MINOR: Wait for embedded clusters to start before using them in Connect OffsetsApiIntegrationTest (apache#16286)

    Reviewers: Greg Harris <greg.harris@aiven.io>
@C0urante C0urante mentioned this pull request Jun 12, 2024
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gharris1727 gharris1727 gharris1727 approved these changes

Assignees

No one assigned

Labels

connect tests Test fixes (including flaky tests)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@C0urante @gharris1727