Skip to content

KAFKA-9855 - return cached Structs for Schemas with no fields#8472

Merged
ijuma merged 1 commit intoapache:trunkfrom
radai-rosenblatt:struct-perf
May 21, 2020
Merged

KAFKA-9855 - return cached Structs for Schemas with no fields#8472
ijuma merged 1 commit intoapache:trunkfrom
radai-rosenblatt:struct-perf

Conversation

@radai-rosenblatt
Copy link
Copy Markdown
Contributor

Signed-off-by: radai-rosenblatt radai.rosenblatt@gmail.com

at the time of this writing there are 6 schemas in kafka APIs with no fields - 3 versions each of LIST_GROUPS and API_VERSIONS.

when reading instances of these schemas off the wire there's little point in returning a unique Struct object (or a unique values array inside that Struct) since there is no payload.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
@radai-rosenblatt
Copy link
Copy Markdown
Contributor Author

@hachikuji @ijuma - this accounted for ~1.3 GB of ~wasted heap space for us for at least one cluster. for your consideration?

radai-rosenblatt added a commit to linkedin/kafka that referenced this pull request Apr 15, 2020
…elds (#77)

TICKET =
LI_DESCERIPTION = committing to our fork until upstream picks this up
EXIT_CRITERIA = when KAFKA-9855 is accepted upstream (specifically - apache#8472)

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
@ijuma
Copy link
Copy Markdown
Member

ijuma commented Apr 16, 2020

ok to test

@radai-rosenblatt
Copy link
Copy Markdown
Contributor Author

@ijuma - any more modifications required?

@ijuma
Copy link
Copy Markdown
Member

ijuma commented May 21, 2020

I have a WIP PR that removes structs altogether, but this is a good improvement in the meantime. Merging to trunk. Thanks.

@ijuma ijuma merged commit aa1b3c1 into apache:trunk May 21, 2020
Kvicii pushed a commit to Kvicii/kafka that referenced this pull request May 22, 2020
* 'trunk' of github.com:apache/kafka:
  KAFKA-9980: Fix bug where alterClientQuotas could not set default client quotas (apache#8658)
  KAFKA-9780: Deprecate commit records without record metadata (apache#8379)
  MINOR: Deploy VerifiableClient in constructor to avoid test timeouts (apache#8651)
  MINOR: Added unit tests for ConnectionQuotas (apache#8650)
  MINOR: Correct MirrorMaker2 integration test configs for Connect internal topics (apache#8653)
  KAFKA-9855 - return cached Structs for Schemas with no fields (apache#8472)
  KAFKA-9950: Construct new ConfigDef for MirrorTaskConfig before defining new properties (apache#8608)
  KAFKA-8869: Remove task configs for deleted connectors from config snapshot (apache#8444)
  KAFKA-9409: Supplement immutability of ClusterConfigState class in Connect (apache#7942)
gitlw pushed a commit to linkedin/kafka that referenced this pull request Jun 13, 2020
…elds (#77)

TICKET =
LI_DESCERIPTION = committing to our fork until upstream picks this up
EXIT_CRITERIA = when KAFKA-9855 is accepted upstream (specifically - apache#8472)

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
gitlw pushed a commit to linkedin/kafka that referenced this pull request Jun 13, 2020
…elds (#77)

TICKET =
LI_DESCERIPTION = committing to our fork until upstream picks this up
EXIT_CRITERIA = when KAFKA-9855 is accepted upstream (specifically - apache#8472)

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
gitlw pushed a commit to linkedin/kafka that referenced this pull request Jun 17, 2020
…elds (#77)

TICKET =
LI_DESCERIPTION = committing to our fork until upstream picks this up
EXIT_CRITERIA = when KAFKA-9855 is accepted upstream (specifically - apache#8472)

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
gitlw pushed a commit to linkedin/kafka that referenced this pull request Jun 17, 2020
…elds (#77)

TICKET =
LI_DESCERIPTION = committing to our fork until upstream picks this up
EXIT_CRITERIA = when KAFKA-9855 is accepted upstream (specifically - apache#8472)

Signed-off-by: radai-rosenblatt <radai.rosenblatt@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants