Skip to content

kafka-2320; Test PR builder#75

Closed
ijuma wants to merge 1 commit intoapache:trunkfrom
ijuma:kafka-2320
Closed

kafka-2320; Test PR builder#75
ijuma wants to merge 1 commit intoapache:trunkfrom
ijuma:kafka-2320

Conversation

@ijuma
Copy link
Copy Markdown
Member

@ijuma ijuma commented Jul 14, 2015

Just trying to trigger a build in Jenkins.

@ijuma
Copy link
Copy Markdown
Member Author

ijuma commented Jul 14, 2015

kafka-2320

@ijuma ijuma closed this Jul 14, 2015
@ijuma ijuma deleted the kafka-2320 branch July 14, 2015 23:35
@ijuma ijuma restored the kafka-2320 branch July 14, 2015 23:35
resetius added a commit to resetius/kafka that referenced this pull request Dec 5, 2016
@resetius
Merge pull request apache#75 from aneporada/logbroker-1637-revert
df83e6c 
Revert "[LOGBROKER-1637] save 10 versions of checkpoints" - changes a…
smccauliff pushed a commit to smccauliff/kafka that referenced this pull request Oct 8, 2020
@gitlw
[LI-HOTFIX] Cache the UpdateMetadataRequest when the inter broker pro…
8adc2d1 
…tocol version >= 2.3-IV2 (apache#75)

TICKET = KAFKA-7186
LI_DESCERIPTION = Cache the UpdateMetadataRequest when using the latest inter broker protocol
version
EXIT_CRITERIA = KAFKA-7186
wyuka pushed a commit to wyuka/kafka that referenced this pull request Mar 4, 2022
@gitlw
[LI-HOTFIX] Cache the UpdateMetadataRequest.Builder when the IBP is >…
307d46c 
…= KAFKA_2_3_IV2 (apache#288)

TICKET = KAFKA-7186
LI_DESCERIPTION =
The original PR for this change is apache#75.
Before this patch, the controller needs to allocate an individual UpdateMetadataRequest.Builder
for each broker. This may incur significant memory overhead if the controller has just started,
and is trying to send the cluster's full metadata to all brokers.

This PR tries to reduce the memory footprint by reusing the same UpdateMetadataRequest.Builder.
This is only achievable when all of the UMR have the same body payload, which means
the common UMR should use the maxBrokerEpoch field instead of individual broker epochs.

EXIT_CRITERIA = KAFKA-7186
wyuka pushed a commit to wyuka/kafka that referenced this pull request Mar 4, 2022
@gitlw
[LI-HOTFIX] Cache the body buffer of UpdateMetadataRequests (apache#287)
a07c125 
TICKET = KAFKA-7186
LI_DESCERIPTION =
This is a rewrite of the original PR apache#75.
In kafka 3.0, structs are removed, and requests can be serialized directly into ByteBuffers.
In vanilla kafka, the header and body are serialized directly into a single ByteBuffer inside the SendBuilder.

However, to reduce the memory footprint of UMRs, we would still want the ByteBufferSends
destined to different brokers to share the same body ByteBuffer. To achieve that, this PR modifies
the Java code generator so that the generated UpdateMetadataRequestData has the following
implementation for its write method

`
    private final ReentrantLock bodyBufferLock = new ReentrantLock();
    private ByteBuffer bodyBuffer = null;
    @OverRide
    public void write(Writable _writable, ObjectSerializationCache _cache, short _version) {
        bodyBufferLock.lock();
        try{
            if (bodyBuffer == null) {
                ObjectSerializationCache serializationCache = new ObjectSerializationCache();
                MessageSizeAccumulator messageSize = new MessageSizeAccumulator();
                addSize(messageSize, serializationCache, _version);
                bodyBuffer = ByteBuffer.allocate(messageSize.totalSize());
                doWrite(new ByteBufferAccessor(bodyBuffer), _cache, _version);
                bodyBuffer.flip();
            }
        } finally {
            bodyBufferLock.unlock();
        }
        _writable.writeByteBuffer(bodyBuffer);
    }
`

All other types of requests would end up with a write implementation as follows, which is
essentially the same as the vanilla implementation.
`
    @OverRide
    public void write(Writable _writable, ObjectSerializationCache _cache, short _version) {
        doWrite(_writable, _cache, _version);
    }

`

EXIT_CRITERIA = KAFKA-7186
wyuka pushed a commit to wyuka/kafka that referenced this pull request Mar 28, 2022
@gitlw @lmr3796
[LI-HOTFIX] Cache the UpdateMetadataRequest.Builder when the IBP is >…
b4ce1a5 
…= KAFKA_2_3_IV2 (apache#288)

TICKET = KAFKA-7186
LI_DESCERIPTION =
The original PR for this change is apache#75.
Before this patch, the controller needs to allocate an individual UpdateMetadataRequest.Builder
for each broker. This may incur significant memory overhead if the controller has just started,
and is trying to send the cluster's full metadata to all brokers.

This PR tries to reduce the memory footprint by reusing the same UpdateMetadataRequest.Builder.
This is only achievable when all of the UMR have the same body payload, which means
the common UMR should use the maxBrokerEpoch field instead of individual broker epochs.

EXIT_CRITERIA = KAFKA-7186
wyuka pushed a commit to wyuka/kafka that referenced this pull request Mar 28, 2022
@gitlw @lmr3796
[LI-HOTFIX] Cache the body buffer of UpdateMetadataRequests (apache#287)
57cc525 
TICKET = KAFKA-7186
LI_DESCERIPTION =
This is a rewrite of the original PR apache#75.
In kafka 3.0, structs are removed, and requests can be serialized directly into ByteBuffers.
In vanilla kafka, the header and body are serialized directly into a single ByteBuffer inside the SendBuilder.

However, to reduce the memory footprint of UMRs, we would still want the ByteBufferSends
destined to different brokers to share the same body ByteBuffer. To achieve that, this PR modifies
the Java code generator so that the generated UpdateMetadataRequestData has the following
implementation for its write method

`
    private final ReentrantLock bodyBufferLock = new ReentrantLock();
    private ByteBuffer bodyBuffer = null;
    @OverRide
    public void write(Writable _writable, ObjectSerializationCache _cache, short _version) {
        bodyBufferLock.lock();
        try{
            if (bodyBuffer == null) {
                ObjectSerializationCache serializationCache = new ObjectSerializationCache();
                MessageSizeAccumulator messageSize = new MessageSizeAccumulator();
                addSize(messageSize, serializationCache, _version);
                bodyBuffer = ByteBuffer.allocate(messageSize.totalSize());
                doWrite(new ByteBufferAccessor(bodyBuffer), _cache, _version);
                bodyBuffer.flip();
            }
        } finally {
            bodyBufferLock.unlock();
        }
        _writable.writeByteBuffer(bodyBuffer);
    }
`

All other types of requests would end up with a write implementation as follows, which is
essentially the same as the vanilla implementation.
`
    @OverRide
    public void write(Writable _writable, ObjectSerializationCache _cache, short _version) {
        doWrite(_writable, _cache, _version);
    }

`

EXIT_CRITERIA = KAFKA-7186
wyuka pushed a commit to wyuka/kafka that referenced this pull request Jun 16, 2022
@gitlw @lmr3796
[LI-HOTFIX] Cache the UpdateMetadataRequest.Builder when the IBP is >…
4777449 
…= KAFKA_2_3_IV2 (apache#288)

TICKET = KAFKA-7186
LI_DESCERIPTION =
The original PR for this change is apache#75.
Before this patch, the controller needs to allocate an individual UpdateMetadataRequest.Builder
for each broker. This may incur significant memory overhead if the controller has just started,
and is trying to send the cluster's full metadata to all brokers.

This PR tries to reduce the memory footprint by reusing the same UpdateMetadataRequest.Builder.
This is only achievable when all of the UMR have the same body payload, which means
the common UMR should use the maxBrokerEpoch field instead of individual broker epochs.

EXIT_CRITERIA = KAFKA-7186
wyuka pushed a commit to wyuka/kafka that referenced this pull request Jun 16, 2022
@gitlw @lmr3796
[LI-HOTFIX] Cache the body buffer of UpdateMetadataRequests (apache#287)
693c518 
TICKET = KAFKA-7186
LI_DESCERIPTION =
This is a rewrite of the original PR apache#75.
In kafka 3.0, structs are removed, and requests can be serialized directly into ByteBuffers.
In vanilla kafka, the header and body are serialized directly into a single ByteBuffer inside the SendBuilder.

However, to reduce the memory footprint of UMRs, we would still want the ByteBufferSends
destined to different brokers to share the same body ByteBuffer. To achieve that, this PR modifies
the Java code generator so that the generated UpdateMetadataRequestData has the following
implementation for its write method

`
    private final ReentrantLock bodyBufferLock = new ReentrantLock();
    private ByteBuffer bodyBuffer = null;
    @OverRide
    public void write(Writable _writable, ObjectSerializationCache _cache, short _version) {
        bodyBufferLock.lock();
        try{
            if (bodyBuffer == null) {
                ObjectSerializationCache serializationCache = new ObjectSerializationCache();
                MessageSizeAccumulator messageSize = new MessageSizeAccumulator();
                addSize(messageSize, serializationCache, _version);
                bodyBuffer = ByteBuffer.allocate(messageSize.totalSize());
                doWrite(new ByteBufferAccessor(bodyBuffer), _cache, _version);
                bodyBuffer.flip();
            }
        } finally {
            bodyBufferLock.unlock();
        }
        _writable.writeByteBuffer(bodyBuffer);
    }
`

All other types of requests would end up with a write implementation as follows, which is
essentially the same as the vanilla implementation.
`
    @OverRide
    public void write(Writable _writable, ObjectSerializationCache _cache, short _version) {
        doWrite(_writable, _cache, _version);
    }

`

EXIT_CRITERIA = KAFKA-7186
rustd pushed a commit to rustd/pranavfinaldemokafka that referenced this pull request Feb 9, 2024
@rutvijmehta-harness @jsancio
MINOR; Add accessor methods to OffsetAndEpoch (apache#12770) (apache#75)
f44e94a 
Accessor are preferred over fields because they compose better with Java's
lambda syntax.

Reviewers: Jason Gustafson <jason@confluent.io>

Co-authored-by: José Armando García Sancio <jsancio@users.noreply.github.com>
davide-armand pushed a commit to aiven/kafka that referenced this pull request Dec 1, 2025
@ivanyu
Make ControlPlane Configurable (apache#75)
31df719 
This is not needed for InMemoryControlPlane, but will be needed for other implementation.
fvaleri pushed a commit to fvaleri/kafka that referenced this pull request Feb 4, 2026
@showuon
Merge pull request apache#75 from showuon/mirrorFetchInFollower
425d53f 
don't pass mirror name for followers in target cluster
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ijuma