Skip to content

MINOR: move RecordReader from org.apache.kafka.tools (client module) to org.apache.kafka.tools.api (tools-api module)#13454

Merged
chia7712 merged 8 commits intoapache:trunkfrom
chia7712:MINOR-13454
Apr 6, 2023
Merged

MINOR: move RecordReader from org.apache.kafka.tools (client module) to org.apache.kafka.tools.api (tools-api module)#13454
chia7712 merged 8 commits intoapache:trunkfrom
chia7712:MINOR-13454

Conversation

@chia7712
Copy link
Copy Markdown
Member

from #13393 (comment)

org.apache.kafka.tools is existent in both tools and client module, and it causes split packages (https://www.logicbig.com/tutorials/core-java-tutorial/modules/split-packages.html)

test aggregatedJavadoc and the output is shown below.

截圖 2023-03-26 下午3 54 16

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@chia7712
Copy link
Copy Markdown
Member Author

@ijuma ping for review :)

I will update KIP if this change is acceptable

@chia7712
Copy link
Copy Markdown
Member Author

@ijuma @mimaison Could you take a look? move RecordReader to the same package as MessageFormatter

Copy link
Copy Markdown
Contributor

@junrao junrao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chia7712 : Thanks for the PR. Left a comment below.

* limitations under the License.
*/
package org.apache.kafka.tools;
package org.apache.kafka.common;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another possibility is to move this to the tools module under the package org.apache.kafka.tools.api, which seems a bit cleaner?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would follow a similar pattern as the storage api classes, which seems reasonable to me.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@junrao @ijuma thanks for feedback.

Another possibility is to move this to the tools module under the package org.apache.kafka.tools.api, which seems a bit cleaner?

tool module depends on core module, so I will add new subproject (tools-api) to tool moulds to avoid circular dependency.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tools doesn't depend on core, only tool tests depend on core. Right?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tools doesn't depend on core, only tool tests depend on core. Right?

ok, will use new package instead.

Copy link
Copy Markdown
Member

@ijuma ijuma Apr 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine to use a separate module as you have.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine to use a separate module as you have.

@ijuma thanks for this response. I prefer to use separate module. It won't bring complex dependency cycle ( core main/test -> tool main, tool test -> core main). Also, we can control the dependencies exposed to users without impacting the tools.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, makes sense.

@chia7712
Copy link
Copy Markdown
Member Author

chia7712 commented Apr 4, 2023

I have test docsJar, aggregatedJavadoc and releaseTarGz

截圖 2023-04-04 下午3 54 14

截圖 2023-04-04 下午4 02 28

截圖 2023-04-04 下午4 05 31

Comment thread build.gradle
implementation project(':group-coordinator')
implementation project(':metadata')
implementation project(':storage:api')
implementation project(':tools:tools-api')
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gradle can't distinguish the subproject name (gradle/gradle#847), so I add the prefix tools to avoid conflict to storage:api

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does core depend on tools-api?

Copy link
Copy Markdown
Contributor

@junrao junrao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chia7712 : Thanks for the updated PR. A couple of more comments.

Comment thread build.gradle
}
}

project(':tools:tools-api') {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change the javadoc part to include the tools.api package?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc-related test is shown in previous comment (#13454 (comment))

Comment thread build.gradle
implementation project(':group-coordinator')
implementation project(':metadata')
implementation project(':storage:api')
implementation project(':tools:tools-api')
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does core depend on tools-api?

@chia7712
Copy link
Copy Markdown
Member Author

chia7712 commented Apr 5, 2023

Why does core depend on tools-api?

ConsoleProducer is still in core module. #13214 can remove the dependency after it moves ConsoleProducer from core to tool

Copy link
Copy Markdown
Contributor

@junrao junrao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chia7712 : Thanks for the explanation. Just one more comment.

public class RecordReaderTest {

@Test
void testDefaultCloseAndConfigure() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we test in this unit test? There is no assertion.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sure the default implementation of those methods don't throw exception.

I will add assertDoesNotThrow to have it more readable.

Copy link
Copy Markdown
Contributor

@junrao junrao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chia7712 : Thanks for the latest PR. LGTM

@chia7712 chia7712 merged commit 637bc92 into apache:trunk Apr 6, 2023
@chia7712 chia7712 changed the title MINOR: move RecordReader from org.apache.kafka.tools to org.apache.co… MINOR: move RecordReader from org.apache.kafka.tools (client module) to org.apache.kafka.tools.api (tools-api module) Apr 6, 2023
@chia7712
Copy link
Copy Markdown
Member Author

chia7712 commented Apr 6, 2023

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866569 has been updated

Comment thread build.gradle
from(project(':streams:test-utils').configurations.runtimeClasspath) { into("libs/") }
from(project(':streams:examples').jar) { into("libs/") }
from(project(':streams:examples').configurations.runtimeClasspath) { into("libs/") }
from(project(':tools:tools-api').jar) { into("libs/") }
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This causes a resolution issue. I have opened https://issues.apache.org/jira/browse/KAFKA-19750 to fix it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants