-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Add support for deduplication testing #10246
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
18831ac to
f6d9802
Compare
merlimat
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the goal is testing the deduping mechanism, we should probably also add something in the performance consumer to validate that no messages with duplicated sequences are received.
d07e66f to
bc293c8
Compare
|
Merlimat, RE the consumer side, I'm sure the consumer side would also benefit as well for a possible demo/software acceptance level validation. However, in my particular use case the validation and scanning for duplicates and lost records was easier implemented in the Data Sink which received the data hence I did not invest extra to build a data validation layer within pulsar test client, only the ability to publish data using both the dedup enabled and dedup disabled. Since, I did not have the requirement, it was not coded. And I did not want to assume that this was a requirement, as a poor implementation may show both strengths and weaknesses in the platform. I can certainly take it in that direction. But would think that may be a separate PR. |
bc293c8 to
636306a
Compare
|
/pulsabot rerun CI-CPP,Python Tests |
Add some basic support for testing deduplication feature.