Throw parse exceptions on schema get errors for SchemaRegistryBasedAvroBytesDecoder#12080
Merged
jon-wei merged 2 commits intoapache:masterfrom Jan 13, 2022
Merged
Conversation
…egistryBasedAvroBytesDecoder
jon-wei
changed the title
clintropolis
approved these changes
Dec 22, 2021
Member
clintropolis
left a comment
clintropolis
left a comment
There was a problem hiding this comment.
i think this change makes sense 👍
my intentions were good when i made this change originally - a bad configuration non-intuitively seemed to run tasks that just couldn't process the data and threw everything out, so if the configuration would never work it made sense to fail fast.
However, I didn't consider the case of a bad record, where throwing a RE means there is no way to process such streams without skipping the offset past such a record, while a parse exception handles this case nicely.
sachinsagare
pushed a commit
to sachinsagare/druid
that referenced
this pull request
Nov 3, 2022
…roBytesDecoder (apache#12080) * Add option to throw parse exceptions on schema get errors for SchemaRegistryBasedAvroBytesDecoder * Remove option (cherry picked from commit 74c876e)
gianm
added a commit
to gianm/druid
that referenced
this pull request
Nov 21, 2022
…stry. The change in apache#12080 lost the original exception context. This patch adds it back.
gianm
added a commit
to gianm/druid
that referenced
this pull request
Nov 21, 2022
…stry. The change in apache#12080 lost the original exception context. This patch adds it back.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adjusts
SchemaRegistryBasedAvroBytesDecoderto throwParseExceptioninstead ofREwhen failure to retrieve a schema occurs.Throwing an
REcan be problematic when this decoder is used in streaming ingestion: if a corrupt record enters the stream with an invalid schema ID, the user cannot easily recover from this as the streaming ingestion tasks will continue to fail and attempt to read the corrupt record.This PR has: