Skip to content

Parquet: Fix NPE in ParquetAvroWriter when schema contains variant type#15934

Merged
huaxingao merged 6 commits intoapache:mainfrom
nssalian:pq-avro-writer-npe-fix
Apr 13, 2026
Merged

Parquet: Fix NPE in ParquetAvroWriter when schema contains variant type#15934
huaxingao merged 6 commits intoapache:mainfrom
nssalian:pq-avro-writer-npe-fix

Conversation

@nssalian
Copy link
Copy Markdown
Contributor

Summary

Fixes: #15933

ParquetAvroWriter.WriteBuilder inherits ParquetTypeVisitor.variant() which returns null. For OPTIONAL variant fields, struct() wraps this null in OptionWriter, which dereferences writer.columns() resulting in an NPE

Changes

Override variant() to throw UnsupportedOperationException, similar to primitive() handles unsupported types in the same class.

Test plan

  • Wrote a test to check the UnsupportedOperationException and ran it locally.
  • Removed the fix in ParquetAvroWriter and verified the NPE is reproducible.

@nssalian nssalian closed this Apr 10, 2026
@nssalian nssalian reopened this Apr 10, 2026
@nssalian
Copy link
Copy Markdown
Contributor Author

nssalian commented Apr 10, 2026

Fixes for the runtime deps and zizmor is #15937. That would need to be merged and I'll rebase after it to get the tests to pass.

@nssalian nssalian marked this pull request as ready for review April 11, 2026 01:22
@nssalian
Copy link
Copy Markdown
Contributor Author

nssalian commented Apr 11, 2026

CC: @manuzhang @aihuaxu @huaxingao

Copy link
Copy Markdown
Contributor

@krvikash krvikash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we squash the commits in a single commit?

Comment thread parquet/src/test/java/org/apache/iceberg/parquet/TestParquet.java Outdated
Copy link
Copy Markdown
Contributor

@huaxingao huaxingao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nssalian
Copy link
Copy Markdown
Contributor Author

Thank you @huaxingao and @singhpk234 for your reviews.

Copy link
Copy Markdown
Contributor

@steveloughran steveloughran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@huaxingao huaxingao merged commit 56092bc into apache:main Apr 13, 2026
36 checks passed
@huaxingao
Copy link
Copy Markdown
Contributor

Thanks @nssalian for the PR! Thanks @singhpk234 @krvikash @steveloughran for the review!

@nssalian nssalian deleted the pq-avro-writer-npe-fix branch April 13, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parquet: Override variant() in ParquetAvroWriter.WriteBuilder to prevent null writer/NPE

5 participants