Support dateCreated expressions in ScanSummary. #2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
LGTM +1 |
yifeih
pushed a commit
to yifeih/incubator-iceberg
that referenced
this pull request
Apr 16, 2019
prodeezy
referenced
this pull request
in rominparekh/incubator-iceberg
Dec 17, 2019
# This is the 1st commit message: Issue-629: Cherrypick Id # This is the commit message #2: Removed redundant methods and changed method name # This is the commit message #3: Fix Imports # This is the commit message #4: Fix Operation Check # This is the commit message apache#5: Fix Error Message # This is the commit message apache#6: Cherry picking operation to apply changes from incoming snapshot on current snapshot # This is the commit message apache#7: Initial working version of cherry-pick operation which applies appends only
mehtaashish23
referenced
this pull request
in mehtaashish23/incubator-iceberg
Jan 28, 2020
Minimize changes and fix commit logic.
CodingCat
referenced
this pull request
in CodingCat/iceberg
Dec 22, 2021
danielcweeks
pushed a commit
that referenced
this pull request
Jan 14, 2023
…flake-managed Iceberg tables (#6428) * Initial read-only Snowflake Catalog implementation by @sfc-gh-mparmar (#1) Initial read-only Snowflake Catalog implementation built on top of the Snowflake JDBC driver, providing support for basic listing of namespaces, listing of tables, and loading/reads of tables. Auth options are passthrough to the JDBC driver. Co-authored-by: Maninder Parmar <maninder.parmar@snowflake.com> Co-authored-by: Maninder Parmar <maninder.parmar+oss@snowflake.com> Co-authored-by: Dennis Huo <dennis.huo+oss@snowflake.com> * Add JdbcSnowflakeClientTest using mocks (#2) Add JdbcSnowflakeClientTest using mocks; provides full coverage of JdbcSnowflakeClient and entities' ResultSetHandler logic. Also update target Spark runtime versions to be included. * Add test { useJUnitPlatform() } tuple to iceberg-snowflake for consistency and future interoperability with inheriting from abstact unittest base classes. * Extract versions into versions.props per PR review * Misc test-related refactors per review suggestions -Convert unittests to all use assertj/Assertions for "fluent assertions" -Refactor test injection into overloaded initialize() method -Add test cases for close() propagation -Use CloseableGroup. * Fix unsupported behaviors of loadNamedpaceMetadata and defaultWarehouseLocation * Move TableIdentifier checks out of newTableOps into the SnowflakTableOperations class itself, add test case. * Refactor out any Namespace-related business logic from the lower SnowflakeClient/JdbcSnowflakeClient layers and merge SnowflakeTable and SnowflakeSchema into a single SnowflakeIdentifier that also encompasses ROOT and DATABASE level identifiers. A SnowflakeIdentifier thus functions like a type-checked/constrained Iceberg TableIdentifier, and eliminates any tight coupling between a SnowflakeClient and Catalog business logic. Parsing of Namespace numerical levels into a SnowflakeIdentifier is now fully encapsulated in NamespaceHelpers so that callsites don't duplicate namespace-handling/validation logic. * Finish migrating JdbcSnowflakeClientTest off any usage of org.junit.Assert in favor of assertj's Assertions. * Style refactorings from review comments, expanded and moved InMemoryFileIO into core with its own unittest. * Fix behavior of getNamespaceMetadata to throw when the namespace doesn't exist. Refactor for naming conventions and consolidating identifier handling into NamespaceHandlers. Make FileIO instantiated fresh for each newTableOps call. * Move private constructor to top, add assertion to test case. * Define minimal ResultSetParser/QueryHarness classes to fully replace any use of commons-dbutils; refactor ResultSet handling fully into JdbcSnowflakeClient.java. * Update snowflake/src/main/java/org/apache/iceberg/snowflake/SnowflakeTableOperations.java Co-authored-by: Eduard Tudenhöfner <etudenhoefner@gmail.com> * Refactor style suggestions; remove debug-level logging, arguments in exceptions, private members if not accessed outside, move precondition checks, add test for NamespaceHelpers. * Fix precondition messages, remove getConf() * Clean up varargs. * Make data members final, include rawJsonVal in toString for debuggability. * Combine some small test cases into roundtrip test cases, misc cleanup * Add comment for why a factory class is exposed for testing purposes. Co-authored-by: Dennis Huo <dennis.huo@snowflake.com> Co-authored-by: Maninder Parmar <maninder.parmar@snowflake.com> Co-authored-by: Maninder Parmar <maninder.parmar+oss@snowflake.com> Co-authored-by: Eduard Tudenhöfner <etudenhoefner@gmail.com>
adamyasharma2797
pushed a commit
to adamyasharma2797/iceberg
that referenced
this pull request
Jul 19, 2024
LAMBERT-83: Made Iceberg Commit Single Phase
Closed
3 tasks
3 tasks
steveloughran
added a commit
to steveloughran/iceberg
that referenced
this pull request
Apr 25, 2025
The changes made earlier to the hadoop exclusions should ensure that no artifacts of earlier releases get onto the test classpath of other modules.
fabio-rizzo-01
added a commit
to fabio-rizzo-01/iceberg
that referenced
this pull request
May 6, 2025
….google.errorprone-error_prone_annotations-2.38.0 Build: Bump com.google.errorprone:error_prone_annotations from 2.37.0 to 2.38.0
steveloughran
added a commit
to steveloughran/iceberg
that referenced
this pull request
May 30, 2025
The changes made earlier to the hadoop exclusions should ensure that no artifacts of earlier releases get onto the test classpath of other modules.
steveloughran
added a commit
to steveloughran/iceberg
that referenced
this pull request
Jul 4, 2025
The changes made earlier to the hadoop exclusions should ensure that no artifacts of earlier releases get onto the test classpath of other modules.
steveloughran
added a commit
to steveloughran/iceberg
that referenced
this pull request
Jul 7, 2025
The changes made earlier to the hadoop exclusions should ensure that no artifacts of earlier releases get onto the test classpath of other modules.
wangyum
added a commit
to wangyum/iceberg
that referenced
this pull request
Jan 22, 2026
This commit addresses critical performance issues identified in the EDV implementation and adds safety checks for large bitmaps. Issue apache#1: Read Path Memory Optimization (CRITICAL FIX) - Problem: BaseDeleteLoader was converting efficient bitmaps back into millions of Java objects, defeating the memory optimization purpose - Impact: 1M deletes would use ~100MB instead of ~1MB (100x overhead) - Fix: Changed DeleteLoader.loadEqualityDeletes() return type from StructLikeSet to Set<StructLike> to allow BitmapBackedStructLikeSet - Result: Achieves true 100x memory reduction with O(1) bitmap lookups - Files: DeleteLoader.java, BaseDeleteLoader.java, DeleteFilter.java Issue apache#2: Writer Safety with Large Bitmaps (SAFETY FIX) - Problem: Unsafe cast to int could overflow if bitmap exceeds 2GB - Fix: Added validation in EqualityDeleteVectorWriter.toBlob() - Result: Clear error message if bitmap size > Integer.MAX_VALUE - Files: EqualityDeleteVectorWriter.java BitmapBackedStructLikeSet Improvements: - Implemented iterator() for mixed-format scenarios (EDV + traditional) - Required for Iterables.addAll() in mixed delete file merging - Fixed compilation issue with StructLikeWrapper return type - Updated test from "unsupported" to "supported" iterator - Files: BitmapBackedStructLikeSet.java, TestBitmapBackedStructLikeSet.java Memory Performance: - 1M deletes: 100MB -> 1MB (100x reduction) ✅ - 10M deletes: 1GB -> 10MB (100x reduction) ✅ - Lookup: O(1) bitmap check (no object creation) ✅ All 37 tests passing. Co-Authored-By: Claude <<EMAIL_ADDRESS>>
wangyum
added a commit
to wangyum/iceberg
that referenced
this pull request
Jan 22, 2026
Document resolution of critical compilation blockers: Resolved Issues: ✅ Critical apache#1: Spark compilation fixed + 4 integration tests ✅ Critical apache#2: Flink compilation fixed + 3 integration tests Progress: - Before: 25% ready for Apache PR - After: 40% ready for Apache PR - Build: Fully passing - Tests: 46 total (7 new integration tests) Remaining Work: - JMH benchmarks (2 days) - Complete spec (2 days) - User docs (1 day) - Community process (2-4 weeks) Timeline to merge: 6-10 weeks (down from 8-13 weeks) Merge probability: 75% (up from 70%)
steveloughran
added a commit
to steveloughran/iceberg
that referenced
this pull request
Jan 23, 2026
1. verify setting schemas to "" means default hdfs value goes 2. verify that a failure in moveToTrash() is caught and downgraded. Test case apache#2 is the key one, as it shows that delete works even if trash move somehow failed. + fix doc trailing space failure.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.