Skip to content

refactor!: deprecate TFRecord support#4593

Merged
Xuanwo merged 9 commits intolance-format:mainfrom
jackye1995:deprecate-tf
Nov 19, 2025
Merged

refactor!: deprecate TFRecord support#4593
Xuanwo merged 9 commits intolance-format:mainfrom
jackye1995:deprecate-tf

Conversation

@jackye1995
Copy link
Copy Markdown
Contributor

@jackye1995 jackye1995 commented Aug 28, 2025

This PR removes TFRecord support from Lance while maintaining TensorFlow tf.data integration. Closes #4586

Comment thread README.md Outdated
Copy link
Copy Markdown
Contributor

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the things we want to delete are the tfrecord stuff, which is the only part we have Rust bindings for.

The other tensorflow stuff in Python I think we can keep.

Comment thread python/python/lance/tf/data.py
@jackye1995 jackye1995 changed the title refactor!: deprecate tensorflow support refactor!: deprecate TFRecord support Aug 28, 2025
@jackye1995 jackye1995 marked this pull request as draft August 28, 2025 23:27
@jackye1995 jackye1995 force-pushed the deprecate-tf branch 2 times, most recently from 6a0cd3f to 8f7217e Compare August 29, 2025 05:05
@timsaucer
Copy link
Copy Markdown
Contributor

I added a work around for the unit test failure in #4598

@timsaucer
Copy link
Copy Markdown
Contributor

My guess is that for this PR if you reverted your cargo lock's arrow and datafusion it would pass.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Aug 29, 2025

Codecov Report

❌ Patch coverage is 0% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.89%. Comparing base (8bb8aa8) to head (312f50f).

Files with missing lines Patch % Lines
rust/lance/src/dataset/transaction.rs 0.00% 10 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4593      +/-   ##
==========================================
+ Coverage   80.58%   80.89%   +0.30%     
==========================================
  Files         317      316       -1     
  Lines      119671   119054     -617     
  Branches   119671   119054     -617     
==========================================
- Hits        96439    96308     -131     
+ Misses      19773    19294     -479     
+ Partials     3459     3452       -7     
Flag Coverage Δ
unittests 80.89% <0.00%> (+0.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jackye1995 jackye1995 force-pushed the deprecate-tf branch 2 times, most recently from 3d33fbb to 3c52c49 Compare September 10, 2025 21:35
@jackye1995 jackye1995 marked this pull request as ready for review September 10, 2025 21:37
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo Xuanwo merged commit cf81a1e into lance-format:main Nov 19, 2025
23 of 25 checks passed
jackye1995 added a commit to jackye1995/lance that referenced this pull request Jan 21, 2026
This PR removes TFRecord support from Lance while maintaining TensorFlow
tf.data integration. Closes lance-format#4586

---------

Signed-off-by: Xuanwo <github@xuanwo.io>
Co-authored-by: Xuanwo <github@xuanwo.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

async-std has been discontinued based on RUSTSEC-2025-0052

5 participants