Skip to content

Conversation

@simicd
Copy link
Contributor

@simicd simicd commented Jan 19, 2024

Which issue does this PR close?

Closes #8208

Rationale for this change

Migrate tests from Rust to sqllogictest

What changes are included in this PR?

  • Remove duplicated code from partitioned_csv.rs
  • Create new unit test with partitioned csv files in csv_files.slt

Are these changes tested?

Yes

Are there any user-facing changes?

No

@github-actions github-actions bot added core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) labels Jan 19, 2024
let tmp_dir = TempDir::new()?;
let partition_count = 4;
let ctx = partitioned_csv::create_ctx(&tmp_dir, partition_count).await?;
let ctx = create_ctx_with_partition(&tmp_dir, partition_count).await?;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same code existed in partitioned_csv.rs and mod.rs - dropped the former and replaced with function calls to mod.rs

id9 value"9


# Read partitioned csv
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partitioned_csv.rs did not have any unit tests so I created a new one to test reading from partitioned csv files.

use tempfile::TempDir;

/// Generate CSV partitions within the supplied directory
fn populate_csv_partitions(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @simicd -- this looks great

@alamb alamb merged commit e986e15 into apache:main Jan 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Port tests in partitioned_csv.rs to sqllogictest

2 participants