Skip to content

MINOR: [C++][Parquet] Fix column_reader_test#35233

Merged
wjones127 merged 1 commit intoapache:mainfrom
wgtmac:fix_column_reader_test
Apr 19, 2023
Merged

MINOR: [C++][Parquet] Fix column_reader_test#35233
wjones127 merged 1 commit intoapache:mainfrom
wgtmac:fix_column_reader_test

Conversation

@wgtmac
Copy link
Member

@wgtmac wgtmac commented Apr 19, 2023

Rationale for this change

#14142 implemented the logic to skip parquet records and added a lot of test caese. However, there is a minor issue in the test function RecordReaderPrimitiveTypeTest::CheckReadValues where ! is missing.

    if (descr_->schema_node()->is_required()) {
      std::vector<int16_t> read_defs(
          record_reader_->def_levels(),
          record_reader_->def_levels() + record_reader_->levels_position());
      ASSERT_TRUE(vector_equal(expected_defs, read_defs));
    }

What changes are included in this PR?

Add ! to if (descr_->schema_node()->is_required()) as mentioned above.

Are these changes tested?

This is a fix to the test case.

Are there any user-facing changes?

NO.

@wgtmac wgtmac requested a review from wjones127 as a code owner April 19, 2023 08:30
@wgtmac
Copy link
Member Author

wgtmac commented Apr 19, 2023

cc @fatemehp @emkornfield

mapleFU
mapleFU approved these changes Apr 19, 2023
View reviewed changes
Copy link
Member

@mapleFU mapleFU left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Seems is_required means the non-nested leaf schema not need def-levels

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Apr 19, 2023
@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting committer review Awaiting committer review labels Apr 19, 2023
@wjones127 wjones127 merged commit f01853d into apache:main Apr 19, 2023
@ursabot
Copy link

ursabot commented Apr 21, 2023

Benchmark runs are scheduled for baseline = 8e5f6c8 and contender = f01853d. f01853d is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed] test-mac-arm
[Finished ⬇️1.28% ⬆️1.02%] ursa-i9-9960x
[Finished ⬇️0.42% ⬆️0.03%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] f01853d7 ec2-t3-xlarge-us-east-2
[Failed] f01853d7 test-mac-arm
[Finished] f01853d7 ursa-i9-9960x
[Finished] f01853d7 ursa-thinkcentre-m75q
[Finished] 8e5f6c80 ec2-t3-xlarge-us-east-2
[Failed] 8e5f6c80 test-mac-arm
[Finished] 8e5f6c80 ursa-i9-9960x
[Finished] 8e5f6c80 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot
Copy link

ursabot commented Apr 21, 2023

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x

liujiacheng777 pushed a commit to LoongArch-Python/arrow that referenced this pull request May 11, 2023
@wgtmac @liujiacheng777
MINOR: [C++][Parquet] Fix column_reader_test (apache#35233)
5ad335f 
### Rationale for this change

apache#14142 implemented the logic to skip parquet records and added a lot of test caese. However, there is a minor issue in the test function `RecordReaderPrimitiveTypeTest::CheckReadValues` where `!` is missing.

```c++
    if (descr_->schema_node()->is_required()) {
      std::vector<int16_t> read_defs(
          record_reader_->def_levels(),
          record_reader_->def_levels() + record_reader_->levels_position());
      ASSERT_TRUE(vector_equal(expected_defs, read_defs));
    }

```

### What changes are included in this PR?

Add `!` to `if (descr_->schema_node()->is_required())` as mentioned above.

### Are these changes tested?

This is a fix to the test case.

### Are there any user-facing changes?

NO.

Authored-by: Gang Wu <ustcwg@gmail.com>
Signed-off-by: Will Jones <willjones127@gmail.com>
ArgusLi pushed a commit to Bit-Quill/arrow that referenced this pull request May 15, 2023
@wgtmac @ArgusLi
MINOR: [C++][Parquet] Fix column_reader_test (apache#35233)
199ef20 
### Rationale for this change

apache#14142 implemented the logic to skip parquet records and added a lot of test caese. However, there is a minor issue in the test function `RecordReaderPrimitiveTypeTest::CheckReadValues` where `!` is missing.

```c++
    if (descr_->schema_node()->is_required()) {
      std::vector<int16_t> read_defs(
          record_reader_->def_levels(),
          record_reader_->def_levels() + record_reader_->levels_position());
      ASSERT_TRUE(vector_equal(expected_defs, read_defs));
    }

```

### What changes are included in this PR?

Add `!` to `if (descr_->schema_node()->is_required())` as mentioned above.

### Are these changes tested?

This is a fix to the test case.

### Are there any user-facing changes?

NO.

Authored-by: Gang Wu <ustcwg@gmail.com>
Signed-off-by: Will Jones <willjones127@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wjones127 wjones127 wjones127 approved these changes

@mapleFU mapleFU mapleFU approved these changes

Assignees

No one assigned

Labels

awaiting merge Awaiting merge Component: C++ Component: Parquet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@wgtmac @ursabot @wjones127 @mapleFU