DO NOT MERGE: Testing the MacOS wheel failures

[jorisvandenbossche]

Test for #39562

[github-actions]

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

• Other pull requests
• Contribution Guidelines - Contributing Overview

[jorisvandenbossche]

@github-actions crossbow submit wheel-macos

[github-actions]

Revision: bb48604

Submitted crossbow builds: ursacomputing/crossbow @ actions-512b0e912c

Task	Status
wheel-macos-big-sur-cp310-arm64
wheel-macos-big-sur-cp311-arm64
wheel-macos-big-sur-cp312-arm64
wheel-macos-big-sur-cp38-arm64
wheel-macos-big-sur-cp39-arm64
wheel-macos-catalina-cp310-amd64
wheel-macos-catalina-cp311-amd64
wheel-macos-catalina-cp312-amd64
wheel-macos-catalina-cp38-amd64
wheel-macos-catalina-cp39-amd64

[jorisvandenbossche]

So it's clearly not related to the logging fs wrapper around the filesystem ..

[jorisvandenbossche]

@github-actions crossbow submit wheel-macos

[github-actions]

Revision: a60c5b3

Submitted crossbow builds: ursacomputing/crossbow @ actions-c22db1a11f

Task	Status
wheel-macos-big-sur-cp310-arm64
wheel-macos-big-sur-cp311-arm64
wheel-macos-big-sur-cp312-arm64
wheel-macos-big-sur-cp38-arm64
wheel-macos-big-sur-cp39-arm64
wheel-macos-catalina-cp310-amd64
wheel-macos-catalina-cp311-amd64
wheel-macos-catalina-cp312-amd64
wheel-macos-catalina-cp38-amd64
wheel-macos-catalina-cp39-amd64

[jorisvandenbossche]

@github-actions crossbow submit wheel-macos

[github-actions]

Revision: a60c5b3

Submitted crossbow builds: ursacomputing/crossbow @ actions-3bd0891f6d

Task	Status
wheel-macos-big-sur-cp310-arm64
wheel-macos-big-sur-cp311-arm64
wheel-macos-big-sur-cp312-arm64
wheel-macos-big-sur-cp38-arm64
wheel-macos-big-sur-cp39-arm64
wheel-macos-catalina-cp310-amd64
wheel-macos-catalina-cp311-amd64
wheel-macos-catalina-cp312-amd64
wheel-macos-catalina-cp38-amd64
wheel-macos-catalina-cp39-amd64

[jorisvandenbossche]

@github-actions crossbow submit wheel-macos

[github-actions]

Revision: 86c3018

Submitted crossbow builds: ursacomputing/crossbow @ actions-2cf7237b8b

Task	Status
wheel-macos-big-sur-cp310-arm64
wheel-macos-big-sur-cp311-arm64
wheel-macos-big-sur-cp312-arm64
wheel-macos-big-sur-cp38-arm64
wheel-macos-big-sur-cp39-arm64
wheel-macos-catalina-cp310-amd64
wheel-macos-catalina-cp311-amd64
wheel-macos-catalina-cp312-amd64
wheel-macos-catalina-cp38-amd64
wheel-macos-catalina-cp39-amd64

[jorisvandenbossche]

@github-actions crossbow submit wheel-macos

[github-actions]

Revision: 86c3018

Submitted crossbow builds: ursacomputing/crossbow @ actions-99f7730303

Task	Status
wheel-macos-big-sur-cp310-arm64
wheel-macos-big-sur-cp311-arm64
wheel-macos-big-sur-cp312-arm64
wheel-macos-big-sur-cp38-arm64
wheel-macos-big-sur-cp39-arm64
wheel-macos-catalina-cp310-amd64
wheel-macos-catalina-cp311-amd64
wheel-macos-catalina-cp312-amd64
wheel-macos-catalina-cp38-amd64
wheel-macos-catalina-cp39-amd64

[jorisvandenbossche]

So I could nail down the failure to the following:

arrow/cpp/src/arrow/dataset/file_parquet.cc

Lines 814 to 823 in ac50918

	Status ParquetFileFragment::SetMetadata(
	std::shared_ptr<parquet::FileMetaData> metadata,
	std::shared_ptr<parquet::arrow::SchemaManifest> manifest) {
	DCHECK(row_groups_.has_value());
	metadata_ = std::move(metadata);
	manifest_ = std::move(manifest);
	statistics_expressions_.resize(row_groups_->size(), compute::literal(true));
	statistics_expressions_complete_.resize(manifest_->descr->num_columns(), false);

In the above snippet, sometimes manifest_->descr->num_columns() returns -1, and so then we do a vector resize with -1 which triggers the std::length_error crash.

But I have no idea why that would sometimes return -1, and only on MacOS when running the test from an installed wheel (not any of the other Mac builds where we build Arrow directly)

[pitrou]

Fixed in #39632. We can close now.