Skip to content

Remove dataframe warnings from py38-docs logs #21861

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
TheNeuralBit merged 1 commit into from
Jun 16, 2022
+10 −6
Merged

Remove dataframe warnings from py38-docs logs #21861

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions sdks/python/scripts/generate_pydoc.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,24 +247,28 @@ EOF

# Build the documentation using sphinx
# Reference: http://www.sphinx-doc.org/en/stable/man/sphinx-build.html
# Note we cut out warnings from apache_beam.dataframe, this package uses pandas
# documentation verbatim.
python $(type -p sphinx-build) -v -a -E -q target/docs/source \
target/docs/_build -c target/docs/source \
-w "target/docs/sphinx-build.warnings.log"
|& grep -E -v 'apache_beam\.dataframe.*WARNING:' \
|& tee "target/docs/sphinx-build.log"

# Fail if there are errors or warnings in docs
! grep -q "ERROR:" target/docs/sphinx-build.warnings.log || exit 1
(! grep -v 'apache_beam.dataframe' target/docs/sphinx-build.warnings.log | grep -q "WARNING:") || exit 1
Copy link
Contributor

@AnandInguva AnandInguva Jun 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why apache_beam.dataframe was specified specially here? Just curious

Copy link
Member Author

@TheNeuralBit TheNeuralBit Jun 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment above to explain it. The issue is that for the DataFrame API we copy docstrings verbatim from the equivalent pandas methods, and these have various issues that raise warnings in sphinx

Copy link
Contributor

@AnandInguva AnandInguva Jun 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Thanks

! grep -q "ERROR:" target/docs/sphinx-build.log || exit 1
! grep -q "WARNING:" target/docs/sphinx-build.log || exit 1

# Run tests for code samples, these can be:
# - Code blocks using '.. testsetup::', '.. testcode::' and '.. testoutput::'
# - Interactive code starting with '>>>'
python -msphinx -M doctest target/docs/source \
target/docs/_build -c target/docs/source \
-w "target/docs/sphinx-doctest.warnings.log"
|& grep -E -v 'apache_beam\.dataframe.*WARNING:' \
|& tee "target/docs/sphinx-doctest.log"

# Fail if there are errors or warnings in docs
! grep -q "ERROR:" target/docs/sphinx-doctest.warnings.log || exit 1
(! grep -v 'apache_beam.dataframe' target/docs/sphinx-doctest.warnings.log | grep -q "WARNING:") || exit 1
! grep -q "ERROR:" target/docs/sphinx-doctest.log || exit 1
! grep -q "WARNING:" target/docs/sphinx-doctest.log || exit 1

# Message is useful only when this script is run locally. In a remote
# test environment, this path will be removed when the test completes.
Expand Down