Ignore __pycache__ contents and *.pyc files in external dependencies

[UebelAndre]

PR Checklist

Please check if your PR fulfills the following requirements:

• Does not include precompiled binaries, eg. .par files. See CONTRIBUTING.md for info
• Tests for the changes have been added (for bug fixes / features)
• Docs have been added / updated (for bug fixes / features)

PR Type

What kind of change does this PR introduce?

• Bugfix
• Feature (please, look at the "Scope of the project" section in the README.md file)
• Code style update (formatting, local variables)
• Refactoring (no functional changes, no api changes)
• Build related changes
• CI related changes
• Documentation content changes
• Other... Please describe:

What is the current behavior?

I noticed in some of my builds were getting a bunch of unexpected cache misses and found that some pypi dependencies were including some __pycache__ contents and left-over *.pyc files. I don't believe these are needed or should ever be used by the Bazel rules.

Issue Number: N/A

What is the new behavior?

This PR prevents these files from being included in the pip_parse and pip_install generated py_library targets.

Does this PR introduce a breaking change?

• Yes
• No

Other information

[mattem]

Hm, this only excludes it from data, I'm finding __pycache__ files in srcs

[UebelAndre]

I'm on mobile now but I think it'd make sense to exclude them from srcs as well. If you wanted to push that change to this branch.

[mattem]

Doesn't look like I have permission to push, these were my changes:

diff --git a/python/pip_install/extract_wheels/lib/bazel.py b/python/pip_install/extract_wheels/lib/bazel.py
index d498581..6402d8f 100644
--- a/python/pip_install/extract_wheels/lib/bazel.py
+++ b/python/pip_install/extract_wheels/lib/bazel.py
@@ -103,6 +103,7 @@ def generate_build_file_contents(
     """

     data_exclude = [
+        "*dist-info/RECORD",
         "*.whl",
         "**/__pycache__/**",
         "**/*.py",
@@ -137,7 +138,7 @@ def generate_build_file_contents(

         py_library(
             name = "{name}",
-            srcs = glob(["**/*.py"], exclude=["{entry_point_prefix}*.py"], allow_empty = True),
+            srcs = glob(["**/*.py"], exclude=["{entry_point_prefix}*.py", "**/__pycache__/**"], allow_empty = True),
             data = glob(["**/*"], exclude={data_exclude}),
             # This makes this directory a top-level in the python import
             # search path for anything that depends on this.

[UebelAndre]

Doesn't look like I have permission to push, these were my changes:

@mattem Ah, I thought you were a maintainer here, makes sense then.

I picked up the only the additional exclude for __pycache__. I'm wondering why *dist-info/RECORD would be ignored and not all of *dist-info/**. Could you explain the reasoning there?

[thundergolfer]

Looks correct to me.

[UebelAndre]

Doesn't look like I have permission to push, these were my changes:

@mattem Ah, I thought you were a maintainer here, makes sense then.

I picked up the only the additional exclude for __pycache__. I'm wondering why *dist-info/RECORD would be ignored and not all of *dist-info/**. Could you explain the reasoning there?

Would still like to hear @mattem's response to this if you don't mind 😄