From 991c164b39375636fbf7a082bb289170f01e3117 Mon Sep 17 00:00:00 2001 From: Jonathon Belotti Date: Mon, 21 Dec 2020 21:42:10 +1100 Subject: [PATCH 1/2] remove the rules_python_external README that was copied over during repo merge --- python/pip_install/README.md | 134 ----------------------------------- 1 file changed, 134 deletions(-) delete mode 100644 python/pip_install/README.md diff --git a/python/pip_install/README.md b/python/pip_install/README.md deleted file mode 100644 index 4db18529f8..0000000000 --- a/python/pip_install/README.md +++ /dev/null @@ -1,134 +0,0 @@ -# rules_python_external ![](https://github.com/dillon-giacoppo/rules_python_external/workflows/CI/badge.svg) - -Bazel rules to transitively fetch and install Python dependencies from a requirements.txt file. - -## Features - -The rules address most of the top packaging issues in [`bazelbuild/rules_python`](https://github.com/bazelbuild/rules_python). This means the rules support common packages such -as [`tensorflow`](https://pypi.org/project/tensorflow/) and [`google.cloud`](https://github.com/googleapis/google-cloud-python) natively. - -* Transitive dependency resolution: - [#35](https://github.com/bazelbuild/rules_python/issues/35), - [#102](https://github.com/bazelbuild/rules_python/issues/102) -* Minimal runtime dependencies: - [#184](https://github.com/bazelbuild/rules_python/issues/184) -* Support for [spreading purelibs](https://www.python.org/dev/peps/pep-0491/#installing-a-wheel-distribution-1-0-py32-none-any-whl): - [#71](https://github.com/bazelbuild/rules_python/issues/71) -* Support for [namespace packages](https://packaging.python.org/guides/packaging-namespace-packages/): - [#14](https://github.com/bazelbuild/rules_python/issues/14), - [#55](https://github.com/bazelbuild/rules_python/issues/55), - [#65](https://github.com/bazelbuild/rules_python/issues/65), - [#93](https://github.com/bazelbuild/rules_python/issues/93), - [#189](https://github.com/bazelbuild/rules_python/issues/189) -* Fetches pip packages only for building Python targets: - [#96](https://github.com/bazelbuild/rules_python/issues/96) -* Reproducible builds: - [#154](https://github.com/bazelbuild/rules_python/issues/154), - [#176](https://github.com/bazelbuild/rules_python/issues/176) - -## Usage - -#### Prerequisites - -The rules support Python >= 3.5 (the oldest [maintained release](https://devguide.python.org/#status-of-python-branches)). - -#### Setup `WORKSPACE` - -```python -rules_python_external_version = "{COMMIT_SHA}" - -http_archive( - name = "rules_python_external", - sha256 = "", # Fill in with correct sha256 of your COMMIT_SHA version - strip_prefix = "rules_python_external-{version}".format(version = rules_python_external_version), - url = "https://github.com/dillon-giacoppo/rules_python_external/archive/v{version}.zip".format(version = rules_python_external_version), -) - -# Install the rule dependencies -load("@rules_python_external//:repositories.bzl", "rules_python_external_dependencies") -rules_python_external_dependencies() - -load("@rules_python_external//:defs.bzl", "pip_install") -pip_install( - name = "py_deps", - requirements = "//:requirements.txt", - # (Optional) You can provide a python interpreter (by path): - python_interpreter = "/usr/bin/python3.8", - # (Optional) Alternatively you can provide an in-build python interpreter, that is available as a Bazel target. - # This overrides `python_interpreter`. - # Note: You need to set up the interpreter target beforehand (not shown here). Please see the `example` folder for further details. - #python_interpreter_target = "@python_interpreter//:python_bin", -) -``` - -#### Example `BUILD` file. - -```python -load("@py_deps//:requirements.bzl", "requirement", "whl_requirement") - -py_binary( - name = "main", - srcs = ["main.py"], - deps = [ - requirement("boto3"), - ] -) - -# If you need to depend on the wheel dists themselves, for instance to pass them -# to some other packaging tool, you can get a handle to them with the whl_requirement macro. -filegroup( - name = "whl_files", - data = [ - whl_requirement("boto3"), - ] -) -``` - -Note that above you do not need to add transitively required packages to `deps = [ ... ]` or `data = [ ... ]` - -#### Setup `requirements.txt` - -While `rules_python_external` **does not** require a _transitively-closed_ `requirements.txt` file, it is recommended. -But if you want to just have top-level packages listed, that also will work. - -Transitively-closed requirements specs are very tedious to produce and maintain manually. To automate the process we -recommend [`pip-compile` from `jazzband/pip-tools`](https://github.com/jazzband/pip-tools#example-usage-for-pip-compile). - -For example, `pip-compile` takes a `requirements.in` like this: - -``` -boto3~=1.9.227 -botocore~=1.12.247 -click~=7.0 -``` - -`pip-compile` 'compiles' it so you get a transitively-closed `requirements.txt` like this, which should be passed to -`pip_install` below: - -``` -boto3==1.9.253 -botocore==1.12.253 -click==7.0 -docutils==0.15.2 # via botocore -jmespath==0.9.4 # via boto3, botocore -python-dateutil==2.8.1 # via botocore -s3transfer==0.2.1 # via boto3 -six==1.14.0 # via python-dateutil -urllib3==1.25.8 # via botocore -``` - -### Demo - -You can find a demo in the [example/](./example) directory. - -## Development - -### Testing - -`bazel test //...` - -## Adopters - -Here's a (non-exhaustive) list of companies that use `rules_python_external` in production. Don't see yours? [You can add it in a PR](https://github.com/dillon-giacoppo/rules_python_external/edit/master/README.md)! - -* [Canva](https://www.canva.com/) From 0bc8a1de45a547df9c28cee345732a4d8ad8cf57 Mon Sep 17 00:00:00 2001 From: Jonathon Belotti Date: Sun, 3 Jan 2021 17:52:30 +1100 Subject: [PATCH 2/2] move documentation about 'whl_requirement' macro out of rules_python_external README into root README. ref: https://github.com/bazelbuild/rules_python/pull/391 --- README.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/README.md b/README.md index e7e3955d59..c90ace1306 100644 --- a/README.md +++ b/README.md @@ -196,6 +196,20 @@ are replaced with `_`. While this naming pattern doesn't change often, it is not guaranted to remain stable, so use of the `requirement()` function is recommended. +### Consuming Wheel Dists Directly + +If you need to depend on the wheel dists themselves, for instance to pass them +to some other packaging tool, you can get a handle to them with the `whl_requirement` macro. For example: + +```python +filegroup( + name = "whl_files", + data = [ + whl_requirement("boto3"), + ] +) +``` + ## Migrating from the bundled rules The core rules are currently available in Bazel as built-in symbols, but this