Add dependency pinning for rules_clojure's own Maven dependencies#22
Merged
dancmeyers merged 2 commits intomainfrom Nov 16, 2022
Merged
Add dependency pinning for rules_clojure's own Maven dependencies#22dancmeyers merged 2 commits intomainfrom
rules_clojure's own Maven dependencies#22dancmeyers merged 2 commits intomainfrom
Conversation
dancmeyers
force-pushed
the
infra/zip-pin-deps
branch
8 times, most recently
from
d0db43f to
8bc7233
Compare
dancmeyers
commented
Nov 15, 2022
| RULES_JVM_EXTERNAL_TAG = "4.2" | ||
| RULES_JVM_EXTERNAL_SHA = "cd1a77b7b02e8e008439ca76fd34f5b07aecb8c752961f9640dea15e9e5ba1ca" | ||
| RULES_JVM_EXTERNAL_TAG = "4.5" | ||
| RULES_JVM_EXTERNAL_SHA = "b17d7388feb9bfa7f2fa09031b32707df529f26c91ab9e5d909eb1676badd9a6" |
Contributor
Author
There was a problem hiding this comment.
rules_jvm_external 4.5 is needed to properly support the .netrc stuff we intend to use in banksy.
dancmeyers
commented
Nov 15, 2022
| clojure_library( | ||
| name = "example", | ||
| srcs = ["example.clj"], | ||
| ) |
Contributor
Author
There was a problem hiding this comment.
I've deleted this example, as it seemed outdated. Even before I started changing stuff around, I couldn't find anything in toolchains.bzl that allowed defining a custom toolchain, only the rules_clojure_default_toolchain function, which has now become setup.bzl/rules_clojure_setup.
dancmeyers
changed the title
rules_clojure's own Maven dependencies
Some minor refactors, for consistency: - Add `.cpcache` to `.gitignore`, so you can run `bazel` commands in-repo and not get spurious files. - Rename `BUILD.bazel` files to `BUILD`, as that is the newer pattern, and we had a mixture of both. And now, the main event: We're now following the pattern that the Bazel team themselves use for dependencies of rules, particularly for JVM-based languages making use of `rules_jvm_external`. E.g. this is used in `contrib/rules_jvm`, among others. - We've pinned the dependencies for `rules_clojure` in `frozen_deps_install.json`, using standard options built into `maven_install`. - There's a new `zip_repository` definition, that extracts the files necessary for Bazel to fetch dependencies from a zipfile. - This `zip_repository` is what we reference everywhere when referring to `rules_clojure`'s internal dependencies, so is what is run by users of `rules_clojure` to set it up. - To construct the zipfile, there is a (for now, Python) script under `tools/freeze-deps.py`. Instructions in `README.md` explain how to update deps when necessary. The Python script is pretty simple, and could be changed to a Clojure one in future. For now, while proving the idea, I've kept it as is. Both it and `zip_repository.bzl` were taken from `contrib/rules_jvm`, where they're licensed under Apache 2. One of the devs of `rules_jvm_external` and that (seems to do most of the releases for both) helped me out with all this and pointed me at them.
dancmeyers
force-pushed
the
infra/zip-pin-deps
branch
from
8bc7233 to
24c3055
Compare
dancmeyers
commented
Nov 15, 2022
| load("@rules_clojure//:toolchains.bzl", "rules_clojure_default_toolchain") | ||
| rules_clojure_default_toolchain() | ||
| load("@rules_clojure//:setup.bzl", "rules_clojure_setup") | ||
| rules_clojure_setup() |
Contributor
Author
There was a problem hiding this comment.
This 2 stage 'deps+setup' pattern is apparently very common in Bazel.
Closed
arohner
approved these changes
Nov 16, 2022
WORKSPACE
Outdated
| "org.clojure:java.classpath:1.0.0", | ||
| "org.clojure:tools.namespace:1.1.0", | ||
| "org.clojure:tools.deps.alpha:0.14.1212", | ||
| "org.projectodd.shimdandy:shimdandy-api:1.2.1", |
Contributor
There was a problem hiding this comment.
Shimdandy is no longer required since the rewrite in clojure, but it looks like I failed to remove it from deps.
It's not needed any more.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Some minor refactors, for consistency:
.cpcacheto.gitignore, so you can runbazelcommands in-repo and not get spurious files.BUILD.bazelfiles toBUILD, as that is the newer pattern, and we had a mixture of both.The main event
We're now following the pattern that the Bazel team themselves use for dependencies of rules, particularly for JVM-based languages making use of
rules_jvm_external. E.g. this is used incontrib/rules_jvm, among others.rules_clojureinfrozen_deps_install.json, using standard options built intomaven_install.zip_repositorydefinition, that extracts the files necessary for Bazel to fetch dependencies from a zipfile.zip_repositoryis what we reference everywhere when referring torules_clojure's internal dependencies, so is what is run by users ofrules_clojureto set it up.tools/freeze-deps.py. Instructions inREADME.mdexplain how to update deps when necessary.The Python script is pretty simple, and could be changed to a Clojure one in future. For now, while proving the idea, I've kept it as is. Both it and
zip_repository.bzlwere taken fromcontrib/rules_jvm, where they're licensed under Apache 2. One of the devs ofrules_jvm_externaland that (seems to do most of the releases for both) helped me out with all this and pointed me at them.Why?
By pinning dependencies and making it so that users of
rules_clojurereference the pinned versions, users can make use of standard Bazel options and configuration to switch the mirrors used to fetch files from (https://bazel.build/reference/command-line-reference#flag--experimental_downloader_config), and to configure auth for those new URLs (via.netrc: https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html). This is the recommended way to handle upstream dependencies in Bazel, rather than requiring your users to understand whatever is being used within your rule (Coursier, in the case ofrules_jvm_external, which is why none of our configuration in~/.m2/settings.xmlfor Maven was obeyed. There are also other issues that arise if you use Coursier's own mirror config, as it breaksrules_jvm_external).See https://github.com/griffinbank/banksy/pull/5395 for am example of it 'in action'.