HBASE-24081 Provide documentation for running Yetus with HBase

dev-support/HOW_TO_YETUS_LOCAL.md

@@ -0,0 +1,166 @@
+# How To Use Apache Yetus with Apache HBase
+
+This document describes how to get up and running with [Apache Yetus][yetus],
+as pertains to the development of Apache HBase. Specifically this covers the use
+of `test-patch`, of which HBase developers make use for tasks related to code
+quality. These are normally run via automation in the foundation's
+[Build][builds] infrastructure. They can also be run locally, which is the
+subject of this document.
+
+The Yetus project provides its own documentation of `test-patch` in
+[Basic PreCommit][yetus-basic-precommit]. By comparison, this document is
+intended to be highly abbreviated, hands-on, and focused on the HBase use-case.
+See that document for more complete explanations and further details.
+
+## Installation
+
+In order to run Yetus, you'll need to first install Yetus and its dependencies.
+This is somewhat simplified when used in Docker mode. Yetus can be retrieved
+from a [distribution artifact][yetus-downloads]. Homebrew/Linuxbrew users can
+install from the tap, the process for which is also described on the downloads
+page.
+
+# Usage Basics
+
+Apache Yetus is comprised of a number of different components. The focus of our
+interest is `test-patch`. `test-patch` is a modular system. Many modules depend
+on some external tool to provide the underlying functionality. For example, the
+`compile` check delegates to a number of provider modules, for example `maven`
+or `gradle` for JVM projects. In order to use these modules, those tools must
+be installed. Yetus calls these modules "plugins".
+
+To see a list of all plugins available to `test-patch`, use
+
+```shell script
+$ test-patch --list-plugins
+```
+
+To see a list of all the options available in both the core modules as well as
+all the plugins, use
+
+```shell script
+$ test-patch --plugins=all --help
+```
+
+An invocation of `test-patch` requires use of one or more plugins. Often times,
+when the full suite of checks are run, specify the meta-plugin "all". Otherwise,
+a limited selection of plugins can be selected using the `--plugins` argument.
+
+## The Workspace, The Patch
+
+`test-patch` operates within a "workspace," a checkout from a source control
+repository. It has a number of options pertaining to this workspace, such as
+the path to the workspace (`--basedir`) and whether it will permit the presence
+of uncommitted changes therein (`--dirty-workspace`).
+
+Onto this workspace, it can optionally apply a change, a.k.a., the "patch" in
+"test patch." The patch can come from a number of sources, including a patch
+file, a JIRA ID, a Pull Request, &c. Or, explicitly inform Yetus that no patch
+file is provided, and the repository should be checked as is, by passing
+`--empty-patch`.
+
+## Personalities
+
+`test-patch` is extremely extensible. Even the functionality of its core
+modules can be extended or overridden. It allows for this type of
+"personalization" by way of "personality" overrides. Yetus ships a number of
+these personalities; a pre-packaged personality can be selected via the
+`--project` parameter. There is a provided HBase personality in Yetus, however
+the HBase project maintains its own within the HBase source repository. Specify
+the path to the personality file using `--personality`. The HBase repository
+places this file under `dev-support/hbase-personality.sh`. 
+
+## Docker mode
+
+Running Yetus in Docker mode simplifies the concerns of dependencies because
+the provided `Dockerfile` handles their installation automatically. However,
+for MacOS users, there are a number of known issues with running Docker on OSX,
+so it may be preferable to instead run outside of the container.
+
+To run in Docker, of course you must install `docker` or some container runtime
+equivalent. [Docker Desktop][docker-desktop] is an option for Mac and Windows
+users.
+
+When running `test-patch` with HBase in docker mode, two parameters must be
+provided. `--docker` activates the `docker` module, enabling the feature.
+Additionally, `--dockerfile` points Yetus at HBase's provided `Dockerfile`,
+instead of using the one provided out-of-the-box by Yetus. The HBase repository
+places this file under `dev-support/docker/Dockerfile`.
+
+## JVM and JAVA_HOME
+
+HBase supports compilation and test execution on a number of different JVM
+versions. To facilitate this, the `Dockerfile` installs multiple JVMs. An
+invocation of `test-patch` requires additional parameters in order to specify
+the target JVM. How you do this is slightly different based on whether you run
+with or without Docker mode.
+
+### Setting `JAVA_HOME` Outside of Docker Mode
+
+Simply specify `JAVA_HOME` in the environment in which `test-patch` is launched.
+
+### Setting `JAVA_HOME` Inside of Docker Mode
+
+The docker image JDKs are available under `/usr/lib/jvm`. Symlinks are provided
+for each supported major version, i.e., `java-11`. Use the `test-patch` argument
+`--java-home`.
+
+# Example: Run SpotBugs
+
+One of the checks supported by Yetus is the static analysis tool
+[SpotBugs][spotbugs]. Let's put together all the above and run SpotBugs with
+`test-patch`.
+
+The SpotBugs check depends on a SpotBugs installation. This is provided by the
+docker file. For running without docker, you'll need to download and unpack the
+SpotBugs binary distribution.
+
+Running the SpotBugs check also depends on [Apache Maven][maven], the build
+system used by HBase. In order for the check to function, both the `maven`
+and `spotbugs` plugins must be specified. If this dependency is omitted, Yetus
+will error with a message similar to
+
+```
+ERROR: you can't specify maven as the buildtool if you don't enable the plugin.
+```
+
+To run just the SpotBugs check, we must explicitly specify `spotbugs` and its
+dependency using `--plugins=maven,spotbugs`.
+
+## Without Docker
+
+Putting it all together, without using docker:
+
+```shell script
+$ cd /path/to/hbase
+$ JAVA_HOME=/path/to/jdk-8 test-patch.sh \
+  --plugins=maven,spotbugs \
+  --spotbugs-home=/path/to/spotbugs/
+  --dirty-workspace \
+  --empty-patch \
+  --personality=./dev-support/hbase-personality.sh
+```
+
+## With Docker
+
+Putting it all together, using docker:
+
+```shell script
+$ cd /path/to/hbase
+$ test-patch.sh \
+  --plugins=maven,spotbugs \
+  --dirty-workspace \
+  --empty-patch \
+  --personality=./dev-support/hbase-personality.sh \
+  --docker \
+  --dockerfile=./dev-support/docker/Dockerfile \
+  --java-home=/usr/lib/jvm/java-8
+```
+
+[builds]: https://builds.apache.org
+[docker-desktop]: https://www.docker.com/products/docker-desktop
+[maven]: https://maven.apache.org
+[spotbugs]: https://spotbugs.github.io
+[yetus]: https://yetus.apache.org
+[yetus-basic-precommit]: https://yetus.apache.org/documentation/0.11.1/precommit-basic/
+[yetus-downloads]: https://yetus.apache.org/downloads/