Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 100 additions & 42 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,23 @@

1. Clone this repo.

2. Copy the Git pre-commit hooks. This will automatically check the build, run
tests, and perform linting before each commit. (Symlinks don't seem to work,
but if you find a way, please add it here!)
2. (OPTIONAL) Copy the Git pre-commit hooks. This will automatically check the build, run
tests, and perform linting before each commit. (Symlinks don't seem to work,
but if you find a way, please add it here!)

```sh
cp .githooks/pre-commit .git/hooks/pre-commit
```
```sh
cp .githooks/pre-commit .git/hooks/pre-commit
```

### Note: You may see this error with the pre-commits due to the monorepo migration:
```
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project gapic-generator-java: Compilation failure: Compilation failure:
[ERROR] gapic-generator-java/gapic-generator-java/src/main/java/com/google/api/generator/gapic/composer/rest/ServiceClientTestClassComposer.java:[19,43] package com.google.api.gax.httpjson.testing does not exist
[ERROR] gapic-generator-java/gapic-generator-java/src/main/java/com/google/api/generator/gapic/composer/rest/ServiceClientTestClassComposer.java:[24,38] package com.google.api.gax.rpc.testing does not exist
[ERROR] gapic-generator-java/gapic-generator-java/src/main/java/com/google/api/generator/gapic/composer/grpc/ServiceClientTestClassComposer.java:[18,39] package com.google.api.gax.grpc.testing does not exist
```
Remove the pre-commit hooks. Tracking the issue in https://github.com/googleapis/gapic-generator-java/issues/1253

3. Install [`bazelisk`](https://github.com/bazelbuild/bazelisk) in your `PATH`.

## Code Formatting
Expand All @@ -32,50 +42,50 @@
- Run all unit and integration tests.

```sh
mvn test # unit tests
mvn install # unit tests, maven test wouldn't work in root folder because gapic-generator-java is dependant on test jars of gax-java
bazel test //... # integration tests
```

- Run all unit tests.

```sh
mvn test
mvn install
```

- Run a single or multiple unit tests:
- For running unit tests in `gapic-generator-java` submodule, first build all modules with `mvn install -DskipTests`, then `cd` into `gapic-generator-java` submodule for the following commands:
- Run a single or multiple unit tests:

```sh
mvn test -Dtest=JavaCodeGeneratorTest

mvn test "-Dtest=Basic*, !%regex[.*.Unstable.*], !%regex[.*.MyTest.class#one.*|two.*], %regex[#fast.*|slow.*]"
```

- Update all unit test golden files:
- Update all unit test golden files:

```sh
mvn test -DupdateUnitGoldens
```
```sh
mvn test -DupdateUnitGoldens
```

- Update a single unit test golden file, for example `JavaCodeGeneratorTest.java`:
- Update a single unit test golden file, for example `JavaCodeGeneratorTest.java`:

```sh
mvn test -DupdateUnitGoldens -Dtest=JavaCodeGeneratorTest
```
```sh
mvn test -DupdateUnitGoldens -Dtest=JavaCodeGeneratorTest
```

- Run a single integration test for API like `Redis`, it generates Java source
code using the Java microgenerator and compares them with the goldens files
in `test/integration/goldens/redis`.

```sh
bazel test //test/integration:redis
```
```sh
bazel test //test/integration:redis
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should all of these bazel commands be using bazelisk instead? (CC @suztomo )

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, use bazelisk where applicable.

```

- Update integration test golden files, for example `Redis`. This clobbers all the
files in `test/integration/goldens/redis`.

```sh
bazel run //test/integration:update_redis
```
```sh
bazel run //test/integration:update_redis
```

## Showcase Integration Testing

Expand Down Expand Up @@ -161,40 +171,67 @@ $ cd showcase
$ mvn compile -P update
```

## Running the Plugin
## Running the Plugin under googleapis with local gapic-generator-java

See also above section "Showcase Integration Testing".
For running the Plugin with showcase protos and local gapic-generator-java, see above section "Showcase Integration Testing".

To generate a production GAPIC API:

1. Clone [googleapis](https://github.com/googleapis/googleapis).

2. Point to local gapic-generator-java
2. Modify `googleapis/WORKSPACE` to point to local gapic-generator-java

Normally, googleapis's build pulls in googleapis/gapic-generator-java from the
Internet:
Normally, googleapis's build pulls in gapic-generator-java from Maven Central.
For a local run, we first need to build a local SNAPSHOT jar of the generator. Then we point googleapis to
both the local SNAPSHOT jar and the local copy of the generator.

Replace the following section in googleapis
```
# Java microgenerator.
_gapic_generator_java_version = "2.1.0"

http_archive(
name = "gapic_generator_java",
urls = ["https://github.com/googleapis/gapic-generator-java/archive/v%s.zip" % _gapic_generator_java_version],
)
_gapic_generator_java_version = "2.13.0"

maven_install(
artifacts = [
"com.google.api:gapic-generator-java:" + _gapic_generator_java_version,
],
#Update this False for local development
fail_on_missing_checksum = True,
repositories = [
"m2Local",
"https://repo.maven.apache.org/maven2/",
]
)

http_archive(
name = "gapic_generator_java",
strip_prefix = "gapic-generator-java-%s" % _gapic_generator_java_version,
urls = ["https://github.com/googleapis/gapic-generator-java/archive/v%s.zip" % _gapic_generator_java_version],
)
```

By replacing this portion using the built-in local_repository rule, you can make
it refer to your local development repo:
to

```
_gapic_generator_java_version = "2.13.1-SNAPSHOT"

maven_install(
artifacts = [
"com.google.api:gapic-generator-java:" + _gapic_generator_java_version,
],
#Update this False for local development
fail_on_missing_checksum = False,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I'm Bazel novice) Can't this True/False be a variable, based on _gapic-generator_java_version has SNAPSHOT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's doable as Bazel scripts are basically a Python variant, but we may want to test non-SNAPSHOT version locally sometimes.

repositories = [
"m2Local",
"https://repo.maven.apache.org/maven2/",
]
)

local_repository(
name = "gapic_generator_java",
path = "/home/<your id>/gapic-generator-java",
name = "gapic_generator_java",
path = "/absolute/path/to/your/local/gapic-generator-java",
)
```

Note: At the time of writing, the gapic-generator version was `2.13.0`. Update the version to the latest version in the pom.xml

3. Build the new target.

Expand All @@ -208,5 +245,26 @@ To generate a production GAPIC API:
bazel build //google/cloud/speech/v2:google-cloud-speech-v2-java
```

Note: If you are running into bazel build issues, you can try to remove gapic-generator-java cached in your local m2
Try running this command:
```
rm -rf ~/.m2/repository/com/google/api/
```
and then rebuild gapic-generator-java (`mvn clean install`).

## FAQ

### Error in workspace: workspace() got unexpected keyword argument 'managed_directories'

Full Error:

```
ERROR: Traceback (most recent call last):
File "/home/alicejli/googleapis/WORKSPACE", line 1, column 10, in <toplevel>
workspace(
Error in workspace: workspace() got unexpected keyword argument 'managed_directories'
ERROR: Error computing the main repository mapping: Encountered error while reading extension file 'tools/build_defs/repo/http.bzl': no such package '@bazel_tools//tools/build_defs/repo': error loading package 'external': Could not load //external package
```

You may be using the latest version of bazel which this project does not support yet. Try installing bazelisk to force
bazel to use the version specified in `.bazeliskrc`