diff --git a/google-cloud-bigquery-jdbc/README.MD b/google-cloud-bigquery-jdbc/README.MD new file mode 100644 index 000000000..12f6c4134 --- /dev/null +++ b/google-cloud-bigquery-jdbc/README.MD @@ -0,0 +1,317 @@ +# Google BigQuery JDBC Client for Java + +Java idiomatic client for [BigQuery JDBC][product-docs]. + +[![Maven][maven-version-image]][maven-version-link] +![Stability][stability-image] + +- [Product Documentation][product-docs] +- [Client Library Documentation][javadocs] + + +## Quickstart + + +If you are using Maven, add this to your pom.xml file: + + + +```xml + + com.google.cloud + google-cloud-bigquery-jdbc + 0.1.0-SNAPSHOT + +``` + +If you are using Gradle without BOM, add this to your dependencies: + +```Groovy +implementation 'com.google.cloud:google-cloud-bigquery-jdbc:0.1.0-SNAPSHOT' +``` + +If you are using SBT, add this to your dependencies: + +```Scala +libraryDependencies += "com.google.cloud" % "google-cloud-bigquery-jdbc" % "0.1.0-SNAPSHOT" +``` + + +## Authentication + +See the [Authentication][authentication] section in the base directory's README. + +## Authorization + +The client application making API calls must be granted [authorization scopes][auth-scopes] required for the desired BigQuery JDBC APIs, and the authenticated principal must have the [IAM role(s)][predefined-iam-roles] required to access GCP resources using the BigQuery JDBC API calls. + +## Developer Guide + +### Prerequisites + +You need to have either Java with Maven installed or Docker. You might want to install [`Make`](https://www.gnu.org/software/make/) to simplify running commands, otherwise please look into Makefile to check for specific configurations. + +### Setup + +`make install` primarily relies on `mvn install` command. All following commands are primarily applicable for the `google-cloud-bigquery-jdbc` project. + +`bigquery-external-jdbc-tests` at this moment is indepdnent and has its own configuration. + +### Running tests + +#### Unittests + +Run all unittests + +`make unittest` + +Run specific unittests + +`make unittest test=` + +Please reference [Maven documentation](https://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html) for details about `` + +Example: `make unittest test=BigQueryArrowStructTest` + +#### Integration tests + +IMPORTANT: Running integration tests will skip unit tests. Primary focus of this command is to run specific set of tests without a lot of overhead. + +Majority of Integration tests rely on certain environment variables: + +``` +# Default gcloud auth setup +export GOOGLE_APPLICATION_CREDENTIALS= + +# Test specfic envs +export SA_EMAIL=email@email.com +export SA_SECRET= +# Alternatively it can be json content: +export SA_SECRET=`cat ` +``` + +Run all integration tests (currently takes 15+ minutes, so this is discouraged). + +`make integration-test` + +Run specific integration test + +`make integration-test test=` + +Please reference [Maven documentation](https://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html) for details about `` + +Example: `make unittest test=ITBigQueryJDBCTest#testValidServiceAccountAuthenticationOAuthPvtKey` + +### Dockerized environment + +If you don't have Java or Maven, or if you want to test changes with a different Java version, you can leverage dockerized environment. + +One-time run commands are similar to local development make commands: + +``` +make docker-build +make docker-unittest +make docker-integration-test +``` + +Please note that running unit or integration tests within docker doesn't leverage maven cache because it is not persisted. +If you want to run multiple commands, you can start a session and treat this shell session as your local environment. + +``` +make docker-session +``` + +All Docker commands rely only on `GOOGLE_APPLICATION_CREDENTIALS` env being present, it will create rest of env vars as needed. + +### Packaging + +There are two ways to package Google JDBC Driver: thin jar and jar with all dependencies. + +#### Thin jar + +Thin jar can be generated by running `make docker-package`. It generates jar and list of dependencies in format compatible with copying into `pom.xml` file. + +#### All dependencies + +Jar with all dependencies can be generated by running `make docker-package-all-dependencies`. Produced jar includes all depenencies and can be used as a standalon jar with tools like R-Studio. + +### Custom query tests + +Thin client using JDBC driver can be found in 'bigquery-jdbc-test-client' folder. It allows you to run custom queries with custom connection string ot generate logs or measure performance. + +It can be built for Google JDBC drivers or any other JDBC drivers. For other JDBC drivers, you need to specify a version and it will download it. + +#### Google test + +For Google, you need to build an uber jar following previous steps. When it is ready, you can use these +sample commands to run it: + +``` +cd bigquery-jdbc-test-client +make docker-package-for-validation JDBC_JAR_PATH= +make run QUERY=small +``` + +### Night builds + +Nightly build is running full set of Integration tests and extended Nightly tests that includes some long-running tests (takes 20+ minutes to complete). + +Nightly Integration Tests include a step to build a full and thin jars and upload them to Google Storage. + +They can be retrieved via following commands: + +``` +gsutil cp gs://bq_devtools_release_private/drivers/jdbc/google-cloud-bigquery-jdbc-latest-full.jar . +gsutil cp gs://bq_devtools_release_private/drivers/jdbc/google-cloud-bigquery-jdbc-latest.zip . +``` + +#### Performance tests + +Cloud Build Pipeline is uploading latest full jar to the internal location for perf tests once a week, at the moment scheduled to do this Sunday night. + +### Code Coverage + +We're using [JaCoCo](https://www.eclemma.org/jacoco/) to track Code Coverage. `Makefile` has 2 separate set of commands for unittests and integration tests reports. + +You can run `make docker-coverage` and it will produce both results, you can find `jacoco-(unittest|full).zip` file in the root with results. + +## Getting Started + +### Prerequisites + +You will need a [Google Cloud Platform Console][developer-console] project with the BigQuery JDBC [API enabled][enable-api]. + +[Follow these instructions][create-project] to get your project set up. You will also need to set up the local development environment by +[installing the Google Cloud Command Line Interface][cloud-cli] and running the following commands in command line: +`gcloud auth login` and `gcloud config set project [YOUR PROJECT ID]`. + +### Installation and setup + +You'll need to obtain the `google-cloud-bigquery-jdbc` library. See the [Quickstart](#quickstart) section +to add `google-cloud-bigquery-jdbc` as a dependency in your code. + +## About BigQuery JDBC + + +[BigQuery JDBC][product-docs] + +See the [BigQuery JDBC client library docs][javadocs] to learn how to +use this BigQuery JDBC Client Library. + + + + + + +## Troubleshooting + +To get help, follow the instructions in the [shared Troubleshooting document][troubleshooting]. + +## Supported Java Versions + +Java 8 or above is required for using this client. + +Google's Java client libraries, +[Google Cloud Client Libraries][cloudlibs] +and +[Google Cloud API Libraries][apilibs], +follow the +[Oracle Java SE support roadmap][oracle] +(see the Oracle Java SE Product Releases section). + +### For new development + +In general, new feature development occurs with support for the lowest Java +LTS version covered by Oracle's Premier Support (which typically lasts 5 years +from initial General Availability). If the minimum required JVM for a given +library is changed, it is accompanied by a [semver][semver] major release. + +Java 11 and (in September 2021) Java 17 are the best choices for new +development. + +### Keeping production systems current + +Google tests its client libraries with all current LTS versions covered by +Oracle's Extended Support (which typically lasts 8 years from initial +General Availability). + +#### Legacy support + +Google's client libraries support legacy versions of Java runtimes with long +term stable libraries that don't receive feature updates on a best efforts basis +as it may not be possible to backport all patches. + +Google provides updates on a best efforts basis to apps that continue to use +Java 7, though apps might need to upgrade to current versions of the library +that supports their JVM. + +#### Where to find specific information + +The latest versions and the supported Java versions are identified on +the individual GitHub repository `github.com/GoogleAPIs/java-SERVICENAME` +and on [google-cloud-java][g-c-j]. + +## Versioning + + +This library follows [Semantic Versioning](http://semver.org/). + + + +## Contributing + + +Contributions to this library are always welcome and highly encouraged. + +See [CONTRIBUTING][contributing] for more information how to get started. + +Please note that this project is released with a Contributor Code of Conduct. By participating in +this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more +information. + + +## License + +Apache 2.0 - See [LICENSE][license] for more information. + +## CI Status + +Java Version | Status +------------ | ------ +Java 8 | [![Kokoro CI][kokoro-badge-image-2]][kokoro-badge-link-2] +Java 8 OSX | [![Kokoro CI][kokoro-badge-image-3]][kokoro-badge-link-3] +Java 8 Windows | [![Kokoro CI][kokoro-badge-image-4]][kokoro-badge-link-4] +Java 11 | [![Kokoro CI][kokoro-badge-image-5]][kokoro-badge-link-5] + +Java is a registered trademark of Oracle and/or its affiliates. + +[product-docs]: https://cloud.google.com/bigquery +[javadocs]: https://cloud.google.com/java/docs/reference/google-cloud-bigquery/latest/history +[kokoro-badge-image-2]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8.svg +[kokoro-badge-link-2]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8.html +[kokoro-badge-image-3]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8-osx.svg +[kokoro-badge-link-3]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8-osx.html +[kokoro-badge-image-4]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8-win.svg +[kokoro-badge-link-4]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java8-win.html +[kokoro-badge-image-5]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java11.svg +[kokoro-badge-link-5]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-bigquery-jdbc/java11.html +[stability-image]: https://img.shields.io/badge/stability-unknown-red +[maven-version-image]: https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-bigquery-jdbc.svg +[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-bigquery-jdbc/0.0.0 +[authentication]: https://github.com/googleapis/google-cloud-java#authentication +[auth-scopes]: https://developers.google.com/identity/protocols/oauth2/scopes +[predefined-iam-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles +[iam-policy]: https://cloud.google.com/iam/docs/overview#cloud-iam-policy +[developer-console]: https://console.developers.google.com/ +[create-project]: https://cloud.google.com/resource-manager/docs/creating-managing-projects +[cloud-cli]: https://cloud.google.com/cli +[troubleshooting]: https://github.com/googleapis/google-cloud-java/blob/main/TROUBLESHOOTING.md +[contributing]: https://github.com/googleapis/java-bigquery/blob/main/CONTRIBUTING.md +[code-of-conduct]: https://github.com/googleapis/java-bigquery/blob/main/CODE_OF_CONDUCT.md +[license]: https://github.com/googleapis/java-bigquery/blob/main/LICENSE + +[semver]: https://semver.org/ +[cloudlibs]: https://cloud.google.com/apis/docs/client-libraries-explained +[apilibs]: https://cloud.google.com/apis/docs/client-libraries-explained#google_api_client_libraries +[oracle]: https://www.oracle.com/java/technologies/java-se-support-roadmap.html +[g-c-j]: http://github.com/googleapis/google-cloud-java \ No newline at end of file