From c7667909d5c76d0e231af16a6bf5ecaee425979e Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 21 Mar 2024 16:40:26 -0400 Subject: [PATCH 01/20] docs: Add Retry Guide to README.md --- README.md | 232 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 232 insertions(+) diff --git a/README.md b/README.md index c31dc4a91291..44447a89b37b 100644 --- a/README.md +++ b/README.md @@ -396,6 +396,238 @@ public CloudTasksClient getService() throws IOException { } ``` +## Retries +Client Libraries use retries to handle unexpected, transient failures (i.e. Server temporary unavailable). +Multiple attempts, hopefully, will result in a successful response from the server. + +By default, retries are configured by the Cloud Service. The configured retry parameters are defined per RPC. +A service *may* choose to only enable retries for a subset of RPCs. It is possible that for a single service, +each RPC is configured differently. + +### Google Cloud Client Library Retry Concepts +Enabling retries allow RPCs multiple attempts to try and achieve a successful call. A successful call +is a response from a server that returns an `OK` Status Code (from gRPC) or a `2xx` Status Code (from HttpJson). + +Retries will occur when either of the following scenarios occur: +- Non-successful status code is received by the library and the code is marked as retryable* +- An RPC invocation exceeds the individual RPC bounds, but still falls within total RPC bounds** + +*The client library will mark a status code as retryable internally. It is marked as retryable if the response's +status code matches with any of an RPC's configure retryable status codes. + +**When configuring the RPC bounds, you may configure the bounds for each attempt as well as the +total RPC's bounds. The retry algorithm will ensure that an individual attempt's bounds falls within +the total RPC's bounds. + +#### Configurable Retry Params +- Retry Status Code: Set of failure Status Codes to retry on +- Retry Time/ Attempt Bounds: Configurable retry settings (via RetrySettings class) to define the retry bounds + +#### Jitter +Jitter is added variance via randomness to spread out when the RPCs are invoked. By default, Google Cloud +Client Libraries enable jitter for retries. When jitter is enabled with exponential backoff, the client libraries +are able to spread out the retry calls without overloading the server. + +#### RetrySettings Configurable Params +- Total Timeout: The overall total limit before the remote call gives up completely. The higher the total timeout, the more retries can be attempted. +- Initial Retry Delay: The delay before the first retry. +- Retry Delay Multiplier: The change in retry delay. This value is multiplied by the previous call’s retry delay to calculate the retry delay for the next call. +- Max Retry Delay: The limit on the value of the retry delay, so that the RetryDelayMultiplier can't increase the retry delay higher than this amount. +- Max Attempts: The maximum number of attempts to perform. If this value is greater than 0, and the number of attempts reaches this limit, the logic will give up retrying even if the total retry time is still lower than TotalTimeout. +- Initial RPC Timeout: The timeout for the initial RPC. +- RPC Timeout Multiplier: The change in RPC timeout. This value is multiplied by the previous call’s timeout to calculate the timeout for the next call. +- Max RPC Timeout: The limit of the RPC timeout, so that the RpcTimeoutMultiplier can't increase the RPC timeout higher than this amount + +See the Official Google Cloud Docs: https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings + +### How to find the Retry Configurations for an RPC +Default retry params are configured inside the client's generated StubSettings' class. + +#### For example, using v3.41.0 of Java-Asset: +Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) + +Example: +```java + definitions.put( + "retry_policy_1_codes", + ImmutableSet.copyOf( + Lists.newArrayList( + StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE))); +``` + +Retry Params are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) + +Example: +```java + settings = + RetrySettings.newBuilder() + .setInitialRpcTimeout(Duration.ofMillis(60000L)) + .setRpcTimeoutMultiplier(1.0) + .setMaxRpcTimeout(Duration.ofMillis(60000L)) + .setTotalTimeout(Duration.ofMillis(60000L)) + .build(); +``` + +The default configurations are mapping [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) + +Example: +```java + builder + .exportAssetsSettings() + .setRetryableCodes(RETRYABLE_CODE_DEFINITIONS.get("no_retry_0_codes")) + .setRetrySettings(RETRY_PARAM_DEFINITIONS.get("no_retry_0_params")); +``` + +### Retry Examples +The following examples below show the behavior of some retry configurations. + +Note: These examples below assume that jitter isn't enabled and that the retry delay is computed +to be the maximum value at the end of each round. With jitter enabled, the actual computed retry +delay values are random. + +#### No Retry +```java +RetrySettings defaultNoRetrySettings = + RetrySettings.newBuilder() + // ... Other configurations do not matter + .setTotalTimeout(Duration.ofMillis(5000L)) + // Explicitly set retries as disabled (maxAttempts == 1) + .setMaxAttempts(1) + .build(); +``` + +The following table shows the attempts: + +| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | +|---------------- |------------- |------------- |-------------- |------------ | +| 1 | 5000ms | 0ms | 0ms | 50000ms | + +#### Retry +##### Example 1 +```java +RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(200L)) + .setRetryDelayMultiplier(2.0) + .setMaxRetryDelay(Duration.ofMillis(500L)) + .setInitialRpcTimeout(Duration.ofMillis(1500L)) + .setRpcTimeoutMultiplier(2.0) + .setMaxRpcTimeout(Duration.ofMillis(3000L)) + .setTotalTimeout(Duration.ofMillis(5000L)) + .build(); +``` + +The following table shows the attempts: + +| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | +|------------------------- |------------- |------------- |-------------- |------------ | +| 1 | 1500ms | 0ms | 0ms | 1500ms | +| 2 (Retry) | 3000ms | 200ms | 1700ms | 4700ms | +| 3 (Retry Not Attempted) | - | 400ms | - | - | + +The third retry attempt is not attempted because the computed retry delay (400ms) would invoke the after +the total timeout (400 + 4700 > 5000). + +##### Example 2 +```java +RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(200L)) + .setRetryDelayMultiplier(2.0) + .setMaxRetryDelay(Duration.ofMillis(500L)) + .setInitialRpcTimeout(Duration.ofMillis(1500L)) + .setRpcTimeoutMultiplier(2.0) + .setMaxRpcTimeout(Duration.ofMillis(3000L)) + .setTotalTimeout(Duration.ofMillis(10000L)) + .build(); +``` + +The following table shows the attempts: + +| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | +|---------------- |------------- |------------- |-------------- |------------ | +| 1 | 1500ms | 0ms | 0ms | 1500ms | +| 2 (Retry) | 3000ms | 200ms | 1700ms | 4700ms | +| 3 (Retry) | 4900ms | 400ms | 5100ms | 10000ms | + +The third retry’s RPC Timeout value is limited due the Total Timeout value. Using the multiplier (2.0) with the +previous timeout value (3000ms) should result in an RPC Timeout of 6000ms. However, the RPC Timeout should not +exceed the Total Timeout and is reduced to be the "time left" (10000 - 5100 = 4900). + +##### Example 3 +```java +RetrySettings defaultRetrySettings = + RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(200L)) + .setRetryDelayMultiplier(2.0) + .setMaxRetryDelay(Duration.ofMillis(500L)) + .setInitialRpcTimeout(Duration.ofMillis(500L)) + .setRpcTimeoutMultiplier(2.0) + .setMaxRpcTimeout(Duration.ofMillis(2000L)) + .setTotalTimeout(Duration.ofMillis(4000L)) + .build(); +``` + +The following table shows the attempts: + +| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | +|---------------- |------------- |------------- |-------------- |------------ | +| 1 | 500ms | 0ms | 0ms | 500ms | +| 2 (Retry) | 1000ms | 200ms | 700ms | 1700ms | +| 3 (Retry) | 1900ms | 400ms | 2100ms | 4000ms | + +Another example where the RPC Timeout is capped to not exceed the total timeout. + +### How to configure a custom retry configuration for an RPC +1. Create the RetrySettings class with your custom configurations +```java +RetrySettings customRetrySettings = + RetrySettings.newBuilder() + // ... Retry Configurations + .build(); +``` +2. Create the StubSettings.Builder for your client and configure it for the RPC +```java +EchoStubSettings.Builder echoStubSettingsBuilder = EchoStubSettings.newBuilder(); +echoStubSettingsBuilder + .blockSettings() + // Set your custom Retry Settings + .setRetrySettings(customRetrySettings) + // Set your custom Retryable Codes + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); +``` + +The snippet above is using a mock service (i.e. EchoClient). The code snippet above is setting +custom retry configurations for EchoClient's Block RPC. It configures the Block RPC to use the +retry settings configured in `customRetrySettings` and sets the retryable codes to be +`DEADLINE_EXCEEDED`. + +3. Create the Settings for the Client +```java +EchoSettings echoSettings = EchoSettings.create(echoSettingsBuilder.build()); +``` +4. Create the Client with the Settings +```java +try (EchoClient echoClient = EchoClient.create(echoSettings)) { + ... +} +``` + +Repeat Step #2 for each RPC that you want to configure. For example: +```java +EchoStubSettings.Builder echoStubSettingsBuilder = EchoStubSettings.newBuilder(); +echoStubSettingsBuilder + .blockSettings() + // Set your custom Retry Settings + .setRetrySettings(customRetrySettings) + // Set your custom Retryable Codes + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); +echoStubSettingsBuilder + .echoSettings() + // Set your custom Retry Settings + .setRetrySettings(customRetrySettings2) + // Set your custom Retryable Codes + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); +``` + ## Long Running Operations Long running operations (LROs) are often used for API calls that are expected to From 787cc9481107c1a78551379eaaa515f574decb84 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 21 Mar 2024 17:06:11 -0400 Subject: [PATCH 02/20] docs: Add info about Jitter --- README.md | 51 ++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 36 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 44447a89b37b..c5e7d0f5bef2 100644 --- a/README.md +++ b/README.md @@ -405,13 +405,16 @@ A service *may* choose to only enable retries for a subset of RPCs. It is possib each RPC is configured differently. ### Google Cloud Client Library Retry Concepts -Enabling retries allow RPCs multiple attempts to try and achieve a successful call. A successful call +Enabling retries allow an RPC multiple attempts to try and achieve a successful call. A successful call is a response from a server that returns an `OK` Status Code (from gRPC) or a `2xx` Status Code (from HttpJson). -Retries will occur when either of the following scenarios occur: -- Non-successful status code is received by the library and the code is marked as retryable* +An RPC will be retried when _both_ of the following scenarios occur: +- Non-successful status code is received by the library and the status code is marked as retryable* - An RPC invocation exceeds the individual RPC bounds, but still falls within total RPC bounds** +Note: If only one (or neither) of the scenarios above are true, then the RPC will not be retried. +i.e. If the total timeout has not been exceeded, but the latest attempt receives a non-retryable status code. + *The client library will mark a status code as retryable internally. It is marked as retryable if the response's status code matches with any of an RPC's configure retryable status codes. @@ -420,30 +423,49 @@ total RPC's bounds. The retry algorithm will ensure that an individual attempt's the total RPC's bounds. #### Configurable Retry Params -- Retry Status Code: Set of failure Status Codes to retry on +- Retry Status Code: Set of failure status codes to retry on - Retry Time/ Attempt Bounds: Configurable retry settings (via RetrySettings class) to define the retry bounds -#### Jitter -Jitter is added variance via randomness to spread out when the RPCs are invoked. By default, Google Cloud -Client Libraries enable jitter for retries. When jitter is enabled with exponential backoff, the client libraries -are able to spread out the retry calls without overloading the server. - #### RetrySettings Configurable Params +- Max Attempts: The maximum number of attempts to perform. If this value is greater than 0, and the number of attempts reaches this limit, the logic will give up retrying even if the total retry time is still lower than TotalTimeout. - Total Timeout: The overall total limit before the remote call gives up completely. The higher the total timeout, the more retries can be attempted. - Initial Retry Delay: The delay before the first retry. - Retry Delay Multiplier: The change in retry delay. This value is multiplied by the previous call’s retry delay to calculate the retry delay for the next call. - Max Retry Delay: The limit on the value of the retry delay, so that the RetryDelayMultiplier can't increase the retry delay higher than this amount. -- Max Attempts: The maximum number of attempts to perform. If this value is greater than 0, and the number of attempts reaches this limit, the logic will give up retrying even if the total retry time is still lower than TotalTimeout. - Initial RPC Timeout: The timeout for the initial RPC. - RPC Timeout Multiplier: The change in RPC timeout. This value is multiplied by the previous call’s timeout to calculate the timeout for the next call. - Max RPC Timeout: The limit of the RPC timeout, so that the RpcTimeoutMultiplier can't increase the RPC timeout higher than this amount -See the Official Google Cloud Docs: https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings +Note: It is recommended to only set _either_ the Max Attempt or the Total Timeout. + +See the official Google Cloud Docs: https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings + +#### Jitter +Jitter is added variance via randomness to spread out when the RPCs are invoked. By default, Google Cloud +Client Libraries enable jitter for retries. When jitter is enabled with exponential backoff, the client libraries +are able to spread out the retry attempts without overwhelming the server. + +The jitter randomness is computed with on the retry delay. On every attempt, the retry algorithm will compute +a random value with the between [1, RETRY_DELAY]. + +For example, with the following retry configurations: +``` +Initial Retry Delay: 100ms +Retry Delay Multiplier: 2.0 +Max Retry Delay: 500ms +``` + +- Attempt 1: Random value between (1, 100] +- Attempt 2: Random value between (1, 200] +- Attempt 3: Random value between (1, 400] +- Attempt 3: Random value between (1, 500] +- ... +- Attempt X: Random value between (1, 500] ### How to find the Retry Configurations for an RPC Default retry params are configured inside the client's generated StubSettings' class. -#### For example, using v3.41.0 of Java-Asset: +#### For example, using Java-Asset v3.41.0: Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) Example: @@ -481,9 +503,8 @@ Example: ### Retry Examples The following examples below show the behavior of some retry configurations. -Note: These examples below assume that jitter isn't enabled and that the retry delay is computed -to be the maximum value at the end of each round. With jitter enabled, the actual computed retry -delay values are random. +Note: These examples below assume that jitter is disabled. The retry delay is computed to be the maximum value +at the end of each attempt. #### No Retry ```java From 95e4731eb560637a392670922f3dec9e2235b789 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 21 Mar 2024 17:10:21 -0400 Subject: [PATCH 03/20] docs: Fix typos --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index c5e7d0f5bef2..facb24e2e5f0 100644 --- a/README.md +++ b/README.md @@ -397,7 +397,7 @@ public CloudTasksClient getService() throws IOException { ``` ## Retries -Client Libraries use retries to handle unexpected, transient failures (i.e. Server temporary unavailable). +Client Libraries use retries to handle unexpected, transient failures (i.e. Server temporarily unavailable). Multiple attempts, hopefully, will result in a successful response from the server. By default, retries are configured by the Cloud Service. The configured retry parameters are defined per RPC. @@ -455,12 +455,12 @@ Retry Delay Multiplier: 2.0 Max Retry Delay: 500ms ``` -- Attempt 1: Random value between (1, 100] -- Attempt 2: Random value between (1, 200] -- Attempt 3: Random value between (1, 400] -- Attempt 3: Random value between (1, 500] +- Attempt 1: Random value between [1, 100] +- Attempt 2: Random value between [1, 200] +- Attempt 3: Random value between [1, 400] +- Attempt 3: Random value between [1, 500] - ... -- Attempt X: Random value between (1, 500] +- Attempt X: Random value between [1, 500] ### How to find the Retry Configurations for an RPC Default retry params are configured inside the client's generated StubSettings' class. From e1b2b6b9f8847c0932061e24dc0e2279c3568053 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 21 Mar 2024 17:43:17 -0400 Subject: [PATCH 04/20] Update README.md Co-authored-by: Tomo Suzuki --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index facb24e2e5f0..96de55525110 100644 --- a/README.md +++ b/README.md @@ -405,7 +405,7 @@ A service *may* choose to only enable retries for a subset of RPCs. It is possib each RPC is configured differently. ### Google Cloud Client Library Retry Concepts -Enabling retries allow an RPC multiple attempts to try and achieve a successful call. A successful call +Enabling retries allows an RPC multiple attempts to try and achieve a successful call. A successful call is a response from a server that returns an `OK` Status Code (from gRPC) or a `2xx` Status Code (from HttpJson). An RPC will be retried when _both_ of the following scenarios occur: From 9e6df27a2e6e0e0361f4c19ab51b445feadcd7d7 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 21 Mar 2024 18:34:40 -0400 Subject: [PATCH 05/20] docs: Address PR comments --- README.md | 313 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 164 insertions(+), 149 deletions(-) diff --git a/README.md b/README.md index 96de55525110..4f928e1291d8 100644 --- a/README.md +++ b/README.md @@ -241,7 +241,7 @@ In cases where the library may expect a project ID explicitly, we provide a help ```java import com.google.cloud.ServiceOptions; ... - String projectId = ServiceOptions.getDefaultProjectId(); + String projectId = ServiceOptions.getDefaultProjectId(); ``` ## Authentication @@ -304,20 +304,20 @@ access token will not be automatically refreshed): ```java Credentials credentials = GoogleCredentials.create(new AccessToken(accessToken, expirationTime)); -Storage storage = StorageOptions.newBuilder() - .setCredentials(credentials) - .build() - .getService(); + Storage storage = StorageOptions.newBuilder() + .setCredentials(credentials) + .build() + .getService(); ``` or: ```java Credentials credentials = GoogleCredentials.create(new AccessToken(accessToken, expirationTime)); -CloudTasksSettings cloudTasksSettings = CloudTasksSettings.newBuilder() - .setCredentialProvider(FixedCredentialsProvider.create(credentials)) - .build(); -CloudTasksClient cloudTasksClient = CloudTasksClient.create(cloudTasksSettings); + CloudTasksSettings cloudTasksSettings = CloudTasksSettings.newBuilder() + .setCredentialProvider(FixedCredentialsProvider.create(credentials)) + .build(); + CloudTasksClient cloudTasksClient = CloudTasksClient.create(cloudTasksSettings); ``` ### Application Default Credentials @@ -365,49 +365,115 @@ import java.net.InetSocketAddress; import java.net.SocketAddress; public CloudTasksClient getService() throws IOException { - TransportChannelProvider transportChannelProvider = - CloudTasksStubSettings.defaultGrpcTransportProviderBuilder() - .setChannelConfigurator( - new ApiFunction() { - @Override - public ManagedChannelBuilder apply(ManagedChannelBuilder managedChannelBuilder) { - return managedChannelBuilder.proxyDetector( - new ProxyDetector() { - @Nullable - @Override - public ProxiedSocketAddress proxyFor(SocketAddress socketAddress) - throws IOException { - return HttpConnectProxiedSocketAddress.newBuilder() - .setUsername(PROXY_USERNAME) - .setPassword(PROXY_PASSWORD) - .setProxyAddress(new InetSocketAddress(PROXY_HOST, PROXY_PORT)) - .setTargetAddress((InetSocketAddress) socketAddress) - .build(); - } - }); - } - }) - .build(); - CloudTasksSettings cloudTasksSettings = - CloudTasksSettings.newBuilder() - .setTransportChannelProvider(transportChannelProvider) - .build(); - return CloudTasksClient.create(cloudTasksSettings); -} + TransportChannelProvider transportChannelProvider = + CloudTasksStubSettings.defaultGrpcTransportProviderBuilder() + .setChannelConfigurator( + new ApiFunction() { +@Override +public ManagedChannelBuilder apply(ManagedChannelBuilder managedChannelBuilder) { + return managedChannelBuilder.proxyDetector( + new ProxyDetector() { +@Nullable +@Override +public ProxiedSocketAddress proxyFor(SocketAddress socketAddress) + throws IOException { + return HttpConnectProxiedSocketAddress.newBuilder() + .setUsername(PROXY_USERNAME) + .setPassword(PROXY_PASSWORD) + .setProxyAddress(new InetSocketAddress(PROXY_HOST, PROXY_PORT)) + .setTargetAddress((InetSocketAddress) socketAddress) + .build(); + } + }); + } + }) + .build(); + CloudTasksSettings cloudTasksSettings = + CloudTasksSettings.newBuilder() + .setTransportChannelProvider(transportChannelProvider) + .build(); + return CloudTasksClient.create(cloudTasksSettings); + } ``` ## Retries -Client Libraries use retries to handle unexpected, transient failures (i.e. Server temporarily unavailable). +Client Libraries use retries to handle unexpected, transient failures (i.e. server temporarily unavailable). Multiple attempts, hopefully, will result in a successful response from the server. By default, retries are configured by the Cloud Service. The configured retry parameters are defined per RPC. -A service *may* choose to only enable retries for a subset of RPCs. It is possible that for a single service, +A service *may* choose to only enable retries for a subset of RPCs. It is possible that for a single service, each RPC is configured differently. +For example, using Java-Asset v3.41.0, the default configurations: + +Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) + +Example: +```java + definitions.put( + "retry_policy_1_codes", + ImmutableSet.copyOf( + Lists.newArrayList( + StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE))); +``` + +Retry Params are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) + +Example: +```java + settings = + RetrySettings.newBuilder() + .setInitialRpcTimeout(Duration.ofMillis(60000L)) + .setRpcTimeoutMultiplier(1.0) + .setMaxRpcTimeout(Duration.ofMillis(60000L)) + .setTotalTimeout(Duration.ofMillis(60000L)) + .build(); +``` + +The default configurations are mapping [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) + +Example: +```java + builder + .exportAssetsSettings() + .setRetryableCodes(RETRYABLE_CODE_DEFINITIONS.get("no_retry_0_codes")) + .setRetrySettings(RETRY_PARAM_DEFINITIONS.get("no_retry_0_params")); +``` + ### Google Cloud Client Library Retry Concepts -Enabling retries allows an RPC multiple attempts to try and achieve a successful call. A successful call +Enabling retries allow an RPC multiple attempts to try and achieve a successful call. A successful call is a response from a server that returns an `OK` Status Code (from gRPC) or a `2xx` Status Code (from HttpJson). +#### Configurable Retry Params +- Retry Status Code: Set of failure status codes to retry on +- Retry Time/ Attempt Bounds: Configurable retry settings (via [RetrySettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings) class) to define the retry bounds + +#### When is an RPC retried +Take a sample RetrySettings configuration +```java + settings = + RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(100L)) + .setRetryDelayMultiplier(1.3) + .setMaxRetryDelay(Duration.ofMillis(60000L)) + .setInitialRpcTimeout(Duration.ofMillis(60000L)) + .setRpcTimeoutMultiplier(1.0) + .setMaxRpcTimeout(Duration.ofMillis(60000L)) + .setTotalTimeout(Duration.ofMillis(60000L)) + .build(); +``` +Individual RPC Bounds (an attempt) are controlled by the following settings: +- setInitialRetryDelay +- setRetryDelayMultiplier +- setMaxRetryDelay +- setInitialRpcTimeout +- setRpcTimeoutMultiplier +- setMaxRpcTimeout + +Total RPC Bounds are controlled by the following settings: +- setTotalTimeout +- setAttemptCount + An RPC will be retried when _both_ of the following scenarios occur: - Non-successful status code is received by the library and the status code is marked as retryable* - An RPC invocation exceeds the individual RPC bounds, but still falls within total RPC bounds** @@ -422,24 +488,6 @@ status code matches with any of an RPC's configure retryable status codes. total RPC's bounds. The retry algorithm will ensure that an individual attempt's bounds falls within the total RPC's bounds. -#### Configurable Retry Params -- Retry Status Code: Set of failure status codes to retry on -- Retry Time/ Attempt Bounds: Configurable retry settings (via RetrySettings class) to define the retry bounds - -#### RetrySettings Configurable Params -- Max Attempts: The maximum number of attempts to perform. If this value is greater than 0, and the number of attempts reaches this limit, the logic will give up retrying even if the total retry time is still lower than TotalTimeout. -- Total Timeout: The overall total limit before the remote call gives up completely. The higher the total timeout, the more retries can be attempted. -- Initial Retry Delay: The delay before the first retry. -- Retry Delay Multiplier: The change in retry delay. This value is multiplied by the previous call’s retry delay to calculate the retry delay for the next call. -- Max Retry Delay: The limit on the value of the retry delay, so that the RetryDelayMultiplier can't increase the retry delay higher than this amount. -- Initial RPC Timeout: The timeout for the initial RPC. -- RPC Timeout Multiplier: The change in RPC timeout. This value is multiplied by the previous call’s timeout to calculate the timeout for the next call. -- Max RPC Timeout: The limit of the RPC timeout, so that the RpcTimeoutMultiplier can't increase the RPC timeout higher than this amount - -Note: It is recommended to only set _either_ the Max Attempt or the Total Timeout. - -See the official Google Cloud Docs: https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings - #### Jitter Jitter is added variance via randomness to spread out when the RPCs are invoked. By default, Google Cloud Client Libraries enable jitter for retries. When jitter is enabled with exponential backoff, the client libraries @@ -465,41 +513,6 @@ Max Retry Delay: 500ms ### How to find the Retry Configurations for an RPC Default retry params are configured inside the client's generated StubSettings' class. -#### For example, using Java-Asset v3.41.0: -Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) - -Example: -```java - definitions.put( - "retry_policy_1_codes", - ImmutableSet.copyOf( - Lists.newArrayList( - StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE))); -``` - -Retry Params are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) - -Example: -```java - settings = - RetrySettings.newBuilder() - .setInitialRpcTimeout(Duration.ofMillis(60000L)) - .setRpcTimeoutMultiplier(1.0) - .setMaxRpcTimeout(Duration.ofMillis(60000L)) - .setTotalTimeout(Duration.ofMillis(60000L)) - .build(); -``` - -The default configurations are mapping [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) - -Example: -```java - builder - .exportAssetsSettings() - .setRetryableCodes(RETRYABLE_CODE_DEFINITIONS.get("no_retry_0_codes")) - .setRetrySettings(RETRY_PARAM_DEFINITIONS.get("no_retry_0_params")); -``` - ### Retry Examples The following examples below show the behavior of some retry configurations. @@ -520,8 +533,8 @@ RetrySettings defaultNoRetrySettings = The following table shows the attempts: | Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | -|---------------- |------------- |------------- |-------------- |------------ | -| 1 | 5000ms | 0ms | 0ms | 50000ms | +|---------------- |------------- |------------- |-------------- |----------- | +| 1 | 5000ms | 0ms | 0ms | 5000ms | #### Retry ##### Example 1 @@ -549,6 +562,8 @@ The third retry attempt is not attempted because the computed retry delay (400ms the total timeout (400 + 4700 > 5000). ##### Example 2 +This example is similar to Example #1, but has a longer total timeout to showcase an additional +retry attempt and the capped RPC Timeout for the last retry attempt. ```java RetrySettings.newBuilder() .setInitialRetryDelay(Duration.ofMillis(200L)) @@ -570,7 +585,7 @@ The following table shows the attempts: | 3 (Retry) | 4900ms | 400ms | 5100ms | 10000ms | The third retry’s RPC Timeout value is limited due the Total Timeout value. Using the multiplier (2.0) with the -previous timeout value (3000ms) should result in an RPC Timeout of 6000ms. However, the RPC Timeout should not +previous timeout value (3000ms) should result in an RPC Timeout of 6000ms. However, the RPC Timeout should not exceed the Total Timeout and is reduced to be the "time left" (10000 - 5100 = 4900). ##### Example 3 @@ -607,9 +622,9 @@ RetrySettings customRetrySettings = ``` 2. Create the StubSettings.Builder for your client and configure it for the RPC ```java -EchoStubSettings.Builder echoStubSettingsBuilder = EchoStubSettings.newBuilder(); -echoStubSettingsBuilder - .blockSettings() +AssetServiceStubSettings.Builder assetStubSettingsBuilder = AssetServiceStubSettings.newBuilder(); +assetStubSettingsBuilder + .exportAssetsSettings() // Set your custom Retry Settings .setRetrySettings(customRetrySettings) // Set your custom Retryable Codes @@ -623,30 +638,30 @@ retry settings configured in `customRetrySettings` and sets the retryable codes 3. Create the Settings for the Client ```java -EchoSettings echoSettings = EchoSettings.create(echoSettingsBuilder.build()); +AssetServiceSettings assetSettings = AssetServiceSettings.create(assetStubSettingsBuilder.build()); ``` 4. Create the Client with the Settings ```java -try (EchoClient echoClient = EchoClient.create(echoSettings)) { +try (AssetServiceClient assetClient = AssetServiceClient.create(assetSettings)) { ... } ``` Repeat Step #2 for each RPC that you want to configure. For example: ```java -EchoStubSettings.Builder echoStubSettingsBuilder = EchoStubSettings.newBuilder(); -echoStubSettingsBuilder - .blockSettings() - // Set your custom Retry Settings - .setRetrySettings(customRetrySettings) - // Set your custom Retryable Codes - .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); -echoStubSettingsBuilder - .echoSettings() - // Set your custom Retry Settings - .setRetrySettings(customRetrySettings2) - // Set your custom Retryable Codes - .setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); +AssetServiceStubSettings.Builder assetStubSettingsBuilder = AssetServiceStubSettings.newBuilder(); + assetStubSettingsBuilder + .exportAssetsSettings() + // Set your custom Retry Settings + .setRetrySettings(customRetrySettings) + // Set your custom Retryable Codes + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); + assetStubSettingsBuilder + .listAssetsSettings() + // Set your custom Retry Settings + .setRetrySettings(customRetrySettings2) + // Set your custom Retryable Codes + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); ``` ## Long Running Operations @@ -666,23 +681,23 @@ check the operation's status. For example, take a sample `createCluster` Operation in google-cloud-dataproc v4.20.0: ```java try (ClusterControllerClient clusterControllerClient = ClusterControllerClient.create()) { - CreateClusterRequest request = - CreateClusterRequest.newBuilder() - .setProjectId("{PROJECT_ID}") - .setRegion("{REGION}") - .setCluster(Cluster.newBuilder().build()) - .setRequestId("{REQUEST_ID}") - .setActionOnFailedPrimaryWorkers(FailureAction.forNumber(0)) - .build(); - OperationFuture future = - clusterControllerClient.createClusterOperationCallable().futureCall(request); - // Do something. - Cluster response = future.get(); -} catch (CancellationException e) { - // Exceeded the default RPC timeout without the Operation completing. - // Library is no longer polling for the Operation status. Consider - // increasing the timeout. -} + CreateClusterRequest request = + CreateClusterRequest.newBuilder() + .setProjectId("{PROJECT_ID}") + .setRegion("{REGION}") + .setCluster(Cluster.newBuilder().build()) + .setRequestId("{REQUEST_ID}") + .setActionOnFailedPrimaryWorkers(FailureAction.forNumber(0)) + .build(); + OperationFuture future = + clusterControllerClient.createClusterOperationCallable().futureCall(request); + // Do something. + Cluster response = future.get(); + } catch (CancellationException e) { + // Exceeded the default RPC timeout without the Operation completing. + // Library is no longer polling for the Operation status. Consider + // increasing the timeout. + } ``` ### LRO Timeouts @@ -697,15 +712,15 @@ Note: The client library handles the Operation's polling mechanism for you. By d to manually poll the status yourself. ### Default LRO Values -Each LRO RPC has a pre-configured default values. You can find these values by +Each LRO RPC has a pre-configured default values. You can find these values by searching in each Client's `StubSettings`'s class. The default LRO settings are initialized inside the `initDefaults()` method in the nested Builder class. -For example, in google-cloud-aiplatform v3.24.0, the default [OperationTimedPollAlgorithm](https://github.com/googleapis/google-cloud-java/blob/9ae786d1acdc7354adf86b78691570668caa293d/java-aiplatform/google-cloud-aiplatform/src/main/java/com/google/cloud/aiplatform/v1/stub/EndpointServiceStubSettings.java#L755-L765) +For example, in google-cloud-aiplatform v3.24.0, the default [OperationTimedPollAlgorithm](https://github.com/googleapis/google-cloud-java/blob/9ae786d1acdc7354adf86b78691570668caa293d/java-aiplatform/google-cloud-aiplatform/src/main/java/com/google/cloud/aiplatform/v1/stub/EndpointServiceStubSettings.java#L755-L765) has these default values: ```java OperationTimedPollAlgorithm.create( - RetrySettings.newBuilder() + RetrySettings.newBuilder() .setInitialRetryDelay(Duration.ofMillis(5000L)) .setRetryDelayMultiplier(1.5) .setMaxRetryDelay(Duration.ofMillis(45000L)) @@ -729,23 +744,23 @@ To configure the LRO values, create an OperationTimedPollAlgorithm object and up RPC's polling algorithm. For example: ```java ClusterControllerSettings.Builder settingsBuilder = ClusterControllerSettings.newBuilder(); -TimedRetryAlgorithm timedRetryAlgorithm = OperationTimedPollAlgorithm.create( - RetrySettings.newBuilder() - .setInitialRetryDelay(Duration.ofMillis(500L)) - .setRetryDelayMultiplier(1.5) - .setMaxRetryDelay(Duration.ofMillis(5000L)) - .setInitialRpcTimeout(Duration.ZERO) // ignored - .setRpcTimeoutMultiplier(1.0) // ignored - .setMaxRpcTimeout(Duration.ZERO) // ignored - .setTotalTimeout(Duration.ofHours(24L)) // set polling timeout to 24 hours - .build()); -settingsBuilder.createClusterOperationSettings() - .setPollingAlgorithm(timedRetryAlgorithm); -ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settingsBuilder.build()); + TimedRetryAlgorithm timedRetryAlgorithm = OperationTimedPollAlgorithm.create( + RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(500L)) + .setRetryDelayMultiplier(1.5) + .setMaxRetryDelay(Duration.ofMillis(5000L)) + .setInitialRpcTimeout(Duration.ZERO) // ignored + .setRpcTimeoutMultiplier(1.0) // ignored + .setMaxRpcTimeout(Duration.ZERO) // ignored + .setTotalTimeout(Duration.ofHours(24L)) // set polling timeout to 24 hours + .build()); + settingsBuilder.createClusterOperationSettings() + .setPollingAlgorithm(timedRetryAlgorithm); + ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settingsBuilder.build()); ``` Note: The configuration above *only* modifies the LRO values for the `createClusterOperation` RPC. -The other RPCs in the Client will still use each RPC's pre-configured LRO values. +The other RPCs in the Client will still use each RPC's pre-configured LRO values. ## Managing Dependencies From 06bdaf7e9851636f1b16fce7a8f76a0e5b01756a Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 21 Mar 2024 18:49:47 -0400 Subject: [PATCH 06/20] docs: Move Retry Guide md file to docs/ folder --- README.md | 268 ------------------------------------------------ docs/retries.md | 267 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 267 insertions(+), 268 deletions(-) create mode 100644 docs/retries.md diff --git a/README.md b/README.md index 4f928e1291d8..a384fe252c20 100644 --- a/README.md +++ b/README.md @@ -396,274 +396,6 @@ public ProxiedSocketAddress proxyFor(SocketAddress socketAddress) } ``` -## Retries -Client Libraries use retries to handle unexpected, transient failures (i.e. server temporarily unavailable). -Multiple attempts, hopefully, will result in a successful response from the server. - -By default, retries are configured by the Cloud Service. The configured retry parameters are defined per RPC. -A service *may* choose to only enable retries for a subset of RPCs. It is possible that for a single service, -each RPC is configured differently. - -For example, using Java-Asset v3.41.0, the default configurations: - -Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) - -Example: -```java - definitions.put( - "retry_policy_1_codes", - ImmutableSet.copyOf( - Lists.newArrayList( - StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE))); -``` - -Retry Params are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) - -Example: -```java - settings = - RetrySettings.newBuilder() - .setInitialRpcTimeout(Duration.ofMillis(60000L)) - .setRpcTimeoutMultiplier(1.0) - .setMaxRpcTimeout(Duration.ofMillis(60000L)) - .setTotalTimeout(Duration.ofMillis(60000L)) - .build(); -``` - -The default configurations are mapping [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) - -Example: -```java - builder - .exportAssetsSettings() - .setRetryableCodes(RETRYABLE_CODE_DEFINITIONS.get("no_retry_0_codes")) - .setRetrySettings(RETRY_PARAM_DEFINITIONS.get("no_retry_0_params")); -``` - -### Google Cloud Client Library Retry Concepts -Enabling retries allow an RPC multiple attempts to try and achieve a successful call. A successful call -is a response from a server that returns an `OK` Status Code (from gRPC) or a `2xx` Status Code (from HttpJson). - -#### Configurable Retry Params -- Retry Status Code: Set of failure status codes to retry on -- Retry Time/ Attempt Bounds: Configurable retry settings (via [RetrySettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings) class) to define the retry bounds - -#### When is an RPC retried -Take a sample RetrySettings configuration -```java - settings = - RetrySettings.newBuilder() - .setInitialRetryDelay(Duration.ofMillis(100L)) - .setRetryDelayMultiplier(1.3) - .setMaxRetryDelay(Duration.ofMillis(60000L)) - .setInitialRpcTimeout(Duration.ofMillis(60000L)) - .setRpcTimeoutMultiplier(1.0) - .setMaxRpcTimeout(Duration.ofMillis(60000L)) - .setTotalTimeout(Duration.ofMillis(60000L)) - .build(); -``` -Individual RPC Bounds (an attempt) are controlled by the following settings: -- setInitialRetryDelay -- setRetryDelayMultiplier -- setMaxRetryDelay -- setInitialRpcTimeout -- setRpcTimeoutMultiplier -- setMaxRpcTimeout - -Total RPC Bounds are controlled by the following settings: -- setTotalTimeout -- setAttemptCount - -An RPC will be retried when _both_ of the following scenarios occur: -- Non-successful status code is received by the library and the status code is marked as retryable* -- An RPC invocation exceeds the individual RPC bounds, but still falls within total RPC bounds** - -Note: If only one (or neither) of the scenarios above are true, then the RPC will not be retried. -i.e. If the total timeout has not been exceeded, but the latest attempt receives a non-retryable status code. - -*The client library will mark a status code as retryable internally. It is marked as retryable if the response's -status code matches with any of an RPC's configure retryable status codes. - -**When configuring the RPC bounds, you may configure the bounds for each attempt as well as the -total RPC's bounds. The retry algorithm will ensure that an individual attempt's bounds falls within -the total RPC's bounds. - -#### Jitter -Jitter is added variance via randomness to spread out when the RPCs are invoked. By default, Google Cloud -Client Libraries enable jitter for retries. When jitter is enabled with exponential backoff, the client libraries -are able to spread out the retry attempts without overwhelming the server. - -The jitter randomness is computed with on the retry delay. On every attempt, the retry algorithm will compute -a random value with the between [1, RETRY_DELAY]. - -For example, with the following retry configurations: -``` -Initial Retry Delay: 100ms -Retry Delay Multiplier: 2.0 -Max Retry Delay: 500ms -``` - -- Attempt 1: Random value between [1, 100] -- Attempt 2: Random value between [1, 200] -- Attempt 3: Random value between [1, 400] -- Attempt 3: Random value between [1, 500] -- ... -- Attempt X: Random value between [1, 500] - -### How to find the Retry Configurations for an RPC -Default retry params are configured inside the client's generated StubSettings' class. - -### Retry Examples -The following examples below show the behavior of some retry configurations. - -Note: These examples below assume that jitter is disabled. The retry delay is computed to be the maximum value -at the end of each attempt. - -#### No Retry -```java -RetrySettings defaultNoRetrySettings = - RetrySettings.newBuilder() - // ... Other configurations do not matter - .setTotalTimeout(Duration.ofMillis(5000L)) - // Explicitly set retries as disabled (maxAttempts == 1) - .setMaxAttempts(1) - .build(); -``` - -The following table shows the attempts: - -| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | -|---------------- |------------- |------------- |-------------- |----------- | -| 1 | 5000ms | 0ms | 0ms | 5000ms | - -#### Retry -##### Example 1 -```java -RetrySettings.newBuilder() - .setInitialRetryDelay(Duration.ofMillis(200L)) - .setRetryDelayMultiplier(2.0) - .setMaxRetryDelay(Duration.ofMillis(500L)) - .setInitialRpcTimeout(Duration.ofMillis(1500L)) - .setRpcTimeoutMultiplier(2.0) - .setMaxRpcTimeout(Duration.ofMillis(3000L)) - .setTotalTimeout(Duration.ofMillis(5000L)) - .build(); -``` - -The following table shows the attempts: - -| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | -|------------------------- |------------- |------------- |-------------- |------------ | -| 1 | 1500ms | 0ms | 0ms | 1500ms | -| 2 (Retry) | 3000ms | 200ms | 1700ms | 4700ms | -| 3 (Retry Not Attempted) | - | 400ms | - | - | - -The third retry attempt is not attempted because the computed retry delay (400ms) would invoke the after -the total timeout (400 + 4700 > 5000). - -##### Example 2 -This example is similar to Example #1, but has a longer total timeout to showcase an additional -retry attempt and the capped RPC Timeout for the last retry attempt. -```java -RetrySettings.newBuilder() - .setInitialRetryDelay(Duration.ofMillis(200L)) - .setRetryDelayMultiplier(2.0) - .setMaxRetryDelay(Duration.ofMillis(500L)) - .setInitialRpcTimeout(Duration.ofMillis(1500L)) - .setRpcTimeoutMultiplier(2.0) - .setMaxRpcTimeout(Duration.ofMillis(3000L)) - .setTotalTimeout(Duration.ofMillis(10000L)) - .build(); -``` - -The following table shows the attempts: - -| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | -|---------------- |------------- |------------- |-------------- |------------ | -| 1 | 1500ms | 0ms | 0ms | 1500ms | -| 2 (Retry) | 3000ms | 200ms | 1700ms | 4700ms | -| 3 (Retry) | 4900ms | 400ms | 5100ms | 10000ms | - -The third retry’s RPC Timeout value is limited due the Total Timeout value. Using the multiplier (2.0) with the -previous timeout value (3000ms) should result in an RPC Timeout of 6000ms. However, the RPC Timeout should not -exceed the Total Timeout and is reduced to be the "time left" (10000 - 5100 = 4900). - -##### Example 3 -```java -RetrySettings defaultRetrySettings = - RetrySettings.newBuilder() - .setInitialRetryDelay(Duration.ofMillis(200L)) - .setRetryDelayMultiplier(2.0) - .setMaxRetryDelay(Duration.ofMillis(500L)) - .setInitialRpcTimeout(Duration.ofMillis(500L)) - .setRpcTimeoutMultiplier(2.0) - .setMaxRpcTimeout(Duration.ofMillis(2000L)) - .setTotalTimeout(Duration.ofMillis(4000L)) - .build(); -``` - -The following table shows the attempts: - -| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | -|---------------- |------------- |------------- |-------------- |------------ | -| 1 | 500ms | 0ms | 0ms | 500ms | -| 2 (Retry) | 1000ms | 200ms | 700ms | 1700ms | -| 3 (Retry) | 1900ms | 400ms | 2100ms | 4000ms | - -Another example where the RPC Timeout is capped to not exceed the total timeout. - -### How to configure a custom retry configuration for an RPC -1. Create the RetrySettings class with your custom configurations -```java -RetrySettings customRetrySettings = - RetrySettings.newBuilder() - // ... Retry Configurations - .build(); -``` -2. Create the StubSettings.Builder for your client and configure it for the RPC -```java -AssetServiceStubSettings.Builder assetStubSettingsBuilder = AssetServiceStubSettings.newBuilder(); -assetStubSettingsBuilder - .exportAssetsSettings() - // Set your custom Retry Settings - .setRetrySettings(customRetrySettings) - // Set your custom Retryable Codes - .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); -``` - -The snippet above is using a mock service (i.e. EchoClient). The code snippet above is setting -custom retry configurations for EchoClient's Block RPC. It configures the Block RPC to use the -retry settings configured in `customRetrySettings` and sets the retryable codes to be -`DEADLINE_EXCEEDED`. - -3. Create the Settings for the Client -```java -AssetServiceSettings assetSettings = AssetServiceSettings.create(assetStubSettingsBuilder.build()); -``` -4. Create the Client with the Settings -```java -try (AssetServiceClient assetClient = AssetServiceClient.create(assetSettings)) { - ... -} -``` - -Repeat Step #2 for each RPC that you want to configure. For example: -```java -AssetServiceStubSettings.Builder assetStubSettingsBuilder = AssetServiceStubSettings.newBuilder(); - assetStubSettingsBuilder - .exportAssetsSettings() - // Set your custom Retry Settings - .setRetrySettings(customRetrySettings) - // Set your custom Retryable Codes - .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); - assetStubSettingsBuilder - .listAssetsSettings() - // Set your custom Retry Settings - .setRetrySettings(customRetrySettings2) - // Set your custom Retryable Codes - .setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); -``` - ## Long Running Operations Long running operations (LROs) are often used for API calls that are expected to diff --git a/docs/retries.md b/docs/retries.md new file mode 100644 index 000000000000..ca548f9dd141 --- /dev/null +++ b/docs/retries.md @@ -0,0 +1,267 @@ +# Retries +Client Libraries use retries to handle unexpected, transient failures (i.e. server temporarily unavailable). +Multiple attempts, hopefully, will result in a successful response from the server. + +By default, retries are configured by the Cloud Service. The configured retry parameters are defined per RPC. +A service *may* choose to only enable retries for a subset of RPCs. It is possible that for a single service, +each RPC is configured differently. + +For example, using Java-Asset v3.41.0, the default configurations are configured in the following files: + +Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) + +Example: +```java + definitions.put( + "retry_policy_1_codes", + ImmutableSet.copyOf( + Lists.newArrayList( + StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE))); +``` + +Retry Params are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) + +Example: +```java + settings = + RetrySettings.newBuilder() + .setInitialRpcTimeout(Duration.ofMillis(60000L)) + .setRpcTimeoutMultiplier(1.0) + .setMaxRpcTimeout(Duration.ofMillis(60000L)) + .setTotalTimeout(Duration.ofMillis(60000L)) + .build(); +``` + +The default configurations are mapping [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) + +Example: +```java + builder + .exportAssetsSettings() + .setRetryableCodes(RETRYABLE_CODE_DEFINITIONS.get("no_retry_0_codes")) + .setRetrySettings(RETRY_PARAM_DEFINITIONS.get("no_retry_0_params")); +``` + +## Google Cloud Client Library Retry Concepts +Enabling retries allow an RPC multiple attempts to try and achieve a successful call. A successful call +is a response from a server that returns an `OK` Status Code (from gRPC) or a `2xx` Status Code (from HttpJson). + +### Configurable Retry Params +- Retry Status Code: Set of failure status codes to retry on +- Retry Time/ Attempt Bounds: Configurable retry settings (via [RetrySettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings) class) to define the retry bounds + +### When is an RPC retried +Take a sample RetrySettings configuration +```java + settings = + RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(100L)) + .setRetryDelayMultiplier(1.3) + .setMaxRetryDelay(Duration.ofMillis(60000L)) + .setInitialRpcTimeout(Duration.ofMillis(60000L)) + .setRpcTimeoutMultiplier(1.0) + .setMaxRpcTimeout(Duration.ofMillis(60000L)) + .setTotalTimeout(Duration.ofMillis(60000L)) + .build(); +``` +Individual RPC Bounds (an attempt) are controlled by the following settings: +- setInitialRetryDelay +- setRetryDelayMultiplier +- setMaxRetryDelay +- setInitialRpcTimeout +- setRpcTimeoutMultiplier +- setMaxRpcTimeout + +Total RPC Bounds are controlled by the following settings: +- setTotalTimeout +- setAttemptCount + +An RPC will be retried when _both_ of the following scenarios occur: +- Non-successful status code is received by the library and the status code is marked as retryable* +- An RPC invocation exceeds the individual RPC bounds, but still falls within total RPC bounds** + +Note: If only one (or neither) of the scenarios above are true, then the RPC will not be retried. +i.e. If the total timeout has not been exceeded, but the latest attempt receives a non-retryable status code. + +*The client library will mark a status code as retryable internally. It is marked as retryable if the response's +status code matches with any of an RPC's configure retryable status codes. + +**When configuring the RPC bounds, you may configure the bounds for each attempt as well as the +total RPC's bounds. The retry algorithm will ensure that an individual attempt's bounds falls within +the total RPC's bounds. + +### Jitter +Jitter is added variance via randomness to spread out when the RPCs are invoked. By default, Google Cloud +Client Libraries enable jitter for retries. When jitter is enabled with exponential backoff, the client libraries +are able to spread out the retry attempts without overwhelming the server. + +The jitter randomness is computed with on the retry delay. On every attempt, the retry algorithm will compute +a random value with the between [1, RETRY_DELAY]. + +For example, with the following retry configurations: +``` +Initial Retry Delay: 100ms +Retry Delay Multiplier: 2.0 +Max Retry Delay: 500ms +``` + +- Attempt 1: Random value between [1, 100] +- Attempt 2: Random value between [1, 200] +- Attempt 3: Random value between [1, 400] +- Attempt 3: Random value between [1, 500] +- ... +- Attempt X: Random value between [1, 500] + +## How to find the Retry Configurations for an RPC +Default retry params are configured inside the client's generated StubSettings' class. + +## Retry Examples +The following examples below show the behavior of some retry configurations. + +Note: These examples below assume that jitter is disabled. The retry delay is computed to be the maximum value +at the end of each attempt. + +### No Retry +```java +RetrySettings defaultNoRetrySettings = + RetrySettings.newBuilder() + // ... Other configurations do not matter + .setTotalTimeout(Duration.ofMillis(5000L)) + // Explicitly set retries as disabled (maxAttempts == 1) + .setMaxAttempts(1) + .build(); +``` + +The following table shows the attempts: + +| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | +|---------------- |------------- |------------- |-------------- |----------- | +| 1 | 5000ms | 0ms | 0ms | 5000ms | + +### Retry +#### Example 1 +```java +RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(200L)) + .setRetryDelayMultiplier(2.0) + .setMaxRetryDelay(Duration.ofMillis(500L)) + .setInitialRpcTimeout(Duration.ofMillis(1500L)) + .setRpcTimeoutMultiplier(2.0) + .setMaxRpcTimeout(Duration.ofMillis(3000L)) + .setTotalTimeout(Duration.ofMillis(5000L)) + .build(); +``` + +The following table shows the attempts: + +| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | +|------------------------- |------------- |------------- |-------------- |------------ | +| 1 | 1500ms | 0ms | 0ms | 1500ms | +| 2 (Retry) | 3000ms | 200ms | 1700ms | 4700ms | +| 3 (Retry Not Attempted) | - | 400ms | - | - | + +The third retry attempt is not attempted because the computed retry delay (400ms) would invoke the after +the total timeout (400 + 4700 > 5000). + +#### Example 2 +This example is similar to Example #1, but has a longer total timeout to showcase an additional +retry attempt and the capped RPC Timeout for the last retry attempt. +```java +RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(200L)) + .setRetryDelayMultiplier(2.0) + .setMaxRetryDelay(Duration.ofMillis(500L)) + .setInitialRpcTimeout(Duration.ofMillis(1500L)) + .setRpcTimeoutMultiplier(2.0) + .setMaxRpcTimeout(Duration.ofMillis(3000L)) + .setTotalTimeout(Duration.ofMillis(10000L)) + .build(); +``` + +The following table shows the attempts: + +| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | +|---------------- |------------- |------------- |-------------- |------------ | +| 1 | 1500ms | 0ms | 0ms | 1500ms | +| 2 (Retry) | 3000ms | 200ms | 1700ms | 4700ms | +| 3 (Retry) | 4900ms | 400ms | 5100ms | 10000ms | + +The third retry’s RPC Timeout value is limited due the Total Timeout value. Using the multiplier (2.0) with the +previous timeout value (3000ms) should result in an RPC Timeout of 6000ms. However, the RPC Timeout should not +exceed the Total Timeout and is reduced to be the "time left" (10000 - 5100 = 4900). + +#### Example 3 +```java +RetrySettings defaultRetrySettings = + RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(200L)) + .setRetryDelayMultiplier(2.0) + .setMaxRetryDelay(Duration.ofMillis(500L)) + .setInitialRpcTimeout(Duration.ofMillis(500L)) + .setRpcTimeoutMultiplier(2.0) + .setMaxRpcTimeout(Duration.ofMillis(2000L)) + .setTotalTimeout(Duration.ofMillis(4000L)) + .build(); +``` + +The following table shows the attempts: + +| Attempt Number | RPC Timeout | Retry Delay | Call Invoked | Call Ended | +|---------------- |------------- |------------- |-------------- |------------ | +| 1 | 500ms | 0ms | 0ms | 500ms | +| 2 (Retry) | 1000ms | 200ms | 700ms | 1700ms | +| 3 (Retry) | 1900ms | 400ms | 2100ms | 4000ms | + +Another example where the RPC Timeout is capped to not exceed the total timeout. + +## How to configure a custom retry configuration for an RPC +1. Create the RetrySettings class with your custom configurations +```java +RetrySettings customRetrySettings = + RetrySettings.newBuilder() + // ... Retry Configurations + .build(); +``` +2. Create the StubSettings.Builder for your client and configure it for the RPC +```java +AssetServiceStubSettings.Builder assetStubSettingsBuilder = AssetServiceStubSettings.newBuilder(); +assetStubSettingsBuilder + .exportAssetsSettings() + // Set your custom Retry Settings + .setRetrySettings(customRetrySettings) + // Set your custom Retryable Codes + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); +``` + +The snippet above is using a mock service (i.e. EchoClient). The code snippet above is setting +custom retry configurations for EchoClient's Block RPC. It configures the Block RPC to use the +retry settings configured in `customRetrySettings` and sets the retryable codes to be +`DEADLINE_EXCEEDED`. + +3. Create the Settings for the Client +```java +AssetServiceSettings assetSettings = AssetServiceSettings.create(assetStubSettingsBuilder.build()); +``` +4. Create the Client with the Settings +```java +try (AssetServiceClient assetClient = AssetServiceClient.create(assetSettings)) { + ... +} +``` + +Repeat Step #2 for each RPC that you want to configure. For example: +```java +AssetServiceStubSettings.Builder assetStubSettingsBuilder = AssetServiceStubSettings.newBuilder(); + assetStubSettingsBuilder + .exportAssetsSettings() + // Set your custom Retry Settings + .setRetrySettings(customRetrySettings) + // Set your custom Retryable Codes + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); + assetStubSettingsBuilder + .listAssetsSettings() + // Set your custom Retry Settings + .setRetrySettings(customRetrySettings2) + // Set your custom Retryable Codes + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); +``` \ No newline at end of file From 1ed75de51749495f124bd858cbff379033825172 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 21 Mar 2024 18:53:19 -0400 Subject: [PATCH 07/20] docs: Fix README.md formatting --- README.md | 138 +++++++++++++++++++++++++++--------------------------- 1 file changed, 69 insertions(+), 69 deletions(-) diff --git a/README.md b/README.md index a384fe252c20..5d8eb60dfdf6 100644 --- a/README.md +++ b/README.md @@ -304,20 +304,20 @@ access token will not be automatically refreshed): ```java Credentials credentials = GoogleCredentials.create(new AccessToken(accessToken, expirationTime)); - Storage storage = StorageOptions.newBuilder() - .setCredentials(credentials) - .build() - .getService(); +Storage storage = StorageOptions.newBuilder() + .setCredentials(credentials) + .build() + .getService(); ``` or: ```java Credentials credentials = GoogleCredentials.create(new AccessToken(accessToken, expirationTime)); - CloudTasksSettings cloudTasksSettings = CloudTasksSettings.newBuilder() - .setCredentialProvider(FixedCredentialsProvider.create(credentials)) - .build(); - CloudTasksClient cloudTasksClient = CloudTasksClient.create(cloudTasksSettings); +CloudTasksSettings cloudTasksSettings = CloudTasksSettings.newBuilder() + .setCredentialProvider(FixedCredentialsProvider.create(credentials)) + .build(); +CloudTasksClient cloudTasksClient = CloudTasksClient.create(cloudTasksSettings); ``` ### Application Default Credentials @@ -365,35 +365,35 @@ import java.net.InetSocketAddress; import java.net.SocketAddress; public CloudTasksClient getService() throws IOException { - TransportChannelProvider transportChannelProvider = - CloudTasksStubSettings.defaultGrpcTransportProviderBuilder() - .setChannelConfigurator( - new ApiFunction() { -@Override -public ManagedChannelBuilder apply(ManagedChannelBuilder managedChannelBuilder) { - return managedChannelBuilder.proxyDetector( - new ProxyDetector() { -@Nullable -@Override -public ProxiedSocketAddress proxyFor(SocketAddress socketAddress) - throws IOException { - return HttpConnectProxiedSocketAddress.newBuilder() - .setUsername(PROXY_USERNAME) - .setPassword(PROXY_PASSWORD) - .setProxyAddress(new InetSocketAddress(PROXY_HOST, PROXY_PORT)) - .setTargetAddress((InetSocketAddress) socketAddress) - .build(); - } - }); - } - }) - .build(); - CloudTasksSettings cloudTasksSettings = - CloudTasksSettings.newBuilder() - .setTransportChannelProvider(transportChannelProvider) - .build(); - return CloudTasksClient.create(cloudTasksSettings); - } + TransportChannelProvider transportChannelProvider = + CloudTasksStubSettings.defaultGrpcTransportProviderBuilder() + .setChannelConfigurator( + new ApiFunction() { + @Override + public ManagedChannelBuilder apply(ManagedChannelBuilder managedChannelBuilder) { + return managedChannelBuilder.proxyDetector( + new ProxyDetector() { + @Nullable + @Override + public ProxiedSocketAddress proxyFor(SocketAddress socketAddress) + throws IOException { + return HttpConnectProxiedSocketAddress.newBuilder() + .setUsername(PROXY_USERNAME) + .setPassword(PROXY_PASSWORD) + .setProxyAddress(new InetSocketAddress(PROXY_HOST, PROXY_PORT)) + .setTargetAddress((InetSocketAddress) socketAddress) + .build(); + } + }); + } + }) + .build(); + CloudTasksSettings cloudTasksSettings = + CloudTasksSettings.newBuilder() + .setTransportChannelProvider(transportChannelProvider) + .build(); + return CloudTasksClient.create(cloudTasksSettings); +} ``` ## Long Running Operations @@ -413,23 +413,23 @@ check the operation's status. For example, take a sample `createCluster` Operation in google-cloud-dataproc v4.20.0: ```java try (ClusterControllerClient clusterControllerClient = ClusterControllerClient.create()) { - CreateClusterRequest request = - CreateClusterRequest.newBuilder() - .setProjectId("{PROJECT_ID}") - .setRegion("{REGION}") - .setCluster(Cluster.newBuilder().build()) - .setRequestId("{REQUEST_ID}") - .setActionOnFailedPrimaryWorkers(FailureAction.forNumber(0)) - .build(); - OperationFuture future = - clusterControllerClient.createClusterOperationCallable().futureCall(request); - // Do something. - Cluster response = future.get(); - } catch (CancellationException e) { - // Exceeded the default RPC timeout without the Operation completing. - // Library is no longer polling for the Operation status. Consider - // increasing the timeout. - } + CreateClusterRequest request = + CreateClusterRequest.newBuilder() + .setProjectId("{PROJECT_ID}") + .setRegion("{REGION}") + .setCluster(Cluster.newBuilder().build()) + .setRequestId("{REQUEST_ID}") + .setActionOnFailedPrimaryWorkers(FailureAction.forNumber(0)) + .build(); + OperationFuture future = + clusterControllerClient.createClusterOperationCallable().futureCall(request); + // Do something. + Cluster response = future.get(); +} catch (CancellationException e) { + // Exceeded the default RPC timeout without the Operation completing. + // Library is no longer polling for the Operation status. Consider + // increasing the timeout. +} ``` ### LRO Timeouts @@ -452,7 +452,7 @@ For example, in google-cloud-aiplatform v3.24.0, the default [OperationTimedPoll has these default values: ```java OperationTimedPollAlgorithm.create( - RetrySettings.newBuilder() + RetrySettings.newBuilder() .setInitialRetryDelay(Duration.ofMillis(5000L)) .setRetryDelayMultiplier(1.5) .setMaxRetryDelay(Duration.ofMillis(45000L)) @@ -476,19 +476,19 @@ To configure the LRO values, create an OperationTimedPollAlgorithm object and up RPC's polling algorithm. For example: ```java ClusterControllerSettings.Builder settingsBuilder = ClusterControllerSettings.newBuilder(); - TimedRetryAlgorithm timedRetryAlgorithm = OperationTimedPollAlgorithm.create( - RetrySettings.newBuilder() - .setInitialRetryDelay(Duration.ofMillis(500L)) - .setRetryDelayMultiplier(1.5) - .setMaxRetryDelay(Duration.ofMillis(5000L)) - .setInitialRpcTimeout(Duration.ZERO) // ignored - .setRpcTimeoutMultiplier(1.0) // ignored - .setMaxRpcTimeout(Duration.ZERO) // ignored - .setTotalTimeout(Duration.ofHours(24L)) // set polling timeout to 24 hours - .build()); - settingsBuilder.createClusterOperationSettings() - .setPollingAlgorithm(timedRetryAlgorithm); - ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settingsBuilder.build()); +TimedRetryAlgorithm timedRetryAlgorithm = OperationTimedPollAlgorithm.create( + RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(500L)) + .setRetryDelayMultiplier(1.5) + .setMaxRetryDelay(Duration.ofMillis(5000L)) + .setInitialRpcTimeout(Duration.ZERO) // ignored + .setRpcTimeoutMultiplier(1.0) // ignored + .setMaxRpcTimeout(Duration.ZERO) // ignored + .setTotalTimeout(Duration.ofHours(24L)) // set polling timeout to 24 hours + .build()); +settingsBuilder.createClusterOperationSettings() + .setPollingAlgorithm(timedRetryAlgorithm); +ClusterControllerClient clusterControllerClient = ClusterControllerClient.create(settingsBuilder.build()); ``` Note: The configuration above *only* modifies the LRO values for the `createClusterOperation` RPC. @@ -621,4 +621,4 @@ Apache 2.0 - See [LICENSE] for more information. [TESTING]: https://github.com/googleapis/google-cloud-java/blob/main/TESTING.md [cloud-platform]: https://cloud.google.com/ [cloud-platform-docs]: https://cloud.google.com/docs/ -[client-lib-docs]: https://cloud.google.com/java/docs/reference +[client-lib-docs]: https://cloud.google.com/java/docs/reference \ No newline at end of file From e7fc2396cdbeb23bfad201d743cf05df7cafbe85 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Fri, 22 Mar 2024 14:09:59 -0400 Subject: [PATCH 08/20] docs: Fix PR comments --- docs/{retries.md => client_retries.md} | 33 +++++++++++++------------- 1 file changed, 17 insertions(+), 16 deletions(-) rename docs/{retries.md => client_retries.md} (92%) diff --git a/docs/retries.md b/docs/client_retries.md similarity index 92% rename from docs/retries.md rename to docs/client_retries.md index ca548f9dd141..4e36d055347c 100644 --- a/docs/retries.md +++ b/docs/client_retries.md @@ -1,4 +1,4 @@ -# Retries +# Client Side Retries Client Libraries use retries to handle unexpected, transient failures (i.e. server temporarily unavailable). Multiple attempts, hopefully, will result in a successful response from the server. @@ -233,9 +233,9 @@ assetStubSettingsBuilder .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); ``` -The snippet above is using a mock service (i.e. EchoClient). The code snippet above is setting -custom retry configurations for EchoClient's Block RPC. It configures the Block RPC to use the -retry settings configured in `customRetrySettings` and sets the retryable codes to be +The example above is configuring the java-asset library. The code snippet above is setting custom +retry configurations for AssetServiceClient's ExportAssets RPC. It configures the ExportAssets RPC +to use the retry settings configured in `customRetrySettings` and sets the retryable codes to be `DEADLINE_EXCEEDED`. 3. Create the Settings for the Client @@ -252,16 +252,17 @@ try (AssetServiceClient assetClient = AssetServiceClient.create(assetSettings)) Repeat Step #2 for each RPC that you want to configure. For example: ```java AssetServiceStubSettings.Builder assetStubSettingsBuilder = AssetServiceStubSettings.newBuilder(); - assetStubSettingsBuilder - .exportAssetsSettings() - // Set your custom Retry Settings - .setRetrySettings(customRetrySettings) - // Set your custom Retryable Codes - .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); - assetStubSettingsBuilder - .listAssetsSettings() - // Set your custom Retry Settings - .setRetrySettings(customRetrySettings2) - // Set your custom Retryable Codes - .setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); + assetStubSettingsBuilder + .exportAssetsSettings() + // Set your custom Retry Settings + .setRetrySettings(customRetrySettings) + // Set your custom Retryable Codes + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); + + assetStubSettingsBuilder + .listAssetsSettings() + // Set your custom Retry Settings + .setRetrySettings(customRetrySettings2) + // Set your custom Retryable Codes + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); ``` \ No newline at end of file From a36ba21df49cc3d839e7b2e86ad28397a41876b1 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Fri, 22 Mar 2024 19:13:16 -0400 Subject: [PATCH 09/20] docs: Update retry guide --- docs/client_retries.md | 45 +++++++++++++++++++++--------------------- 1 file changed, 22 insertions(+), 23 deletions(-) diff --git a/docs/client_retries.md b/docs/client_retries.md index 4e36d055347c..af560bd6c4da 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -6,7 +6,8 @@ By default, retries are configured by the Cloud Service. The configured retry pa A service *may* choose to only enable retries for a subset of RPCs. It is possible that for a single service, each RPC is configured differently. -For example, using Java-Asset v3.41.0, the default configurations are configured in the following files: +## RPC Retry Configurations +Using Java-Asset v3.41.0 as an example, the default configurations are configured in the following files: Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) @@ -53,16 +54,16 @@ is a response from a server that returns an `OK` Status Code (from gRPC) or a `2 ### When is an RPC retried Take a sample RetrySettings configuration ```java - settings = - RetrySettings.newBuilder() - .setInitialRetryDelay(Duration.ofMillis(100L)) - .setRetryDelayMultiplier(1.3) - .setMaxRetryDelay(Duration.ofMillis(60000L)) - .setInitialRpcTimeout(Duration.ofMillis(60000L)) - .setRpcTimeoutMultiplier(1.0) - .setMaxRpcTimeout(Duration.ofMillis(60000L)) - .setTotalTimeout(Duration.ofMillis(60000L)) - .build(); +settings = + RetrySettings.newBuilder() + .setInitialRetryDelay(Duration.ofMillis(100L)) + .setRetryDelayMultiplier(1.3) + .setMaxRetryDelay(Duration.ofMillis(60000L)) + .setInitialRpcTimeout(Duration.ofMillis(60000L)) + .setRpcTimeoutMultiplier(1.0) + .setMaxRpcTimeout(Duration.ofMillis(60000L)) + .setTotalTimeout(Duration.ofMillis(60000L)) + .build(); ``` Individual RPC Bounds (an attempt) are controlled by the following settings: - setInitialRetryDelay @@ -84,7 +85,7 @@ Note: If only one (or neither) of the scenarios above are true, then the RPC wil i.e. If the total timeout has not been exceeded, but the latest attempt receives a non-retryable status code. *The client library will mark a status code as retryable internally. It is marked as retryable if the response's -status code matches with any of an RPC's configure retryable status codes. +status code matches with any of an RPC's configured retryable status codes. **When configuring the RPC bounds, you may configure the bounds for each attempt as well as the total RPC's bounds. The retry algorithm will ensure that an individual attempt's bounds falls within @@ -95,8 +96,9 @@ Jitter is added variance via randomness to spread out when the RPCs are invoked. Client Libraries enable jitter for retries. When jitter is enabled with exponential backoff, the client libraries are able to spread out the retry attempts without overwhelming the server. -The jitter randomness is computed with on the retry delay. On every attempt, the retry algorithm will compute -a random value with the between [1, RETRY_DELAY]. +The jitter randomness is computed on the retry delay. Before each attempt, the retry algorithm will compute +a random value with the between [1, RETRY_DELAY]. This computed value is the *approximate* delay before the request +is sent to the server. For example, with the following retry configurations: ``` @@ -112,9 +114,6 @@ Max Retry Delay: 500ms - ... - Attempt X: Random value between [1, 500] -## How to find the Retry Configurations for an RPC -Default retry params are configured inside the client's generated StubSettings' class. - ## Retry Examples The following examples below show the behavior of some retry configurations. @@ -124,12 +123,12 @@ at the end of each attempt. ### No Retry ```java RetrySettings defaultNoRetrySettings = - RetrySettings.newBuilder() - // ... Other configurations do not matter - .setTotalTimeout(Duration.ofMillis(5000L)) - // Explicitly set retries as disabled (maxAttempts == 1) - .setMaxAttempts(1) - .build(); + RetrySettings.newBuilder() + // ... Other configurations do not matter + .setTotalTimeout(Duration.ofMillis(5000L)) + // Explicitly set retries as disabled (maxAttempts == 1) + .setMaxAttempts(1) + .build(); ``` The following table shows the attempts: From 1ae4060a742b708bc05d438fc8a1082077a8f6bb Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Tue, 26 Mar 2024 17:28:07 -0400 Subject: [PATCH 10/20] chore: Address PR comments --- README.md | 2 +- docs/client_retries.md | 23 ++++++++++++++--------- 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 5d8eb60dfdf6..30a81f0b1dd4 100644 --- a/README.md +++ b/README.md @@ -444,7 +444,7 @@ Note: The client library handles the Operation's polling mechanism for you. By d to manually poll the status yourself. ### Default LRO Values -Each LRO RPC has a pre-configured default values. You can find these values by +Each LRO RPC has a set of pre-configured default values. You can find these values by searching in each Client's `StubSettings`'s class. The default LRO settings are initialized inside the `initDefaults()` method in the nested Builder class. diff --git a/docs/client_retries.md b/docs/client_retries.md index af560bd6c4da..92f3bbb4d4c8 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -1,10 +1,10 @@ # Client Side Retries -Client Libraries use retries to handle unexpected, transient failures (i.e. server temporarily unavailable). +Client Libraries use retries to handle unexpected, transient failures (i.e. server is temporarily unavailable). Multiple attempts, hopefully, will result in a successful response from the server. -By default, retries are configured by the Cloud Service. The configured retry parameters are defined per RPC. -A service *may* choose to only enable retries for a subset of RPCs. It is possible that for a single service, -each RPC is configured differently. +Default retry values are selected by the team operating the cloud service. These retry parameters are defined +for each RPC. A service *may* choose to only enable retries for a subset of RPCs. It is possible that for a single +service, each RPC is configured differently. ## RPC Retry Configurations Using Java-Asset v3.41.0 as an example, the default configurations are configured in the following files: @@ -20,7 +20,7 @@ Example: StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE))); ``` -Retry Params are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) +Retry parameters are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) Example: ```java @@ -33,7 +33,7 @@ Example: .build(); ``` -The default configurations are mapping [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) +The default configurations are mapped [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) Example: ```java @@ -65,6 +65,10 @@ settings = .setTotalTimeout(Duration.ofMillis(60000L)) .build(); ``` +The configuration above modifies the retry settings for both an RPC's operation and attempt. An RPC operation +is collection of all attempts made and an RPC attempt is the individual attempt made. A single RPC invocation will +have a single operation and one or more attempts. + Individual RPC Bounds (an attempt) are controlled by the following settings: - setInitialRetryDelay - setRetryDelayMultiplier @@ -73,7 +77,7 @@ Individual RPC Bounds (an attempt) are controlled by the following settings: - setRpcTimeoutMultiplier - setMaxRpcTimeout -Total RPC Bounds are controlled by the following settings: +Total RPC Bounds (an operation) are controlled by the following settings: - setTotalTimeout - setAttemptCount @@ -84,8 +88,9 @@ An RPC will be retried when _both_ of the following scenarios occur: Note: If only one (or neither) of the scenarios above are true, then the RPC will not be retried. i.e. If the total timeout has not been exceeded, but the latest attempt receives a non-retryable status code. -*The client library will mark a status code as retryable internally. It is marked as retryable if the response's -status code matches with any of an RPC's configured retryable status codes. +*The client library will check RPC's list of retryable status codes and mark a status code accordingly. +It is marked as retryable if the response's status code matches with any of the RPC's configured retryable status +codes. **When configuring the RPC bounds, you may configure the bounds for each attempt as well as the total RPC's bounds. The retry algorithm will ensure that an individual attempt's bounds falls within From c18a88d2a71ed6391a308f23b8bcf1151c048f65 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Wed, 27 Mar 2024 10:19:42 -0400 Subject: [PATCH 11/20] docs: Clean up the retry guide --- docs/client_retries.md | 70 +++++++++++++++++++++--------------------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/docs/client_retries.md b/docs/client_retries.md index 92f3bbb4d4c8..9d6d7c411f53 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -6,51 +6,52 @@ Default retry values are selected by the team operating the cloud service. These for each RPC. A service *may* choose to only enable retries for a subset of RPCs. It is possible that for a single service, each RPC is configured differently. -## RPC Retry Configurations +## Retry Parameters +Client libraries have two types of retry parameters to configure: +1. Retry Status Code: Set of failure status codes to retry on +2. Retry Time/ Attempt Bounds: Configurable [RetrySettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings) to define the bounds + +### RPC Retry Configurations Using Java-Asset v3.41.0 as an example, the default configurations are configured in the following files: Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) Example: ```java - definitions.put( - "retry_policy_1_codes", - ImmutableSet.copyOf( - Lists.newArrayList( - StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE))); +definitions.put( + "retry_policy_1_codes", + ImmutableSet.copyOf( + Lists.newArrayList( + StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE))); ``` Retry parameters are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) Example: ```java - settings = - RetrySettings.newBuilder() - .setInitialRpcTimeout(Duration.ofMillis(60000L)) - .setRpcTimeoutMultiplier(1.0) - .setMaxRpcTimeout(Duration.ofMillis(60000L)) - .setTotalTimeout(Duration.ofMillis(60000L)) - .build(); +settings = + RetrySettings.newBuilder() + .setInitialRpcTimeout(Duration.ofMillis(60000L)) + .setRpcTimeoutMultiplier(1.0) + .setMaxRpcTimeout(Duration.ofMillis(60000L)) + .setTotalTimeout(Duration.ofMillis(60000L)) + .build(); ``` The default configurations are mapped [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) Example: ```java - builder - .exportAssetsSettings() - .setRetryableCodes(RETRYABLE_CODE_DEFINITIONS.get("no_retry_0_codes")) - .setRetrySettings(RETRY_PARAM_DEFINITIONS.get("no_retry_0_params")); +builder + .exportAssetsSettings() + .setRetryableCodes(RETRYABLE_CODE_DEFINITIONS.get("no_retry_0_codes")) + .setRetrySettings(RETRY_PARAM_DEFINITIONS.get("no_retry_0_params")); ``` -## Google Cloud Client Library Retry Concepts +## Client Library Retry Concepts Enabling retries allow an RPC multiple attempts to try and achieve a successful call. A successful call is a response from a server that returns an `OK` Status Code (from gRPC) or a `2xx` Status Code (from HttpJson). -### Configurable Retry Params -- Retry Status Code: Set of failure status codes to retry on -- Retry Time/ Attempt Bounds: Configurable retry settings (via [RetrySettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings) class) to define the retry bounds - ### When is an RPC retried Take a sample RetrySettings configuration ```java @@ -246,7 +247,7 @@ to use the retry settings configured in `customRetrySettings` and sets the retry ```java AssetServiceSettings assetSettings = AssetServiceSettings.create(assetStubSettingsBuilder.build()); ``` -4. Create the Client with the Settings +4.Create the Client with the Settings ```java try (AssetServiceClient assetClient = AssetServiceClient.create(assetSettings)) { ... @@ -256,17 +257,16 @@ try (AssetServiceClient assetClient = AssetServiceClient.create(assetSettings)) Repeat Step #2 for each RPC that you want to configure. For example: ```java AssetServiceStubSettings.Builder assetStubSettingsBuilder = AssetServiceStubSettings.newBuilder(); - assetStubSettingsBuilder - .exportAssetsSettings() - // Set your custom Retry Settings - .setRetrySettings(customRetrySettings) - // Set your custom Retryable Codes - .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); - assetStubSettingsBuilder - .listAssetsSettings() - // Set your custom Retry Settings - .setRetrySettings(customRetrySettings2) - // Set your custom Retryable Codes - .setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); +// Modify the retry params for ExportAssets RPC +assetStubSettingsBuilder +.exportAssetsSettings() +.setRetrySettings(customRetrySettings) +.setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); + +// Modify the retry params for ListAssets RPC +assetStubSettingsBuilder +.listAssetsSettings() +.setRetrySettings(customRetrySettings2) +.setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); ``` \ No newline at end of file From 9f369fc9d10180f6b326761024ec7ee1c3811bec Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Wed, 27 Mar 2024 11:37:06 -0400 Subject: [PATCH 12/20] docs: Add links for RetrySettings --- README.md | 2 +- docs/client_retries.md | 61 +++++++++++++++++++++++++++++------------- 2 files changed, 43 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 30a81f0b1dd4..bc5b056dfdeb 100644 --- a/README.md +++ b/README.md @@ -241,7 +241,7 @@ In cases where the library may expect a project ID explicitly, we provide a help ```java import com.google.cloud.ServiceOptions; ... - String projectId = ServiceOptions.getDefaultProjectId(); + String projectId = ServiceOptions.getDefaultProjectId(); ``` ## Authentication diff --git a/docs/client_retries.md b/docs/client_retries.md index 9d6d7c411f53..54dd3e59e911 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -2,9 +2,9 @@ Client Libraries use retries to handle unexpected, transient failures (i.e. server is temporarily unavailable). Multiple attempts, hopefully, will result in a successful response from the server. -Default retry values are selected by the team operating the cloud service. These retry parameters are defined -for each RPC. A service *may* choose to only enable retries for a subset of RPCs. It is possible that for a single -service, each RPC is configured differently. +Default retry values are selected by the team operating the cloud service. These retry values are configured +for each RPC. A service *may* choose to only enable retries for a subset of RPCs. It is possible that each RPC +for a service is configured differently. ## Retry Parameters Client libraries have two types of retry parameters to configure: @@ -12,7 +12,7 @@ Client libraries have two types of retry parameters to configure: 2. Retry Time/ Attempt Bounds: Configurable [RetrySettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings) to define the bounds ### RPC Retry Configurations -Using Java-Asset v3.41.0 as an example, the default configurations are configured in the following files: +Using Java-Asset v3.41.0 as an example, the default retry configurations are defined in the following places: Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) @@ -38,7 +38,7 @@ settings = .build(); ``` -The default configurations are mapped [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) +The configurations above are set [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) Example: ```java @@ -66,27 +66,27 @@ settings = .setTotalTimeout(Duration.ofMillis(60000L)) .build(); ``` -The configuration above modifies the retry settings for both an RPC's operation and attempt. An RPC operation -is collection of all attempts made and an RPC attempt is the individual attempt made. A single RPC invocation will +The configuration above modifies the retry settings for both an RPC's attempt and operation. An RPC attempt is the +individual attempt made and an RPC operation is collection of all attempts made. A single RPC invocation will have a single operation and one or more attempts. Individual RPC Bounds (an attempt) are controlled by the following settings: -- setInitialRetryDelay -- setRetryDelayMultiplier -- setMaxRetryDelay -- setInitialRpcTimeout -- setRpcTimeoutMultiplier -- setMaxRpcTimeout +- [setInitialRetryDelay](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setInitialRetryDelay_org_threeten_bp_Duration_) +- [setRetryDelayMultiplier](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setRetryDelayMultiplier_double_) +- [setMaxRetryDelay](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setMaxRetryDelay_org_threeten_bp_Duration_) +- [setInitialRpcTimeout](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setInitialRpcTimeout_org_threeten_bp_Duration_) +- [setRpcTimeoutMultiplier](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setRpcTimeoutMultiplier_double_) +- [setMaxRpcTimeout](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setMaxRpcTimeout_org_threeten_bp_Duration_) Total RPC Bounds (an operation) are controlled by the following settings: -- setTotalTimeout -- setAttemptCount +- [setTotalTimeout](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setTotalTimeout_org_threeten_bp_Duration_) +- [setMaxAttempts](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setMaxAttempts_int_) An RPC will be retried when _both_ of the following scenarios occur: - Non-successful status code is received by the library and the status code is marked as retryable* - An RPC invocation exceeds the individual RPC bounds, but still falls within total RPC bounds** -Note: If only one (or neither) of the scenarios above are true, then the RPC will not be retried. +Note: If only one (or neither) of the scenarios above are true, then the RPC will _NOT_ be retried. i.e. If the total timeout has not been exceeded, but the latest attempt receives a non-retryable status code. *The client library will check RPC's list of retryable status codes and mark a status code accordingly. @@ -97,13 +97,30 @@ codes. total RPC's bounds. The retry algorithm will ensure that an individual attempt's bounds falls within the total RPC's bounds. +### Exponential Backoff +Exponential backoff will retry requests with an increasing delay between each retry attempt. This retry delay value +can be capped with a maximum retry delay value. + +For example, with the following retry configurations: +``` +Initial Retry Delay: 100ms +Retry Delay Multiplier: 2.0 +Max Retry Delay: 500ms +``` +- Attempt 1: Delay 100ms +- Attempt 2: Delay 200ms +- Attempt 3: Delay 400ms +- Attempt 4: Delay 500ms +- ... +- Attempt X: Delay 500ms + ### Jitter Jitter is added variance via randomness to spread out when the RPCs are invoked. By default, Google Cloud Client Libraries enable jitter for retries. When jitter is enabled with exponential backoff, the client libraries are able to spread out the retry attempts without overwhelming the server. The jitter randomness is computed on the retry delay. Before each attempt, the retry algorithm will compute -a random value with the between [1, RETRY_DELAY]. This computed value is the *approximate* delay before the request +a random value with the between `[1, RETRY_DELAY]`. This computed value is the *approximate* delay before the request is sent to the server. For example, with the following retry configurations: @@ -112,11 +129,10 @@ Initial Retry Delay: 100ms Retry Delay Multiplier: 2.0 Max Retry Delay: 500ms ``` - - Attempt 1: Random value between [1, 100] - Attempt 2: Random value between [1, 200] - Attempt 3: Random value between [1, 400] -- Attempt 3: Random value between [1, 500] +- Attempt 4: Random value between [1, 500] - ... - Attempt X: Random value between [1, 500] @@ -136,6 +152,13 @@ RetrySettings defaultNoRetrySettings = .setMaxAttempts(1) .build(); ``` +Alternatively, this can be configured with +```java +RetrySettings defaultNoRetrySettings = + RetrySettings.newBuilder() + .setLogicalTimeout(Duration.ofMillis(5000L)) + .build(); +``` The following table shows the attempts: From c0aed1ebbe2f9141c91142972445e6d231df8fe9 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Wed, 27 Mar 2024 12:02:08 -0400 Subject: [PATCH 13/20] docs: Add FAQ section --- docs/client_retries.md | 46 +++++++++++++++++++++++++++++++++--------- 1 file changed, 37 insertions(+), 9 deletions(-) diff --git a/docs/client_retries.md b/docs/client_retries.md index 54dd3e59e911..9fe86cc134cd 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -188,8 +188,10 @@ The following table shows the attempts: | 2 (Retry) | 3000ms | 200ms | 1700ms | 4700ms | | 3 (Retry Not Attempted) | - | 400ms | - | - | -The third retry attempt is not attempted because the computed retry delay (400ms) would invoke the after -the total timeout (400 + 4700 > 5000). +The third retry attempt is not attempted because the computed retry delay (400ms) would invoke the +RPC after the total timeout (400 + 4700 > 5000). + +The RPC will return a failed message after 4700ms. #### Example 2 This example is similar to Example #1, but has a longer total timeout to showcase an additional @@ -242,7 +244,9 @@ The following table shows the attempts: Another example where the RPC Timeout is capped to not exceed the total timeout. -## How to configure a custom retry configuration for an RPC +## How to configure a custom retry parameters for an RPC +The following example is using the Java-Asset client library: + 1. Create the RetrySettings class with your custom configurations ```java RetrySettings customRetrySettings = @@ -260,11 +264,9 @@ assetStubSettingsBuilder // Set your custom Retryable Codes .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); ``` - -The example above is configuring the java-asset library. The code snippet above is setting custom -retry configurations for AssetServiceClient's ExportAssets RPC. It configures the ExportAssets RPC -to use the retry settings configured in `customRetrySettings` and sets the retryable codes to be -`DEADLINE_EXCEEDED`. +The code snippet above is setting custom retry configurations for AssetServiceClient's ExportAssets +RPC. It configures the ExportAssets RPC to use the retry settings configured in `customRetrySettings` +and sets the retryable codes to be `DEADLINE_EXCEEDED`. 3. Create the Settings for the Client ```java @@ -292,4 +294,30 @@ assetStubSettingsBuilder .listAssetsSettings() .setRetrySettings(customRetrySettings2) .setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); -``` \ No newline at end of file +``` + +## FAQ +### I expected X retry attempts, but it attempted Y times +Unless you explicitly specify the number of max attempts (along with disabling the timeout configurations), +you will not consistently see the same number of retry attempts made. + +[Jitter's](#jitter) random values for RPC delay make it difficult predict when the request is actually sent. +You should only expect that retries attempts are made and have no expectations of when the retry attempts are +made. + +### The RPC returned a failure before the Total Timeout value was reached +The retry algorithm will calculate the jittered retry delay value during each retry attempt. The calculated +retry delay will be scheduled to run in the future (i.e. `currentTime() + jitteredRetryDelay`). If that attempt's +scheduled attempt exceeds the total timeout, the "final" retry attempt will not be made. + +See this [example](#example-1) as an example of this behavior. + +### I configured custom settings and am seeing quota issues +You may have configured the RetrySettings to run too aggressively. The default retry values are chosen by +the team operation the service. + +Consider increasing the retry delay (initial retry delay and retry multiplier) so that the retry attempts +are spaced out. Note, this may result in a slower response. + +Your use case may require a quicker response and/or more frequent retry attempts. If that is the case, try to +increase the quota limits. \ No newline at end of file From 6665e12ff02c32e5b493d9636f472793f03c932b Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Wed, 27 Mar 2024 13:15:07 -0400 Subject: [PATCH 14/20] docs: Update guide --- docs/client_retries.md | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/docs/client_retries.md b/docs/client_retries.md index 9fe86cc134cd..62336e0e6500 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -3,8 +3,8 @@ Client Libraries use retries to handle unexpected, transient failures (i.e. serv Multiple attempts, hopefully, will result in a successful response from the server. Default retry values are selected by the team operating the cloud service. These retry values are configured -for each RPC. A service *may* choose to only enable retries for a subset of RPCs. It is possible that each RPC -for a service is configured differently. +per RPC. A service *may* choose to only enable retries for a subset of RPCs. It is possible that each RPC for +a service is configured differently. ## Retry Parameters Client libraries have two types of retry parameters to configure: @@ -68,7 +68,7 @@ settings = ``` The configuration above modifies the retry settings for both an RPC's attempt and operation. An RPC attempt is the individual attempt made and an RPC operation is collection of all attempts made. A single RPC invocation will -have a single operation and one or more attempts. +have one or more attempts in a single operation. Individual RPC Bounds (an attempt) are controlled by the following settings: - [setInitialRetryDelay](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setInitialRetryDelay_org_threeten_bp_Duration_) @@ -152,7 +152,7 @@ RetrySettings defaultNoRetrySettings = .setMaxAttempts(1) .build(); ``` -Alternatively, this can be configured with +Alternatively, this same behavior can be configured with ```java RetrySettings defaultNoRetrySettings = RetrySettings.newBuilder() @@ -166,7 +166,7 @@ The following table shows the attempts: |---------------- |------------- |------------- |-------------- |----------- | | 1 | 5000ms | 0ms | 0ms | 5000ms | -### Retry +### Retry Examples #### Example 1 ```java RetrySettings.newBuilder() @@ -299,16 +299,13 @@ assetStubSettingsBuilder ## FAQ ### I expected X retry attempts, but it attempted Y times Unless you explicitly specify the number of max attempts (along with disabling the timeout configurations), -you will not consistently see the same number of retry attempts made. - -[Jitter's](#jitter) random values for RPC delay make it difficult predict when the request is actually sent. -You should only expect that retries attempts are made and have no expectations of when the retry attempts are -made. +you may not consistently see the same number of retry attempts made. [Jitter's](#jitter) random values for +RPC delay make it difficult predict when the request is actually sent. ### The RPC returned a failure before the Total Timeout value was reached The retry algorithm will calculate the jittered retry delay value during each retry attempt. The calculated retry delay will be scheduled to run in the future (i.e. `currentTime() + jitteredRetryDelay`). If that attempt's -scheduled attempt exceeds the total timeout, the "final" retry attempt will not be made. +scheduled attempt time exceeds the total timeout, the "final" retry attempt will not be made. See this [example](#example-1) as an example of this behavior. @@ -317,7 +314,7 @@ You may have configured the RetrySettings to run too aggressively. The default r the team operation the service. Consider increasing the retry delay (initial retry delay and retry multiplier) so that the retry attempts -are spaced out. Note, this may result in a slower response. +are spaced out and less frequent. Note, this may result in a slower response. Your use case may require a quicker response and/or more frequent retry attempts. If that is the case, try to increase the quota limits. \ No newline at end of file From fd095c0776c2ae2687fb8d711237d400769f298b Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Wed, 27 Mar 2024 17:50:04 -0400 Subject: [PATCH 15/20] docs: Address PR comments --- docs/client_retries.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/docs/client_retries.md b/docs/client_retries.md index 62336e0e6500..86d46789fa85 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -11,22 +11,23 @@ Client libraries have two types of retry parameters to configure: 1. Retry Status Code: Set of failure status codes to retry on 2. Retry Time/ Attempt Bounds: Configurable [RetrySettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings) to define the bounds -### RPC Retry Configurations -Using Java-Asset v3.41.0 as an example, the default retry configurations are defined in the following places: - -Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) - +### Default RPC Retry Configuration Location +The default retry configurations are defined in the generated {Client}StubSettings file. Using Java-Asset v3.41.0 as +an example, the default retry configurations are defined in the following places: +
+- Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) +
Example: ```java -definitions.put( +defimnitions.put( "retry_policy_1_codes", ImmutableSet.copyOf( Lists.newArrayList( StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE))); ``` -Retry parameters are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) - +- Retry parameters are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) +
Example: ```java settings = @@ -38,8 +39,8 @@ settings = .build(); ``` -The configurations above are set [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) - +- The configurations above are mapped to the RPC [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) +
Example: ```java builder From b179adc22a5c3d42c6feeb6644d7c191e952a6f3 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 28 Mar 2024 10:44:41 -0400 Subject: [PATCH 16/20] docs: Update the ExportAssets example --- docs/client_retries.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/client_retries.md b/docs/client_retries.md index 86d46789fa85..67851a46c78d 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -12,31 +12,31 @@ Client libraries have two types of retry parameters to configure: 2. Retry Time/ Attempt Bounds: Configurable [RetrySettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings) to define the bounds ### Default RPC Retry Configuration Location -The default retry configurations are defined in the generated {Client}StubSettings file. Using Java-Asset v3.41.0 as -an example, the default retry configurations are defined in the following places: +The default retry configurations are defined in the generated {Client}StubSettings file. Using the ExportAssets RPC in +Java-Asset v3.41.0 as an example, the default retry configurations are defined in the following places:
- Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082)
Example: ```java -defimnitions.put( - "retry_policy_1_codes", - ImmutableSet.copyOf( - Lists.newArrayList( - StatusCode.Code.DEADLINE_EXCEEDED, StatusCode.Code.UNAVAILABLE))); +ImmutableMap.Builder> definitions = ImmutableMap.builder(); +definitions.put("no_retry_0_codes", ImmutableSet.copyOf(Lists.newArrayList())); ``` - Retry parameters are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155)
Example: ```java +ImmutableMap.Builder definitions = ImmutableMap.builder(); +RetrySettings settings = null; settings = - RetrySettings.newBuilder() - .setInitialRpcTimeout(Duration.ofMillis(60000L)) - .setRpcTimeoutMultiplier(1.0) - .setMaxRpcTimeout(Duration.ofMillis(60000L)) - .setTotalTimeout(Duration.ofMillis(60000L)) - .build(); + RetrySettings.newBuilder() + .setInitialRpcTimeout(Duration.ofMillis(60000L)) + .setRpcTimeoutMultiplier(1.0) + .setMaxRpcTimeout(Duration.ofMillis(60000L)) + .setTotalTimeout(Duration.ofMillis(60000L)) + .build(); +definitions.put("no_retry_0_params", settings); ``` - The configurations above are mapped to the RPC [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) From 73c81c64bd2412d46ff1a88be7c6fd23feb59bca Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 28 Mar 2024 11:52:11 -0400 Subject: [PATCH 17/20] doc: Clean up the sections --- docs/client_retries.md | 60 +++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/docs/client_retries.md b/docs/client_retries.md index 67851a46c78d..1bc287e3995b 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -8,24 +8,22 @@ a service is configured differently. ## Retry Parameters Client libraries have two types of retry parameters to configure: -1. Retry Status Code: Set of failure status codes to retry on -2. Retry Time/ Attempt Bounds: Configurable [RetrySettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings) to define the bounds +1. Retry Status Code: Set of status codes to retry on +2. Retry Timeout/ Attempt Bounds: Configurable [RetrySettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings) to define the bounds ### Default RPC Retry Configuration Location -The default retry configurations are defined in the generated {Client}StubSettings file. Using the ExportAssets RPC in +The default retry configurations are defined in the generated `{Client}StubSettings` file. Using the ExportAssets RPC in Java-Asset v3.41.0 as an example, the default retry configurations are defined in the following places:
-- Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082) -
-Example: +- Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082). Example: ```java ImmutableMap.Builder> definitions = ImmutableMap.builder(); definitions.put("no_retry_0_codes", ImmutableSet.copyOf(Lists.newArrayList())); +// ... More StatusCode configurations +RETRYABLE_CODE_DEFINITIONS = definitions.build(); ``` -- Retry parameters are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155) -
-Example: +- Retry parameters are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155). Example: ```java ImmutableMap.Builder definitions = ImmutableMap.builder(); RetrySettings settings = null; @@ -37,11 +35,11 @@ settings = .setTotalTimeout(Duration.ofMillis(60000L)) .build(); definitions.put("no_retry_0_params", settings); +// ... More RetrySettings configurations +RETRY_PARAM_DEFINITIONS = definitions.build(); ``` -- The configurations above are mapped to the RPC [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474) -
-Example: +- The configurations above are mapped to the RPC [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474). Example: ```java builder .exportAssetsSettings() @@ -53,7 +51,7 @@ builder Enabling retries allow an RPC multiple attempts to try and achieve a successful call. A successful call is a response from a server that returns an `OK` Status Code (from gRPC) or a `2xx` Status Code (from HttpJson). -### When is an RPC retried +### Attempt vs Operation Take a sample RetrySettings configuration ```java settings = @@ -83,6 +81,7 @@ Total RPC Bounds (an operation) are controlled by the following settings: - [setTotalTimeout](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setTotalTimeout_org_threeten_bp_Duration_) - [setMaxAttempts](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setMaxAttempts_int_) +### When is an RPC retried An RPC will be retried when _both_ of the following scenarios occur: - Non-successful status code is received by the library and the status code is marked as retryable* - An RPC invocation exceeds the individual RPC bounds, but still falls within total RPC bounds** @@ -139,15 +138,15 @@ Max Retry Delay: 500ms ## Retry Examples The following examples below show the behavior of some retry configurations. - -Note: These examples below assume that jitter is disabled. The retry delay is computed to be the maximum value -at the end of each attempt. +
+Note: These examples below assume that jitter is disabled. The retry delay is computed to be the attempt's maximum +value at the end of each attempt. ### No Retry ```java RetrySettings defaultNoRetrySettings = RetrySettings.newBuilder() - // ... Other configurations do not matter + // Use the default configurations for other settings .setTotalTimeout(Duration.ofMillis(5000L)) // Explicitly set retries as disabled (maxAttempts == 1) .setMaxAttempts(1) @@ -188,11 +187,8 @@ The following table shows the attempts: | 1 | 1500ms | 0ms | 0ms | 1500ms | | 2 (Retry) | 3000ms | 200ms | 1700ms | 4700ms | | 3 (Retry Not Attempted) | - | 400ms | - | - | - The third retry attempt is not attempted because the computed retry delay (400ms) would invoke the -RPC after the total timeout (400 + 4700 > 5000). - -The RPC will return a failed message after 4700ms. +RPC after the total timeout (400 + 4700 > 5000). The RPC will return a failure message after 4700ms. #### Example 2 This example is similar to Example #1, but has a longer total timeout to showcase an additional @@ -252,8 +248,12 @@ The following example is using the Java-Asset client library: ```java RetrySettings customRetrySettings = RetrySettings.newBuilder() - // ... Retry Configurations - .build(); + // ... Retry Configurations + .build(); +RetrySettings customRetrySettings2 = + RetrySettings.newBuilder() + // ... Retry Configurations + .build(); ``` 2. Create the StubSettings.Builder for your client and configure it for the RPC ```java @@ -286,15 +286,15 @@ AssetServiceStubSettings.Builder assetStubSettingsBuilder = AssetServiceStubSett // Modify the retry params for ExportAssets RPC assetStubSettingsBuilder -.exportAssetsSettings() -.setRetrySettings(customRetrySettings) -.setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); + .exportAssetsSettings() + .setRetrySettings(customRetrySettings) + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.DEADLINE_EXCEEDED)); // Modify the retry params for ListAssets RPC assetStubSettingsBuilder -.listAssetsSettings() -.setRetrySettings(customRetrySettings2) -.setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); + .listAssetsSettings() + .setRetrySettings(customRetrySettings2) + .setRetryableCodes(ImmutableSet.of(StatusCode.Code.UNAVAILABLE)); ``` ## FAQ @@ -315,7 +315,7 @@ You may have configured the RetrySettings to run too aggressively. The default r the team operation the service. Consider increasing the retry delay (initial retry delay and retry multiplier) so that the retry attempts -are spaced out and less frequent. Note, this may result in a slower response. +are spaced out and less frequent. Note that this *may* result in a slower response. Your use case may require a quicker response and/or more frequent retry attempts. If that is the case, try to increase the quota limits. \ No newline at end of file From 4e50c508660a5995c101080120bc0775ae009687 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Tue, 2 Apr 2024 14:49:43 -0400 Subject: [PATCH 18/20] docs: Address PR comments --- docs/client_retries.md | 56 +++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/docs/client_retries.md b/docs/client_retries.md index 1bc287e3995b..d46827ca1832 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -17,34 +17,34 @@ Java-Asset v3.41.0 as an example, the default retry configurations are defined i
- Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082). Example: ```java -ImmutableMap.Builder> definitions = ImmutableMap.builder(); -definitions.put("no_retry_0_codes", ImmutableSet.copyOf(Lists.newArrayList())); -// ... More StatusCode configurations -RETRYABLE_CODE_DEFINITIONS = definitions.build(); + ImmutableMap.Builder> definitions = ImmutableMap.builder(); + definitions.put("no_retry_0_codes", ImmutableSet.copyOf(Lists.newArrayList())); + // ... More StatusCode configurations + RETRYABLE_CODE_DEFINITIONS = definitions.build(); ``` - Retry parameters are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155). Example: ```java -ImmutableMap.Builder definitions = ImmutableMap.builder(); -RetrySettings settings = null; -settings = - RetrySettings.newBuilder() - .setInitialRpcTimeout(Duration.ofMillis(60000L)) - .setRpcTimeoutMultiplier(1.0) - .setMaxRpcTimeout(Duration.ofMillis(60000L)) - .setTotalTimeout(Duration.ofMillis(60000L)) - .build(); -definitions.put("no_retry_0_params", settings); -// ... More RetrySettings configurations -RETRY_PARAM_DEFINITIONS = definitions.build(); + ImmutableMap.Builder definitions = ImmutableMap.builder(); + RetrySettings settings = null; + settings = + RetrySettings.newBuilder() + .setInitialRpcTimeout(Duration.ofMillis(60000L)) + .setRpcTimeoutMultiplier(1.0) + .setMaxRpcTimeout(Duration.ofMillis(60000L)) + .setTotalTimeout(Duration.ofMillis(60000L)) + .build(); + definitions.put("no_retry_0_params", settings); + // ... More RetrySettings configurations + RETRY_PARAM_DEFINITIONS = definitions.build(); ``` - The configurations above are mapped to the RPC [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474). Example: ```java -builder - .exportAssetsSettings() - .setRetryableCodes(RETRYABLE_CODE_DEFINITIONS.get("no_retry_0_codes")) - .setRetrySettings(RETRY_PARAM_DEFINITIONS.get("no_retry_0_params")); + builder + .exportAssetsSettings() + .setRetryableCodes(RETRYABLE_CODE_DEFINITIONS.get("no_retry_0_codes")) + .setRetrySettings(RETRY_PARAM_DEFINITIONS.get("no_retry_0_params")); ``` ## Client Library Retry Concepts @@ -70,16 +70,16 @@ individual attempt made and an RPC operation is collection of all attempts made. have one or more attempts in a single operation. Individual RPC Bounds (an attempt) are controlled by the following settings: -- [setInitialRetryDelay](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setInitialRetryDelay_org_threeten_bp_Duration_) -- [setRetryDelayMultiplier](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setRetryDelayMultiplier_double_) -- [setMaxRetryDelay](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setMaxRetryDelay_org_threeten_bp_Duration_) -- [setInitialRpcTimeout](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setInitialRpcTimeout_org_threeten_bp_Duration_) -- [setRpcTimeoutMultiplier](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setRpcTimeoutMultiplier_double_) -- [setMaxRpcTimeout](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setMaxRpcTimeout_org_threeten_bp_Duration_) +- [setInitialRetryDelay](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setInitialRetryDelay_org_threeten_bp_Duration_): Delay before the first attempt +- [setRetryDelayMultiplier](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setRetryDelayMultiplier_double_) - Delay multiplier applied between each attempt +- [setMaxRetryDelay](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setMaxRetryDelay_org_threeten_bp_Duration_) - Max Delay possible for an attempt +- [setInitialRpcTimeout](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setInitialRpcTimeout_org_threeten_bp_Duration_) - Timeout for the first attempt +- [setRpcTimeoutMultiplier](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setRpcTimeoutMultiplier_double_) - Timeout multiplier applied between each attempt +- [setMaxRpcTimeout](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setMaxRpcTimeout_org_threeten_bp_Duration_) - Max Timeout possible for an attempt Total RPC Bounds (an operation) are controlled by the following settings: -- [setTotalTimeout](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setTotalTimeout_org_threeten_bp_Duration_) -- [setMaxAttempts](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setMaxAttempts_int_) +- [setTotalTimeout](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setTotalTimeout_org_threeten_bp_Duration_) - Total timeout allowed the entire operation +- [setMaxAttempts](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.retrying.RetrySettings.Builder#com_google_api_gax_retrying_RetrySettings_Builder_setMaxAttempts_int_) - Max number of attempts allowed ### When is an RPC retried An RPC will be retried when _both_ of the following scenarios occur: From e876c32f0a6a44de2b4cb730a479eaa35d4ebe16 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Tue, 2 Apr 2024 14:51:24 -0400 Subject: [PATCH 19/20] docs: Fix indentation issue --- docs/client_retries.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/client_retries.md b/docs/client_retries.md index d46827ca1832..f2909cbfff79 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -16,15 +16,15 @@ The default retry configurations are defined in the generated `{Client}StubSetti Java-Asset v3.41.0 as an example, the default retry configurations are defined in the following places:
- Retry Status Codes are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1058-L1082). Example: -```java + ```java ImmutableMap.Builder> definitions = ImmutableMap.builder(); definitions.put("no_retry_0_codes", ImmutableSet.copyOf(Lists.newArrayList())); // ... More StatusCode configurations RETRYABLE_CODE_DEFINITIONS = definitions.build(); -``` + ``` - Retry parameters are configured [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1086-L1155). Example: -```java + ```java ImmutableMap.Builder definitions = ImmutableMap.builder(); RetrySettings settings = null; settings = @@ -37,15 +37,15 @@ Java-Asset v3.41.0 as an example, the default retry configurations are defined i definitions.put("no_retry_0_params", settings); // ... More RetrySettings configurations RETRY_PARAM_DEFINITIONS = definitions.build(); -``` + ``` - The configurations above are mapped to the RPC [here](https://github.com/googleapis/google-cloud-java/blob/d9da511b4b56302e509abe8b2d919a15ea7dcae7/java-asset/google-cloud-asset/src/main/java/com/google/cloud/asset/v1/stub/AssetServiceStubSettings.java#L1306-L1474). Example: -```java + ```java builder .exportAssetsSettings() .setRetryableCodes(RETRYABLE_CODE_DEFINITIONS.get("no_retry_0_codes")) .setRetrySettings(RETRY_PARAM_DEFINITIONS.get("no_retry_0_params")); -``` + ``` ## Client Library Retry Concepts Enabling retries allow an RPC multiple attempts to try and achieve a successful call. A successful call From cc5ef2893421d7a02996f345b9a79ce2d26d95d0 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Wed, 24 Apr 2024 16:25:20 -0400 Subject: [PATCH 20/20] chore: Address PR comments --- docs/client_retries.md | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/docs/client_retries.md b/docs/client_retries.md index f2909cbfff79..bc3d7dde6fd7 100644 --- a/docs/client_retries.md +++ b/docs/client_retries.md @@ -101,7 +101,7 @@ the total RPC's bounds. Exponential backoff will retry requests with an increasing delay between each retry attempt. This retry delay value can be capped with a maximum retry delay value. -For example, with the following retry configurations: +For example, the following retry configurations may result in the following delay times: ``` Initial Retry Delay: 100ms Retry Delay Multiplier: 2.0 @@ -114,27 +114,33 @@ Max Retry Delay: 500ms - ... - Attempt X: Delay 500ms +Note: The example above is configured to retry indefinitely. Do set an RPC operation cap by setting the TotalTimeout +and/or MaxAttempts value. + ### Jitter -Jitter is added variance via randomness to spread out when the RPCs are invoked. By default, Google Cloud -Client Libraries enable jitter for retries. When jitter is enabled with exponential backoff, the client libraries -are able to spread out the retry attempts without overwhelming the server. +Jitter is added variance via randomness to spread out when the RPCs are invoked. Google Cloud Client Libraries +always enable jitter for retries. This is to help spread out the retry attempts without overwhelming the server. -The jitter randomness is computed on the retry delay. Before each attempt, the retry algorithm will compute -a random value with the between `[1, RETRY_DELAY]`. This computed value is the *approximate* delay before the request -is sent to the server. +The jitter random value is computed as the retry delay. Before each attempt, the retry algorithm will compute a random +value between `[1, RETRY_DELAY]`. This computed value is the *approximate* delay before the request is sent to the +server. -For example, with the following retry configurations: +For example, the following retry configurations utilizes Jitter and Exponential Backoff, which may result in the +following delays times: ``` Initial Retry Delay: 100ms Retry Delay Multiplier: 2.0 Max Retry Delay: 500ms ``` -- Attempt 1: Random value between [1, 100] -- Attempt 2: Random value between [1, 200] -- Attempt 3: Random value between [1, 400] -- Attempt 4: Random value between [1, 500] +- Attempt 1: Delay a random value between `[1, 100]`ms +- Attempt 2: Delay a random value between `[1, 200]`ms +- Attempt 3: Delay a random value between `[1, 400]`ms +- Attempt 4: Delay a random value between `[1, 500]`ms - ... -- Attempt X: Random value between [1, 500] +- Attempt X: Delay a random value between `[1, 500]`ms + +Note: The example above is configured to retry indefinitely. Do set an RPC operation cap by setting the TotalTimeout +and/or MaxAttempts value. ## Retry Examples The following examples below show the behavior of some retry configurations.