Connections stay in CLOSE_WAIT state on using kms_v1.KeyManagementServiceClient

[rachari]

On using kms_v1.KeyManagementServiceClient in one of the scripts that I'm working on, I noticed the following in lsof output for the process

 TCP <instance_ip>:42108->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42118->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42130->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42140->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42152->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42162->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42202->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42212->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42224->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42234->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42246->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42256->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42266->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42278->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42288->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42300->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42310->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42322->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42332->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42344->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42354->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42366->169.254.169.254:http (CLOSE_WAIT)
 TCP <instance_ip>:42376->169.254.169.254:http (CLOSE_WAIT)

This happens irrespective of whether the KMS client is called in a loop or outside of a loop

If the script is being run as a scheduled job, then these CLOSE_WAIT sockets keep piling up eventually leading to a "Too many open files" error for the running process.

Is there a way to close these connections ?

NOTE -
Following packages are installed and being used :-

google-api-core                       1.15.0
google-api-python-client              1.7.11
google-auth                           1.6.3
google-auth-httplib2                  0.0.3
google-cloud-kms                      1.2.1
googleapis-common-protos              1.6.0
grpc-google-iam-v1                    0.12.3
grpcio                                1.26.0

Modified sample from https://cloud.google.com/kms/docs/reference/libraries (Python example) that can be used to check this :-

## Imports the Google APIs client library
import time
from google.cloud import kms_v1

# Your Google Cloud Platform project ID
project_id = 'YOUR_PROJECT_ID'

# Lists keys in the "global" location.
location = 'global'

# Creates an API client for the KMS API.
# client = kms_v1.KeyManagementServiceClient()

# The resource name of the location associated with the key rings.
# parent = client.location_path(project_id, location)

# Lists key rings
while True:
    # Creates an API client for the KMS API.
    client = kms_v1.KeyManagementServiceClient()

    # The resource name of the location associated with the key rings.
    parent = client.location_path(project_id, location)
    response = client.list_key_rings(parent, timeout=5)
    response_list = list(response)

    if len(response_list) > 0:
        print('Key rings:')
        for key_ring in response_list:
            print(key_ring.name)
    else:
        print('No key rings found.')

    time.sleep(30)

[tswast]

This sounds a lot like issue googleapis/google-cloud-python#9457

I've filed googleapis/gapic-generator-python#575 to track implementing closable client objects in the code generator.

[tswast]

In the meantime, I believe you'll need to call client._transport.channel.close() to clean up sockets.

[vchudnov-g]

Note that googleapis/gapic-generator-python#575 has been implemented, so it should be even easier to close connections.