Implement pre-emptive credential refresh

[igorbernstein2]

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Is your feature request related to a problem? Please describe.
In the current implementation OAuth credentials get refreshed inline when auth headers are requested by gRPC. This causes latency spikes that are hard to debug.

Describe the solution you'd like
Implement a stale state for OAuth tokens where the existing token is returned, and a refresh is triggered in the background. Prior art: googleapis/google-auth-library-java#646

Describe alternatives you've considered
Other possibilities are:

• Have an always running background thread to always refresh the token. This will have lifetime issues since credentials dont currently have a way to close them/stop the thread

[clundin25]

Implemented for sync credentials in googleapis/google-auth-library-python#1368.

Async implementation is blocked until the API / Inheritance issues are sorted.

[parthea]

Transferring to google-cloud-python as the code for google-auth-library-python will be migrated there in 1-3 weeks.