Manually implement client.Wait backoffs

[jedevc]

🛠️ Fixes regression #4164, while preserving the spirit of #4015.
◀️ Reverts #4015.

When calling client.Wait, we want to avoid the default backoff behavior, because we want to achieve a quick response back once the server becomes active.

To do this, without modifying the entire client's exponential backoff configuration, we can use conn.ResetConnectBackoff, while attempting to reconnect every second.

Here are some common scenarios under the new client.Wait implementation:

• Server is listening: the call to Info succeeds quickly, and we return.
• Server is listening, but is behind several proxies and so latency is high: the call to Info succeeds slowly (up to minConnectTimeout=20s), and we return.
  • Note: if the server cannot be reached in fewer than minConnectTimeout, then client.Wait will never return.
• Server is not listening and gets "connection refused": the call to Info fails quickly, and we wait a second before retrying.
• Server is not listening and does not respond (e.g. firewall dropping packets): the call to Info fails slowly (by default after minConnectTimeout=20s). After the call fails, we wait a second before retrying.

This PR is split into two commits:

• The first resolves After upgrade local CLI to 0.12 remote build stop working #4164, and fixes the regression from buildctl: set max backoff delay to 1 second #4015, and will need to be cherry-picked to buildx (since it affects the remote driver).
• The second re-implements buildctl: set max backoff delay to 1 second #4015 by making more substantial changes to client.Wait.

[crazy-max]

The automatic retries for transient errors and recover gracefully when the service is unavailable temporarily looks good enough but I wonder if we should consider a maximum retry limit as well.

[jedevc]

I wonder if we should consider a maximum retry limit as well.

I think the --timeout flag controls a maximum duration of the context that eventually gets passed through into client.Wait. I think that makes more sense than an explicit number of retries?

[crazy-max]

I wonder if we should consider a maximum retry limit as well.

I think the --timeout flag controls a maximum duration of the context that eventually gets passed through into client.Wait. I think that makes more sense than an explicit number of retries?

Looks good 👍