Mark a revision ready at the right time

[akyyy]

Expected Behavior

Once a revision is marked ready, we should be able to send requests to the revision.

Actual Behavior

7 seconds after a revision is marked ready, the forwarded request got timeout error.
http: proxy error: dial tcp 10.31.249.52:80: i/o timeout
This doesn't happen 100%.

Currently, we mark a revision ready when there is at least one endpoint available.
https://github.com/elafros/elafros/blob/0be70191e5021bfeb8e29593490e5d0a2c24d8cc/pkg/controller/revision/revision.go#L492

Steps to Reproduce the Problem

This is best to repro with the activator development work. When a revision is inactive, send a request, the activator should activate the revision and hold the request until it's ready.

Additional Info

We may consider the readinessProbe definition if customer defined it like here.
https://github.com/elafros/elafros/blob/9ec25c157ee67116d2cb9e5a20519f93eda3e250/sample/helloworld/configuration.yaml#L31-L36

https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#define-readiness-probes
"The kubelet uses readiness probes to know when a Container is ready to start accepting traffic."

[mattmoor]

This looks similar to an issue @tcnghia is working on.

[tcnghia]

Yes, I am adding when working on #429 . That issue is scoped to scale down, but I was looking at scaling in general and saw that what you describe here also applies to scaling up.

[mattmoor]

@tcnghia Does your change fix this? If not, can you at least try to root cause it in M4?

[tcnghia]

I made some changes to #341 to use the readied addresses from the Endpoint directly instead of going through the service IP, and requests succeed every time revision is Ready.

This suggests that the problem is due to the bridge between Service <-> Endpoints not being ready fast enough after addition of new ready addresses.

[tcnghia]

This is true even if I forward the requests to the Endpoint's ready addresses immediately after Revision becoming Ready (aka, without the small sleep in #341).

[tcnghia]

Reusing the Service (instead of recreating) show some small reduction in the delay (1-2 seconds), but it still take more than 5 seconds for requests made to the Service ClusterIP to succeed.

[tcnghia]

This doesn't just affect autoscaling: newly created revision also shows a similar latency. However usually with newly created revision the Ingress IP takes very long to appear, thus masking the (smaller) delay in Service ClusterIP availability.

[tcnghia]

Running 'curl $SERVICE_IP' also sees similar delay, so this isn't particular to the way we proxy from Activator to Service.