Consider to report stats in Queue-proxy only when it's user container is ready

[lvjing2]

In what area(s)?

/area autoscale

/kind proposal

Ask your question here:

Current, the queue-proxy will start to report the stats once it started even if the user container is not ready.
But the stats when the user-container is still not ready is not believable to decide to whether should scale up or down, in fact there is no request be routed to the pod when the user container is not ready, that's mean those stats should be all zeros.
So I propose that the queue proxy start to report stats only when the user-container is ready.
@markusthoemmes @dgerd WDYT

[vagababov]

I think I saw a PR to that end recently.

[lvjing2]

Do you mean #3493 ?

[vagababov]

No, I don't think so.
I think this: #3538

[lvjing2]

@vagababov Sorry, I can't get the same point with me from #3538 , it feels like to reduce the time in activator by poke the activator when the pod is ready when scaling up from 0 which is different with my propose.
My propose is applied to the stats from all user pod to autoscaler when creating a new pod, only to report the stats to prometheus exporter when the user-container is ready.
Can you help to review #3583 ? and check whether it is needed. thanks

[markusthoemmes]

I commented under the PR, but cross-posting for posterity:

Which problem are we solving by doing this? The scraper scrapes a K8s service, which will only receive endpoints if the respective pods are ready. The pod will only go ready if the user-container has already been successfully probed.

That means: The scraper won't see the (arguably wrong) metrics until the user-container becomes ready, which is what you want to fix by this issue already I think.

[lvjing2]

@markusthoemmes Yes, that's what i want to fix. the PR would only be useful in the previous directly sending stats from queue-proxy to autoscaler mode, but currently it had been change to scraper mode, then this PR is no need anymore.