Don't scale to 1 upon deploy

[duglin]

In what area(s)?

/area API
/area autoscale

Other classifications:
/kind good-first-issue
/kind spec

Describe the feature

Right now when a new KnService is deployed it is automatically scaled up to one instance. I believe this is incorrect, it should be scaled to zero by default. For a couple of reasons:

• when not being used, w/o minScale > 0, services will scale to zero.... the service has not been used yet
• when $ is involved with the running of the pods associated with a service, end-users will not expect to be charged when no one called their service... starting with a scale of 1 means they will be charged even though no one called their service, violating that "only pay for what you use" contract
• the current behavior treats "deploy" time differently from any other time the service is alive. I see no reason to think that the 90 seconds immediately after a service is created is any different from any other time
• if people really think "deploy time" is special, then we should call it out explicitly by allowing the user to specify a default scale value (either on the Service itself or via some Kn-wide config) rather than assuming the same needs apply to all users. Then they can also choose something other than "1" if they want.

/cc @nimakaviani

edit: proposal: provide a config knob so people can choose if they want to scale to 1 on deploy.

[markusthoemmes]

This is referred to in the runtime contract as a deployment probe. It's there to make sure your app is even viable and can start up in the first place.

We could enhance it and force scale down to 0 as soon as we see it becoming ready successfully. It served its purpose at this point.

[duglin]

Maybe... I can certainly see why it's done the way it is, but technically there could be lots of reasons why it won't start-up later in time too due to things changing in the env, and so I'm still thinking that treating the "deploy time" as special is probably not appropriate.

But, as I mentioned, giving the user the choice when money might be involved would be nice.

[rgregg]

Without scaling the service to 1 during deployment, there are a number of failure conditions that we won't be able to detect at deployment time, and will postpone those to being discovered at runtime. That seems like a bad user experience.

What's the downside about creating an instance of the app initially?

[duglin]

In an environment where a customer only wants to be charged when their service is actually invoked, they're now going to be charged just because it was deployed, not invoked.

As I mentioned above, I think having some kind of initialScale type of flag (and we can bike-shed on whether it's default is 1 or 0) would solve this. At least then the user (or cloud provider) can choose.

[mbehrendt]

adding to what @duglin wrote: If a customer wants to test whether what they deployed works, they'd just curl it. That would scaling from 0 to 1, and then immediately back to 0.

[mattmoor]

If a customer wants to test whether what they deployed works, they'd just curl it.

They could if they are manually rolling revisions out, but this elides a fairly fundamental check for "run latest" style services.

[duglin]

The runtime contract says:

---

On the initial deployment, platform providers SHOULD start an instance of the container to validate that the container is valid and will become ready. This startup SHOULD occur even if the container would not serve any user requests.

---

Notice the SHOULDs in there. They are not MUSTs. Therefore it is optional. We should allow for the user to leverage this optionality. I'm not sure I agree with the SHOULDs in the contract, I think even that's too strong, I'd prefer a MAY, but SHOULD still gives the flexibility that we need.

[mattmoor]

ok, I'll elaborate on "fundamental".

I'm reminded of the time that someone on the cloud dataflow team checked in a python syntax error that caused their container to crash on startup. You might not think that's such a big deal, but their integration testing environment decided to let 1000 or so VMs sit in this state overnight set to imagePullPolicy: Always (as in, pull the image every container restart). All was good on the GCR side, but I was pretty stressed out for a bit because GCR's QPS went up 10x for about 20 hours (for them it was devel, so who cares?). This was also in the fun days before the kubelet had rate limits.

I'm also reminded of a time before GCR was public that a PR was merged into the github repo backing the google/docker-registry image, which at the time was configured with automated builds to :latest. At the time this was used by a fledgling App Engine Flex product (before it was called that), which noticed the change and proceeded to pull it across the fleet. Turns out this was in the days of docker registry v1, and the OSS docker-registry was Python. Another syntax error, but this one actually managed to take down DockerHub for a few hours. This outage was effectively the reason imagePullPolicy: exists in Kubernetes, and has such a terrible default value (that makes me sad).

This type of error is really real and is all too common in dynamic languages, which are quite popular in the serverless space. However, statically typed languages aren't immune to boot-time failures either. You could omit a required flag, listen on the wrong port, or even do something as simple as the notorious multiple definitions of flag "log_dir".

We can even attack this from a higher level than language: what if I request a new revision with resource limits larger than can be possible scheduled on node in my cluster? What if I deploy a windows container to a linux cluster?

I could probably go on all night. The point of all of these is that if you don't boot the container to the point of a successful readiness probe before declaring it Ready and rolling it out, you'd miss every last one of these errors, and if no traffic ever comes in then the prior working versions might get GC'd before the user even realizes something is amiss.

---

MAY clearly sends the wrong message here, frankly SHOULD sends the wrong message (and it does surprise me that we've written SHOULD vs. MUST). However, in my experience you should take this SHOULD to mean: you'd be totally crazy not to, and I think we should seriously consider making it MUST.

That said, changing SHOULD -> MUST is easier (breaking for vendor conformance), but MUST -> SHOULD is harder (breaking for customer applications).