Support both Knative and standalone

[jroper]

We should support either working within Knative, or as standalone.

Currently, an operator that works with Knative revisions has been implemented, though it is probably in a non working state since we stopped work on integrating with Knative. We've created a patch for Knative that allows our operator to take over managing the deployment:

knative/serving#4152

That patch has been rejected due to undocumented tight coupling with the autoscaler, and we have found that the Knative autoscaler isn't suitable for scaling Akka clusters anyway, so work needs to be continued on Knative to disable the autoscaler for certain revisions too (and possibly the activator). But we will also need to make our proxy possible to work with either Knative or stand alone. To work with Knative, we need the following:

• The autoscaler needs to be multi deployment aware. Currently, it only works with one deployment, but every new version of a Knative service that is deployed results in a new deployment. To support this, the autoscaler needs to probably do something along the lines of work out the proportion of traffic going to each node, and distribute scaling across the deployments accordingly. So, if 25% of traffic is going to deployment A, and 75% to deployment B, and the autoscaler decides that it needs 8 nodes to handle its load, then deployment A should be scaled to 2 nodes, and deployment B should be scaled to 6 nodes.
• We need a way to be able to switch Knative vs standalone modes for the proxy.

[viktorklang]

@jroper Is this issue still relevant?

[kwehden]

I think we should pick this back up, now that there is discussions on pluggable autoscaling, also custom metrics support: https://knative.dev/docs/serving/custom-metrics-api/