From fff8074ccf3142c373629f80db07799f5482bcba Mon Sep 17 00:00:00 2001 From: Shriram Rajagopalan Date: Sun, 29 Jan 2017 21:40:25 -0500 Subject: [PATCH 1/2] merge traffic splitting/shifting docs --- .../http_conn_man/route_config/route.rst | 6 +- .../route_config/route_config.rst | 3 +- .../route_config/traffic_shifting.rst | 49 ----- .../route_config/traffic_splitting.rst | 192 ++++++++++++++++++ .../route_config/weighted_routing.rst | 120 ----------- docs/intro/arch_overview/http_routing.rst | 6 +- 6 files changed, 201 insertions(+), 175 deletions(-) delete mode 100644 docs/configuration/http_conn_man/route_config/traffic_shifting.rst create mode 100644 docs/configuration/http_conn_man/route_config/traffic_splitting.rst delete mode 100644 docs/configuration/http_conn_man/route_config/weighted_routing.rst diff --git a/docs/configuration/http_conn_man/route_config/route.rst b/docs/configuration/http_conn_man/route_config/route.rst index 9872b8cfb1fd6..7e3a735cbf4da 100644 --- a/docs/configuration/http_conn_man/route_config/route.rst +++ b/docs/configuration/http_conn_man/route_config/route.rst @@ -49,7 +49,7 @@ cluster *path_redirect* is not specified), one of *cluster* or *weighted_clusters* must be specified. With the *weighted_clusters* option, multiple upstream clusters can be specified for a given route. The request is forwarded to one of the upstream clusters based on weights assigned - to each cluster. See :ref:`traffic splitting ` + to each cluster. See :ref:`traffic splitting ` for additional documentation. .. _config_http_conn_man_route_table_route_host_redirect: @@ -122,7 +122,9 @@ Runtime ------- A :ref:`runtime ` route configuration can be used to roll out route changes -in a gradual manner without full code/config deploys. +in a gradual manner without full code/config deploys. Refer to +:ref:`traffic shifting ` docs for examples +for additional documentation. .. code-block:: json diff --git a/docs/configuration/http_conn_man/route_config/route_config.rst b/docs/configuration/http_conn_man/route_config/route_config.rst index a37ef752f3fb6..d8317c8069a89 100644 --- a/docs/configuration/http_conn_man/route_config/route_config.rst +++ b/docs/configuration/http_conn_man/route_config/route_config.rst @@ -55,5 +55,4 @@ response_headers_to_remove vcluster rate_limits route_matching - traffic_shifting - weighted_routing + traffic_splitting diff --git a/docs/configuration/http_conn_man/route_config/traffic_shifting.rst b/docs/configuration/http_conn_man/route_config/traffic_shifting.rst deleted file mode 100644 index 69c4e1911fd24..0000000000000 --- a/docs/configuration/http_conn_man/route_config/traffic_shifting.rst +++ /dev/null @@ -1,49 +0,0 @@ -.. _config_http_conn_man_route_table_traffic_shifting: - -Traffic Shifting -================ - -Envoy's router can accomplish traffic shifting between routes of a virtual host -based on the :ref:`runtime ` object in the route configuration. -A common use case is shifting traffic between clusters with different versions of a service deployed on them. - -.. code-block:: json - - { - "route_config": { - "virtual_hosts": [ - { - "name": "service", - "domains": ["*"], - "routes": [ - { - "prefix": "/", - "cluster": "service_v1", - "runtime": { - "key": "routing.traffic_shift.service", - "default": 50 - } - }, - { - "prefix": "/", - "cluster": "service_v2", - } - ] - } - ] - } - } - -Envoy matches routes with a :ref:`first match ` policy. -If the route has a runtime object, the request will be additionally matched based on the runtime -:ref:`value ` -(or the default, if no value is specified). Thus, by placing routes back-to-back in the above example and specifying -a runtime object in the first route, traffic shifting can be accomplished by changing the runtime value. The flow would -look something like this: - -1. Set ``routing.traffic_shift.service`` to ``100``. This would mean that all requests to the ``service`` virtual host - would match with the v1 route. -2. Set ``routing.traffic_shift.service`` to values ``0 < x < 100``. For instance at ``50``, half of the requests to the - ``service`` virtual host will not match the v1 route and fall through to the v2 route. -3. Set ``routing.traffic_shift.service`` to ``0``. This means no requests to the ``service`` virtual host will match - to the v1 route and they will all fall through to the v2 route. diff --git a/docs/configuration/http_conn_man/route_config/traffic_splitting.rst b/docs/configuration/http_conn_man/route_config/traffic_splitting.rst new file mode 100644 index 0000000000000..8d37040e0173d --- /dev/null +++ b/docs/configuration/http_conn_man/route_config/traffic_splitting.rst @@ -0,0 +1,192 @@ +.. _config_http_conn_man_route_table_traffic_splitting: + +Traffic Shifting/Splitting +=========================================== + +Envoy's router can split traffic to a route in a virtual host across +two or more upstream clusters. There are two common use cases. The first use +case is version upgrades, where traffic to a route is shifted gradually +from one cluster to another. The +:ref:`traffic shifting ` +section describes this scenario in more detail. + +The second use case, as discussed in the +:ref:`traffic splitting ` +section is A/B testing or multivariate testing, where ``two or more versions`` of +the same service are tested simultaneously. The traffic to the route has to +be *split* between clusters running different versions of the same service. + +.. _config_http_conn_man_route_table_traffic_splitting_shift: + +Traffic shifting between two upstreams +====================================== + +The :ref:`runtime ` object +in the route configuration determines the probability of selecting a +particular route (and hence its cluster). By using the runtime +configuration, traffic to a particular route in a virtual host can be +gradually shifted from one cluster to another. Consider the following +example configuration, where two versions ``helloworld_v1`` and +``helloworld_v2`` of a service named ``helloworld`` are declared in the +envoy configuration file. + +.. code-block:: json + + { + "route_config": { + "virtual_hosts": [ + { + "name": "helloworld", + "domains": ["*"], + "routes": [ + { + "prefix": "/", + "cluster": "helloworld_v1", + "runtime": { + "key": "routing.traffic_shift.helloworld", + "default": 50 + } + }, + { + "prefix": "/", + "cluster": "helloworld_v2", + } + ] + } + ] + } + } + +Envoy matches routes with a :ref:`first match ` policy. +If the route has a runtime object, the request will be additionally matched based on the runtime +:ref:`value ` +(or the default, if no value is specified). Thus, by placing routes +back-to-back in the above example and specifying a runtime object in the +first route, traffic shifting can be accomplished by changing the runtime +value. The following are the approximate sequence of actions required to +accomplish the task. + +1. In the beginning, set ``routing.traffic_shift.helloworld`` to ``100``, + so that all requests to the ``helloworld`` virtual host would match with + the v1 route and be served by the ``helloworld_v1`` cluster. +2. To start shifting traffic to ``helloworld_v2`` cluster, set + ``routing.traffic_shift.helloworld`` to values ``0 < x < 100``. For + instance at ``90``, 1 out of every 10 requests to the ``helloworld`` + virtual host will not match the v1 route and will fall through to the v2 + route. +3. Gradually decrease the value set in ``routing.traffic_shift.helloworld`` + so that a larger percentage of requests match the v2 route. +4. When ``routing.traffic_shift.helloworld`` is set to ``0``, no requests + to the ``helloworld`` virtual host will match to the v1 route. All + traffic would now fall through to the v2 route and be served by the + ``helloworld_v2`` cluster. + + +.. _config_http_conn_man_route_table_traffic_splitting_split: + +Traffic splitting across multiple upstreams +=========================================== + +Consider the ``helloworld`` example again with three versions (v1, v2 and +v3). Envoy provides two ways to split traffic evenly across the three +versions (i.e., ``33%, 33%, 34%``): + +.. _config_http_conn_man_route_table_traffic_splitting_split_percentages: + +(a) Weight-based cluster selection +---------------------------------- + +The first option is to use a **single** :ref:`route ` with +:ref:`weighted_clusters `, +where multiple upstream cluster targets are specified for a single route, +along with weights that indicate the **percentage** of traffic to be sent +to each upstream cluster. + +.. code-block:: json + + { + "route_config": { + "virtual_hosts": [ + { + "name": "helloworld", + "domains": ["*"], + "routes": [ + { + "prefix": "/", + "weighted_clusters": { + "runtime_key_prefix" : "routing.traffic_split.helloworld", + "clusters" : [ + { "name" : "helloworld_v1", "weight" : 33 }, + { "name" : "helloworld_v2", "weight" : 33 }, + { "name" : "helloworld_v3", "weight" : 34 } + ] + } + } + ] + } + ] + } + } + +The weights assigned to each cluster can be dynamically adjusted using the +following runtime variables: ``routing.traffic_split.helloworld.helloworld_v1``, +``routing.traffic_split.helloworld.helloworld_v2`` and +``routing.traffic_split.helloworld.helloworld_v3``. + +.. _config_http_conn_man_route_table_traffic_splitting_split_probabilities: + +(b) Probabilistic route selection +--------------------------------- + +The second option is to use **multiple** :ref:`routes ` +as shown in the traffic shifting example, with :ref:`runtimes ` +that specify the **probability** of selecting a route. +Since Envoy matches routes with a :ref:`first match ` +policy, the related routes (one for each upstream cluster) must be placed back-to-back, +along with a runtime in all but the last route. + +.. code-block:: json + + { + "route_config": { + "virtual_hosts": [ + { + "name": "helloworld", + "domains": ["*"], + "routes": [ + { + "prefix": "/", + "cluster": "helloworld_v1", + "runtime": { + "key": "routing.traffic_split.helloworld.helloworld_v1", + "default": 33 + } + }, + { + "prefix": "/", + "cluster": "helloworld_v2", + "runtime": { + "key": "routing.traffic_split.helloworld.helloworld_v2", + "default": 50 + } + }, + { + "prefix": "/", + "cluster": "helloworld_v3", + } + ] + } + ] + } + } + +In the configuration above, + +1. ``routing.traffic_split.helloworld.helloworld_v1`` is set to ``33``, so that there is a + *33\% probability* that the v1 route will be selected by Envoy. +2. ``routing.traffic_split.helloworld.helloworld_v2`` is set to ``50``, so that if the v1 route + is not selected, between v2 and v3, there is a *50\% probability* that the v2 route will + be selected by Envoy. If v2 is not selected the traffic falls through to the v3 route. + +This distribution of probabilities ensures that the traffic will be split evenly across +all three routes (i.e. ``33%, 33%, 34%``). diff --git a/docs/configuration/http_conn_man/route_config/weighted_routing.rst b/docs/configuration/http_conn_man/route_config/weighted_routing.rst deleted file mode 100644 index c6f252b0ae60c..0000000000000 --- a/docs/configuration/http_conn_man/route_config/weighted_routing.rst +++ /dev/null @@ -1,120 +0,0 @@ -.. _config_http_conn_man_route_table_weighted_routing: - -Traffic splitting across multiple upstreams -=========================================== - -Envoy's router can split traffic to a route in a virtual host across -multiple upstream clusters. There are two common use cases. The first use -case is version upgrades, where traffic to a route is shifted gradually -from one cluster to another. The -:ref:`traffic shifting ` -describes this scenario in more detail. - -The second use case, discussed in this section, is A/B testing or multivariate -testing, where ``two or more versions`` of the same service are tested simultaneously. -The traffic to the route has to be *split* between clusters running different versions -of the same service. - -Consider a simple example ``service`` with three versions (v1, v2 and -v3). Envoy provides two ways to split traffic evenly across the three -versions (i.e., ``33%, 33%, 34%``): - -.. _config_http_conn_man_route_table_weighted_routing_percentages: - -(a) Weight-based cluster selection ----------------------------------- - -The first option is to use a **single** :ref:`route ` with -:ref:`weighted_clusters `, -where multiple upstream cluster targets are specified for a single route, -along with weights that indicate the **percentage** of traffic to be sent -to each upstream cluster. - -.. code-block:: json - - { - "route_config": { - "virtual_hosts": [ - { - "name": "service", - "domains": ["*"], - "routes": [ - { - "prefix": "/", - "weighted_clusters": { - "runtime_key_prefix" : "routing.traffic_split.service", - "clusters" : [ - { "name" : "service_v1", "weight" : 33 }, - { "name" : "service_v2", "weight" : 33 }, - { "name" : "service_v3", "weight" : 34 } - ] - } - } - ] - } - ] - } - } - -The weights assigned to each cluster can be dynamically adjusted using the -following runtime variables: ``routing.traffic_split.service.service_v1``, -``routing.traffic_split.service.service_v2`` and -``routing.traffic_split.service.service_v3``. - -.. _config_http_conn_man_route_table_weighted_routing_probabilities: - -(b) Probabilistic route selection ---------------------------------- - -The second option is to use **multiple** :ref:`routes ` -with :ref:`runtimes ` that specify the -**probability** of selecting a route. -Since Envoy matches routes with a :ref:`first match ` -policy, the related routes (one for each upstream cluster) must be placed back-to-back, -along with a runtime in all but the last route. - -.. code-block:: json - - { - "route_config": { - "virtual_hosts": [ - { - "name": "service", - "domains": ["*"], - "routes": [ - { - "prefix": "/", - "cluster": "service_v1", - "runtime": { - "key": "routing.traffic_split.service.service_v1", - "default": 33 - } - }, - { - "prefix": "/", - "cluster": "service_v2", - "runtime": { - "key": "routing.traffic_split.service.service_v2", - "default": 50 - } - }, - { - "prefix": "/", - "cluster": "service_v3", - } - ] - } - ] - } - } - -In the configuration above, - -1. ``routing.traffic_split.service.service_v1`` is set to ``33``, so that there is a - *33\% probability* that the v1 route will be selected by Envoy. -2. ``routing.traffic_split.service.service_v2`` is set to ``50``, so that if the v1 route - is not selected, between v2 and v3, there is a *50\% probability* that the v2 route will - be selected by Envoy. If v2 is not selected the traffic falls through to the v3 route. - -This distribution of probabilities ensures that the traffic will be split evenly across -all three routes (i.e. ``33%, 33%, 34%``). diff --git a/docs/intro/arch_overview/http_routing.rst b/docs/intro/arch_overview/http_routing.rst index 5a93ce91c2c7e..2804c7910a356 100644 --- a/docs/intro/arch_overview/http_routing.rst +++ b/docs/intro/arch_overview/http_routing.rst @@ -27,8 +27,10 @@ request. The router filter supports the following features: * Request timeout specified either via :ref:`HTTP header ` or via :ref:`route configuration `. -* Runtime configuration routing rules. Runtime configuration can be used to create :ref:`traffic - shifting `. +* Probability-based route selection via :ref:`runtimes ` + (see :ref:`traffic shifting/shifting `). +* :ref:`Weight/percentage-based ` upstream cluster selection + for a given route (see :ref:`traffic shifting/splitting `). * Arbitrary headers :ref:`routing rules `. * Virtual cluster specifications. A virtual cluster is specified at the virtual host level and is used by Envoy to generate additional statistics on top of the standard cluster level ones. Virtual From 101c0a5e8aa811c451ee8e43a6610779a57d4b9d Mon Sep 17 00:00:00 2001 From: Shriram Rajagopalan Date: Mon, 30 Jan 2017 16:37:50 -0500 Subject: [PATCH 2/2] addressing comments --- .../http_conn_man/route_config/route.rst | 2 +- .../route_config/traffic_splitting.rst | 14 ++++++++------ docs/intro/arch_overview/http_routing.rst | 2 +- 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/configuration/http_conn_man/route_config/route.rst b/docs/configuration/http_conn_man/route_config/route.rst index 7e3a735cbf4da..8a5f42830dffe 100644 --- a/docs/configuration/http_conn_man/route_config/route.rst +++ b/docs/configuration/http_conn_man/route_config/route.rst @@ -123,7 +123,7 @@ Runtime A :ref:`runtime ` route configuration can be used to roll out route changes in a gradual manner without full code/config deploys. Refer to -:ref:`traffic shifting ` docs for examples +:ref:`traffic shifting ` docs for additional documentation. .. code-block:: json diff --git a/docs/configuration/http_conn_man/route_config/traffic_splitting.rst b/docs/configuration/http_conn_man/route_config/traffic_splitting.rst index 8d37040e0173d..c8d90b58b14b2 100644 --- a/docs/configuration/http_conn_man/route_config/traffic_splitting.rst +++ b/docs/configuration/http_conn_man/route_config/traffic_splitting.rst @@ -4,17 +4,19 @@ Traffic Shifting/Splitting =========================================== Envoy's router can split traffic to a route in a virtual host across -two or more upstream clusters. There are two common use cases. The first use -case is version upgrades, where traffic to a route is shifted gradually +two or more upstream clusters. There are two common use cases. + +1. Version upgrades: traffic to a route is shifted gradually from one cluster to another. The :ref:`traffic shifting ` section describes this scenario in more detail. -The second use case, as discussed in the -:ref:`traffic splitting ` -section is A/B testing or multivariate testing, where ``two or more versions`` of +2. A/B testing or multivariate testing: ``two or more versions`` of the same service are tested simultaneously. The traffic to the route has to -be *split* between clusters running different versions of the same service. +be *split* between clusters running different versions of the same +service. The +:ref:`traffic splitting ` +section describes this scenario in more detail. .. _config_http_conn_man_route_table_traffic_splitting_shift: diff --git a/docs/intro/arch_overview/http_routing.rst b/docs/intro/arch_overview/http_routing.rst index 2804c7910a356..52795e2807e69 100644 --- a/docs/intro/arch_overview/http_routing.rst +++ b/docs/intro/arch_overview/http_routing.rst @@ -27,7 +27,7 @@ request. The router filter supports the following features: * Request timeout specified either via :ref:`HTTP header ` or via :ref:`route configuration `. -* Probability-based route selection via :ref:`runtimes ` +* Probability-based route selection via :ref:`runtime values ` (see :ref:`traffic shifting/shifting `). * :ref:`Weight/percentage-based ` upstream cluster selection for a given route (see :ref:`traffic shifting/splitting `).