From 7f103abe6055e4dfe9ad4c4063f3e0565a855f83 Mon Sep 17 00:00:00 2001 From: Jose Nino Date: Mon, 30 Jan 2017 13:19:08 -0800 Subject: [PATCH 1/7] Add README --- README.md | 268 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 268 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 000000000..d8738dfcc --- /dev/null +++ b/README.md @@ -0,0 +1,268 @@ + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [Overview](#overview) +- [Building and Testing](#building-and-testing) +- [Configuration](#configuration) + - [The configuration format](#the-configuration-format) + - [Definitions](#definitions) + - [Format overview](#format-overview) + - [Example 1](#example-1) + - [Example 2](#example-2) + - [Example 3](#example-3) + - [Descriptor list definition](#descriptor-list-definition) + - [Rate limit definition](#rate-limit-definition) + - [Loading Configuration](#loading-configuration) +- [Rate limit statistics](#rate-limit-statistics) +- [Debug Port](#debug-port) + + + +# Overview + +The rate limit service is a Go/GRPC service designed to enable generic rate limit scenarios from different types of +applications. Applications request a rate limit decision based on a domain and a set of descriptors. The service +reads the configuration from disk via [runtime](https://github.com/lyft/goruntime), composes a cache key, and talks to the redis cache. A +decision is then returned to the caller. + +# Building and Testing + +* Install redis-server +* Make sure go is setup correctly and checkout rate limit service into your go path +* In order to run the integration tests using a local default redis install you will also need these environment variables set: +``` +export REDIS_SOCKET_TYPE=tcp +export REDIS_URL=localhost:6379 +``` +* To setup for the first time (only done once): +``` +make bootstrap +``` +* To compile: +``` +make compile +``` +* To compile and run tests: +``` +make tests +``` +* To run the server locally using some sensible default settings you can do this (this will setup the server to read the configuration files from the path you specify): +``` +USE_STATSD=false LOG_LEVEL=debug REDIS_SOCKET_TYPE=tcp REDIS_URL=localhost:6379 RUNTIME_ROOT=/home/user/src/runtime/data RUNTIME_SUBDIRECTORY=ratelimit +``` + +# Configuration + +## The configuration format + +### Definitions + +* **Domain:** A domain is a container for a set of rate limits. All domains known to the rate limit service must be +globally unique. They serve as a way for different teams/projects to have rate limit configurations that don't conflict. +* **Descriptor:** A descriptor is a list of key/value pairs owned by a domain that the rate limit service uses to +select the correct rate limit to use when limiting. Descriptors are case-sensitive. Examples of descriptors are: + * ("database", "users") + * ("message_type", "marketing"),("to_number","2061234567") + * ("to_cluster", "service_a") + * ("to_cluster", "service_a"),("from_cluster", "service_b") + +### Format overview + +#### Example 1 + +Let's start with a simple example: + +``` +domain: mongo_cps +descriptors: + - key: database + value: users + rate_limit: + unit: second + requests_per_unit: 500 + + - key: database + value: default + rate_limit: + unit: second + requests_per_unit: 500 +``` + +The rate limit configuration file format is YAML (mainly so that comments are supported). In the configuration above +the domain is "mongo_cps" and we setup 2 different rate limits in the top level descriptor list. Each of the limits +have the same key ("database"). They have a different value ("users", and "default"), and each of them setup a 500 +request per second rate limit. + +#### Example 2 + +A slightly more complex example: + +``` +domain: messaging +descriptors: + # Only allow 5 marketing messages a day + - key: message_type + value: marketing + descriptors: + - key: to_number + rate_limit: + unit: day + requests_per_unit: 5 + + # Only allow 100 messages a day to any unique phone number + - key: to_number + rate_limit: + unit: day + requests_per_unit: 100 +``` + +In the preceding example, the domain is "messaging" and we setup two different scenarios that illustrate more +complex functionality. First, we want to limit on marketing messages to a specific number. To enable this, we make +use of *nested descriptor lists.* The top level descriptor is ("message_type", "marketing"). However this descriptor +does not have a limit assigned so it's just a placeholder. Contained within this entry we have another descriptor list +that includes an entry with key "to_number". However, notice that no value is provided. This means that the service +will match against any value supplied for "to_number" and generate a unique limit. Thus, ("message_type", "marketing"), +("to_number", "2061111111") and ("message_type", "marketing"),("to_number", "2062222222") will each get 5 requests +per day. + +The configuration also sets up another rule without a value. This one creates an overall limit for messages sent to +any particular number during a 1 day period. Thus, ("to_number", "2061111111") and ("to_number", "2062222222") both +get 100 requests per day. + +When calling the rate limit service, the client can specify *multiple descriptors* to limit on in a single call. This +limits round trips and allows limiting on aggregate rule definitions. For example, using the preceding configuration, +the client could send this complete request (in pseudo IDL): + +``` +RateLimitRequest: + domain: messaging + descriptor: ("message_type", "marketing"),("to_number", "2061111111") + descriptor: ("to_number", "2061111111") +``` + +And the service with rate limit against *all* matching rules and return an aggregate result. + +#### Example 3 + +One last example to illustrate matching order. + +``` +domain: edge_proxy_per_ip +descriptors: + - key: ip_address + rate_limit: + unit: second + requests_per_unit: 10 + + # Black list IP + - key: ip_address + value: 50.0.0.5 + rate_limit: + unit: second + requests_per_unit: 0 +``` + +In the preceding example, we setup a generic rate limit for individual IP addresses. The architecture's edge proxy can +be configured to make a rate limitservice call with the descriptor ("ip_address", "50.0.0.1") for example. This IP would +get 10 requests per second as +would any other IP. However, the configuration also contains a second configuration that explicitly defines a +value along with the same key. If the descriptor ("ip_address", "50.0.0.5") is received, the service will +*attempt the most specific match possible*. This means both the most nested matching descriptor entry, as well as +the most specific at any descriptor list level. Thus, key/value is always attempted as a match before just key. + +### Descriptor list definition + +Each configuration contains a top level descriptor list and potentially multiple nested lists beneath that. The format is: + +``` +domain: +descriptors: + - key: + value: + rate_limit: (optional block) + unit: + requests_per_unit: + descriptors: (optional block) + ... (nested repitiion of above) +``` + +Each descriptor in a descriptor list must have a key. It can also optionally have a value to enable a more specific +match. The "rate_limit" block is optional and if present sets up an actual rate limit rule. See below for how the rule +is defined. The reason a rule might not be present is typically if a descriptor is a container for a 2nd level +descriptor list. Each descriptor can optionally contain a nested descriptor list that allows for more complex matches +and rate limit scenarios. + +### Rate limit definition + +``` +rate_limit: + unit: + requests_per_unit: +``` + +The rate limit block specifies the actual rate limit that will be used when there is a match. +Currently the service supports per second, minute, hour, and day limits. More types of limits may be added in the +future based on customer demand. + +## Loading Configuration + +The ratelimit service uses a library written by Lyft called goruntime to do configuration loading. Goruntime monitors +a designated path, and watches for symlink swaps to files in the directory tree to reload configuration files. + +The path to watch can be configured via the [settings](https://github.com/lyft/ratelimit/blob/master/src/settings/settings.go) +package with the following environment variables: + +``` +RUNTIME_ROOT default:"/srv/runtime_data/current"` +RUNTIME_SUBDIRECTORY +``` + +For more information on how runtime works you can read its [README](https://github.com/lyft/goruntime). + +# Rate limit statistics + +The rate limit service generates various statistics for each configured rate limit rule that will be useful for end +users both for visibility and for setting alarms. + +Rate Limit Statistic Path: + +``` +ratelimit.service.rate_limit.DOMAIN.KEY_VALUE.STAT +``` + +DOMAIN: +* As specified in the domain value in the YAML runtime file + +KEY_VALUE: +* A combination of the key value +* Nested descriptors would be suffixed in the stats path + +STAT: +* near_limit: Number of rule hits over the NearLimit ratio threshold (currently 80%) but under the threshold rate. +* over_limit: Number of rule hits exceeding the threshold rate +* total_hits: Number of rule hits in total + +These are examples of generated stats for some configured rate limit rules from the above examples: + +``` +ratelimit.service.rate_limit.mongo_cps.database_default.over_limit: 0 +ratelimit.service.rate_limit.mongo_cps.database_default.total_hits: 2846 +ratelimit.service.rate_limit.mongo_cps.database_users.over_limit: 0 +ratelimit.service.rate_limit.mongo_cps.database_users.total_hits: 2939 +ratelimit.service.rate_limit.messaging.message_type_marketing.to_number.over_limit: 0 +ratelimit.service.rate_limit.messaging.message_type_marketing.to_number.total_hits: 0 +``` + +# Debug Port + +The debug port can be used to interact with the running process. + +``` +$ curl 0:6070/ +/debug/pprof/: root of various pprof endpoints. hit for help. +/rlconfig: print out the currently loaded configuration for debugging +/stats: print out stats +``` + +You can specify the debug port with the `DEBUG_PORT` environment variable. It defaults to `6070`. From 4c7ea53f3e0bef0ff4f43fd768ff4ea6f480493d Mon Sep 17 00:00:00 2001 From: Jose Nino Date: Tue, 31 Jan 2017 17:32:13 -0800 Subject: [PATCH 2/7] Fix matching scheme --- README.md | 57 +++++++++++++++++++++++++++++++------- src/config/config_impl.go | 8 ++++-- test/config/config_test.go | 10 +++++++ 3 files changed, 63 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index d8738dfcc..030f1e037 100644 --- a/README.md +++ b/README.md @@ -73,7 +73,7 @@ select the correct rate limit to use when limiting. Descriptors are case-sensiti Let's start with a simple example: -``` +```yaml domain: mongo_cps descriptors: - key: database @@ -98,7 +98,7 @@ request per second rate limit. A slightly more complex example: -``` +```yaml domain: messaging descriptors: # Only allow 5 marketing messages a day @@ -145,15 +145,15 @@ And the service with rate limit against *all* matching rules and return an aggre #### Example 3 -One last example to illustrate matching order. +An example to illustrate matching order. -``` +```yaml domain: edge_proxy_per_ip descriptors: - key: ip_address - rate_limit: - unit: second - requests_per_unit: 10 + rate_limit: + unit: second + requests_per_unit: 10 # Black list IP - key: ip_address @@ -164,12 +164,49 @@ descriptors: ``` In the preceding example, we setup a generic rate limit for individual IP addresses. The architecture's edge proxy can -be configured to make a rate limitservice call with the descriptor ("ip_address", "50.0.0.1") for example. This IP would +be configured to make a rate limit service call with the descriptor ("ip_address", "50.0.0.1") for example. This IP would get 10 requests per second as would any other IP. However, the configuration also contains a second configuration that explicitly defines a value along with the same key. If the descriptor ("ip_address", "50.0.0.5") is received, the service will -*attempt the most specific match possible*. This means both the most nested matching descriptor entry, as well as -the most specific at any descriptor list level. Thus, key/value is always attempted as a match before just key. +*attempt the most specific match possible*. This means +the most specific descriptor at the same level as your request. Keep in mind that equally specific descriptors are matched on a first match basis. Thus, key/value is always attempted as a match before just key. + +#### Example 4 + +The Ratelimit service matches requests to configuration entries with the same depth level. For instance, the following request: + +``` +RateLimitRequest: + domain: example4 + descriptor: ("key", "value"),("subkey", "subvalue") +``` + +Would **not** match the following configuration even though the first descriptor in +the request matches the descriptor in the configuration. + +```yaml +domain: example4 +descriptors: + - key: key + value: value + rate_limit: + - requests_per_unit: 300 + unit: second +``` + +However, it would match the following configuration: + +```yaml +domain: example4 +descriptors: + - key: key + value: value + descriptors: + - key: subkey + rate_limit: + - requests_per_unit: 300 + unit: second +``` ### Descriptor list definition diff --git a/src/config/config_impl.go b/src/config/config_impl.go index a1fab23d6..fff76ea9d 100644 --- a/src/config/config_impl.go +++ b/src/config/config_impl.go @@ -251,7 +251,7 @@ func (this *rateLimitConfigImpl) GetLimit( } descriptorsMap := value.descriptors - for _, entry := range descriptor.Entries { + for i, entry := range descriptor.Entries { // First see if key_value is in the map. If that isn't in the map we look for just key // to check for a default value. finalKey := entry.Key + "_" + entry.Value @@ -265,7 +265,11 @@ func (this *rateLimitConfigImpl) GetLimit( if nextDescriptor != nil && nextDescriptor.limit != nil { logger.Debugf("found rate limit: %s", finalKey) - rateLimit = nextDescriptor.limit + if (i == len(descriptor.Entries) - 1) { + rateLimit = nextDescriptor.limit + } else { + logger.Debugf("request depth does not match config depth, there are more entries in the request's descriptor") + } } if nextDescriptor != nil && len(nextDescriptor.descriptors) > 0 { diff --git a/test/config/config_test.go b/test/config/config_test.go index a788c8a88..0443521e9 100644 --- a/test/config/config_test.go +++ b/test/config/config_test.go @@ -37,6 +37,16 @@ func TestBasicConfig(t *testing.T) { &pb.RateLimitDescriptor{[]*pb.RateLimitDescriptor_Entry{{"key1", "value1"}}}) assert.Nil(rl) + rl = rlConfig.GetLimit( + nil, "test-domain", + &pb.RateLimitDescriptor{[]*pb.RateLimitDescriptor_Entry{{"key2", "value2"}, {"subkey", "subvalue"}}}) + assert.Nil(rl) + + rl = rlConfig.GetLimit( + nil, "test-domain", + &pb.RateLimitDescriptor{[]*pb.RateLimitDescriptor_Entry{{"key5", "value5"}, {"subkey5", "subvalue"}}}) + assert.Nil(rl) + rl = rlConfig.GetLimit( nil, "test-domain", &pb.RateLimitDescriptor{ From 51c94ab2947f7497c91a0db1b0bd6daa1f838615 Mon Sep 17 00:00:00 2001 From: Jose Nino Date: Wed, 1 Feb 2017 11:15:35 -0800 Subject: [PATCH 3/7] address comments --- README.md | 9 ++++++--- test/config/basic_config.yaml | 12 ++++++++++++ 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 030f1e037..d7870673a 100644 --- a/README.md +++ b/README.md @@ -173,7 +173,9 @@ the most specific descriptor at the same level as your request. Keep in mind tha #### Example 4 -The Ratelimit service matches requests to configuration entries with the same depth level. For instance, the following request: +The Ratelimit service matches requests to configuration entries with the same depth level, i.e +same number of tuples in the request's descriptor as nested levels of descriptors +in the configuration file. For instance, the following request: ``` RateLimitRequest: @@ -181,8 +183,9 @@ RateLimitRequest: descriptor: ("key", "value"),("subkey", "subvalue") ``` -Would **not** match the following configuration even though the first descriptor in -the request matches the descriptor in the configuration. +Would **not** match the following configuration, even though the first descriptor in +the request matches the descriptor in the configuration, because the request has +two tuples in the descriptor. ```yaml domain: example4 diff --git a/test/config/basic_config.yaml b/test/config/basic_config.yaml index 4dec3c07d..24e43d091 100644 --- a/test/config/basic_config.yaml +++ b/test/config/basic_config.yaml @@ -40,3 +40,15 @@ descriptors: rate_limit: unit: day requests_per_unit: 1 + + - key: key5 + value: value5 + rate_limit: + unit: day + requests_per_unit: 15 + descriptors: + key: subkey5 + value: subvalue5 + rate_limit: + unit: day + requests_per_unit: 25 From 53b34a6a58832e51b77aa6c858d53127f1070b1e Mon Sep 17 00:00:00 2001 From: Jose Nino Date: Wed, 1 Feb 2017 11:19:56 -0800 Subject: [PATCH 4/7] fix --- test/config/basic_config.yaml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/test/config/basic_config.yaml b/test/config/basic_config.yaml index 24e43d091..29d3d56db 100644 --- a/test/config/basic_config.yaml +++ b/test/config/basic_config.yaml @@ -47,8 +47,8 @@ descriptors: unit: day requests_per_unit: 15 descriptors: - key: subkey5 - value: subvalue5 - rate_limit: - unit: day - requests_per_unit: 25 + - key: subkey5 + value: subvalue5 + rate_limit: + unit: day + requests_per_unit: 25 From 90fd9fb4de68bce73d8542d3a67c61d47dded23b Mon Sep 17 00:00:00 2001 From: Jose Nino Date: Thu, 2 Feb 2017 15:13:44 -0800 Subject: [PATCH 5/7] Update from comments --- README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 00686315c..3f0c91174 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ - [Example 1](#example-1) - [Example 2](#example-2) - [Example 3](#example-3) + - [Example 4](#example-4) - [Loading Configuration](#loading-configuration) - [Rate limit statistics](#rate-limit-statistics) - [Debug Port](#debug-port) @@ -31,7 +32,6 @@ decision is then returned to the caller. * Install redis-server. * Make sure go is setup correctly and checkout rate limit service into your go path. More information about installing go [here](https://golang.org/doc/install). - * In order to run the integration tests using a local default redis install you will also need these environment variables set: ``` export REDIS_SOCKET_TYPE=tcp @@ -62,9 +62,9 @@ The rate limit configuration file format is YAML (mainly so that comments are su ### Definitions -* **Domain:** A domain is a container for a set of rate limits. All domains known to the rate limit service must be +* **Domain:** A domain is a container for a set of rate limits. All domains known to the Ratelimit service must be globally unique. They serve as a way for different teams/projects to have rate limit configurations that don't conflict. -* **Descriptor:** A descriptor is a list of key/value pairs owned by a domain that the rate limit service uses to +* **Descriptor:** A descriptor is a list of key/value pairs owned by a domain that the Ratelimit service uses to select the correct rate limit to use when limiting. Descriptors are case-sensitive. Examples of descriptors are: * ("database", "users") * ("message_type", "marketing"),("to_number","2061234567") @@ -179,7 +179,8 @@ RateLimitRequest: descriptor: ("to_number", "2061111111") ``` -And the service with rate limit against *all* matching rules and return an aggregate result. +And the service with rate limit against *all* matching rules and return an aggregate result; a logical OR of all +the individual rate limit decisions. #### Example 3 @@ -207,11 +208,11 @@ get 10 requests per second as would any other IP. However, the configuration also contains a second configuration that explicitly defines a value along with the same key. If the descriptor ("ip_address", "50.0.0.5") is received, the service will *attempt the most specific match possible*. This means -the most specific descriptor at the same level as your request. Keep in mind that equally specific descriptors are matched on a first match basis. Thus, key/value is always attempted as a match before just key. +the most specific descriptor at the same level as your request. Thus, key/value is always attempted as a match before just key. #### Example 4 -The Ratelimit service matches requests to configuration entries with the same depth level, i.e +The Ratelimit service matches requests to configuration entries with the same level, i.e same number of tuples in the request's descriptor as nested levels of descriptors in the configuration file. For instance, the following request: From 0d1f99606b2eb1d44b771276d413ba73f1a5d40a Mon Sep 17 00:00:00 2001 From: Jose Nino Date: Fri, 3 Feb 2017 09:40:55 -0800 Subject: [PATCH 6/7] remove merge conflict --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 3f0c91174..aa2cb5318 100644 --- a/README.md +++ b/README.md @@ -253,7 +253,6 @@ descriptors: ## Loading Configuration The Ratelimit service uses a library written by Lyft called [goruntime](https://github.com/lyft/goruntime) to do configuration loading. Goruntime monitors ->>>>>>> master a designated path, and watches for symlink swaps to files in the directory tree to reload configuration files. The path to watch can be configured via the [settings](https://github.com/lyft/ratelimit/blob/master/src/settings/settings.go) From fe0e5dacf4d5bd1e8d6f4d5a7504ea8c968c6c32 Mon Sep 17 00:00:00 2001 From: Jose Nino Date: Fri, 3 Feb 2017 16:12:37 -0800 Subject: [PATCH 7/7] Update from comments --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index aa2cb5318..295e808bf 100644 --- a/README.md +++ b/README.md @@ -179,7 +179,7 @@ RateLimitRequest: descriptor: ("to_number", "2061111111") ``` -And the service with rate limit against *all* matching rules and return an aggregate result; a logical OR of all +And the service will rate limit against *all* matching rules and return an aggregate result; a logical OR of all the individual rate limit decisions. #### Example 3 @@ -222,8 +222,8 @@ RateLimitRequest: descriptor: ("key", "value"),("subkey", "subvalue") ``` -Would **not** match the following configuration, even though the first descriptor in -the request matches the descriptor in the configuration, because the request has +Would **not** match the following configuration. Even though the first descriptor in +the request matches the 1st level descriptor in the configuration, the request has two tuples in the descriptor. ```yaml