Add CORS filter#1528
Conversation
|
@alyssawilk do you mind driving review of the CORS filter? |
alyssawilk
left a comment
There was a problem hiding this comment.
Looks good overall! Please correct any points of mine where I'm misunderstanding CORS - I'm not familar with it and only wanted to delay your review reading docs for so long :-P
| CORS filter | ||
| ==================== | ||
|
|
||
| This is a filter which handles CORS requests based on route or virtual host settings. |
There was a problem hiding this comment.
Mind spelling Cross-origin resource sharing out once before the first mention for those who haven't reached the link yet?
| { | ||
| "cors": { | ||
| "enabled": false, | ||
| "allow_origin": "", |
There was a problem hiding this comment.
I see other filters filling in pseudo values so folks can see an example of what it would look like (e.g. grpc_json_transcoder_filter.rst). Mind doing the same?
There was a problem hiding this comment.
Good point! Will amend.
| *(optional, boolean)* Defaults to false. | ||
|
|
||
| allow_origin | ||
| *(required, string)* The content of the access-control-allow-origin header. |
There was a problem hiding this comment.
super nitty nit, consider replacing "of the" with "for the" here and below since it's the content envoy will be adding not the content which is present int he header on arrival.
| */ | ||
| virtual const std::string& maxAge() const PURE; | ||
|
|
||
| /** |
| CorsFilter::~CorsFilter() {} | ||
|
|
||
| FilterHeadersStatus CorsFilter::decodeHeaders(HeaderMap& headers, bool) { | ||
| initialize(); |
There was a problem hiding this comment.
would it maks sense to make this initializeCorsPolicy and only call if the origin header is present on an OPTIONS request?
There was a problem hiding this comment.
That is definitely better!
I had it inside setDecoderFilterCallbacks first but got an exception because the route was not yet available. Then I moved it there and forgot about it >_<
| enabled | ||
| *(optional, boolean)* Defaults to false. | ||
|
|
||
| allow_origin |
There was a problem hiding this comment.
Curious why is this required when the code allows it to be empty?
There was a problem hiding this comment.
I wanted to get some feedback of what you think should be the required parameters. I think making enabled required would make sense too.
True, I didn't think about that much. I should have checked if it's the empty string.
| void CorsFilter::initialize() { | ||
| auto routeEntry = decoder_callbacks_->route()->routeEntry(); | ||
| const Envoy::Router::CorsPolicy& routeCorsPolicy = routeEntry->corsPolicy(); | ||
| const Envoy::Router::CorsPolicy& virtualHostCorsPolicy = routeEntry->virtualHost().corsPolicy(); |
There was a problem hiding this comment.
What about making these shared pointers to avoid all of the string copies? You could instead have helpers allowOrigin() allowCredentials() etc. which would pull the appropriate value from the config with precedence.
There was a problem hiding this comment.
I was worrying about the copying. Good point. I'll make some changes. Having the helpers I could make the code cleaner and get rid of the initializer. The reason I didn't do it was that I thought calling it several times would maybe add too much overhead, but thinking about it, origin() would be called twice and others would be called only once so maybe not that much of a problem after all!
| } | ||
| } | ||
|
|
||
| void CorsFilter::onDestroy() {} |
There was a problem hiding this comment.
Might as well leave this inline in the .h file
| const Envoy::Router::CorsPolicy& virtualHostCorsPolicy = routeEntry->virtualHost().corsPolicy(); | ||
|
|
||
| if (routeCorsPolicy.enabled() && virtualHostCorsPolicy.enabled()) { | ||
| cors_enabled_ = true; |
There was a problem hiding this comment.
If we opt to stick with the code below, any reason to not early return if cors is not enabled?
| Settings | ||
| -------- | ||
|
|
||
| Settings on a route take precedence over settings on the virtual host. |
There was a problem hiding this comment.
That's surprising to me. I would have assumed global config was overriden by per-cluster was overriden by per-host in most cases. Unfortunately I'm having trouble telling the order from existing docs. @mattklein123 can you confirm this is consistent?
There was a problem hiding this comment.
Maybe I mistook it. Here is how I understood it:
{
"virtual_host": {
...
"cors": {},
"routes": [{"prefix": "...", "cors": {...}}]
}
The CORS inside would take precedence.
No problem to change it though. What I didn't like much about the description in the issue is that you have to enable CORS explicitly on the virtual_host and on the route because it says if either of them is off => it's off.
Maybe there is a better way?
There was a problem hiding this comment.
Route should override virtual host. IMO if either of them is set it's on.
| bool allowCredentials() const override { return allow_credentials_; }; | ||
| bool enabled() const override { return enabled_; }; | ||
| /* | ||
| MOCK_CONST_METHOD0(allowOrigin, const std::string&()); |
There was a problem hiding this comment.
TODO: remove this commented out code
| Http::TestHeaderMapImpl request_headers{{"method", "options"}}; | ||
|
|
||
| EXPECT_EQ(FilterHeadersStatus::Continue, filter_.decodeHeaders(request_headers, false)); | ||
| // EXPECT_EQ(false, filter_.is_cors_request_); |
There was a problem hiding this comment.
@alyssawilk is there any way to do this check? Maybe only by having a public accessor?
There was a problem hiding this comment.
Thinking about it i'll go with friend class CorsFilterTest; is that ok?
There was a problem hiding this comment.
Good question - I'm new enough I'm not entirely sure of the answer :-P I'm fan of test peers and I'd be inclined to claim DnsResolverImplPeer is precedent (and I prefer having Test in the name) enough to say go for it, but @mattklein123 may have a different opinion :-)
There was a problem hiding this comment.
A friend class is fine if you really need it, however, in this case my preference would be to just test the outward behaviors. Meaning, you don't really care how the filter works, you just care that it's not handling in this case and returning continue. So that would be what I would do. I don't have a very strong preference though.
There was a problem hiding this comment.
I was doing that first but then I had the problem that in almost all cases it returns continue and I cannot test with confidence if it really went into the specific if case. I also think black box testing is ok in general. But since there is a spec for when to return I wanted to test that somehow. If it's just an implementation detail I wouldn't want to test it.
I have some problems with the tests now. While testing in my dev environment works some tests fail.
I never heard the term test peers and Google didn't bring anything up. But I guess it means friending the class? Judging from the name. :)
| const ConfigImpl& globalRouteConfig() const { return global_route_config_; } | ||
|
|
||
| // Router::VirtualHost | ||
| const CorsPolicy& corsPolicy() const override { return cors_policy_; } |
There was a problem hiding this comment.
I was never sure if the ordering is important. BUILD files where alphabetical but the code wasn't.
There was a problem hiding this comment.
I don't think we bother for function names. includes and buildfiles yes though :-)
|
|
||
| HeaderMapPtr response_headers{ | ||
| new HeaderMapImpl{{Headers::get().Status, std::to_string(enumToInt(Http::Code::OK))}}}; | ||
|
|
There was a problem hiding this comment.
@codesuki check here if the Origin header value matches the one in the config, no?
There was a problem hiding this comment.
something like:
if (allow_origin_.find(*origin_) == std::string::npos) {
return FilterHeadersStatus::Continue;
}
response_headers->insertAccessControlAllowOrigin().value(*origin_);
if (allow_credentials_) {
response_headers->insertAccessControlAllowCredentials().value(
Http::Headers::get().CORSValues.True);
}There was a problem hiding this comment.
Thanks! Your comment made me re-read the RFC. I first thought we can use regexes but doesn't seem so after all. I'll read the spec once more and make some changes.
There was a problem hiding this comment.
Per w3c the header must contain the domain that was sent as the Origin header for the request to be accepted. How the server decides how to accept that string is implementation dependent.
|
I am on vacation this week. Will push the changes next week! |
alyssawilk
left a comment
There was a problem hiding this comment.
Looking much cleaner!
Have you considered adding an integration test? It would be good to have some end to end coverage of things working as expected. I don't think it's necessary to have end-to-end coverage of the interplay between global and route config but iff you want to I'd suggest extending the proto_integration_test once your API changes are merged, since it's easier to add multiple tests with the same config in that one. If you just want to test one config end to end (which is totally fine given the thorough unit testing) you can just extend the appropriate test/config/integration/*.json and the corresponding integration test.
| CorsFilterConfigConstSharedPtr config_; | ||
| StreamDecoderFilterCallbacks* decoder_callbacks_{}; | ||
| StreamEncoderFilterCallbacks* encoder_callbacks_{}; | ||
| const Envoy::Router::CorsPolicy* routeCorsPolicy_{}; |
There was a problem hiding this comment.
member variables get snake case so route_cors_policy_{} and virtual_host_cors_policy_{}
Meanwhile Envoy isn't big on raw pointers where avoidable* and I'm unsure this would be safe if there were a config reload. Should they be shared pointers instead?
*there's totally precedent for the Encoder/Decoder callbacks being raw pointers so don't worry about those.
There was a problem hiding this comment.
I have a shared pointer version stashed. Let me apply and push it. I hesitated adding it because no other policy / config used it. I added it as shared pointers all the way from config_impl router::routeentry and router::virtualhost. Not sure that's all correct.
| const Envoy::Router::CorsPolicy* virtualHostCorsPolicy_{}; | ||
| bool is_cors_request_{}; | ||
| Http::HeaderEntry* origin_{}; | ||
| std::string origin__{}; |
There was a problem hiding this comment.
I think this is now unused. If you end up using, I'd suggest origin_header_ and origin_value_ or some such - if you have conflicting names it's better to make them more descriptive :-)
There was a problem hiding this comment.
Sorry, accidentally stashed those changes.
| void initialize(); | ||
|
|
||
| // Http::StreamFilterBase | ||
| void onDestroy() override{}; |
There was a problem hiding this comment.
I think clang_format will require a space after override.
There was a problem hiding this comment.
nit: remove ';' after definition (will fix Alyssa's comment)
| const ConfigImpl& globalRouteConfig() const { return global_route_config_; } | ||
|
|
||
| // Router::VirtualHost | ||
| const CorsPolicy& corsPolicy() const override { return cors_policy_; } |
There was a problem hiding this comment.
I don't think we bother for function names. includes and buildfiles yes though :-)
|
Very good, I'll add an integration test :) |
Now it is possible to disable CORS for specific routes even if there is a CORS policy on the virtual host.
|
I tried adding integration tests but got some compile errors. Any ideas? I am inheriting from the BaseIntegrationTest that has the members. Maybe I am too tired. BUILD |
|
Have you pulled from and merged with upstream/master recently? You can
check to see if your local copy of integration.h has the additions from PR
1532 <#1532>
…On Wed, Sep 6, 2017 at 4:42 AM, Neri Marschik ***@***.***> wrote:
I tried adding integration tests but got some compile errors. Any ideas? I
am inheriting from the BaseIntegrationTest that has the members. Maybe I am
too tired.
test/integration/cors_filter_integration_test.cc:32:23: error: use of undeclared identifier 'codec_client_'
codec_client_ =
^
test/integration/cors_filter_integration_test.cc:36:23: error: use of undeclared identifier 'request_encoder_'
request_encoder_ = &codec_client_->startRequest(
^
test/integration/cors_filter_integration_test.cc:36:43: error: use of undeclared identifier 'codec_client_'
request_encoder_ = &codec_client_->startRequest(
^
test/integration/cors_filter_integration_test.cc:41:28: error: use of undeclared identifier 'response_'
*response_);
^
test/integration/cors_filter_integration_test.cc:44:37: error: use of undeclared identifier 'codec_client_'
[&]() -> void { codec_client_->sendData(*request_encoder_, 0, true); },
^
test/integration/cors_filter_integration_test.cc:44:62: error: use of undeclared identifier 'request_encoder_'
[&]() -> void { codec_client_->sendData(*request_encoder_, 0, true); },
^
test/integration/cors_filter_integration_test.cc:45:37: error: use of undeclared identifier 'response_'
[&]() -> void { response_->waitForEndStream(); },
^
test/integration/cors_filter_integration_test.cc:46:37: error: use of undeclared identifier 'codec_client_'
[&]() -> void { codec_client_->close(); }});
^
test/integration/cors_filter_integration_test.cc:48:17: error: use of undeclared identifier 'response_'
EXPECT_TRUE(response_->complete());
^
test/integration/cors_filter_integration_test.cc:49:25: error: use of undeclared identifier 'response_'
EXPECT_STREQ("200", response_->headers().Status()->value().c_str());
#include "test/integration/integration.h"
#include "test/mocks/http/mocks.h"
#include "gtest/gtest.h"
namespace Envoy {
class CorsFilterIntegrationTest : public BaseIntegrationTest,
public testing::TestWithParam<Network::Address::IpVersion> {
public:
CorsFilterIntegrationTest() : BaseIntegrationTest(GetParam()) {}
/**
* Global initializer for all integration tests.
*/
void SetUp() override {
fake_upstreams_.emplace_back(new FakeUpstream(0, FakeHttpConnection::Type::HTTP2, version_));
registerPort("upstream_0", fake_upstreams_.back()->localAddress()->ip()->port());
createTestServer("test/config/integration/server_cors_filter.json", {"http"});
}
/**
* Global destructor for all integration tests.
*/
void TearDown() override {
test_server_.reset();
fake_upstreams_.clear();
}
protected:
void testTest() {
executeActions({[&]() -> void {
codec_client_ =
makeHttpConnection(lookupPort("http"), Http::CodecClient::Type::HTTP2);
},
[&]() -> void {
request_encoder_ = &codec_client_->startRequest(
Http::TestHeaderMapImpl{{":method", "GET"},
{":path", "/healthcheck"},
{":scheme", "http"},
{":authority", "host"}},
*response_);
},
[&]() -> void { codec_client_->sendData(*request_encoder_, 0, true); },
[&]() -> void { response_->waitForEndStream(); },
[&]() -> void { codec_client_->close(); }});
EXPECT_TRUE(response_->complete());
EXPECT_STREQ("200", response_->headers().Status()->value().c_str());
}
}; // namespace Envoy
INSTANTIATE_TEST_CASE_P(IpVersions, CorsFilterIntegrationTest,
testing::ValuesIn(TestEnvironment::getIpVersionsForTest()));
TEST_P(CorsFilterIntegrationTest, TestCors) { testTest(); }
} // namespace Envoy
BUILD
envoy_cc_test(
name = "cors_filter_integration_test",
srcs = [
"cors_filter_integration_test.cc"
],
data = [
"//test/config/integration:server_cors_filter.json",
],
deps = [
":integration_lib",
"//source/common/buffer:buffer_lib",
"//source/common/http:header_map_lib",
"//test/mocks/http:http_mocks",
"//test/mocks/upstream:upstream_mocks",
"//test/test_common:utility_lib",
],
)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1528 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ARYFvfV-ZuVAuxDLw632xVNTFXwSI46Lks5sflrsgaJpZM4PBM4J>
.
|
|
Thanks! I noticed there were some issues with the integration tests. I'll try later! |
|
Tests passing now. |
alyssawilk
left a comment
There was a problem hiding this comment.
LGTM. @mattklein123 mind taking a look now?
mattklein123
left a comment
There was a problem hiding this comment.
Overall excellent work, thank you. Some random comments to get started.
| The first route that matches will be used. | ||
|
|
||
| :ref:`cors <config_http_filters_cors>` | ||
| *(optional, object)* Specifies the route's CORS policy. |
There was a problem hiding this comment.
"Specifies the virtual host's ..."
|
|
||
| This is a filter which handles Cross-Origin Resource Sharing requests based on route or virtual host settings. | ||
| For the meaning of the headers please refer to the pages below. | ||
| https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS |
There was a problem hiding this comment.
nit: I would potentially use bulleted list here. (Either way take a look at generated docs locally to make sure they are to your liking).
| .. code-block:: json | ||
|
|
||
| { | ||
| "type": "both", |
There was a problem hiding this comment.
"type" is deprecated now so just go ahead and kill from docs.
| } | ||
|
|
||
| enabled | ||
| *(optional, boolean)* Defaults to false. Setting *enabled* to false on a route disables CORS |
There was a problem hiding this comment.
Q: Is this the most common case for this field? Or do you think we should potentially default to true? (I'm fine with whatever just want to make sure we default to the most common thing).
There was a problem hiding this comment.
Sorry! This should say defaults to true.
| for this route only. The setting has no effect on a virtual host. | ||
|
|
||
| allow_origin | ||
| *(optional, string)* The origins that will be allowed to do CORS request. |
There was a problem hiding this comment.
Is this a comma delimited list? Can you specify? Same for other fields below where this applies.
| }; | ||
|
|
||
| void CorsFilter::initialize() { | ||
| route_cors_policy_ = decoder_callbacks_->route()->routeEntry()->corsPolicy(); |
There was a problem hiding this comment.
Both of these calls aren't always safe. The route() might be empty. It might also be a redirect. You need to handle both of these cases.
There was a problem hiding this comment.
I will skip the filter in case it's a redirect, since the final host should probably handle that. Is that ok?
There was a problem hiding this comment.
Yes, that's what I would do. (Skip of no route or redirect)
| return FilterHeadersStatus::Continue; | ||
| } | ||
|
|
||
| auto requestMethod = headers.AccessControlRequestMethod(); |
There was a problem hiding this comment.
nit: const auto request_method
|
|
||
| is_cors_request_ = true; | ||
|
|
||
| auto method = headers.Method(); |
|
|
||
| bool CorsFilter::isOriginAllowed(Http::HeaderString& origin) { | ||
| for (const auto& o : allowOrigin()) { | ||
| if (o == "*" || origin == o.c_str()) { |
There was a problem hiding this comment.
Looking at this review now. If you want to use a map here, please use a map. I'm not sure where anyone is getting the impression that I don't like maps! :)
| // Default timeout is 15s if nothing is specified in the route config. | ||
| static const uint64_t DEFAULT_ROUTE_TIMEOUT_MS = 15000; | ||
|
|
||
| CorsPolicySharedPtr cors_policy_; |
There was a problem hiding this comment.
per other comments, I think if we remove shared_ptr, this can become unique_ptr
There was a problem hiding this comment.
With unique pointer do you mean the CorsPolicy* you wrote above or unique_ptr?
There was a problem hiding this comment.
I just meant that internally you can store as unique_ptr<CorsPolicy) then just return cors_policy_.get()
There was a problem hiding this comment.
Understood. If using new then use unique_ptr. But since I will just initialize it in the constructor I don't need new so I will just make it a member like RetryPolicy etc.
( https://github.com/lyft/envoy/blob/master/source/common/router/config_impl.h#L327-L333 )
|
Thanks for the review! The points I didn't reply to I will fix directly. |
|
All points should be addressed. |
mattklein123
left a comment
There was a problem hiding this comment.
Looks good. Few small comments and then good to go. Note that there are 2 PRs that will merge today (integration test and access log one which will cause you to have to do a few small merges. Sorry).
| expose_headers_ = config.expose_headers(); | ||
| max_age_ = config.max_age(); | ||
| allow_credentials_ = PROTOBUF_GET_WRAPPED_OR_DEFAULT(config, allow_credentials, false); | ||
| enabled_ = PROTOBUF_GET_WRAPPED_OR_DEFAULT(config, enabled, true); |
There was a problem hiding this comment.
Unless I'm missing something, isn't this going to make a fully empty CORS config be "enabled" which will kick on the filter? It seems like if the user does't specify any CORS config at all the default instantiation should be for it to be disabled?
There was a problem hiding this comment.
True! ... is there any way to detect absence of config? Maybe by making it an unique_ptr after all?
Setting it to false would mean that a user has to enable it on every route even if all the settings are contained on the vhost.
There was a problem hiding this comment.
You could make it a unique_ptr and return a const pointer. The issue though is when you translate v1 JSON config to v2. You will probably have to pass a pointer to a v2 object vs. reference, and then pivot on whether it is nullptr or not.
| @@ -0,0 +1,31 @@ | |||
| #include "server/config/http/cors.h" | |||
|
|
|||
| #include <chrono> | |||
| #include "envoy/registry/registry.h" | ||
|
|
||
| #include "common/http/filter/cors_filter.h" | ||
| #include "common/json/config_schemas.h" |
There was a problem hiding this comment.
nit: not needed (maybe others also, please clean up BUILD file also)
There was a problem hiding this comment.
Oh, sorry. I'll clean that up. Totally used to go cleaning up imports by itself.
| HttpFilterFactoryCb createFilterFactory(const Json::Object& json_config, | ||
| const std::string& stats_prefix, | ||
| FactoryContext& context) override; | ||
| std::string name() override { return "cors"; } |
| decoder_callbacks_ = &callbacks; | ||
| }; | ||
|
|
||
| void CorsFilter::initialize() { |
There was a problem hiding this comment.
nit: Since this is only called from decodeHeaders(), can we inline this code there?
|
|
||
| void CorsFilter::initialize() { | ||
| if (decoder_callbacks_->route() == nullptr || | ||
| decoder_callbacks_->route()->redirectEntry() != nullptr || |
There was a problem hiding this comment.
nit: redirectEntry() vs. routeEntry() is guaranteed to be XOR, so you can just check that routeEntry() != nullptr.
| return FilterHeadersStatus::StopIteration; | ||
| } | ||
|
|
||
| FilterHeadersStatus CorsFilter::encodeHeaders(HeaderMap& headers, bool) { |
There was a problem hiding this comment.
Can you add a small comment here on why we need to do this in encodeHeaders()? (Either link to spec or just tiny explanation for future readers).
| CorsFilter::CorsFilter() : is_cors_request_(false) {} | ||
|
|
||
| FilterHeadersStatus CorsFilter::decodeHeaders(HeaderMap& headers, bool) { | ||
| origin_ = headers.Origin(); |
There was a problem hiding this comment.
Per my other comment about default init of the policy, IMO, the first thing we should do here is the current contents of initialize(), then, if not enabled, quickly continue.
|
Addressed all of the above I think. |
mattklein123
left a comment
There was a problem hiding this comment.
Looks solid. Great contribution. Few small comments then ready to go.
| } | ||
|
|
||
| origin_ = headers.Origin(); | ||
| if (origin_ == nullptr || origin_->value() == "") { |
There was a problem hiding this comment.
nit: origin_->value().empty()
There was a problem hiding this comment.
Sorry, have to focus more.
| } | ||
|
|
||
| const auto requestMethod = headers.AccessControlRequestMethod(); | ||
| if (requestMethod == nullptr || requestMethod->value() == "") { |
|
|
||
| StreamDecoderFilterCallbacks* decoder_callbacks_{}; | ||
| StreamEncoderFilterCallbacks* encoder_callbacks_{}; | ||
| std::vector<const Envoy::Router::CorsPolicy*> policies_{}; |
There was a problem hiding this comment.
Perf nit: I would make this an std::array of fixed size 2, then just initialize in decodeHeaders().
There was a problem hiding this comment.
I had it as an array first because of that reason. Then I switched to vector to use push_back. I'll switch back.
| expose_headers_ = config.expose_headers(); | ||
| max_age_ = config.max_age(); | ||
| if (config.has_allow_credentials()) { | ||
| allow_credentials_.value(PROTOBUF_GET_WRAPPED_REQUIRED(config, allow_credentials)); |
There was a problem hiding this comment.
I'm a little confused as to why we need to make this optional. Can't you just default this to false?
There was a problem hiding this comment.
Premise: we check route first then fall back to vhost if not set.
If I default to false I cannot check if it's set or not. I could accept that it's false so the user would have to set true every time he adds a route config to change the headers or such. So I just made this behave the same.
We could change it back to default false and document it. Might be unexpected though behavior though :)
There was a problem hiding this comment.
No that's fine, was just curious. Sounds good.
There was a problem hiding this comment.
I might even want to make expose headers the same. Because it could make sense to set it to an empty string. Have to confirm. The others make no sense to be empty I think. (Headers+methods+age)
mattklein123
left a comment
There was a problem hiding this comment.
awesome work. Thanks for working on this.
**Description** https://aigateway.envoyproxy.io/docs/0.3/getting-started/prerequisites has error in the doc mentioning v0.3.0 version for envoy gateway helm chart, it needs to be 1.5.0 instead **Related Issues/PRs (if applicable)** N/A **Special notes for reviewers (if applicable)** N/A --------- Signed-off-by: Deepak Deore <deepakdeore2004@gmail.com>
Fixes #300
Implemented according to the issue.
There are some things to discuss. For example should max_age be an integer and should the headers, origins be a list or a string.
Also I am not sure the C++ code is nice, didn't use it for a couple of years.