Buffer: Implement tracking for BufferMemoryAccounts in WatermarkFactory. by KBaichoo · Pull Request #17093 · envoyproxy/envoy

KBaichoo · 2021-06-22T21:22:30Z

Signed-off-by: Kevin Baichoo kbaichoo@google.com

Commit Message: Implement Tracking for BufferMemoryAccounts in Watermark Factory.
Additional Description: This PR tracks memory accounts using >1MB of allocated space, with feedback mechanisms based on credits and debits on accounts. It further creates the handle from which the BufferMemoryAccount can reset the stream, and has the WatermarkBufferFactory also produce the particular BufferMemoryAccountImpl used for tracking.
Risk Level: Medium
Testing: Unit and Integration test
Docs Changes: NA
Release Notes: NA -- not yet user facing
Platform Specific Features: NA
Runtime guard: Yes, envoy.test_only.per_stream_buffer_accounting from #16218 sufficient
Related Issue #15791

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo · 2021-06-22T21:22:52Z

/assign @antoniovicente

antoniovicente

Here's some initial feedback. Looks like a good start.

/wait

antoniovicente · 2021-06-22T22:33:03Z

+   *
+   * @param reason the reason for reseting the stream.
+   */
+  virtual void resetStream(Http::StreamResetReason reason) PURE;


Consider changing the name to resetDownstream(reason) or resetDownstreamDueToOverload()

Changed to resetDownstream

antoniovicente · 2021-06-22T22:39:24Z

+  /**
+   * Unregister a buffer memory account.
+   */
+  virtual void unregisterAccount(const BufferMemoryAccountSharedPtr& account) PURE;


I don't think these two methods need to be part of the general factory interface. I think accounts can depend on the concrete implementation in order to do a more optimized account balance update only in cases where there's a significant change in account balance which requires moving the account from one bucket to another. By having some of the update logic in the account proper we can optimize away unnecessary calls to update the balance.

SGTM. Moved the BufferMemoryAccountImpl to the WatermarkBuffer impl's file, since it's the factory that produces that object. This inturn allows us to more easily couple between the two, while external classes use there interfaces.

antoniovicente · 2021-06-22T22:41:19Z

    hdrs = ["buffer_impl.h"],
    deps = [
        "//envoy/buffer:buffer_interface",
+        "//envoy/http:codec_interface",


I think this depends on stream_reset_handler_interface, not codec_interface

Whoops, must of got mixed up when breaking this out. Done.

antoniovicente · 2021-06-22T23:16:28Z

+    ++class_idx;
+  }
+
+  return class_idx;


Consider the following version. Also, consider that class_idx in your implementation is simply the location of the most significant bit that is set. See std::bit_width or std::countl_zero or the abseil equivalents (since I think c++20 is not allowed yet) for optimized O(1) MSB computation functions.

int class_idx = absl::bit_width(shifted_balance); return std::min(class_idx, size_class_account_sets_.size() - 1);

Neat, done.

antoniovicente · 2021-06-22T23:19:42Z

  done_notification.WaitForNotification();
 }

+void TrackedWatermarkBufferFactory::inspectMemoryClasses(


function seems unused.

Used in test/common/buffer/buffer_memory_account_test.cc

antoniovicente · 2021-06-22T23:20:43Z

 };

 using WatermarkBufferPtr = std::unique_ptr<WatermarkBuffer>;
+using MemoryClassesToAccountsSet = std::array<absl::flat_hash_set<BufferMemoryAccountSharedPtr>, 8>;


Please move type definition to the protected section below, just before it is used.

using MemoryClassesToAccountsSet = std::array<absl::flat_hash_set<BufferMemoryAccountSharedPtr>, 8>; MemoryClassesToAccountsSet size_class_account_sets_;

antoniovicente · 2021-06-22T23:23:13Z

+
+protected:
+  // Enable subclasses to inspect the mapping.
+  MemoryClassesToAccountsSet size_class_account_sets_;


Verify in WatermarkBufferFactory destructor that size_class_account_sets_ is empty

abstract interfaces coupling the concrete impls. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo

Thanks for the feedback @antoniovicente, addressed most of your comments

KBaichoo · 2021-06-24T12:50:25Z

+   *
+   * @param reason the reason for reseting the stream.
+   */
+  virtual void resetStream(Http::StreamResetReason reason) PURE;


Changed to resetDownstream

KBaichoo · 2021-06-24T12:54:09Z

  done_notification.WaitForNotification();
 }

+void TrackedWatermarkBufferFactory::inspectMemoryClasses(


Used in test/common/buffer/buffer_memory_account_test.cc

KBaichoo · 2021-06-24T13:00:22Z

+
+protected:
+  // Enable subclasses to inspect the mapping.
+  MemoryClassesToAccountsSet size_class_account_sets_;


KBaichoo · 2021-06-24T13:02:33Z

 };

 using WatermarkBufferPtr = std::unique_ptr<WatermarkBuffer>;
+using MemoryClassesToAccountsSet = std::array<absl::flat_hash_set<BufferMemoryAccountSharedPtr>, 8>;


KBaichoo · 2021-06-24T13:17:31Z

+    ++class_idx;
+  }
+
+  return class_idx;


Neat, done.

KBaichoo · 2021-06-24T14:35:37Z

    hdrs = ["buffer_impl.h"],
    deps = [
        "//envoy/buffer:buffer_interface",
+        "//envoy/http:codec_interface",


Whoops, must of got mixed up when breaking this out. Done.

KBaichoo · 2021-06-24T17:41:30Z

+  /**
+   * Unregister a buffer memory account.
+   */
+  virtual void unregisterAccount(const BufferMemoryAccountSharedPtr& account) PURE;


SGTM. Moved the BufferMemoryAccountImpl to the WatermarkBuffer impl's file, since it's the factory that produces that object. This inturn allows us to more easily couple between the two, while external classes use there interfaces.

KBaichoo · 2021-06-24T19:17:32Z

+  return BufferMemoryAccountImpl::createAccount(this, reset_handler);
+}
+
+void WatermarkBufferFactory::onAccountBalanceUpdate(const BufferMemoryAccountSharedPtr& account,


Would be worried about lifetime issues that might occur from this.

KBaichoo · 2021-06-24T19:42:34Z

+                                                    uint64_t prior_balance) {
+
+  int prev_idx = accountBalanceToClassIndex(prior_balance);
+  int new_idx = accountBalanceToClassIndex(account->balance());


Now use something like updateAccountClass.

cases. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

antoniovicente · 2021-06-28T17:44:29Z

Please address the magic number issues

/wait

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo · 2021-06-29T20:45:03Z

/retest

repokitteh-read-only · 2021-06-29T20:45:07Z

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #17093 (comment) was created by @KBaichoo.

see: more, trace.

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

antoniovicente · 2021-06-30T05:43:45Z

+  /**
+   *@return the balance of the account.
+   */
+  virtual uint64_t balance() const PURE;


I see no non-test uses of this API method. Consider removing from the interface.

antoniovicente · 2021-06-30T05:53:22Z

+  ASSERT(reset_handler_ != nullptr);
+  reset_handler_ = nullptr;
+
+  factory_->unregisterAccount(shared_this_, current_bucket_idx_);


current_bucket_idx_ does not contain the right value after unregistration, please add:

current_bucket_idx_ = -1;

antoniovicente · 2021-06-30T06:01:33Z

+  if (shared_this_ && new_class != current_bucket_idx_) {
+    factory_->updateAccountClass(shared_this_, current_bucket_idx_, new_class);
+    current_bucket_idx_ = new_class;
+  }


The previous 5 lines are repeated in the charge method below. Consider refactoring to an updateAccountClass() private method which you can call in the two places.

antoniovicente · 2021-06-30T06:09:56Z

+   *
+   * @param reason the reason for reseting the stream.
+   */
+  virtual void resetDownstream(Http::StreamResetReason reason) PURE;


nit: reset reason could be omitted, resetting via this interface implies that the termination is related to overload.

antoniovicente · 2021-06-30T06:17:59Z

+  uint64_t shifted_balance = buffer_memory_allocated_ >> 20; // shift by 1MB.
+
+  if (shifted_balance == 0) {
+    return -1; // Not worth tracking anything < 1MB.


You're not allowing for a lot of configuration for this accounting. Will we need some configuration in the future?

Note that the recommended stream buffer limits for edge configurations are around 64kb. Tracking high memory connections at 1MB or larger leaves a fairly wide gap of ~16x before this mechanism can terminate expensive streams. It seems very unlikely that we'll have streams using >1MB given the 64kb edge config stream limit.

We will need some configuration of this probably. Shall I do it in this PR or a follow up? Plumbing from the bootstrap to the watermark factory might require a good amount of changes.

I'm thinking that exposing the starting_bucket_size which is rounded to nearest power of two should be sufficient, as with 8 buckets the largest bucket will be 128x the smallest. WDYT?

TODO(kbaichoo): Either implement in this PR or follow up PR.

I would consider requiring that the config be powers of two. I do think that the addition of config will force you to undo some of the micro optimizations in this component.

Implemented this in KBaichoo#105

antoniovicente · 2021-06-30T06:22:07Z

+  // and shared_this_.
+  static BufferMemoryAccountSharedPtr createAccount(WatermarkBufferFactory* factory,
+                                                    Http::StreamResetHandler* reset_handler);
+  ~BufferMemoryAccountImpl() override { ASSERT(buffer_memory_allocated_ == 0); }


ASSERT(reset_handler_ == nullptr); despite it being impossible for this to not be the case given use of shared_this_.

antoniovicente · 2021-06-30T06:26:01Z

+
+  void resetDownstream(Http::StreamResetReason reason) override {
+    if (reset_handler_ != nullptr) {
+      reset_handler_->resetStream(reason);


You may be making an assumption that reset_handler_->resetStream(reason); calls clearDownstream()

You may run into some problems in the watermark factory while iterating through accounts and calling resetDownstream if some implementation doesn't invoke clearDownstream() from resetStream. I'm fairly sure that some of the stream implementations like HTTP2 do not call clearDownstream() from resetStream since they use the deferred delete mechanism provided by the dispatcher. Another potential danger is you calling reset_handler_->resetStream(); on a stream that is waiting to be deleted via deferred deletion. If the stream doesn't correctly handle the case by ignoring the resetStream we may trigger a crash.

Great point. Will have it such that resetStream(reason) always calls clearDownstream() to avoid situations where a stream might be reset for a non-OM situation, and is not yet destroyed, but Envoy Overload resets it again.

To avoid the deferred deletion issue whenever the stream is added to a zombie stream list, make sure that clearDownstream has been called.

TODO(kbaichoo): implement.

antoniovicente · 2021-06-30T06:34:51Z

+  // be called, but the downstream stream isn't going to terminate soon
+  // such as StreamDecoderFilterCallbacks::recreateStream().
+  if (buffer_memory_account_) {
+    buffer_memory_account_->clearDownstream();


Doing this from the destructor may be too late and result in calls to resetStream after deferred deletion started. Consider hooking into StreamImpl::destroy which is called before scheduling the deferred deletion.

Great catch! Yes, this should happen in destroy s.t. we unlink the stream from the account at that point as we don't want it to reset the stream at that point.

antoniovicente · 2021-06-30T06:36:14Z

+  // be called, but the downstream stream isn't going to terminate soon
+  // such as StreamDecoderFilterCallbacks::recreateStream().
+  if (buffer_memory_account_) {
+    buffer_memory_account_->clearDownstream();


See comment in the H2 codec, it's not clear to me if this happens early enough.

You are correct, the QUIC streams go into a deferred deletion mechanism that periodically is cleared by an Envoy QUIC alarm. Will instead call clearDownstream when the stream is added to the list of zombie streams that no data should be re-transmitted for it.

TODO(kbaichoo): implement

interface cleanups. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

by clearing downstream when writeSideCloses (the stream shouldn't have additional writes on it). Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo · 2021-07-12T18:41:35Z

/retest

repokitteh-read-only · 2021-07-12T18:41:39Z

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #17093 (comment) was created by @KBaichoo.

see: more, trace.

KBaichoo · 2021-07-13T13:30:34Z

/retest

repokitteh-read-only · 2021-07-13T13:30:38Z

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #17093 (comment) was created by @KBaichoo.

see: more, trace.

KBaichoo · 2021-07-19T15:29:23Z

Anything else @antoniovicente ? Thanks

antoniovicente · 2021-07-19T19:12:24Z

+
+  // Clear the downstream on the account since we're resetting the downstream.
+  if (buffer_memory_account_) {
+    buffer_memory_account_->clearDownstream();


Consider doing this before calling the base class method for safety.

Good idea, done.

antoniovicente · 2021-07-19T19:14:02Z

+  // Clear the downstream on the account since we're resetting the downstream.
+  if (buffer_memory_account_) {
+    buffer_memory_account_->clearDownstream();
+  }


clear before calling base class method may make sense. But isn't this covered by the call in ServerStreamImpl::destroy()?

Now call clear before the base class method.

StreamImpl::resetStream can sometimes defer the reset (if end stream already but not end stream sent). By having this in the SeverStreamImpl::resetStream, we know the BufferFactory won't try to reset the stream again since clearing downstream unregister's the account.

antoniovicente · 2021-07-19T19:42:34Z

  }
+  if (buffer_memory_account_) {
+    buffer_memory_account_->clearDownstream();
+  }


Consider doing this before reset callbacks.

Good idea, done.

derived classes for safety. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo

Thanks for the review @antoniovicente

KBaichoo · 2021-07-22T00:28:57Z

+
+  // Clear the downstream on the account since we're resetting the downstream.
+  if (buffer_memory_account_) {
+    buffer_memory_account_->clearDownstream();


Good idea, done.

KBaichoo · 2021-07-22T00:31:47Z

  }
+  if (buffer_memory_account_) {
+    buffer_memory_account_->clearDownstream();
+  }


Good idea, done.

KBaichoo · 2021-07-22T00:42:55Z

+  // Clear the downstream on the account since we're resetting the downstream.
+  if (buffer_memory_account_) {
+    buffer_memory_account_->clearDownstream();
+  }


Now call clear before the base class method.

StreamImpl::resetStream can sometimes defer the reset (if end stream already but not end stream sent). By having this in the SeverStreamImpl::resetStream, we know the BufferFactory won't try to reset the stream again since clearing downstream unregister's the account.

antoniovicente

Thanks for the improvement. Off to senior-maintainers for final review.

/assign-from @envoyproxy/senior-maintainers

antoniovicente · 2021-07-22T18:22:11Z

+  // be called, but the downstream stream isn't going to terminate soon
+  // such as StreamDecoderFilterCallbacks::recreateStream().
+  if (buffer_memory_account_) {
+    buffer_memory_account_->clearDownstream();


Also move before base class call

repokitteh-read-only · 2021-07-22T18:23:29Z

@envoyproxy/senior-maintainers assignee is @htuch

🐱

Caused by: a #17093 (review) was submitted by @antoniovicente.

see: more, trace.

htuch · 2021-07-23T02:28:24Z

+   * invoke to reset the stream.
+   * @return a BufferMemoryAccountSharedPtr of the newly created account.
+   */
+  virtual BufferMemoryAccountSharedPtr createAccount(Http::StreamResetHandler* reset_handler) PURE;


Nit: StreamResetHandler& (unless optional?)

htuch · 2021-07-23T02:34:28Z

+    }
+  }
+
+  // The number of memory classes the Account expects to exists.


Where is the memory class size scheme explained?

Added additional comments to WatermarkBufferFactory and BufferMemoryAccountImpl::balanceToClassIndex

htuch · 2021-07-23T02:39:06Z

-    downstream_request_account = std::make_shared<Buffer::BufferMemoryAccountImpl>();
+    // Create account, wiring the stream to use it.
+    auto& buffer_factory = read_callbacks_->connection().dispatcher().getWatermarkFactory();
+    downstream_request_account = buffer_factory.createAccount(&response_encoder.getStream());


Very dumb question - why are we talking about a downstream request account but all the wiring is happening for the response encoder?

The primary owner of the account right is the server-side(e.i. downstream) stream. Perhaps we should change "request" to "stream" in this variable name.

There's a desire to split the accounts used for the request and response directions, but have both of those accounts associated primarily with the downstream stream. But that change will happen in a future PR.

I've changed it to downstream_stream_account as suggested.

document WatermarkBufferFactory tracking. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo

Thanks for the reviews @antoniovicente and @htuch.

KBaichoo · 2021-07-26T15:16:27Z

+  // be called, but the downstream stream isn't going to terminate soon
+  // such as StreamDecoderFilterCallbacks::recreateStream().
+  if (buffer_memory_account_) {
+    buffer_memory_account_->clearDownstream();


KBaichoo · 2021-07-26T16:25:31Z

+    }
+  }
+
+  // The number of memory classes the Account expects to exists.


Added additional comments to WatermarkBufferFactory and BufferMemoryAccountImpl::balanceToClassIndex

KBaichoo · 2021-07-26T17:52:20Z

+   * invoke to reset the stream.
+   * @return a BufferMemoryAccountSharedPtr of the newly created account.
+   */
+  virtual BufferMemoryAccountSharedPtr createAccount(Http::StreamResetHandler* reset_handler) PURE;


KBaichoo · 2021-07-26T18:21:17Z

-    downstream_request_account = std::make_shared<Buffer::BufferMemoryAccountImpl>();
+    // Create account, wiring the stream to use it.
+    auto& buffer_factory = read_callbacks_->connection().dispatcher().getWatermarkFactory();
+    downstream_request_account = buffer_factory.createAccount(&response_encoder.getStream());


I've changed it to downstream_stream_account as suggested.

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo · 2021-07-29T21:07:57Z

/retest

repokitteh-read-only · 2021-07-29T21:08:00Z

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #17093 (comment) was created by @KBaichoo.

see: more, trace.

htuch

LGTM, thanks!

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo · 2021-07-30T18:33:20Z

verified examples kept failing, but merging in #17549 fixed it. think this is ready for merge, so I can put this follow up: KBaichoo#105 for review.

…ry. (envoyproxy#17093) This PR tracks memory accounts using >1MB of allocated space, with feedback mechanisms based on credits and debits on accounts. It further creates the handle from which the BufferMemoryAccount can reset the stream, and has the WatermarkBufferFactory also produce the particular BufferMemoryAccountImpl used for tracking. Risk Level: Medium Testing: Unit and Integration test Docs Changes: NA Release Notes: NA -- not yet user facing Platform Specific Features: NA Runtime guard: Yes, envoy.test_only.per_stream_buffer_accounting from envoyproxy#16218 sufficient Related Issue envoyproxy#15791 Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

Implement tracking for BufferMemoryAccounts in WatermarkFactory.

f811ac7

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo requested review from asraa, cpakulski, ggreenway and lizan as code owners June 22, 2021 21:22

repokitteh-read-only Bot assigned antoniovicente Jun 22, 2021

antoniovicente reviewed Jun 23, 2021

View reviewed changes

repokitteh-read-only Bot added the waiting label Jun 23, 2021

KBaichoo mentioned this pull request Jun 23, 2021

Refactor WatermarkFactory::create to WatermarkFactory::createBuffer. #17108

Merged

KBaichoo added 2 commits June 24, 2021 21:26

Move BufferMemoryAccountImpl with WatermarkBufferFactory to simplify the

9a06423

abstract interfaces coupling the concrete impls. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

merged main.

ea31002

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

repokitteh-read-only Bot removed the waiting label Jun 24, 2021

KBaichoo commented Jun 24, 2021

View reviewed changes

Clang-tidy, asan fixes, updateAccountClass broken into the 3 possible

2766dfa

cases. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo changed the title ~~WIP: Implement tracking for BufferMemoryAccounts in WatermarkFactory.~~ Buffer: Implement tracking for BufferMemoryAccounts in WatermarkFactory. Jun 25, 2021

KBaichoo commented Jun 25, 2021

View reviewed changes

Comment thread source/common/buffer/watermark_buffer.cc Outdated

repokitteh-read-only Bot added the waiting label Jun 28, 2021

KBaichoo added 2 commits June 29, 2021 17:12

Unmagic numbers related to the number of Memory Classes for accounts.

633aac4

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

Merge remote-tracking branch 'upstream/main' into acct-streams

1c3bfde

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

repokitteh-read-only Bot removed the waiting label Jun 29, 2021

Merge remote-tracking branch 'upstream/main' into acct-streams

8c30467

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

antoniovicente reviewed Jun 30, 2021

View reviewed changes

antoniovicente added the waiting label Jun 30, 2021

Iterate on feedback: no longer expose balance at Account Interface,

917e40f

interface cleanups. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

repokitteh-read-only Bot removed the waiting label Jul 7, 2021

resetStream clears downstream for H1/H2/H3. Handle H3 deferred deletion

b74e9bc

by clearing downstream when writeSideCloses (the stream shouldn't have additional writes on it). Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo mentioned this pull request Jul 13, 2021

Buffer Factory: Add configurability of BufferFactory. KBaichoo/envoy#105

Closed

yanavlasov self-assigned this Jul 14, 2021

antoniovicente reviewed Jul 19, 2021

View reviewed changes

antoniovicente added the waiting label Jul 19, 2021

KBaichoo added 2 commits July 22, 2021 00:47

Moved base method invokations in reset after clearing downstream in

ad8ff2d

derived classes for safety. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

Merge remote-tracking branch 'upstream/main' into acct-streams

a3dd9d8

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

repokitteh-read-only Bot removed the waiting label Jul 22, 2021

KBaichoo commented Jul 22, 2021

View reviewed changes

antoniovicente previously approved these changes Jul 22, 2021

View reviewed changes

repokitteh-read-only Bot assigned htuch Jul 22, 2021

htuch reviewed Jul 23, 2021

View reviewed changes

Minor changes: Keep optref to StreamResetHandler instead of pointer,

e2d1f50

document WatermarkBufferFactory tracking. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

KBaichoo dismissed antoniovicente’s stale review via e2d1f50 July 26, 2021 18:23

KBaichoo commented Jul 26, 2021

View reviewed changes

htuch suggested changes Jul 29, 2021

View reviewed changes

Comment thread source/common/buffer/watermark_buffer.h Outdated

Comment thread source/common/buffer/watermark_buffer.h Outdated

Comment thread source/common/buffer/watermark_buffer.cc Outdated

Comment thread source/common/buffer/watermark_buffer.cc

Update comment on memory classes.

5f62aae

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

htuch approved these changes Jul 30, 2021

View reviewed changes

KBaichoo added 2 commits July 30, 2021 13:16

Merge remote-tracking branch 'upstream/main' into acct-streams

afa1930

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

Merge remote-tracking branch 'upstream/main' into acct-streams

406bce4

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>

htuch merged commit 4506199 into envoyproxy:main Jul 30, 2021

Conversation

KBaichoo commented Jun 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KBaichoo commented Jun 22, 2021

Uh oh!

antoniovicente left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

KBaichoo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

antoniovicente commented Jun 28, 2021

Uh oh!

KBaichoo commented Jun 29, 2021

Uh oh!

repokitteh-read-only Bot commented Jun 29, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KBaichoo commented Jun 22, 2021 •

edited

Loading