Skip to content

Decouple proxy ingestion and interface waiting#1157

Closed
smola wants to merge 1 commit intomainfrom
smola/decouple-wait-and-ingest
Closed

Decouple proxy ingestion and interface waiting#1157
smola wants to merge 1 commit intomainfrom
smola/decouple-wait-and-ingest

Conversation

@smola
Copy link
Copy Markdown
Member

@smola smola commented May 5, 2023

Description

Motivation

Rather than stopping ingestion on each interface after each wait, we can separate both concerns by waiting for all conditions, then stop all watchdog events at once.

Additional notes

Rather than stopping ingestion on each interface after each wait, we can
separate both concerns by waiting for all conditions, then stop all
watchdog events at once.
Copy link
Copy Markdown
Collaborator

@cbeauchesne cbeauchesne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

I'm not sure to understand the motivation of this change.

All the purpose of chained wait/stop ingestion is a feature. There is a chain of component a -> b -> c; by chain, I mean the data flow is globally unidirectional, a sends to b, b sends to c.

The idea is that we wait for a, then actively stops a and then wait for b, etc

So every component will have its grace time to process all event received.

If we stops all watchdogs at once, we can have an event sent by a, that we won't see in b, if for any reason it kept it i a queue process.

@smola
Copy link
Copy Markdown
Member Author

smola commented May 10, 2023

I guess the change doesn't make sense standalone. I split this out while doing this POC: #1150

A grace period defined in time is prone to error: it's either too long most of the time and/or sometimes it's too short under some circumstances. So I'm experimenting with having an approach that is mostly time-independent and relies on precondition events happening, rather than hoping that implicit preconditions are met after a certain amount of time.

@smola
Copy link
Copy Markdown
Member Author

smola commented May 10, 2023

Closing, since there's no value in this without further changes to the setup/wait model.

@smola smola closed this May 10, 2023
@smola smola deleted the smola/decouple-wait-and-ingest branch May 10, 2023 12:44
@cbeauchesne
Copy link
Copy Markdown
Collaborator

You can adjust this time for any scenario. Now, about this :

So I'm experimenting with having an approach that is mostly time-independent and relies on precondition events happening

It's not that easy. There are two different use cases :

  1. I'm waiting for something to happen => if it does not happen, when should I stop waiting and consider it's a failure?
  2. I want to be sure that something does not happen => at which point should I stop waiting , and consider it's a success?

The golden bullet for this is having a way to know that tracer/agent have processed everything they have to process. But as now, AFAIK, such a thing does not exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants