Skip to content

[docker log] Adjust matcher for corner cases + UT#6296

Merged
prognant merged 4 commits intomasterfrom
prognant/docker-matcher-adjusmtent
Sep 2, 2020
Merged

[docker log] Adjust matcher for corner cases + UT#6296
prognant merged 4 commits intomasterfrom
prognant/docker-matcher-adjusmtent

Conversation

@prognant
Copy link
Copy Markdown
Contributor

@prognant prognant commented Aug 28, 2020

What does this PR do?

Rewrite the tailing-from-docker-socket-end-of-line-matcher

Motivation

The old implementation was missing some cases leading to split lines.
It was also doing append(...) on potentially large chunk of data.

Additional Notes

Not 100% sure, but when tailing from the docker socket, instead of
splitting message based on \n it may sound better to split message
every docker header and do the line re-assembling later (as it may
be required in other cases, as per #6265 & #6266)

Describe your test plan

Added various UT, attempted to cover most of realistic use cases.
See AC-184 for a Dockerfile that was exhibiting the issue and gets fixed by this PR.

@prognant prognant requested review from a team as code owners August 28, 2020 14:39
@prognant prognant added this to the 7.23.0 milestone Aug 28, 2020
@prognant prognant added changelog/no-changelog No changelog entry needed [deprecated] team/agent-core Deprecated. Use metrics-logs / shared-components labels instead.. labels Aug 28, 2020
Copy link
Copy Markdown
Member

@truthbk truthbk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice stuff, tricky little bug. Just a couple questions, nits.

Comment thread pkg/logs/input/docker/matcher.go Outdated
Comment thread pkg/logs/input/docker/matcher.go Outdated
}
return bs[i-len(exists)]
}
return 0xFF
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the only thing I don't like about this - and I totally understand why we're doing this because the function interface is so much more usable than a (byte, error) return type - and it might honestly never be a problem given how this function is used in matchHeader(), is that 0xFF could be considered a valid byte value. I will leave this to your best judgement, but in any case we should add a comment to reflect 0xFF is the return code when overflowing exists+bs.

Copy link
Copy Markdown
Contributor Author

@prognant prognant Aug 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I switched getByte to checkByte and wrapped the equality assertion inside the function. IMHO it keeps the code simple while avoiding the case where it could return 0xFF for a non-existing byte.

d.Stop()
}

func TestDecoderDetectMultipleDockerHeader(t *testing.T) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍰

Comment thread pkg/logs/input/docker/matcher_test.go Outdated
@prognant prognant requested a review from truthbk August 31, 2020 09:57
@prognant prognant merged commit 79d5546 into master Sep 2, 2020
@prognant prognant deleted the prognant/docker-matcher-adjusmtent branch September 2, 2020 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/no-changelog No changelog entry needed [deprecated] team/agent-core Deprecated. Use metrics-logs / shared-components labels instead..

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants