test: load testing tooling and initial observations by Kavindu-Dodan · Pull Request #225 · open-feature/flagd

Kavindu-Dodan · 2022-11-25T21:19:52Z

This PR

PR introduce tools to support performance testing for flagd. Content include,

A random feature flag generator
Sample K6 script
go pprof enabled flagd build with a dedicated Dockerfile (profile.Dockerfile)
Initial performance observations

Related Issues

PR is related to #212

How to test

Consider the readme contents at /test/loadtest/README.MD

Signed-off-by: Kavindu Dodanduwa <KavinduDodanduwa@gmail.com>

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>

beeme1mr

How was the loadTestResults.png created?

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>

Kavindu-Dodan · 2022-11-28T15:25:54Z

How was the loadTestResults.png created?

This is an extraction from the load test results [1]. If you are interested in how numbers were derived, it was by running a containerised flagd with random FFs (generated with ff_gen.go) and using the K6 script (included in the PR). Final result is the average of 3 test runs. And there was a warm-up time provided to flagd.

[1] https://docs.google.com/spreadsheets/d/1YLtkA-WryJXaZ4GWrbKhT_Izx3dh4spS/edit?usp=sharing&ouid=109303231969053490300&rtpof=true&sd=true

toddbaert · 2022-11-28T17:59:45Z

I think you should note somewhere that HTTP is used as the interface. Based on what I'm seeing, I think encoding and (de)serialization is probably where we are spending a lot of time. I think it would be ideal to run a suite in gRPC as well, using https://github.com/grpc/grpc/blob/master/doc/command_line_tool.md or https://github.com/fullstorydev/grpcurl, potentially. If we do this we'll want to make sure the same connection is used for all the RPCs. Do you think we should create an issue for this? Should we do it as part of this issue?

cc @beeme1mr

beeme1mr · 2022-11-28T18:08:15Z

It would be worth extending the tests to also include gRPC. It looks like K6 supports it natively.

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>

Kavindu-Dodan · 2022-11-28T22:17:39Z

I think you should note somewhere that HTTP is used as the interface. Based on what I'm seeing, I think encoding and (de)serialization is probably where we are spending a lot of time. I think it would be ideal to run a suite in gRPC as well, using https://github.com/grpc/grpc/blob/master/doc/command_line_tool.md or https://github.com/fullstorydev/grpcurl, potentially. If we do this we'll want to make sure the same connection is used for all the RPCs. Do you think we should create an issue for this? Should we do it as part of this issue?

cc @beeme1mr

Great input. I highlighted the API type used in the observations section (updated with latest commit).

Yes, testing and comparing against gRPC would be great. And yes, pprof proved the same point: most of the cpu time and memory is spent on reading and writing request

Co-authored-by: Skye Gill <gill.skye95@gmail.com> Signed-off-by: Kavindu Dodanduwa <Kavindu-Dodan@users.noreply.github.com>

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>

beeme1mr · 2022-12-01T17:58:23Z

@Kavindu-Dodan please resolve any conversations that have been addressed.

Kavindu-Dodan · 2022-12-01T18:36:47Z

@Kavindu-Dodan please resolve any conversations that have been addressed.

ACK. There's one more conversation to resolve and I am waiting for the response from @AlexsJones .

AlexsJones · 2022-12-05T10:11:34Z

@Kavindu-Dodan please resolve any conversations that have been addressed.

ACK. There's one more conversation to resolve and I am waiting for the response from @AlexsJones .

Sorry just remarked on it

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>

Kavindu-Dodan · 2022-12-06T16:04:37Z

@Kavindu-Dodan please resolve any conversations that have been addressed.

ACK. There's one more conversation to resolve and I am waiting for the response from @AlexsJones .

Sorry just remarked on it

Thanks and I updated the PR with the suggested change

@Kavindu-Dodan

@Kavindu-Dodan has contributed multiple significant changes and proposals to flagd: - multiple refactors: #291, #307 - ci/security improvements: #338, #337 - architectural proposals (some of which got some attention from outside parties!): open-feature/ofep#45, open-feature/flagd-schemas#78, #249 (comment) - load testing: #225 - documentation improvements For these reasons, I believe he should be made a CODEOWNER in this repository. NOTE: before this is merged, @Kavindu-Dodan should be added with at least `maintainer` permissions to the repo. Signed-off-by: Todd Baert <toddbaert@gmail.com>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Kavindu-Dodan requested a review from AlexsJones as a code owner November 25, 2022 21:19

Kavindu-Dodan added 2 commits November 25, 2022 13:22

init testing

07f833b

Signed-off-by: Kavindu Dodanduwa <KavinduDodanduwa@gmail.com>

cleanup

43574a6

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>

Kavindu-Dodan force-pushed the feat/load-test-init branch from d89531a to 43574a6 Compare November 25, 2022 21:23

tcarrio reviewed Nov 28, 2022

View reviewed changes

Comment thread loadtest/ff_gen.go Outdated

beeme1mr reviewed Nov 28, 2022

View reviewed changes

review correction

dcf222f

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>

beeme1mr self-requested a review November 28, 2022 15:53

beeme1mr approved these changes Nov 28, 2022

View reviewed changes

Specify API type tests were run against

3a4a2d4

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>

AlexsJones reviewed Nov 29, 2022

View reviewed changes

Comment thread Dockerfile Outdated

Comment thread loadtest/README.MD Outdated

Comment thread loadtest/README.MD Outdated

skyerus reviewed Nov 29, 2022

View reviewed changes

Comment thread loadtest/README.MD Outdated

Kavindu-Dodan and others added 2 commits November 29, 2022 08:05

Update loadtest/README.MD

13ebd47

Co-authored-by: Skye Gill <gill.skye95@gmail.com> Signed-off-by: Kavindu Dodanduwa <Kavindu-Dodan@users.noreply.github.com>

New readme content and restructure tests

59bc037

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>

beeme1mr mentioned this pull request Nov 30, 2022

flagD resource limits open-feature/open-feature-operator#222

Closed

Merge branch 'main' into feat/load-test-init

702a59a

Kavindu-Dodan requested a review from AlexsJones November 30, 2022 22:45

beeme1mr changed the title ~~feat: load testing tooling and initial observations~~ test: load testing tooling and initial observations Dec 1, 2022

Kavindu-Dodan added 2 commits December 5, 2022 08:14

Merge branch 'main' into feat/load-test-init

ba11838

Review change-seperate dockerfile for profiling

8e8f29e

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>

Kavindu-Dodan force-pushed the feat/load-test-init branch from 20bdba6 to 8e8f29e Compare December 5, 2022 18:34

Merge branch 'main' into feat/load-test-init

90405b7

AlexsJones requested changes Dec 7, 2022

View reviewed changes

Comment thread profiler.go

Kavindu-Dodan requested a review from AlexsJones December 8, 2022 15:43

skyerus approved these changes Dec 13, 2022

View reviewed changes

skyerus mentioned this pull request Dec 13, 2022

feat: introduce configurable resource limits for flagd sidecar open-feature/open-feature-operator#241

Merged

Merge branch 'main' into feat/load-test-init

4a231d7

beeme1mr merged commit 26de63e into open-feature:main Dec 13, 2022

Kavindu-Dodan deleted the feat/load-test-init branch December 13, 2022 17:52

toddbaert mentioned this pull request Feb 6, 2023

chore: add @kavindu-dodan to CODEOWNERS #377

Merged

raphael-wigoutschnigg-dt pushed a commit to open-feature-forking/flagd that referenced this pull request Mar 11, 2025

chore(main): release tests/flagd 1.2.3 (open-feature#225)

04184f9

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: load testing tooling and initial observations #225

test: load testing tooling and initial observations #225
beeme1mr merged 11 commits intoopen-feature:mainfrom
Kavindu-Dodan:feat/load-test-init

Kavindu-Dodan commented Nov 25, 2022 •

edited

Loading

Uh oh!

Uh oh!

beeme1mr left a comment

Uh oh!

Kavindu-Dodan commented Nov 28, 2022

Uh oh!

toddbaert commented Nov 28, 2022 •

edited

Loading

Uh oh!

beeme1mr commented Nov 28, 2022

Uh oh!

Kavindu-Dodan commented Nov 28, 2022 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

beeme1mr commented Dec 1, 2022

Uh oh!

Kavindu-Dodan commented Dec 1, 2022

Uh oh!

AlexsJones commented Dec 5, 2022

Uh oh!

Kavindu-Dodan commented Dec 6, 2022

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

Kavindu-Dodan commented Nov 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This PR

Related Issues

How to test

Uh oh!

Uh oh!

beeme1mr left a comment

Choose a reason for hiding this comment

Uh oh!

Kavindu-Dodan commented Nov 28, 2022

Uh oh!

toddbaert commented Nov 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

beeme1mr commented Nov 28, 2022

Uh oh!

Kavindu-Dodan commented Nov 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

beeme1mr commented Dec 1, 2022

Uh oh!

Kavindu-Dodan commented Dec 1, 2022

Uh oh!

AlexsJones commented Dec 5, 2022

Uh oh!

Kavindu-Dodan commented Dec 6, 2022

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Kavindu-Dodan commented Nov 25, 2022 •

edited

Loading

toddbaert commented Nov 28, 2022 •

edited

Loading

Kavindu-Dodan commented Nov 28, 2022 •

edited

Loading