-
-
Notifications
You must be signed in to change notification settings - Fork 368
Closed
Description
Describe the bug
When we reset an experiment from dashboard, starting numbered versions of experiments in data, it makes confusion in User.max_experiments_reached? and considers the current experiment as already taken.
The symptom is that subsequent calls to ab_test will always return control, once the user starts to be excluded from alternative choice by max_experiments_reached.
To Reproduce
Steps to reproduce the behavior:
I wrote an example app that reproduces the error. It's in rails because it was easier to me.
- Go to home page. You'll see the alternatives returned by multiple calls to
ab_test. This time it works as usual; - Go to split dashboard and click in
reset data, so to start a new version in the experiment - Now refreshes the app until the first alternative is choosed as
other. - In this case, the result you see is
oher, control, controlshowing that, after that first call subsequent ones will always returncontrol
Additional context
I've already debugged it. The problem is in User#max_experiments_reached?. The count in L31 doesn't clean up experiment key, that now has :<version> at the end.
I can submit the PR if you like so.
wladimirgramacho
Metadata
Metadata
Assignees
Labels
No labels