feat: kpi forecasting add funnel_forecast unit tests#248
Conversation
change signatures of `fit` and `predict` to take arguments that default to attributes Co-authored-by: Brad Ochocki Szasz <bochocki@mozilla.com>
Co-authored-by: Julio Cezar Moscon <jcmoscon@gmail.com>
Co-authored-by: Julio Cezar Moscon <jcmoscon@gmail.com>
|
Made a notebook to compare outputs. For most of the rows (where the optimization lands on the same parameters), the difference is within 1% but there are some significant outliers. Not sure if this is inevitable due to the stochastic nature of the prediction or not. Notably we didn't have this issue with https://colab.research.google.com/drive/1NxD5eVwZ0Vw3UhZtZoIev5Q44qEEjSTJ#scrollTo=CjguJpSKsv7k |
|
The seed is set in both branches, so the results should be the same. Investigating |
|
Validation NB is updated and all tests pass. Two changes were necessary:
|
m-d-bowerman
left a comment
There was a problem hiding this comment.
The changes to funnel_forecast look good. The test cases, too, look sufficient to catch errors in the methods that might arise from future changes to the classes. I left a couple of comments for my clarity, but let's go ahead with it rather than having this persist as a blocker.
@jaredsnyder I added a bit to the notebook, checking for differences between values, and all differences were 0. The lil bit is right before the components section; have a look, make sure I didn't goof up the merge or sets I used? |
@m-d-bowerman : that's because the tables you were looking at ran the data with that change reverted. I loaded the tables with the change and made some plots to compare the differences here https://colab.research.google.com/drive/1NxD5eVwZ0Vw3UhZtZoIev5Q44qEEjSTJ#scrollTo=Look_at_distribution_of_differences_when_changing_1_to_0 |
|
Updates look good |
Adds unit tests for funnel_forecast and does some light refactoring
Checklist for reviewer:
referenced, the pull request should include the bug number in the title)
.circleci/config.yml) will cause environment variables (particularlycredentials) to be exposed in test logs
telemetry-airflow
responsibly.