Setting epsilon for early stopping #88
Description
What would you like to be added:
For early stopping, we just compare the exact result of the new experiment and the best result, but we should add. a tolerance for this, like the early_stopping_epsilon. then
abs(current_result - best-result) > early_stopping_epsilon, we take it as not qualified for the measurement.
Why is this needed:
Completion requirements:
This feature requires the following artifacts:
- Design doc
- API change
- Docs update
The artifacts should be linked in subsequent comments.