leaderboard scores limited to provided decimal points.

[ihsaan-ullah]

@ mention of reviewers

@Didayolo @bbearce

A brief description of the purpose of the changes contained in this PR.

In leaderboard scores are rounded to precision decimal points. precision can be provided in the yaml file as below

leaderboards:
  - index: 0 
    title: Results
    key: main
    submission_rule: "Force_Last"
    columns:
      - index: 0
        title: average auc
        key: auc
        sorting: desc
        precision: 3
      - index: 1
        title: average bac
        key: bac
        sorting: desc
        precision: 4

Issues this PR resolves

Solving the long decimal number for scores in leaderboard

Screenshot of solved problem

A checklist for hand testing

• run makemigrations, migrate and collect-static for testing

Checklist

• Code review by me
• Hand tested by me
• I'm proud of my work
• Code review by reviewer
• Hand tested by reviewer
• CircleCi tests are passing
• Ready to merge

[Didayolo]

@ihsaan-ullah

Do we have also a new field in the editor for the precision?

[ihsaan-ullah]

New field in the editor added

[Didayolo]

@ihsaan-ullah

• For some reason I can't see the "Precision"column in the editor.
  I have run the migration and collectstatic.

• In CodaLab Competitions (v1), this field is called numeric_format in the YAML files. I think the v1-unpacker should be edited to convert numeric_format into precision.

• When I tried to set the precision in a v1 bundle, it did not work. The precision was set to the default value. This should be fixed by taking into account the previous point.

[ihsaan-ullah]

I followed these steps and I can see precision :

• pull this branch
• docker down and up
• collect static docker-compose exec django ./manage.py collectstatic --noinput
• hard refresh MAC: command + shift + R
• uploaded a fresh competition which does not have this field in YAML

A bit confused about the problem in V1 bundle

[Didayolo]

Last point: updating the documentation https://github.com/codalab/codabench/wiki/Yaml-Structure

[ihsaan-ullah]

@Didayolo I think I have covered all the points

• Documentation updated for precision
• V1 unpacker updated to take into account numeric_format
• V1 packer tested on V1 Iris bundle

YAML from Iris bundle:

numeric_format = 2and 4

Converted into precision as shown in the editor:

[ihsaan-ullah]

@bbearce

The rounding to 2 decimal places we added fails in this situation because one of the V1 bundle has numeric_format = 4 rather than 2.

What should be done in this case?

[Didayolo]

@ihsaan-ullah Your last commit seems a bit strange.

• First, you change a test that was replaced in recent PR (Add v15 Iris e2e tests #828, Remove the SNCF bundle e2e test #830)
• Also, it seems that you setup the precision in the test file? That should be done by the bundle no?

[ihsaan-ullah]

I changed this function _run_submission_and_add_to_leaderboard to add one more parameter precision which by default is 2. In all bundles it is 2 so default value will be used when rounding the score from leaderboard

Now in the specific case of bundle v15, I passed precision=4 to the function _run_submission_and_add_to_leaderboard because for this specific bundle precision/numeric_format is 4.

I just checked the same file here: #828 and it looks like the difference is just the addition of test_v15_iris_result_submission_end_to_end and test_v15_iris_code_submission_end_to_end.

My changes to the test will not affect the new additions as these will use the default parameter precision=2.

To clarify a bit more, the precision passed as a parameter was already there but was hardcoded like this:

assert Decimal(self.find('leaderboards table tbody tr:nth-of-type(1) td:nth-of-type(3)').text) == round(Decimal(prediction_score), 2)

And now this is changed to

assert Decimal(self.find('leaderboards table tbody tr:nth-of-type(1) td:nth-of-type(3)').text) == round(Decimal(prediction_score), precision)