Skip to content

v1 competition unpacker not creating computation columns #368

@Didayolo

Description

@Didayolo

I have a V1/V2 challenge with multiple tasks. The final rank of a participant is computed regarding its average score on all tasks.

It works on Codalab V1:

ave_score_1

But the exact same bundle and submission on Codalab V2 gives:

ave_score_2

Here is the YAML configuration of the leaderboard:

leaderboard:
  columns:
    auc_classification:
      label: Classification (AUC ROC)
      leaderboard: &id001
        label: RESULTS
        rank: 1
      rank: 2
      sort: desc
    acc_selection:
      label: Feature Selection (accuracy)
      leaderboard: *id001
      rank: 3
      sort: desc
    acc_causality:
      label: Causal Inference (accuracy)
      leaderboard: *id001
      rank: 4
      sort: desc
    ave_score:
      label: < Rank >
      leaderboard: *id001
      rank: 1
      sort: asc
      computed:
        operation: Avg
        fields: auc_classification, acc_selection, acc_causality
  leaderboards:
    RESULTS: *id001

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions