Skip to content

Releases: HashakGik/KANDYBenchmark

Preliminary datasets

28 Oct 20:25

Choose a tag to compare

Preliminary version of easy and hard curricula.
The filename has the following format: {curriculum name}_{number of samples for each task}x{number of tasks}_{gamma}-{beta}.zip

The easy curriculum is provided with only 100 samples for each of the 20 tasks and it is fully supervised ($\gamma = \beta = 1.0$), the hard curriculum is provided with 100, 1000 and 10000 samples for each of the 18 tasks, with 3 supervision schedules:

  • $\gamma = \beta = 1.0$
  • $\gamma = 0.8, \beta = 0.2$
  • $\gamma = \beta = 0.5$

For the easy curriculum, splits are: $train = 0.5, val = test = 0.25$.
For hard curriculum, splits are: $train = 0.8, val = test = 0.1$.

The ilp_annotations.zip archive contains the train and test annotations for easy and hard curricula with 100 samples to allow symbolic processing without downloading the entire datasets.