Make default metrics be the output of a function, rather than a stored data set

[nikosbosse]

Update: we reached the conclusion to make a function instead of stored data object. This needs to be implemented.

---

Default metrics for score() are currently stored as a data object in the package and get created like this:

metrics_quantile <- list(
  "wis" = wis,
  "overprediction" = overprediction,
  "underprediction" = underprediction,
  "dispersion" = dispersion,
  "bias" = bias_quantile,
  "coverage_50" = interval_coverage_quantile,
  "coverage_90" = \(...) {run_safely(..., range = 90, fun = interval_coverage_quantile)},
  "coverage_deviation" = interval_coverage_deviation_quantile,
  "ae_median" = ae_median_quantile
)
usethis::use_data(metrics_quantile, overwrite = TRUE)

@seabbs noted that he didn't like this and would prefer this to be a function instead of a stored data object. Could you please elaborate a bit why you think that and what the trade-offs are?

[seabbs]

I think the question is why a list of package functions should be data and not a function? The standard would be for them to be a function so rather than make an argument for why they should be I think we should be making an argument for why they shouldn't be

[nikosbosse]

🤷‍♂️ Good with me! :) I can make it a function

[nikosbosse]

Only thing is that it's more intuitive to do
metrics <- metrics_quantile[names(metrics_quantile) == "wis"], rather than
metrics <- metrics_quantile()[names(metrics_quantile()) == "wis"].

or metrics_quantile[1] rather than metrics_quantile()[1]

But probably it's not a big thing either way and I'm happy make the change.

[sbfnk]

I may not understand the use case correctly but can't we implement it as metrics <- metrics_quantile("wis")?

[seabbs]

yes we could

[nikosbosse]

Seems like a good idea. Will implement that.