Add generate_OAT_SA_design() for sensitivity analysis input design #3729
Conversation
mdietze
left a comment
mdietze
left a comment
There was a problem hiding this comment.
A good overall test that this function is working correctly would be to verify that the run.write.configs module runs correctly with the OAT design as an input and continues to generate output that's readable by the SA postprocessing and graphing functions. If it doesn't, then this function is generating inputs that are not serving any useful purpose. Along the way, we may want/get to simplify the run write configs logic
| #' all inputs vary together. | ||
| #' | ||
| #' @param settings PEcAn settings object | ||
| #' @param sa_samples Optional. Pre-loaded SA samples from samples.Rdata. |
There was a problem hiding this comment.
This argument suggests a fundamental design misunderstanding. sa_samples should be generated by this function, not read by it. I'm OK with taking this as an optional input if one wants to reuse a previous sa_samples, but if it's not provided it needs to be generated
There was a problem hiding this comment.
the parameter now clearly indicates it's optional for reusing existing samples, not the expected input
| if (is.null(sa_samples)) { | ||
| samples_file <- file.path(settings$outdir, "samples.Rdata") | ||
|
|
||
| # generate samples if they don't exist (safety fallback) |
There was a problem hiding this comment.
This should be the default, not the safety fallback
There was a problem hiding this comment.
removed "safety fallback" comment
|
|
||
| PEcAn.uncertainty::get.parameter.samples( | ||
| settings, | ||
| ensemble.size = 1, # SA doesn't need ensemble samples |
There was a problem hiding this comment.
not needed as this is the default
There was a problem hiding this comment.
removed, good catch!
| settings, | ||
| ensemble.size = 1, # SA doesn't need ensemble samples | ||
| posterior.files, | ||
| ens.sample.method |
There was a problem hiding this comment.
also not needed as this isn't an ensemble sample
| #' @export | ||
| #' @author Akash B V | ||
| #' @importFrom rlang %||% | ||
| generate_OAT_SA_design <- function(settings, sa_samples = NULL) { |
There was a problem hiding this comment.
It might be too much at this PR, but it would be good to move away from settings as a dependency unless the underlying number of pieces of information is too large to meaningfully pass to the function. But in that case it would be good to document exactly what part of the settings is required. Here I think it might just be settings$outdir, settings$pfts, settings$sensitivity.analysis, and settings$ensemble (i.e. I wonder if you could get away with a function that has outdir, pfts, sensitivity.analysis, ensemble, and sa_samples as arguments?). Would also be good to better document semi-hidden dependencies (e.g. what does the pft$posterior.files need to point to for the function to actually sample parameters correctly)
There was a problem hiding this comment.
I agree with reducing dependency of settings for input desing by passing specific argument, but after analyzing both functions to keep the architecture consistent, i found that the parameter requirements are not same; generate_OAT_SA_design needs outdir, pfts, samplingspace, sensitivity.analysis and generate_joint_ensemble_design additionally needs run$inputs (via input.ens.gen() which samples from settings$run$inputs[[input]]$path).
However there is a deeper blocker -- both functions call get.parameter.samples(settings, ...) which itself uses many settings fields (database$bety, host$name, sensitivity.analysis, etc.). So even with explicit parameters in the design functions, we'd still pass settings through to get.parameter.samples. And then that involves refactoring SDA and sobol callers.
anyways i have documented what setting it uses and semi-hidden dependiences in both desing function. I happy to know ur thoughts.
There was a problem hiding this comment.
I think it's fine if the parameters requirements are not the same. I also think it's fine to push the refactor of the generate design functions and get.parameter.samples to a future PR
|
── Failed tests ──────────────────────────────────────────────────────────────── |
we are passing |
|
https://github.com/PecanProject/pecan/actions/runs/20528083080/job/58974923917?pr=3729
|
2 participants
originally discussed -- comment
Description
Add
generate_OAT_SA_design()function to create input design matrices for OAT sensitivity analysis.SA requires isolating the effect of each parameter by holding all other inputs (met, IC, soil) constant. The existing
generate_joint_ensemble_design()randomizes these inputs, which is correct for ensemble runs but invalidates SA variance decomposition.added tests in
test.input_design.Rto validate the OAT sensitivity design, including checks :Motivation and Context
Review Time Estimate
Types of changes
Checklist: