Use case
one reason we allow missing values in the dart state is because CLM stores some values in arrays of layers and different ensemble members have different numbers of layers. however, i'd like to step back and ask if impacting individual layer values with the assimilation is the only solution here. i believe observations are of snow depth, or water content, or albedo, or other quantities which are modeled by the layers. could we instead preprocess the layers to construct one or more derived fields which are impacted by the observations during the assimilation and then redistributed to the layers before being passed back to CLM?
Is your feature request related to a problem?
allowing missing values in the model state has a code footprint in: forward operators, inflation, assimilation, smoother code. but possibly more important is that our current solution skips updating values for any items where one or more ensemble members have missing values. that means members without missing values contain their original values for those items at the end of the assimilation. that makes them inconsistent with layers which were successfully impacted by the observations. also adaptive inflation values for those items aren't updated.
Describe your preferred solution
can we look at what quantities (water content, crystal size, age, albedo, i don't know?) the model actually cares about, rather than the implementation details? i believe there is a 'redistribution function' in the model which spreads quantities across levels. can we impact the aggregated quantities with observations and then redistribute the updated values before returning to the model? the clm_to_dart program constructs derived fields, the assimilation updates the derived fields, and the dart_to_clm program uses them to update the prognostic fields in the restart files. we already have these programs to deal with CLM's inconsistent treatment of missing values in the restart files, so this would be an enhancement to what's already there.
Describe any alternatives you have considered
what is there is working but seems like it could be improved, both to simplify the dart code and to possibly give a better assimilation result.
Use case
one reason we allow missing values in the dart state is because CLM stores some values in arrays of layers and different ensemble members have different numbers of layers. however, i'd like to step back and ask if impacting individual layer values with the assimilation is the only solution here. i believe observations are of snow depth, or water content, or albedo, or other quantities which are modeled by the layers. could we instead preprocess the layers to construct one or more derived fields which are impacted by the observations during the assimilation and then redistributed to the layers before being passed back to CLM?
Is your feature request related to a problem?
allowing missing values in the model state has a code footprint in: forward operators, inflation, assimilation, smoother code. but possibly more important is that our current solution skips updating values for any items where one or more ensemble members have missing values. that means members without missing values contain their original values for those items at the end of the assimilation. that makes them inconsistent with layers which were successfully impacted by the observations. also adaptive inflation values for those items aren't updated.
Describe your preferred solution
can we look at what quantities (water content, crystal size, age, albedo, i don't know?) the model actually cares about, rather than the implementation details? i believe there is a 'redistribution function' in the model which spreads quantities across levels. can we impact the aggregated quantities with observations and then redistribute the updated values before returning to the model? the clm_to_dart program constructs derived fields, the assimilation updates the derived fields, and the dart_to_clm program uses them to update the prognostic fields in the restart files. we already have these programs to deal with CLM's inconsistent treatment of missing values in the restart files, so this would be an enhancement to what's already there.
Describe any alternatives you have considered
what is there is working but seems like it could be improved, both to simplify the dart code and to possibly give a better assimilation result.