cell inspector

[jtwhite79]

I’ve noticed that whenever digging into model convergence issues, the workflow for me is almost always the same: Find which cell or cells are offending the solver, then dig thru package after package, array after array, list after list, looking for input errors (its an input error >99.99% of the time for me). So I wonder what it would take to have a “cell inspector” in flopy that takes a cell (or cells) index (or indices), and traverses all packages and writes any input associated with these cells (and possibly connected neighbors) into a csv file for easy inspection. This would include static properties as well as forcings across time. Maybe there could also be an option to include heads and cell budget file entries as well? @spaulins-usgs how work would this be for mf6? I think it could be really useful for rooting out input errors....

[spaulins-usgs]

@jtwhite79 It is definitely doable. There is already code that iterates through a model's datasets (set_all_data_external). FloPy stores the necessary information to determine what data is associated with each cell. In many cases (arrays, basic gwf stress packages) it would be fairly easy to find all data associated with a particular cell id.

It gets a little more complicated with the advanced stress packages, each having their own way of associating data to a cell id. For most of that packages this would be straightforward, search for cell id data in the package, find the relevant id associated with that cell id (wellno, rno, ...) and then return any data using that associated id. However, there would need to be some special cases, for example handling lake tables.

It also gets more complicated when handling simulations with multiple models. There would be a number of special cases to handle, including exchange packages, mover packages, and handling gwf-gwt simulations where models "share" a grid and most transport packages do not directly use cellids (LKT and MWT ). Also, in the case of multiple gwf models the "cell inspector" would need to specify which model the cell is from.

Regarding the heads and cell budget files, the code we already have can fairly easily be used to get output data associated with a particular cell id, with one caveat. I think the head and cell budget reader first builds an index of the file before returning any data. If the model is stopped in the middle of a run due to convergence issues, building an index of an incomplete output file may fail, and in that case the existing code can not be used to retrieve data from a partial output file. The code could potentially be modified to fail more gracefully and allow access to the incomplete output data.

Hopefully that gives you an idea of the major issues (at least the ones I have thought of so far) and how much work it would be.

[jdhughes-dev]

@langevin-usgs @spaulins-usgs any thoughts on the level of effort and priority for this effort? Assigning to next version for now.

[spaulins-usgs]

@jdhughes-usgs Doing this for single cells and supporting the basic packages is fairly easy. Adding in support for connected neighbors and the advanced packages is a moderate amount of work. Note though that the "cell inspector" may not automatically work on new packages, so this could involve some ongoing effort as new packages and models are added to MF6.

[spaulins-usgs]

@jtwhite79 I just checked in some code on my branch for the cell inspector feature. It is still preliminary, but I wanted to get your feedback before I go back and test it on every package. The cell inspector feature can be found on my fork here:

https://github.com/spaulins-usgs/flopy

The cell inspector feature is run at the model level and can be run by passing a list of cells to inspect along with a file path to store the results, as follows:

import flopy
sim = flopy.mf6.MFSimulation.load("name", "mf6", "mf6.exe", ".")
model = sim.get_model()
inspect_list = [(2, 3, 2), (0, 4, 2), (0, 2, 4)]
out_file = os.path.join("temp", "inspect_AdvGW_tidal.csv")
model.inspect_cells(inspect_list, output_file_path=out_file)

See t504_test,py and t505_test.py in autotest for some examples. There are examples of cell inspector in test_np001, test028_sfr, test_gwf3, test006_gwf3, test028_sfr, and AdvGW_tidal. Cell inspector output from these tests go to the "autotest\temp" folder. Output files are .csv and can be opened in Excel. I have attached the file that test028_sfr generates below.

inspect_test028_sfr.csv

Here is the documentation for inspect_cells:

    Inspect model cells.  Returns model data associated with cells.

    Parameters
    ----------
    cell_list : list of tuples
        List of model cells.  Each model cell is a tuple of integers.
        ex: [(1,1,1), (2,4,3)]
    stress_period : int
        For transient data only return data from this stress period.  If
        not specified or None, all stress period data will be returned.
    output_file_path: str
        Path to output file that will contain the inspection results
    inspect_budget: bool
        Inspect budget file
    inspect_dependent_var: bool
        Inspect head file
		
    Returns
    -------
    output : dict
        Dictionary containing inspection results

I have made a few implementation decisions to keep it simple. All IDs are kept as zero-based numbers, as flopy stores them. I can change it to convert the IDs to 1-based numbers for the file, but that will require a little more coding. Inspect cells does not try to resolve time series and time array series data. Instead, it outputs the values as it finds them in the package.

The cell inspector feature does support output dependent variables (head, concentration) and budget information. Though, it currently does not output flow-ja-face since translating flow-ja-face is not trivial and I don't think we currently have code that translates it. This could be included though I would like to first implement general-use flow-ja-face translation code for each of the model discretization types.

Also, instead of looking for "nearby cells", inspect cells lets you provide it with a list of cells. This way you can define what a nearby cell is and pass in the appropriate cell list.

I have tested groundwater flow and transport models, including some basic and advanced packages with dis, disv, and disu. I will need to go back and test every package before releasing this, but first wanted you to take a look at it to confirm this is what you are looking for. Please take a look at this implementation and let me know what you think.