feat(datafile): add .headers property with data frame#2221
Merged
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #2221 +/- ##
=========================================
+ Coverage 69.8% 70.4% +0.5%
=========================================
Files 294 294
Lines 59008 59025 +17
=========================================
+ Hits 41228 41578 +350
+ Misses 17780 17447 -333
|
christianlangevin
approved these changes
Jun 11, 2024
wpbonelli
pushed a commit
that referenced
this pull request
Jun 14, 2024
…2232) Now that data files have a .headers data frame property (#2221), there is no need to have methods that print parts of this information to stdout (they return None). This PR deprecates the following: * list_records(): use headers instead * list_unique_records(): use headers[["text", "imeth"]].drop_duplicates() instead * list_unique_packages(to=True/False): use headers.paknam.drop_duplicates() or headers.paknam2.unique() (there are a few ways) The last two only apply to CellBudgetFile. This PR does not apply to flopy.mf6.utils.mfobservation.list_records().
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This feature adds a
.headersproperty for a pandas data frame of the headers of various data files, including HeadFile, FormattedHeadFile, UcnFile and CellBudgetFile. This is a modern accessor to the headers, which is created internally using:where the index is the file position to the start of each array. Text fields are decoded to
strtypes (rather than dealing with bytes types, which is probably a hang-over from Python2). Int32 types are left as-is, except for iposarray which should be int64 to read large files >2GB. Float32 types are also left as-is, since these would otherwise have a lossy conversion to float64.With the CellBudgetFile, the headers vary depending on the type of budget file. For instance, "classic" files (created without "COMPACT BUDGET" option) don't have imeth, delt, pertim, or totim columns. Furthermore, only files with imeth=6 have the extra text columns modelnam, paknam, modelnam2, and paknam2, since these fields are always empty.
This PR also adds more checks to the outputs created with
._build_index()methods. Eventually, I'm planning to deprecate other properties and functions that are using the.recordarraystructured array, so it is important to have a detailed trace of these outputs before replacing them.This PR also moves the
__enter__/__exit__methods fromBinaryLayerFiletoLayerFile(this feature was from #669). This is soFormattedHeadFilecan also use the "with" context statement to also auto-close the file.Examples
E.g. show data frames to markdown with:
python -c "from flopy.utils import HeadFile; print(HeadFile('examples/data/mf6/create_tests/test004_bcfss/expected_output/bcf2ss.hds').headers.to_markdown())examples/data/mf6/create_tests/test004_bcfss/expected_output/bcf2ss.hds
examples/data/mt3d_test/mf2kmt3d/reinject/MT3D001.UCN
examples/data/mfusg_test/01A_nestedgrid_nognc/output/flow.cbc
examples/data/mfusg_test/03A_conduit_unconfined/output/ex3A.cbb
examples/data/mf6/create_tests/test004_bcfss/expected_output/bcf2ss.cbb