Open SWMM Version 6.0.0 Debut Release

[cbuahin]

OpenSWMM Engine v6.0.0-alpha.1

Summary

This pull request delivers the OpenSWMM Engine v6.0.0-alpha.1 — a ground-up modernization of the EPA SWMM computational engine. The work spans 12 implementation phases covering architecture, algorithms, API design, plugin infrastructure, Python bindings, testing, CI/CD, and documentation.

The legacy EPA SWMM 5.x solver is preserved unmodified in src/legacy/ as a regression baseline. The new engine is a complete, parallel implementation that produces as baseline, near identical results using the same algorithms, constants, and convergence criteria with significantly improved performance. The data structures implement pave the way for a more community driven development effort through the implementation of the SWMM code in a modular manner. The API and python bindings allow for a fully end to end modeling workflow fully done by code.

---

What Changed

New Engine Architecture (146 C++ source files)

• Data-Oriented Design — All model data stored in Structure of Arrays (SoA) layout for cache efficiency and SIMD vectorization. Core state encapsulated in SimulationContext.
• Reentrant Engine — Global state eliminated. An opaque SWMM_Engine handle encapsulates all simulation state, enabling multiple independent simulations per process.
• Engine Lifecycle State Machine — Explicit states (CREATED → OPENED → BUILDING → INITIALIZED → STARTED → RUNNING → ENDED → CLOSED) with compile-time state guards on all API calls.
• C++20 / C17 — New engine uses C++20 with -fno-fast-math and -fexcess-precision=standard for numerical fidelity.

Comprehensive C API (19 domain-split headers, ~250+ functions)

The monolithic legacy interface is replaced by a domain-organized C89-compatible API:

Header	Domain
openswmm_engine.h	Engine lifecycle, error codes, state machine
openswmm_model.h	Model building, validation, .inp serialization, options
openswmm_nodes.h	Junctions, outfalls, storage nodes, dividers
openswmm_links.h	Conduits, pumps, orifices, weirs, outlets
openswmm_subcatchments.h	Subcatchments, infiltration, landuse coverage
openswmm_gages.h	Rain gages (timeseries and file sources)
openswmm_pollutants.h	Pollutant definitions and runtime quality injection
openswmm_tables.h	Time series, curves, patterns with cursor-optimized lookups
openswmm_inflows.h	External inflows, dry weather flow, RDII
openswmm_controls.h	Control rules and direct link setting/status actions
openswmm_infrastructure.h	Transects, streets, inlets, LID controls
openswmm_spatial.h	CRS, coordinates, polylines, polygons
openswmm_quality.h	Landuse, buildup/washoff, treatment expressions
openswmm_massbalance.h	Continuity errors and cumulative flux totals
openswmm_callbacks.h	Progress, warning, step-begin/end callbacks
openswmm_hotstart.h	Hot start file save/load/modify/query
openswmm_statistics.h	Node, link, and subcatchment statistics

Models can be built entirely via the C API without an .inp file.

Process Formulation Enhancements

The following physics-based improvements are being designed for backward-compatible integration. Each addresses a known simplification in the current SWMM formulation.

• Spatially Explicit Overland Flow & Groundwater — SWMM currently has one-way feedback between hydrology and hydraulics: flooded nodes pond over a user-specified area and are reintroduced locally. In reality, surcharge flows traverse the terrain and may re-enter the network at a downstream node. The new module couples a 2D overland flow grid with the 1D pipe network, enabling spatially explicit green infrastructure placement, terrain-routed surcharge, and lateral groundwater flow exchanges, etc.

• Dynamic Preissmann Slot — Standard SWMM uses a fixed-width slot for pressurized conduit transitions, which can cause numerical instability at the open-channel/pressure interface. A dynamic slot formulation smooths the transition with a geometry-dependent slot width, improving stability for rapidly filling/draining conduits.

• Spatially Explicit Inlets — Current inlets are attributes of conduits, limiting bypass routing and preventing backflow or series-inlet configurations. The redesign promotes inlets to mode-switching junction nodes that actively capture street flow when the gutter spread exceeds a threshold and revert to passive junctions otherwise.

• LID as Storage Nodes — Current LID units lack hydraulic feedback: no backpressure from the drainage network, no bidirectional flow, and no control-structure integration. The redesign maps LID layers (surface, media, gravel) onto an extended storage node using a reduced-physics kinematic Richards ODE — a simplified 1D formulation that captures gravity-driven percolation and capillary redistribution between discrete layers without the computational cost of a full 3D variably-saturated solver. This provides physically meaningful moisture dynamics while remaining compatible with SWMM's routing timestep, and enables elevation-mapped underdrain links with full network coupling.

• Physics-Based Initial Abstraction Recovery — SWMM's seasonal RTK calibration for RDII confounds infrastructure leakage fraction with soil moisture state, requiring 12 monthly parameter sets. The new formulation tracks initial abstraction capacity as an exponential decay/recovery process. During dry periods, available capacity recovers with additive time-based and temperature-dependent rate components:

  $$IA_{avail}(t+\Delta t) = IA_{max} - \bigl(IA_{max} - IA_{avail}(t)\bigr) \cdot e^{-k_{rec}(T),\Delta t}, \qquad k_{rec}(T) = k_0 + k_T \cdot e^{,\theta,(T - T_{ref})}$$

  The base rate $k_0$ captures gravity drainage and capillary redistribution that occur regardless of temperature, while $k_T \cdot e^{\theta(T-T_{ref})}$ captures thermally-driven evapotranspiration. During storms, capacity is depleted: $IA_{avail}(t+\Delta t) = IA_{avail}(t) \cdot e^{-k_{dep},\Delta P}$. Recovery is suppressed when $T < T_{freeze}$, producing emergent seasonal variation (high winter RDII, low summer RDII) from a single RTK set per sewershed — no monthly parameter tables required.

Plugin SDK (5 headers)

• IOutputPlugin / IReportPlugin — Abstract C++ interfaces for writing time-series results and summary reports to custom formats (HDF5, NetCDF, CSV, etc.).
• IPluginComponentInfo — Metadata and factory interface with reverse-DNS identifiers.
• PluginState — Lifecycle state machine: LOADED → INITIALIZED → VALIDATED → PREPARED → UPDATING → FINALIZED → CLOSED.
• SimulationSnapshot — Read-only deep-copy of simulation state passed to plugins on a dedicated I/O thread.
• New [PLUGINS] input file section for registering shared library plugins with initialization arguments.

Computational Algorithms (Numerical Parity with Legacy SWMM)

All SWMM computational modules ported to SoA/batch/SIMD-friendly design with near identical numerical behavior as an initial starting point :

Module	Legacy Source	Key Capability
Cross-sections	xsect.c	19 shapes, shape-grouped batch kernels (AD-13)
Kinematic wave	kinwave.c	WX=0.6, WT=0.6, EPSIL=0.001 Newton solver
Dynamic wave	dwflow.c, dynwave.c	Theta-implicit momentum, Preissmann slot, adaptive CFL timestep
Routing	routing.c, flowrout.c	KW/DW/steady dispatch, topological sort, storage iteration
Hydrology	runoff.c, subcatch.c	Nonlinear reservoir, batch depth update
Infiltration	infil.c	Horton, Green-Ampt, SCS Curve Number
Climate	climate.c	Hargreaves ET, monthly/timeseries evaporation
Snow	snow.c	ATI, degree-day, rain-on-snow
Groundwater	gwater.c	Two-zone model, lateral flow to nodes
LID	lid.c, lidproc.c	Bio-retention, perm. pavement, green roof, rain barrel, swale, roof disconnection
Quality routing	qualrout.c	Complete mixing, first-order decay, evaporation concentration
Treatment	treatmnt.c, mathexpr.c	Shunting-yard expression parser (C=/R= equations)
Controls	controls.c	Rule parser, PID controller, curve/timeseries actions
Pumps/Orifices/Weirs/Outlets	link.c	Type-grouped SoA with batch flow computation
Inflows	inflow.c	External + DWF with 4-pattern modulation
RDII	rdii.c	3-response UH convolution with circular buffer
Culverts	culvert.c	58-type FHWA HEC-5 coefficient table
Inlets/Streets	inlet.c, street.c	HEC-22 grate/curb/combo/slotted capture
Force mains	forcmain.c	Hazen-Williams + Darcy-Weisbach (Swamee-Jain)
Exfiltration	exfil.c	Bottom/bank Green-Ampt for storage nodes
Transects	transect.c	Station-elevation → normalized A/R/W tables
Model serialization	(new)	Full .inp writer for all 37 SWMM sections

Explicit Timestep Control

• dt_next = min(dt_output, dt_cfl, dt_controls, dt_rdii) — simulation stops exactly at output boundaries with no interpolation.
• Separate I/O thread with lock-free ring queue for non-blocking output writes.

Hot Start API

• Binary format OPENSWMM_HS_V1 with CRC32, object UUID map for missing-object detection.
• Full C API: swmm_hotstart_save/open/apply/modify/close with per-object depth/flow overrides.

Input System Enhancements

• Multi-delimiter tokenizer — Accepts comma, tab, and multi-space delimiters (mixed per line).
• CSV rain files — New USER_CSV format with column selection: FILE "rain.csv:EAST_GAGE".
• Extension options — Unknown [OPTIONS] keys stored in a string map accessible to plugins via swmm_options_get_ext().
• CRS — New CRS option (EPSG code or PROJ string) for spatial data.
• User flags — [USER_FLAGS] and [USER_FLAG_VALUES] sections for BOOLEAN/INTEGER/REAL/STRING metadata on any object.
• Optional sections — SectionRegistry with custom function pointer readers.

Python Bindings (11 Cython modules)

• _solver.pyx — Solver lifecycle context manager with iterative stepping.
• _model.pyx — ModelBuilder for programmatic model construction.
• _nodes.pyx / _links.pyx / _subcatchments.pyx / _gages.pyx — Domain access with numpy bulk arrays.
• _hotstart.pyx — Hot start save/open/apply with context manager.
• _massbalance.pyx — Continuity error queries.
• _enums.py — NodeType, LinkType, XSectShape, FlowUnits, RouteModel int enums.

Legacy Isolation

• All 80 original EPA SWMM C source files physically moved to src/legacy/engine/ and src/legacy/output/ — zero modifications to legacy code.
• Separate CMake targets: openswmm_legacy_engine, openswmm_legacy_output.
• Legacy includes via <openswmm/legacy/engine/openswmm_solver.h>.

Testing

• Google Test migration — All unit tests converted from Boost.Test to Google Test 1.15.2.
• Expanded legacy tests — 73+ engine tests, 41 output tests (up from ~20 original).
• Test directory restructured — tests/unit/legacy/{engine,output} and tests/unit/{engine,output}.
• New engine unit tests — Tokenizer, section registry, user flags, plugin lifecycle, hot start, cross-section geometry, infiltration.
• Regression framework — RunLegacy + RunNew + CompareOutputs with per-object tolerance (depth ±0.001 ft, flow ±0.001 cfs).
• Benchmark framework — Google Benchmark for engine-vs-legacy wall-time and SIMD-vs-scalar comparison.

CI/CD Pipelines (4 workflows rewritten)

• unit_testing.yml — 4-platform matrix (Linux x64, macOS x64, macOS ARM64, Windows x64), engine build+test+package, Python wheel build+test. Fixed critical bug: -DBUILD_TESTS=ON → -DOPENSWMM_BUILD_TESTS=ON.
• regression_testing.yml — Multi-platform regression with vcpkg + Google Test (removed Boost.Test/NuGet dependency).
• documentation.yml — Doxygen build + GitHub Pages deployment; updated to actions/checkout@v4.
• deployment.yml — Tag-triggered release builds + cibuildwheel wheels + GitHub Release creation.

Documentation

• README.md — Complete rewrite with badges, v6 feature overview, project structure, build instructions (C++ and Python), library targets, contributing guide.
• CHANGELOG.md — Populated with comprehensive v6.0.0-alpha.1 entries (Added/Changed/Removed/Fixed).
• Doxygen API docs — All 19 public C API headers documented with @brief, @details, @param, @returns, @see, @note, @warning.
• Author/license metadata — @author Caleb Buahin, @copyright HydroCouple 2026, @license MIT on all 146 new engine source files.
• User manual Chapter 13 — New chapter covering the programmatic C API: architecture, workflow example, callbacks, hot start, Python bindings, CMake integration.
• User manual §13.7–13.10 — User-defined flags, plugin interface, CSV rain-file inputs, extension options.
• Appendix D updates — [OPTIONS] CRS and extension options; [RAINGAGES] CSV column syntax.
• Manuals index — Added C API Reference and Changelog links.
• Master Implementation Plan — 12-phase plan with architecture decisions, task tracking.

---

Stats

Metric	Count
New engine C++ files (src/engine/)	146
Legacy C files preserved (src/legacy/)	80
Public C API headers	19
Plugin SDK headers	5
C API functions	~250+
Python Cython modules	11
Unit test files	25
CI/CD workflows	4
Implementation phases	12 (0–8 complete, 9–11 in progress)

---

Key Design Decisions

• Numerical parity — All algorithms use identical constants, formulas, and convergence criteria as legacy SWMM (OMEGA=0.5, HEADTOL=0.005, GRAVITY=32.2). Differences beyond 2× machine epsilon require documented justification.
• Shape-grouped batch geometry — Conduits grouped by cross-section shape for branchless SIMD-friendly kernels.
• SIMD strategy — __restrict__ + #pragma ivdep for complex loops; explicit simd:: intrinsics for simple array operations.
• IO thread isolation — Deep-copy snapshots; no mutex on the simulation critical path.
• C89 ABI — No C++ types in public API headers; error codes only (no exceptions).
• Plugin SDK compiles without exceptions — Error codes throughout.

---

Breaking Changes

• Project renamed from OpenSWMMCore to openswmm; library output is openswmm.engine.
• CMake options renamed to OPENSWMM_* prefix (legacy OPENSWMMCORE_* aliases preserved).
• CMake minimum raised to 3.21.
• Boost.Test removed; Google Test 1.15.2 is now the test framework.
• vcpkg replaces NuGet for dependency management.

---

Remaining Work (tracked in MASTER_IMPLEMENTATION_PLAN.md)

• Phase 9 — Python binding completion (callbacks, pollutants, controls modules; pytest suite).
• Phase 10 — Regression test data (Example1–3.inp), benchmark suite activation.
• Phase 11 — Doxygen group annotations, complete manual updates, Mainpage.md refresh.

[dickinsonre]

Fantastic work @cbuahin I am getting my Ais to check all of this