Rebase/decompose by fletchapin · Pull Request #49 · we3lab/eeco

fletchapin · 2025-12-09T19:37:14Z

Pull request recommendations:

Name your pull request your-development-type/short-description. Ex: feature/gui
Link to any relevant issue in the PR description. Ex: Resolves [ENH: Support optimization of flexibility metrics #12]
Provide context of changes.

In utils:

Updating sum() util function to use pyo.summation
Adding decompose_consumption function to split a np array, pyo variable, or cvxpy variable into positive and negative components
Adding initialize_decomposed_pyo_vars to help with initializing the decomposed elements in a pyomo model

In costs:

Updating calculate_cost, calculate_itemized_costs to take decomposition_type argument. Currently handles None or "absolute_value", but can be expanded to include binary and other methods
Updating calculate_export_revenue to expect positive values only and warn if negative values are input
Updating calculate_energy_cost to only use convex relaxation if charge limits are present. (otherwise, it under-estimates cost)

Fixes to parametrize_charge_dict:

Updating scale_ratios argument name to percent_change_dict
Removing get_unique_row_name and consolidating with default_varstr_alias_func
Provide relevant tests for your feature or bug fix.
Provide or update documentation for any feature added by your pull request.

Thanks for contributing!

…ed off of feature/parametrize_charges) In utils: - Updating sum() util function to use pyo.summation - Adding decompose_consumption function to split a np array, pyo variable, or cvxpy variable into positive and negative components - Adding initialize_decomposed_pyo_vars to help with initializing the decomposed elements in a pyomo model Modifying existing tests in test_costs.py to include decompositino Adding test_decompose_consumption_np, test_decompose_consumption_cvxpy, test_decompose_consumption_pyo to test_utils.py Adding documentation notes in how_to_advanced Adding test coverage for negative values input to energy / demand calculation Removing ValueError raised in calculate_cost for non-np.array/pyo/cvx object. Since this code is never actually reached because the valueerror will be raised from ut.multiply() first Consolidating consumption_data_dict and consumption_object_dict arguments Adding magnitude constraint so that positive_var[t] + negative_var[t] == abs(expression[t]) to prevent both variables becoming large when export rate is higher than energy rate Reformatted with black Updated test_calculate_cost_pyo to use ipopt rather than gurobi when running decompose_exports, since abs() function is nonlinear / nonconvex

…ct and parametrize_rate_data Adding test_calculate_itemized_costs for np, cvx, and pyo with one non-decomposed and one decomposed example Adding a warning test to calculate_export_revenue Adding helper functions for pyo and cvx problems to be used by their versions of both test_calculate_cost and test_calculate_itemized_cost Adding warning that decompose_exports isn't supported with cvxpy due to non-DCP exports-imports issues Reformatted with black

…take None, "absolute_value", "binary", or other types in the future Using max_pos for creating positive and negative pyomo variables (WIP: Index issue failing test in test_utils)

Adding calculate_conversion_factors Modifying calculate_energy_cost to only apply convex approximation when there are charge tiers TESTS Adding consumption_estimate to test_calculate_cost_cvx to avoid default 0 issues Adding additional test case for cvx problem without charge limit to test_calculate_cost_cvx Updating test expectations in test_max_pos_pyo so that scalar inputs have scalar expectations, vector inputs have vector expectations

Reformatting with black

…position, removing it from divisor in costs.py and renaming divisor to n_per_hour Moving application of conversion factor to helper function get_converted_consumption_data

default_varstr_alias_func takes row index parameter, where index is used if "name" is blank Removing dashes from names in test_costs.py since sanitize_varstr removes dashes and replaces with underscores Cleaning up error expectation in test_calculate_cost_np

Adding back get_unique_row_name (renamed to get_charge_name) Removing initialize_decomposed_pyo_vars. The variables self-initialize during solve. Updating test_decompose_consumption_pyo to just check for the presence of objects, not values. (WIP to add value check in test_calculate_cost_pyo for pre-decomposed inputs) Renaming _get_decomposed_var_names to get_decomposed_var_names Adding test cases for extended consumption data format in test_calculate_cost_np and test_calculate_cost_pyo

Updating minimum pyomo version to 6.8

- Add scalar LinearExpression tests for max_pos_pyo - Add TypeError tests for invalid types in max_pos and decompose_consumption - Add warning tests for unimplemented decomposition types - Add ValueError test for invalid type in calculate_export_revenue - Add negative values warning test in calculate_demand_costs - Add test for consumption_estimate=None and dict consumption_estimate - Add conversion factor test with MW units in test_calculate_itemized_cost_pyo - Fix idempotency in get_converted_consumption_data to prevent duplicate components - Add warnings for unimplemented decomposition_type in utils.decompose_consumption

…composition has already been run on the model

…ized costs

fletchapin · 2025-12-09T19:41:37Z

Closing the old PR (#33) and opening a new PR for rebased branch. Prior notes from @dalyw

Some pseudocode to explain the approach more:

Define variable for electricity consumption (indexed to timeseries resolution)
Disaggregate the positive and negative consumption into separate variables

imports = maximum(consumption, 0)

In Pyomo, this is implemented as:

imports >= 0
imports >= consumption

exports = minimum(consumption, 0)

In Pyomo, this is implemented as:

exports >= 0
exports >= consumption

Add constraint for total consumption to equal sum of components

consumption = imports - exports

In Pyomo, we add an additional constraint to prevent “cheating” when export price is higher than energy rates and the solver tries to increase imports to infinity

abs(Consumption) = imports + exports

Calculate energy, demand and export cost/value based on disaggregated variables

Energy cost = f(imports)
Demand cost = f(imports)
Export revenue = f(exports)

codecov-commenter · 2025-12-16T00:55:02Z

Codecov Report

❌ Patch coverage is 98.55072% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.40%. Comparing base (d86ca9c) to head (03da2db).

Files with missing lines	Patch %	Lines
eeco/costs.py	95.45%	4 Missing ⚠️
eeco/tests/test_costs.py	99.22%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #49      +/-   ##
==========================================
+ Coverage   91.00%   93.40%   +2.40%     
==========================================
  Files          10       10              
  Lines        1356     1593     +237     
==========================================
+ Hits         1234     1488     +254     
+ Misses        122      105      -17

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

arao53 · 2025-12-16T02:21:12Z

eeco/utils.py

+    return f"{utility}_positive", f"{utility}_negative"
+
+
+def decompose_consumption(


If we have a few types of decomposition, I could see how this could be a bit tough to follow. You might consider having a function for each decomposition type (i.e. decompose_absolute_value, decompose_binary, etc.) that are called within decompose_consumption.

That's a good idea to make it more readable. I might work on the binary version so that it will work with gurobi in the next couple of weeks and will separate them

arao53

LGTM

dalyw · 2025-12-16T21:01:16Z

docs/how_to_advanced.rst

 .. code-block:: python

-    from eeco import costs
+    from electric_emission_cost import costs


Is this an older version using electric_emission_cost name rather than eeco?

dalyw · 2025-12-16T21:07:01Z

docs/how_to_advanced.rst

+Options:
+- Default `None`
+- `"binary_variable"`: To be implemented
+- `"absolute_value"`


This formatting isn't working, I think due to indentation?

Should be
Options:

Default None

"absolute_value"

"binary_variable": To be implemented

Fixed! There needs to be a blank line before and after the enumerated list

dalyw · 2025-12-16T21:11:05Z

eeco/costs.py

-                                        new_start.strftime("%Y%m%d"),
-                                        str(limit),
-                                    )
+                                key_str = default_varstr_alias_func(


Good use of this for consistency!

dalyw · 2025-12-16T21:18:18Z

eeco/costs.py

-            "Please switch to new 'charge (metric)' and 'charge (imperial)' format",
+            "Please switch to new 'charge (metric)' and 'charge (imperial)' format. "
+            "Current behavior assumes units of therms (imperial) "
+            "and converts to cubic meters (metric) when unspecified.",


Not necessary here, but could consider if an optional electric_charge_units or gas_charge_units argument could be useful to handle cases where the charge data is not in standard units. (But that would only be if someone brought in their own billing data)

Yeah I guess we could do that, but if you bring your own data you can just convert to one of those two units is my thought. My plan is to actually remove the support for the old format entirely after iterating a couple more versions. it's mostly this way to support the old billing files that Pepe used (hence the DeprecationWarning).

I would just have a single column except for that the raw data is almost always in therms in the US, but scientists usually use metric, so I thought providing that conversion would be nice.

For the majority of users who just calculate electricity bills and not natural gas the imperial and metric columns are the same anyway.

dalyw · 2025-12-16T21:19:49Z

Looks good (assuming the linting failure gets resolved!) Just two minor changes on the how_to_advanced.rst file

dalyw and others added 18 commits December 9, 2025 10:40

Simplifying test_decompose_consumption_pyo with approx

276c0e6

Changing decompose_exports argument to decomposition_type, which can …

d9873aa

…take None, "absolute_value", "binary", or other types in the future Using max_pos for creating positive and negative pyomo variables (WIP: Index issue failing test in test_utils)

Adding missing arguments in test_costs.py

ed883c2

Fixing bug with duplicated conversion_factor in calculate_cost

a16404f

Reformatting with black

Since conversion_factor is applied earlier during import/export decom…

39267ea

…position, removing it from divisor in costs.py and renaming divisor to n_per_hour Moving application of conversion factor to helper function get_converted_consumption_data

Cleaning up calculate_energy_cost

695b880

Adding test calling max_pos on pyo scalar

982dd43

Updating minimum pyomo version to 6.8

Reformatted with black

5b3c611

Removing uneeded else statements in calculate_cost if conversion / de…

b1a2ab2

…composition has already been run on the model

Correcting UserWarning syntax in costs.py

8a67db9

Creating new test to address bug with double unit conversions in item…

c50e6c7

…ized costs

Fixed errors in rebase so that only two tests fail now

df561c0

fletchapin mentioned this pull request Dec 9, 2025

ENH: Automatically decompose consumption variable into positive and negative components for calculating import costs + export revenue #30

Closed

Fixing flake8 errors

399c5cb

fletchapin mentioned this pull request Dec 9, 2025

Feature/decompose consumption #33

Closed

2 tasks

fletchapin added 3 commits December 15, 2025 14:36

Trying to address double unit conversion

502009c

Fixed unit conversion errors and all tests now passing

dd660b1

Autoreformatted with black

1aa760c

fletchapin requested review from arao53 and dalyw December 16, 2025 00:52

arao53 reviewed Dec 16, 2025

View reviewed changes

arao53 approved these changes Dec 16, 2025

View reviewed changes

dalyw reviewed Dec 16, 2025

View reviewed changes

eeco/costs.py

new_start.strftime("%Y%m%d"),

str(limit),

)

key_str = default_varstr_alias_func(

Copy link

Contributor

dalyw Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good use of this for consistency!

dalyw reviewed Dec 16, 2025

View reviewed changes

fletchapin added 4 commits December 16, 2025 13:23

Creating codecov.yml file to try to address patch coverage

b1dc01f

Fixed flake8 errors

3b8cadd

Fixing documentation formatting

f26de35

Fixing whitespace error

03da2db

fletchapin merged commit ad00aea into main Dec 16, 2025
2 checks passed

		return f"{utility}_positive", f"{utility}_negative"


		def decompose_consumption(

Conversation

fletchapin commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fletchapin commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arao53 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dalyw commented Dec 16, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Comments

fletchapin commented Dec 9, 2025 •

edited

Loading

fletchapin commented Dec 9, 2025 •

edited

Loading

codecov-commenter commented Dec 16, 2025 •

edited

Loading