Skip to content

Change default solver to scipy #587

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tsmbland merged 6 commits into from
Dec 3, 2024
Merged

Change default solver to scipy #587

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
75395ac
Change default solver to scipy
tsmbland Nov 29, 2024
4923636
Add warning if lpsolver is not specified
tsmbland Nov 29, 2024
4ffc950
Fix failing test
tsmbland Nov 29, 2024
b886146
Merge branch 'v1.3' into default_solver
tsmbland Nov 29, 2024
e900b5c
Merge branch 'v1.3' into default_solver
tsmbland Dec 2, 2024
b9456f8
Clearer documentation about default solver
tsmbland Dec 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/inputs/toml.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -442,12 +442,12 @@ Sectors contain a number of subsections:
are registered via :py:func:`~muse.investments.register_investment`. At time of
writing, three are available:

- "scipy" solver (default from v1.3): Formulates investment as a true LP problem and solves it using
the `scipy solver`_.

- an "adhoc" solver: Simple in-house solver that ranks the technologies
according to cost and service the demand incrementally.

- "scipy" solver: Formulates investment as a true LP problem and solves it using
the `scipy solver`_.

- "cvxopt" solver: Formulates investment as a true LP problem and solves it
Copy link
Collaborator

@dalonsoa dalonsoa Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this actually working? I've never tried it or had to deal with it.

Copy link
Collaborator Author

@tsmbland tsmbland Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A quick test suggests the answer is no (at least in the current MUSE version - it's not covered by the tests so might have been accidentally broken along the way)

using the python package `cvxopt`_. `cvxopt`_ is *not* installed by default.
Users can install it with ``pip install cvxopt`` or ``conda install cvxopt``.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/add-agent/1-single-objective/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/add-agent/2-multiple-objective/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/add-correlation-demand/1-correlation/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/add-new-technology/1-introduction/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/add-new-technology/2-scenario/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/add-region/1-new-region/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/add-service-demand/1-exogenous-demand/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/carbon-budget/1-carbon-budget/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/min-max-timeslice-constraints/1-min-constraint/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/min-max-timeslice-constraints/2-max-constraint/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/modify-timing-data/1-modify-timeslices/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/modify-timing-data/2-modify-time-framework/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-code/new-decision-metric/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion src/muse/agents/factories.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,7 @@ def _standardize_inputs(

def _standardize_investing_inputs(
search_rules: Optional[Union[str, Sequence[str]]] = None,
investment: Union[Callable, str, Mapping] = "adhoc",
investment: Union[Callable, str, Mapping] = "scipy",
constraints: Optional[
Union[Callable, str, Mapping, Sequence[Union[str, Mapping]]]
] = None,
Expand Down
2 changes: 1 addition & 1 deletion src/muse/data/example/default/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion src/muse/data/example/default_retro/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.retro_and_new]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
2 changes: 1 addition & 1 deletion src/muse/data/example/default_timeslice/settings.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ commodities_out = '{path}/technodata/residential/CommOut.csv'
[sectors.residential.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/residential/ExistingCapacity.csv'
lpsolver = "scipy" # Optional, defaults to "adhoc"
lpsolver = "scipy" # Optional, defaults to "scipy"
constraints = [
# Optional, defaults to the constraints below
"max_production",
Expand Down
11 changes: 10 additions & 1 deletion src/muse/sectors/subsector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,8 @@ def factory(
name: str = "subsector",
timeslice_level: str | None = None,
) -> Subsector:
from logging import getLogger

from muse import constraints as cs
from muse import demand_share as ds
from muse import investments as iv
Expand All @@ -139,6 +141,13 @@ def factory(
msg = "Invalid parameter asset_threshhold. Did you mean asset_threshold?"
raise ValueError(msg)

# Raise warning if lpsolver is not specified (PR #587)
if not hasattr(settings, "lpsolver"):
msg = (
f"lpsolver not specified for subsector '{name}'. Defaulting to 'scipy'"
)
getLogger(__name__).warning(msg)

agents = agents_factory(
settings.agents,
settings.existing_capacity,
Expand All @@ -147,7 +156,7 @@ def factory(
year=current_year or int(technologies.year.min()),
asset_threshold=getattr(settings, "asset_threshold", 1e-12),
# only used by self-investing agents
investment=getattr(settings, "lpsolver", "adhoc"),
investment=getattr(settings, "lpsolver", "scipy"),
forecast=getattr(settings, "forecast", 5),
constraints=getattr(settings, "constraints", ()),
timeslice_level=timeslice_level,
Expand Down
1 change: 1 addition & 0 deletions tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,7 @@ def newcapa_agent(agent_args, technologies, stock) -> Agent:

@fixture
def retro_agent(agent_args, technologies, stock) -> Agent:
agent_args["investment"] = "adhoc" # fails with scipy solver, see # 587
return create_agent(agent_args, technologies, stock.capacity, "retrofit")


Expand Down