Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
43a7820
add matrix to CI to expand tests
TinyMarsh May 19, 2023
51d66b2
Added GitHub workflow to check for broken links
dc2917 May 19, 2023
61e6293
Run workflow on push
dc2917 May 19, 2023
501ec0e
Recursively check for broken links
dc2917 May 19, 2023
70ef6a4
Update .github/workflows/ci.yml
TinyMarsh May 19, 2023
57c677e
Update .github/workflows/ci.yml
TinyMarsh May 19, 2023
e77742e
Update .github/workflows/ci.yml
TinyMarsh May 19, 2023
b4a27ca
Update .github/workflows/ci.yml
TinyMarsh May 19, 2023
f21eeb6
Run test on push to expand_ci_system branch
TinyMarsh May 19, 2023
0a99291
Update .github/workflows/ci.yml
dalonsoa May 19, 2023
098e940
Fix error in workflow file
TinyMarsh May 19, 2023
3d3be62
Merge branch 'expand_ci_system' of https://github.com/ImperialCollege…
TinyMarsh May 19, 2023
762a34f
Fix error in workflow
TinyMarsh May 19, 2023
59bb28f
Moved all that can be moved from setup.cfg to pyproject.toml.
jamesturner246 May 19, 2023
5711d68
Switch from technote-space to ruzickap link checker
dc2917 May 19, 2023
448e5ad
Added back run manually, added correct branches to run on, removed un…
dc2917 May 19, 2023
3a5d124
Remove targeting Python 3.10 and 3.11
TinyMarsh May 19, 2023
f6a6f9b
Remove targeting Python 3.10 and 3.11 from regression-tests
TinyMarsh May 19, 2023
f9d486f
Remove expand_ci_system branch from list of branches in CI workflow
TinyMarsh May 19, 2023
d79f8b4
Merge branch 'setup_pyproject_conversion' into expand_ci_system
TinyMarsh May 19, 2023
0b69bae
Update pyproject.toml with Python versions
TinyMarsh May 19, 2023
c30d6f5
Updated change log
dc2917 May 19, 2023
228b77f
Add entrypoint (via muse_main) to pyproject.toml
jamesturner246 May 19, 2023
5a521b6
Fix flake8 empty line complains
jamesturner246 May 19, 2023
0f4f034
More robust project data inclusion
jamesturner246 May 20, 2023
5e41855
Merge pull request #3 from ImperialCollegeLondon/entrypoint
jamesturner246 May 20, 2023
7211c32
Merge pull request #115 from ImperialCollegeLondon/broken_link_checker
dalonsoa May 22, 2023
ecd998b
Merge pull request #119 from ImperialCollegeLondon/expand_ci_system
dalonsoa May 22, 2023
b2f6c3a
isort and black cleanup.
jamesturner246 May 22, 2023
4f384f6
Moved all that can be moved from setup.cfg to pyproject.toml.
jamesturner246 May 19, 2023
6dab36c
Update pyproject.toml with Python versions
TinyMarsh May 19, 2023
7a057e2
Merge pull request #118 from ImperialCollegeLondon/setup_pyproject_co…
dalonsoa May 22, 2023
905ad1d
Merge branch 'develop_imperial' into pyproject_python_versions
TinyMarsh May 22, 2023
204e208
Merge branch 'pyproject_python_versions' of https://github.com/Imperi…
TinyMarsh May 22, 2023
f43e00b
Remove duplicate key declaration
TinyMarsh May 22, 2023
90cf501
Merge pull request #121 from ImperialCollegeLondon/pyproject_python_v…
dalonsoa May 23, 2023
6085c0b
Update CHANGELOG
dalonsoa May 23, 2023
588b2b5
Move bumpversion config from setup.cfg to .bumpversion.cfg
jamesturner246 May 23, 2023
d15dd0b
Remove pycodestyle config from pyproject.toml, since this is done by …
jamesturner246 May 23, 2023
49c2ebd
Merge pull request #124 from jamesturner246/develop_imperial
dalonsoa May 24, 2023
98720d4
Add python 3.9 to classifiers
dalonsoa May 24, 2023
1e0a20d
Add flake8 file and remove setup.cfg
dalonsoa May 24, 2023
6b5816c
Add setup.py so sphinx can pick config from pyproject.toml
dalonsoa May 24, 2023
0e77cee
Remove alias from pyproject.toml
dalonsoa May 24, 2023
4cf01e8
Update instructions on how to build the docs
dalonsoa May 24, 2023
8f98273
Check links when a PR opens
dalonsoa May 24, 2023
8f058ef
Make link checker verbose
dalonsoa May 24, 2023
b78d4aa
Remove verbose, add timeout
dalonsoa May 24, 2023
512ee59
Update bump2version configuration
dalonsoa May 26, 2023
f0347ee
Bump version: "1.0.0" → 1.0.1
dalonsoa May 26, 2023
bc3a249
Add publishing in TestPyPI and PyPI
dalonsoa May 26, 2023
68791ec
Update instructions in README
dalonsoa May 26, 2023
b4df93e
Fix version numbers
dalonsoa May 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[bumpversion]
current_version = 1.0.1
commit = True
tag = True

[bumpversion:file:src/muse/__init__.py]

[bumpversion:file:docs/conf.py]
11 changes: 11 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[flake8]
max-line-length = 88
exclude =
.tox,
.git,
.venv,
venv,
build,
__pycache__,
extend-ignore =
E203,W503
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ document the change (include PR #) - note reverse order of PR #s.
## Key checklist

- [ ] All tests pass: `$ python -m pytest`
- [ ] The documentation builds and looks OK: `$ python setup.py build_sphinx`
- [ ] The documentation builds and looks OK: `$ python -m sphinx -b html docs docs/build`

## Further checks

Expand Down
21 changes: 21 additions & 0 deletions .github/workflows/broken-link-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
on:
schedule:
- cron: 0 0 1 * * # run monthly
repository_dispatch: # run manually
types: [check-link]
push:
branches: [main, develop]
pull_request:
types: [opened, synchronize, reopened]

name: Broken Link Check
jobs:
check:
name: Broken Link Check
runs-on: ubuntu-latest
steps:
- name: Broken Link Check
uses: ruzickap/action-my-broken-link-checker@v2
with:
url: https://muse-os.readthedocs.io/en/latest/
cmd_params: '--timeout=20 --buffer-size=8192 --max-connections=3 --color=always --skip-tls-verification --header="User-Agent:curl/7.54.0"' # muffet parameters
97 changes: 88 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
name: QA and tests
name: QA, tests and publishing

on:
pull_request:
types: [opened, synchronize, reopened]
push:
branches: [main, develop]

branches: [main, develop]
tags:
- '*'
jobs:
# Checks the style using the pre-commit hooks
qa:
Expand All @@ -17,14 +18,21 @@ jobs:
# Only then, normal testing proceeds
unit-tests:
needs: qa
runs-on: ubuntu-latest
runs-on: ${{ matrix.os }}


strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ["3.8", "3.9"]

steps:
- uses: actions/checkout@v3
- name: Set up Python 3.8
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
Expand All @@ -48,14 +56,20 @@ jobs:

regression-tests:
needs: qa
runs-on: ubuntu-latest
runs-on: ${{ matrix.os }}

strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ["3.8", "3.9"]

steps:
- uses: actions/checkout@v3
- name: Set up Python 3.8
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
Expand All @@ -66,3 +80,68 @@ jobs:
# The regression tests (with non coverage)
- name: Regression tests
run: pytest -m "regression"

# If all tests pass, we try to build a wheel
build-wheel:
# needs: [regression-tests, unit-tests]
name: Build source distribution
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Build sdist
run: |
python -m pip install --upgrade build
python -m build

- uses: actions/upload-artifact@v3
with:
path: dist/MUSE*

# And if we are pushing a tag, then we try to publish it
publish-TestPyPI:
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
needs: build-wheel
name: Publish MUSE to TestPyPI
runs-on: ubuntu-latest
permissions:
id-token: write

steps:

- name: Download sdist artifact
uses: actions/download-artifact@v3
with:
name: artifact
path: dist

- name: Display structure of downloaded files
run: ls -R dist

- name: Publish package distributions to TestPyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/
skip-existing: true

publish-PyPI:
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
needs: publish-TestPyPI
name: Publish MUSE to PyPI
runs-on: ubuntu-latest
permissions:
id-token: write

steps:

- name: Download sdist artifact
uses: actions/download-artifact@v3
with:
name: artifact
path: dist

- name: Display structure of downloaded files
run: ls -R dist

- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Features

- Updating pyproject.toml with valid python versions ([#121](https://github.com/SGIModel/MUSE_OS/pull/121))
- Expand CI workflow ([#119](https://github.com/SGIModel/MUSE_OS/pull/119))
- Convert setup.cfg to pyproject.toml and add entrypoint ([#118](https://github.com/SGIModel/MUSE_OS/pull/118))
- Add GitHub action to check for broken links ([#115](https://github.com/SGIModel/MUSE_OS/pull/115))
- Inconsistent trade case study ([#58](https://github.com/SGIModel/MUSE_OS/pull/58))
- Update README.txt in installation path ([#57](https://github.com/SGIModel/MUSE_OS/pull/57))
- Add tests to check the case of not using retrofit agents ([#53](https://github.com/SGIModel/MUSE_OS/pull/53))
Expand Down
131 changes: 89 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,91 +1,139 @@
# ModUlar energy system Simulation Environment: MUSE

Installation
============
## Installation

Pre-requisite: A virtual environment
------------------------------------
### Recommended way

Although not strictly necessary, creating a [conda](https://www.anaconda.com/what-is-anaconda/)
virtual environment is highly recommended: it will isolate users and developers from changes
occuring on their operating system, and from conflicts between python packages. It ensures
reproducibility from day to day.
The recommended way for **end users** to access and use the tool is via `pipx`.
This will create an isolated environment and install MUSE-OS within in one go,
also letting you to invoke `muse` anywhere in your system.

Create a virtual env including python with:
1. Install and configure [`pipx`](https://pypa.github.io/pipx/) following the
instructions appropriate for your operative system. Make sure it works well before
moving on.
2. Install MUSE-OS with `pipx install MUSE-OS`. It might take a
while to complete, but afterwards updates should be pretty fast.
3. To run MUSE-OS just open a terminal and execute `muse`, with the appropriate input
arguments, if relevant. See section below about usage.

Whenever there is a new version of MUSE-OS, just run `pipx upgrade MUSE-OS` and
it will be downloaded and installed with no fuss.

### Alternative way

If you want to have a bit more control - or you don't want to use `pipx`,
just create a virtual environment first and then install `MUSE-OS`.

Although not strictly necessary, **creating a virtual environment is highly recommended**:
it will isolate users and developers from changes occuring on their operating system,
and from conflicts between python packages. It ensures reproducibility from day to day.

There are several ways of creating a virtual environment - below we list two of them.
Regardless of the method used, **once it has been created and activated**, you can install
`MUSE-OS` within using:

```bash
> conda create -n muse python=3.8
python -m pip install MUSE-OS
```

And then use it by invoking `muse` with the relevant input arguments.

#### Creating a virtual environment using `conda`

Create a virtual environment including python with:

```bash
conda create -n muse_env python=3.9
```

Activate the environment with:

```bash
> conda activate muse
conda activate muse_env
```

Later, to recover the system-wide "normal" python, deactivate the environment with:

```bash
> conda deactivate
conda deactivate
```

Installing muse itself
----------------------
#### Creating a virtual environment using `venv`

Once a virtual environment has been *activated*, as describe above, we can
install muse without fear of interfering with other python jobs. Run:
Create a virtual environment with:

```bash
> python -m pip install https://github.com/SGIModel/MUSE_OS.git#egg=muse
python -m pip install venv
python -m venv muse_env
```

Usage
-----
Activate the environment with:

```powershell
# In Powershell
muse_env\Scripts\Activate.ps1

# In Linux/MacOS
source muse_env/bin/activate
```

Later, to recover the system-wide "normal" python, deactivate the environment with:

```bash
deactivate
```

## Usage

Once installed, users can:

- activate the virtual environment (needed only once per session) as explained
above
- run `python -m muse --model default` to run the default example model
- run `python -m muse --model default --copy XXX` to copy the model to subfolder `XXX`.
- Alternatively, run `python -m muse settings.toml`, where `settings.toml` is an input
- run `muse --model default` to run the default example model
- run `muse --model default --copy XXX` to copy the model to subfolder `XXX`.
- Alternatively, run `muse settings.toml`, where `settings.toml` is an input
file for a custom model
- run `python -m muse --help` to get a description of the command-line arguments,
- run `muse --help` to get a description of the command-line arguments,
including the name of any additional models provided with MUSE.

Development
-----------
## Development

It is strongly recommened to use a conda virtual environment, as above. The simplest approach is to
It is strongly recommened to use a virtual environment, as above. The simplest approach is to
first download the muse code with:

```bash
> git clone https://github.com/SGIModel/MUSE_OS.git muse
> git clone https://github.com/SGIModel/MUSE_OS.git
```

```bash
cd MUSE_OS
# Create virtual environment - for development, this is typically called ".venv"
# Activate virtual environment. Finally, install muse:
python -m pip install -e ."[dev,doc]"
```

And then install the working directory into the conda environment:
Please note the quotation marks. The downloaded code can then be modified. The changes will be
automatically reflected in the environment.

To ensure the consistency of the code with other developers, install the pre-commit hooks with:

```bash
> # after activating the virtual environment with:
> # conda activate muse
> python -m pip install -e ."muse[dev,doc]"
python -m pip install pre-commit
pre-commit install
```

Please note the quotation marks. `muse` in the last line above is the path to source code that was
just downloaded with `git`. The downloaded code can then be modified. The changes will be
automatically reflected in the conda environment.
This will ensure that a series of quality assurance tools are run with every commit you make.

In the developing phase, MUSE can also be used to run test cases to check that the model would reproduce expected results from a defined set of input data.
Tests can be run with the command [pytest](https://docs.pytest.org/en/latest/), from
theb testing framework of the same name.
In the developing phase, MUSE can also be used to run test cases to check that the model would reproduce expected results from a defined set of input data. Tests can be run with the command [pytest](https://docs.pytest.org/en/latest/), from the testing framework of the same name.

The documentation can be built with:

```bash
> python setup.py docs
python -m sphinx -b html docs docs/build
```

The main page for the documentation can then be found at
`build\\sphinx\\html\\index.html` (or `build/sphinx/html/index.html` on Mac and Linux).
`docs\\build\\html\\index.html` (or `docs/build/html/index.html` on Mac and Linux).
The file can viewed from any web browser.

[vscode](https://code.visualstudio.com/) users will find that the repository is setup
Expand All @@ -95,7 +143,6 @@ or conda environment where to run the code. This will change the `.vscode/settin
file and add a user-specific path to it. Users should try and avoid commiting changes to
`.vscode/settings.json` indiscriminately.

Copyright
---------
## Copyrigh

Copyright © 2021 Imperial College London
Copyright © 2023 Imperial College London
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
project = "MUSE"
copyright = "2022, Sustainable Gas Institute"
author = "Imperial College London"
release = "1.0.0"
release = "1.0.1"
version = ".".join(release.split(".")[:2])

# -- General configuration ---------------------------------------------------
Expand Down
Loading