From 645b3e4219bc4134a09f01a7ea6c9e42fb46017c Mon Sep 17 00:00:00 2001
From: Jeff Newman <jeffnewman@camsys.com>
Date: Mon, 12 Sep 2022 18:57:14 -0500
Subject: [PATCH 01/29] update release instructions

---
 HOW_TO_RELEASE.md | 104 +++++++++++++++++-----------------------------
 1 file changed, 39 insertions(+), 65 deletions(-)

diff --git a/HOW_TO_RELEASE.md b/HOW_TO_RELEASE.md
index e8f2a42634..841b6c76f6 100644
--- a/HOW_TO_RELEASE.md
+++ b/HOW_TO_RELEASE.md
@@ -5,9 +5,9 @@
 01. Check that the branch you intend to release is passing tests on Travis.
     If it's not passing there you should not release it.
 
-00. Start from a completely clean conda environment 
-    and git repository.  Assuming you have `conda` installed, you can do so 
-    by starting where ActivitySim is not yet cloned (e.g. in an empty 
+00. Start from a completely clean conda environment
+    and git repository.  Assuming you have `conda` installed, you can do so
+    by starting where ActivitySim is not yet cloned (e.g. in an empty
     directory) and running:
     ```sh
     conda create -n TEMP-ASIM-DEV python=3.9 git gh -c conda-forge --override-channels
@@ -19,7 +19,7 @@
 
 00. Per project policy, code on the main branch should have been released,
     but if you are *preparing* a release then the code should be on the `develop`
-    branch.  Switch to that branch now, and make sure it is synced to the 
+    branch.  Switch to that branch now, and make sure it is synced to the
     version on GitHub:
     ```sh
     git switch develop
@@ -28,45 +28,46 @@
 
 00. Update your Conda environment for testing.  We do not want to use an
     existing environment on your machine, as it may be out-of-date
-    and we want to make sure everything passes muster using the 
+    and we want to make sure everything passes muster using the
     most up-to-date dependencies available.  The following command
-    will update the active environment (we made this to be `TEMP-ASIM-DEV` 
+    will update the active environment (we made this to be `TEMP-ASIM-DEV`
     if you followed the directions above).
     ```sh
     conda env update --file=conda-environments/activitysim-dev.yml
     ```
-    If you add to the ActivitySim dependencies, make sure to also update 
-    the environments in `conda-environments`, which are used for testing 
-    and development.  If they are not updated, these environments will end 
+    If you add to the ActivitySim dependencies, make sure to also update
+    the environments in `conda-environments`, which are used for testing
+    and development.  If they are not updated, these environments will end
     up with dependencies loaded from *pip* instead of *conda-forge*.
 
-00. Run pycodestyle to ensure that the codebase passes all style checks.
+00. Run `black` to ensure that the codebase passes minimal style checks.
     This check should only take a few seconds.  These checks are also done on
-    Travis and are platform independent, so they should not be necessary to
+    GitHub Actions and are platform independent, so they should not be necessary to
     replicate locally, but are listed here for completeness.
     ```sh
-    pycodestyle .
+    black --check --diff .
     ```
 
-00. Run the regular test suite on Windows. Travis tests are done on Linux,
-    but most users are on Windows, and the test suite should also be run
-    on Windows to ensure that it works on that platform as well.  If you
+00. Run the regular test suite on Windows. Most GitHub Actions tests are done on Linux,
+    Linux (it's faster to start up and run a new clean VM for testing) but most
+    users are on Windows, and the test suite should also be run on Windows to
+    ensure that it works on that platform as well.  If you
     are not preparing this release on Windows, you should be sure to run
-    at least through this step on a Windows machine before finalizing a 
-    release.  
-    
-    A few of the tests require pre-created data that is not included in the 
-    repository directly, but rather recreated on the fly before testing. The 
-    regular test suite takes some time to run, between about half an hour and 
+    at least through this step on a Windows machine before finalizing a
+    release.
+
+    A few of the tests require pre-created data that is not included in the
+    repository directly, but rather recreated on the fly before testing. The
+    regular test suite takes some time to run, between about half an hour and
     two hours depending on the specs of your machine.
     ```sh
     python activitysim/examples/placeholder_multiple_zone/scripts/two_zone_example_data.py
     python activitysim/examples/placeholder_multiple_zone/scripts/three_zone_example_data.py
     pytest .
     ```
- 
+
 00. Test the full-scale regional examples. These examples are big, too
-    large to run on Travis, and will take a lot of time (many hours) to 
+    large to run on GitHub Actions, and will take a lot of time (many hours) to
     download and run.
     ```sh
     mkdir tmp-asim
@@ -76,7 +77,7 @@
     call run_all_examples.bat
     ```
     These tests will run through the gamut even if some of them crash, so
-    if you don't sit and watch them go (please don't do this) you'll need 
+    if you don't sit and watch them go (please don't do this) you'll need
     to scan through the results to make sure there are no errors after the
     fact.
     ```sh
@@ -84,60 +85,33 @@
     ```
 
 00. Test the notebooks in `activitysim/examples/prototype_mtc/notebooks`.
-    There are also demo notebooks for estimation, but their functionality  
+    There are also demo notebooks for estimation, but their functionality
     is completely tested in the unit tests run previously.
 
-00. Use bump2version to tag the release commit and update the 
-    version number.  The following code will generate a "patch" release,
-    incrementing the third value in the version number (i.e. "1.2.3" 
-    becomes "1.2.4").  Alternatively, make a "minor" or "major" release. 
-    The `--list` command will generate output to your console to confirm 
-    that the old and new version numbers are what you expect, before you 
-    push the commit (with the changed version in the code) and tags to 
-    GitHub.
-    ```sh
-    bump2version patch --list
-    ```
-    
-    It is also possible to make a development pre-release. To do so, 
-    explicitly set the version number to the next patch plus a ".devN" 
-    suffix:  
-    
-    ```sh
-    bump2version patch --new-version 1.2.3.dev0 --list
-    ```
-    
-    Then, when ready to make a "final" release, set the version by 
-    explicitly removing the suffix:  
-    ```sh
-    bump2version patch --new-version 1.2.3 --list
-    ```
-
-00. Push the tagged commit to GitHub.
-    ```sh
-    git push --tags
-    ```
-
-00. For non-development releases, open a pull request to merge the proposed 
-    release into main. The following command will open a web browser for 
+00. For non-development releases, open a pull request to merge the proposed
+    release into main. The following command will open a web browser for
     you to create the pull request.
     ```sh
     gh pr create --web
     ```
     After creating the PR, confirm with the ActivitySim PMC that the release
     is ready before actually merging it.
-    
+
     Once final approval is granted, merge the PR into main.  The presence
     of the git tags added earlier will trigger automated build steps to
     prepare and deploy the release to pypi and conda-forge.
-    
+
 00. Create a "release" on GitHub.
     ```sh
     gh release create v1.2.3
     ```
+    The process of creating and tagging a release will automatically
+    trigger various GitHub Actions scripts to build, test, and publish the
+    new release to PyPI and conda forge, assuming there are no errors.
+
     For a development pre-release, include the `--prerelease` argument.
     As the project's policy is that only formally released code is merged
-    to the main branch, any pre-release should also be built against a 
+    to the main branch, any pre-release should also be built against a
     non-default branch.  For example, to pre-release from the `develop`
     branch:
     ```sh
@@ -147,11 +121,11 @@
         --notes "this pre-release is for a cool new feature" \
         --title "Development Pre-Release"
     ```
-    
-00. Clean up your workspace, including removing the Conda environment used for 
-    testing (which will prevent you from accidentally using an old 
+
+00. Clean up your workspace, including removing the Conda environment used for
+    testing (which will prevent you from accidentally using an old
     environment when you should have a fresh up-to-date one next time).
     ```sh
     conda deactivate
     conda env remove -n TEMP-ASIM-DEV
-    ```
\ No newline at end of file
+    ```

From fe65ce4aa4707b490ee0a132979753de2687bdef Mon Sep 17 00:00:00 2001
From: Jeff Newman <jeffnewman@camsys.com>
Date: Tue, 13 Sep 2022 10:10:55 -0500
Subject: [PATCH 02/29] auto-build docs on release

---
 .github/workflows/deployment.yml | 53 +++++++++++++++++++++++++++++++-
 HOW_TO_RELEASE.md                |  4 +++
 conda-environments/docbuild.yml  |  8 ++++-
 3 files changed, 63 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/deployment.yml b/.github/workflows/deployment.yml
index 3145ab0665..4b7389ec85 100644
--- a/.github/workflows/deployment.yml
+++ b/.github/workflows/deployment.yml
@@ -75,6 +75,57 @@ jobs:
           repository_url: https://test.pypi.org/legacy/
           verbose: true
 
+  docbuild:
+    needs: test-built-dist
+    if: github.event_name == 'release'
+    name: ubuntu-latest py3.9
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        shell: bash -l {0}
+    steps:
+      - uses: actions/download-artifact@v3
+        with:
+          name: releases
+          path: dist
+      - name: Set up Python 3.9
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.9
+      - name: Install dependencies
+        uses: conda-incubator/setup-miniconda@v2
+        with:
+          miniforge-variant: Mambaforge
+          miniforge-version: latest
+          use-mamba: true
+          environment-file: conda-environments/docbuild.yml
+          python-version: 3.9
+          activate-environment: docbuild
+          auto-activate-base: false
+          auto-update-conda: false
+      - name: Install activitysim
+        run: |
+          python -m pip install dist/activitysim-*.whl
+      - name: Conda checkup
+        run: |
+          conda info -a
+          conda list
+          echo REPOSITORY ${{ github.repository }}
+          echo REF ${{ github.ref }}
+          echo REF_NAME ${{ github.ref_name }}
+      - name: Build the docs
+        run: |
+          cd docs
+          make clean
+          make html
+      - name: Push to GitHub Pages
+        uses: peaceiris/actions-gh-pages@v3.8.0
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          # Token is created automatically by Github Actions, no other config needed
+          publish_dir: ./docs/_build/html
+          destination_dir: ${{ github.ref_name }}
+
   upload-to-pypi:
     needs: test-built-dist
     if: github.event_name == 'release'
@@ -89,4 +140,4 @@ jobs:
         with:
           user: __token__
           password: ${{ secrets.PYPI_TOKEN }}
-          verbose: true
\ No newline at end of file
+          verbose: true
diff --git a/HOW_TO_RELEASE.md b/HOW_TO_RELEASE.md
index 841b6c76f6..f57c91277f 100644
--- a/HOW_TO_RELEASE.md
+++ b/HOW_TO_RELEASE.md
@@ -129,3 +129,7 @@
     conda deactivate
     conda env remove -n TEMP-ASIM-DEV
     ```
+
+00. Change the default redirect page for the ActivitySim documentation to point
+    to the newly released documentation.  The redirect page can be edited
+    [here](https://github.com/ActivitySim/activitysim/blob/gh-pages/index.html).
diff --git a/conda-environments/docbuild.yml b/conda-environments/docbuild.yml
index a4617fcf60..c598d8606c 100644
--- a/conda-environments/docbuild.yml
+++ b/conda-environments/docbuild.yml
@@ -1,4 +1,11 @@
 # Environment for building docs
+#
+#   This conda environment is used to build the documentation.  It includes a
+#   number of dependencies needed for the documentation only, and not to run or
+#   test ActivitySim itself.  Note that ActivitySim itself is *not* installed
+#   in this environment, you must install it separately after using this recipe,
+#   which allows you to install the specific version you want.
+
 name: docbuild
 channels:
 - conda-forge
@@ -43,4 +50,3 @@ dependencies:
 
 - pip:
   - autodoc_pydantic
-  - -e ..

From af8cf94ea497130762443ba650e40ddc981a9c4f Mon Sep 17 00:00:00 2001
From: Jeff Newman <jeffnewman@camsys.com>
Date: Tue, 13 Sep 2022 10:16:16 -0500
Subject: [PATCH 03/29] remind to update switcher

---
 HOW_TO_RELEASE.md | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/HOW_TO_RELEASE.md b/HOW_TO_RELEASE.md
index f57c91277f..edbfee7c22 100644
--- a/HOW_TO_RELEASE.md
+++ b/HOW_TO_RELEASE.md
@@ -133,3 +133,7 @@
 00. Change the default redirect page for the ActivitySim documentation to point
     to the newly released documentation.  The redirect page can be edited
     [here](https://github.com/ActivitySim/activitysim/blob/gh-pages/index.html).
+
+00. Add the new release to the `switch.json` file. Don't delete the references
+    for existing old documentation.  The switcher can be edited
+    [here](https://github.com/ActivitySim/activitysim/blob/gh-pages/switcher.json).

From 7ffb4d8ba734599cc7df669d346fe6668971ec5e Mon Sep 17 00:00:00 2001
From: Jeff Newman <jeffnewman@camsys.com>
Date: Tue, 13 Sep 2022 19:50:11 -0500
Subject: [PATCH 04/29] need checkout

---
 .github/workflows/deployment.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.github/workflows/deployment.yml b/.github/workflows/deployment.yml
index 4b7389ec85..7b41abe603 100644
--- a/.github/workflows/deployment.yml
+++ b/.github/workflows/deployment.yml
@@ -84,6 +84,7 @@ jobs:
       run:
         shell: bash -l {0}
     steps:
+      - uses: actions/checkout@v3
       - uses: actions/download-artifact@v3
         with:
           name: releases

From e662c103abae0772cb9b09a60a77b3f870e8a2c6 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Wed, 21 Sep 2022 13:29:36 -0700
Subject: [PATCH 05/29] Re-applied changes to v1.1.3 code instead of v1.0.4
 code

---
 activitysim/core/config.py                                     | 2 +-
 activitysim/core/random.py                                     | 2 +-
 .../placeholder_sandag/test/configs_1_zone/settings.yaml       | 3 +++
 3 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/activitysim/core/config.py b/activitysim/core/config.py
index 02d6fadea2..bc9935add1 100644
--- a/activitysim/core/config.py
+++ b/activitysim/core/config.py
@@ -65,7 +65,7 @@ def pipeline_file_name(settings):
 
 @inject.injectable()
 def rng_base_seed():
-    return 0
+    return setting("rng_base_seed", 0)
 
 
 @inject.injectable(cache=True)
diff --git a/activitysim/core/random.py b/activitysim/core/random.py
index 12527840f4..ff4b7e7b17 100644
--- a/activitysim/core/random.py
+++ b/activitysim/core/random.py
@@ -531,7 +531,7 @@ def set_base_seed(self, seed=None):
         assert len(list(self.channels.keys())) == 0
 
         if seed is None:
-            self.base_seed = np.random.RandomState().randint(_MAX_SEED)
+            self.base_seed = np.random.RandomState().randint(_MAX_SEED, dtype = np.uint32)
             logger.debug("Set random seed randomly to %s" % self.base_seed)
         else:
             logger.debug("Set random seed base to %s" % seed)
diff --git a/activitysim/examples/placeholder_sandag/test/configs_1_zone/settings.yaml b/activitysim/examples/placeholder_sandag/test/configs_1_zone/settings.yaml
index bf0e418e96..c1ca83f8ff 100644
--- a/activitysim/examples/placeholder_sandag/test/configs_1_zone/settings.yaml
+++ b/activitysim/examples/placeholder_sandag/test/configs_1_zone/settings.yaml
@@ -7,6 +7,9 @@ strict: True
 households_sample_size: 30
 chunk_size: 0
 
+# random seed
+rng_base_seed: null
+
 # - shadow pricing global switches
 use_shadow_pricing: False
 

From d5a33c2b7b5cb8066ef5dac15652eaef51d0f670 Mon Sep 17 00:00:00 2001
From: Joe Flood <joejimflood@gmail.com>
Date: Mon, 24 Oct 2022 16:10:33 -0700
Subject: [PATCH 06/29] Update random.py

---
 activitysim/core/random.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/activitysim/core/random.py b/activitysim/core/random.py
index ff4b7e7b17..229ba022ac 100644
--- a/activitysim/core/random.py
+++ b/activitysim/core/random.py
@@ -531,7 +531,7 @@ def set_base_seed(self, seed=None):
         assert len(list(self.channels.keys())) == 0
 
         if seed is None:
-            self.base_seed = np.random.RandomState().randint(_MAX_SEED, dtype = np.uint32)
+            self.base_seed = np.random.RandomState().randint(_MAX_SEED, dtype=np.uint32)
             logger.debug("Set random seed randomly to %s" % self.base_seed)
         else:
             logger.debug("Set random seed base to %s" % seed)

From 6d4e7eaf1ec585cb636224736f07c21090a73c2c Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Mon, 24 Oct 2022 18:04:35 -0700
Subject: [PATCH 07/29] Updated test environment to reflect changes in pandas

---
 conda-environments/github-actions-tests.yml | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/conda-environments/github-actions-tests.yml b/conda-environments/github-actions-tests.yml
index 01108bdc48..77ab0af99d 100644
--- a/conda-environments/github-actions-tests.yml
+++ b/conda-environments/github-actions-tests.yml
@@ -1,4 +1,7 @@
 # Environment for testing in GitHub Actions
+#  This environment contains a minimal set of dependencies needed to run most tests.
+#  It does not install ActivitySim itself (which is done by the test scripts) and
+#  is not meant for use outside the CI tools.
 name: asim-test
 channels:
 - conda-forge
@@ -9,11 +12,11 @@ dependencies:
 - cytoolz >= 0.8.1
 - isort
 - nbmake
-- numba >= 0.55.2
+- numba = 0.55.2  # see https://github.com/conda-forge/numba-feedstock/pull/104
 - numpy >= 1.16.1,<=1.21
 - openmatrix >= 0.3.4.1
 - orca >= 1.6
-- pandas >= 1.1.0
+- pandas >= 1.1.0,<1.5
 - psutil >= 4.1
 - pyarrow >= 2.0
 - pypyr >= 5.3
@@ -23,7 +26,7 @@ dependencies:
 - pytest-regressions
 - pyyaml >= 5.1
 - requests >= 2.7
-- sharrow >= 2.2.4
+- sharrow >= 2.3.2
 - simwrapper > 1.7
 - xarray >= 0.21
 - zarr

From ef53ad1e653e8b57eb7f2739e3932dd8bc838061 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Mon, 24 Oct 2022 18:43:02 -0700
Subject: [PATCH 08/29] Removed random random seed generation from
 placeholder_sandage 1-zone test configuration to get example to pass
 automated test

---
 .../placeholder_sandag/test/configs_1_zone/settings.yaml       | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/activitysim/examples/placeholder_sandag/test/configs_1_zone/settings.yaml b/activitysim/examples/placeholder_sandag/test/configs_1_zone/settings.yaml
index c1ca83f8ff..bf0e418e96 100644
--- a/activitysim/examples/placeholder_sandag/test/configs_1_zone/settings.yaml
+++ b/activitysim/examples/placeholder_sandag/test/configs_1_zone/settings.yaml
@@ -7,9 +7,6 @@ strict: True
 households_sample_size: 30
 chunk_size: 0
 
-# random seed
-rng_base_seed: null
-
 # - shadow pricing global switches
 use_shadow_pricing: False
 

From 80b6330bdeb2e7f34a760fe36085195620525db6 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Wed, 26 Oct 2022 17:38:22 -0700
Subject: [PATCH 09/29] Added test_random_seed.py

---
 .../prototype_mtc/test/test_random_seed.py    | 116 ++++++++++++++++++
 1 file changed, 116 insertions(+)
 create mode 100644 activitysim/examples/prototype_mtc/test/test_random_seed.py

diff --git a/activitysim/examples/prototype_mtc/test/test_random_seed.py b/activitysim/examples/prototype_mtc/test/test_random_seed.py
new file mode 100644
index 0000000000..586ba8d98f
--- /dev/null
+++ b/activitysim/examples/prototype_mtc/test/test_random_seed.py
@@ -0,0 +1,116 @@
+# ActivitySim
+# See full license in LICENSE.txt.
+import os
+import subprocess
+import yaml
+from shutil import copy
+
+import pandas as pd
+import pandas.testing as pdt
+import pkg_resources
+
+from activitysim.core import inject
+
+def update_settings(settings_file, key, value):
+    with open(settings_file, 'r') as f:
+        settings = yaml.safe_load(f)
+        f.close()
+
+    settings[key] = value
+
+    with open(settings_file, 'w') as f:
+        yaml.safe_dump(settings, f)
+        f.close()
+
+
+
+def run_test_random_seed():
+
+    steps_to_run = [
+        'initialize_landuse',
+        'initialize_households',
+        'compute_accessibility',
+        'workplace_location',
+        'write_tables'
+        ]
+
+    def example_path(dirname):
+        resource = os.path.join("examples", "prototype_mtc", dirname)
+        return pkg_resources.resource_filename("activitysim", resource)
+
+    def test_path(dirname):
+        return os.path.join(os.path.dirname(__file__), dirname)
+
+    def create_rng_configs(rng_base_seed = None):
+        new_configs_dir = test_path('configs_random_seed_=_{}'.format(rng_base_seed))
+        new_settings_file = os.path.join(new_configs_dir, 'settings.yaml')
+        os.mkdir(new_configs_dir)
+        copy(
+            os.path.join(example_path('configs'), 'settings.yaml'),
+            new_configs_dir
+        )
+
+        update_settings(new_settings_file, 'models', steps_to_run)
+        if rng_base_seed != '': #Undefined
+            update_settings(new_settings_file, 'rng_base_seed', rng_base_seed)
+
+    #(run name, rng_base_seed value)
+    runs = [
+        ('0-a', 0),
+        ('0-b', 0),
+        ('1-a', 1),
+        ('1-b', 1),
+        ('None-a', None),
+        ('None-b', None),
+        ('Undefined', '')
+    ]
+
+    seeds = list(set(runs.values()))
+    for seed in seeds:
+        create_rng_configs(seed)
+
+    outputs = {}
+    def compare_frames(df1, df2, should_be_equal = True):
+        """
+        Compares df1 and df2 and raises an AssertionError if they are unequal when `should_be_equal` is True and equal when `should_be_equal` is False
+        """
+        if should_be_equal:
+            pdt.assert_frame_equal(outputs[df1], outputs[df2])
+        else:
+            try:
+                pdt.assert_frame_equal(outputs[df1], outputs[df2])
+            except AssertionError:
+                pass
+            else:
+                raise AssertionError
+
+    file_path = os.path.join(os.path.dirname(__file__), "simulation.py")
+
+    for name, seed in runs:
+
+        run_args = [
+            "-c",
+            test_path("configs_random_seed_=_{}".format(seed)),
+            "-d",
+            example_path("data"),
+            "-o",
+            test_path("output"),
+        ]
+
+        subprocess.run(["coverage", "run", "-a", file_path] + run_args, check=True)
+
+        #Read in output to memory to compare later
+        outputs[name] = pd.read_csv(os.path.join(test_path("output"), "final_persons.csv"))
+
+    check_outputs("0-a", "0-b", True)
+    check_outputs("0-a", "Undefined", True)
+    check_outputs("1-a", "1-b", True)
+    check_outputs("None-a", "None-b", False)
+    check_outputs("0-a", "1-a", False)
+    check_outputs("None-a", "0-a", False)
+    check_outputs("None-a", "1-a", False)
+    check_outputs("None-b", "0-a", False)
+    check_outputs("None-b", "1-a", False)
+
+if __name__ == "__main__":
+    run_test_random_seed()
\ No newline at end of file

From a24e2f74d7c19d6e49beab871b7f87273b584bf3 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Wed, 26 Oct 2022 17:43:22 -0700
Subject: [PATCH 10/29] blacken test_random_seed.py

---
 .../prototype_mtc/test/test_random_seed.py    | 63 ++++++++++---------
 1 file changed, 32 insertions(+), 31 deletions(-)

diff --git a/activitysim/examples/prototype_mtc/test/test_random_seed.py b/activitysim/examples/prototype_mtc/test/test_random_seed.py
index 586ba8d98f..85f8552a9b 100644
--- a/activitysim/examples/prototype_mtc/test/test_random_seed.py
+++ b/activitysim/examples/prototype_mtc/test/test_random_seed.py
@@ -11,28 +11,28 @@
 
 from activitysim.core import inject
 
+
 def update_settings(settings_file, key, value):
-    with open(settings_file, 'r') as f:
+    with open(settings_file, "r") as f:
         settings = yaml.safe_load(f)
         f.close()
 
     settings[key] = value
 
-    with open(settings_file, 'w') as f:
+    with open(settings_file, "w") as f:
         yaml.safe_dump(settings, f)
         f.close()
 
 
-
 def run_test_random_seed():
 
     steps_to_run = [
-        'initialize_landuse',
-        'initialize_households',
-        'compute_accessibility',
-        'workplace_location',
-        'write_tables'
-        ]
+        "initialize_landuse",
+        "initialize_households",
+        "compute_accessibility",
+        "workplace_location",
+        "write_tables",
+    ]
 
     def example_path(dirname):
         resource = os.path.join("examples", "prototype_mtc", dirname)
@@ -41,28 +41,25 @@ def example_path(dirname):
     def test_path(dirname):
         return os.path.join(os.path.dirname(__file__), dirname)
 
-    def create_rng_configs(rng_base_seed = None):
-        new_configs_dir = test_path('configs_random_seed_=_{}'.format(rng_base_seed))
-        new_settings_file = os.path.join(new_configs_dir, 'settings.yaml')
+    def create_rng_configs(rng_base_seed=None):
+        new_configs_dir = test_path("configs_random_seed_=_{}".format(rng_base_seed))
+        new_settings_file = os.path.join(new_configs_dir, "settings.yaml")
         os.mkdir(new_configs_dir)
-        copy(
-            os.path.join(example_path('configs'), 'settings.yaml'),
-            new_configs_dir
-        )
+        copy(os.path.join(example_path("configs"), "settings.yaml"), new_configs_dir)
 
-        update_settings(new_settings_file, 'models', steps_to_run)
-        if rng_base_seed != '': #Undefined
-            update_settings(new_settings_file, 'rng_base_seed', rng_base_seed)
+        update_settings(new_settings_file, "models", steps_to_run)
+        if rng_base_seed != "":  # Undefined
+            update_settings(new_settings_file, "rng_base_seed", rng_base_seed)
 
-    #(run name, rng_base_seed value)
+    # (run name, rng_base_seed value)
     runs = [
-        ('0-a', 0),
-        ('0-b', 0),
-        ('1-a', 1),
-        ('1-b', 1),
-        ('None-a', None),
-        ('None-b', None),
-        ('Undefined', '')
+        ("0-a", 0),
+        ("0-b", 0),
+        ("1-a", 1),
+        ("1-b", 1),
+        ("None-a", None),
+        ("None-b", None),
+        ("Undefined", ""),
     ]
 
     seeds = list(set(runs.values()))
@@ -70,7 +67,8 @@ def create_rng_configs(rng_base_seed = None):
         create_rng_configs(seed)
 
     outputs = {}
-    def compare_frames(df1, df2, should_be_equal = True):
+
+    def compare_frames(df1, df2, should_be_equal=True):
         """
         Compares df1 and df2 and raises an AssertionError if they are unequal when `should_be_equal` is True and equal when `should_be_equal` is False
         """
@@ -99,8 +97,10 @@ def compare_frames(df1, df2, should_be_equal = True):
 
         subprocess.run(["coverage", "run", "-a", file_path] + run_args, check=True)
 
-        #Read in output to memory to compare later
-        outputs[name] = pd.read_csv(os.path.join(test_path("output"), "final_persons.csv"))
+        # Read in output to memory to compare later
+        outputs[name] = pd.read_csv(
+            os.path.join(test_path("output"), "final_persons.csv")
+        )
 
     check_outputs("0-a", "0-b", True)
     check_outputs("0-a", "Undefined", True)
@@ -112,5 +112,6 @@ def compare_frames(df1, df2, should_be_equal = True):
     check_outputs("None-b", "0-a", False)
     check_outputs("None-b", "1-a", False)
 
+
 if __name__ == "__main__":
-    run_test_random_seed()
\ No newline at end of file
+    run_test_random_seed()

From d1087c4adad8f578168b5006cbd6f810b747ca3d Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 08:29:33 -0700
Subject: [PATCH 11/29] Moved random seed generation testing out of
 prototype_mtc folder and edited core_tests.yaml to call it

---
 .github/workflows/core_tests.yml              | 53 +++++++++++++++++++
 test/random_seed/configs/network_los.yaml     |  6 +++
 test/random_seed/configs/settings.yaml        | 25 +++++++++
 test/random_seed/simulation.py                | 15 ++++++
 .../random_seed}/test_random_seed.py          |  0
 5 files changed, 99 insertions(+)
 create mode 100644 test/random_seed/configs/network_los.yaml
 create mode 100644 test/random_seed/configs/settings.yaml
 create mode 100644 test/random_seed/simulation.py
 rename {activitysim/examples/prototype_mtc/test => test/random_seed}/test_random_seed.py (100%)

diff --git a/.github/workflows/core_tests.yml b/.github/workflows/core_tests.yml
index ef4b4acaf4..cbed682522 100644
--- a/.github/workflows/core_tests.yml
+++ b/.github/workflows/core_tests.yml
@@ -251,6 +251,59 @@ jobs:
         run: |
           python -m pytest activitysim/examples/${{ matrix.region }}/test --durations=0
 
+  random_seed_generation:
+    needs: foundation
+    env:
+      mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
+      python-version: 3.9
+      label: linux-64
+    defaults:
+      run:
+        shell: bash -l {0}
+    name: random_seed_generation_test
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Setup Mambaforge
+        uses: conda-incubator/setup-miniconda@v2
+        with:
+          miniforge-variant: Mambaforge
+          miniforge-version: latest
+          activate-environment: asim-test
+          use-mamba: true
+          python-version: ${{ env.python-version }}
+
+      - name: Set cache date for year and month
+        run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV
+
+      - uses: actions/cache@v2
+        with:
+          path: ${{ env.mamba-env-prefix }}
+          key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
+        id: cache
+
+      - name: Update environment
+        run: mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
+        if: steps.cache.outputs.cache-hit != 'true'
+
+      - name: Install activitysim
+        # installing without dependencies is faster, we trust that all needed dependencies
+        # are in the conda environment defined above.  Also, this avoids pip getting
+        # confused and reinstalling tables (pytables).
+        run: |
+          python -m pip install -e . --no-deps
+
+      - name: Conda checkup
+        run: |
+          mamba info -a
+          mamba list
+
+      - name: Test Random Seed Generation
+        run:
+          python -m pytest test/random_seed --durations=0
+
   estimation_mode:
     needs: foundation
     env:
diff --git a/test/random_seed/configs/network_los.yaml b/test/random_seed/configs/network_los.yaml
new file mode 100644
index 0000000000..1c4cd79daf
--- /dev/null
+++ b/test/random_seed/configs/network_los.yaml
@@ -0,0 +1,6 @@
+inherit_settings: True
+
+# read cached skims (using numpy memmap) from output directory (memmap is faster than omx )
+read_skim_cache: False
+# write memmapped cached skims to output directory after reading from omx, for use in subsequent runs
+write_skim_cache: False
diff --git a/test/random_seed/configs/settings.yaml b/test/random_seed/configs/settings.yaml
new file mode 100644
index 0000000000..abaebfa7b6
--- /dev/null
+++ b/test/random_seed/configs/settings.yaml
@@ -0,0 +1,25 @@
+inherit_settings: True
+
+# treat warnings as errors
+strict: True
+
+# number of households to simulate
+households_sample_size:  10
+chunk_size: 0
+
+# - shadow pricing global switches
+use_shadow_pricing: False
+
+# turn writing of sample_tables on and off for all models
+# (if True, tables will be written if DEST_CHOICE_SAMPLE_TABLE_NAME is specified in individual model settings)
+want_dest_choice_sample_tables: False
+
+cleanup_pipeline_after_run: True
+
+output_tables:
+  h5_store: False
+  action: include
+  prefix: final_
+  sort: True
+  tables:
+    - trips
diff --git a/test/random_seed/simulation.py b/test/random_seed/simulation.py
new file mode 100644
index 0000000000..8313dd45e7
--- /dev/null
+++ b/test/random_seed/simulation.py
@@ -0,0 +1,15 @@
+# ActivitySim
+# See full license in LICENSE.txt.
+
+import argparse
+import sys
+
+from activitysim.cli.run import add_run_args, run
+
+if __name__ == "__main__":
+
+    parser = argparse.ArgumentParser()
+    add_run_args(parser)
+    args = parser.parse_args()
+
+    sys.exit(run(args))
diff --git a/activitysim/examples/prototype_mtc/test/test_random_seed.py b/test/random_seed/test_random_seed.py
similarity index 100%
rename from activitysim/examples/prototype_mtc/test/test_random_seed.py
rename to test/random_seed/test_random_seed.py

From 80e291fbd7eff5d183193e945e157d58b9bb957d Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 09:18:24 -0700
Subject: [PATCH 12/29] Corrected name of check_outputs()

---
 test/random_seed/test_random_seed.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/random_seed/test_random_seed.py b/test/random_seed/test_random_seed.py
index 85f8552a9b..4cac36ee83 100644
--- a/test/random_seed/test_random_seed.py
+++ b/test/random_seed/test_random_seed.py
@@ -68,7 +68,7 @@ def create_rng_configs(rng_base_seed=None):
 
     outputs = {}
 
-    def compare_frames(df1, df2, should_be_equal=True):
+    def check_outputs(df1, df2, should_be_equal=True):
         """
         Compares df1 and df2 and raises an AssertionError if they are unequal when `should_be_equal` is True and equal when `should_be_equal` is False
         """

From 03590b834c2d857a67279dad5cf5d0257b6b55d3 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 09:30:03 -0700
Subject: [PATCH 13/29] Added print statements to debug script

---
 test/random_seed/test_random_seed.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/test/random_seed/test_random_seed.py b/test/random_seed/test_random_seed.py
index 4cac36ee83..d5e6681923 100644
--- a/test/random_seed/test_random_seed.py
+++ b/test/random_seed/test_random_seed.py
@@ -48,7 +48,7 @@ def create_rng_configs(rng_base_seed=None):
         copy(os.path.join(example_path("configs"), "settings.yaml"), new_configs_dir)
 
         update_settings(new_settings_file, "models", steps_to_run)
-        if rng_base_seed != "":  # Undefined
+        if rng_base_seed != "": # Undefined
             update_settings(new_settings_file, "rng_base_seed", rng_base_seed)
 
     # (run name, rng_base_seed value)
@@ -64,6 +64,7 @@ def create_rng_configs(rng_base_seed=None):
 
     seeds = list(set(runs.values()))
     for seed in seeds:
+        print('Creating config files for random seed = {}'.format(seed))
         create_rng_configs(seed)
 
     outputs = {}

From 125853204a440d60f49e4e187f444052f2dae259 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 09:33:21 -0700
Subject: [PATCH 14/29] Blacken test_random_seed.py

---
 test/random_seed/test_random_seed.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/random_seed/test_random_seed.py b/test/random_seed/test_random_seed.py
index d5e6681923..053949bab1 100644
--- a/test/random_seed/test_random_seed.py
+++ b/test/random_seed/test_random_seed.py
@@ -48,7 +48,7 @@ def create_rng_configs(rng_base_seed=None):
         copy(os.path.join(example_path("configs"), "settings.yaml"), new_configs_dir)
 
         update_settings(new_settings_file, "models", steps_to_run)
-        if rng_base_seed != "": # Undefined
+        if rng_base_seed != "":  # Undefined
             update_settings(new_settings_file, "rng_base_seed", rng_base_seed)
 
     # (run name, rng_base_seed value)
@@ -64,7 +64,7 @@ def create_rng_configs(rng_base_seed=None):
 
     seeds = list(set(runs.values()))
     for seed in seeds:
-        print('Creating config files for random seed = {}'.format(seed))
+        print("Creating config files for random seed = {}".format(seed))
         create_rng_configs(seed)
 
     outputs = {}

From 062a0de10d58c171288c647f33f8d14c00ff8c2e Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 09:50:06 -0700
Subject: [PATCH 15/29] Added pipe character to call that tests the random
 generator

---
 .github/workflows/core_tests.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/core_tests.yml b/.github/workflows/core_tests.yml
index cbed682522..d48e9db049 100644
--- a/.github/workflows/core_tests.yml
+++ b/.github/workflows/core_tests.yml
@@ -301,7 +301,7 @@ jobs:
           mamba list
 
       - name: Test Random Seed Generation
-        run:
+        run: |
           python -m pytest test/random_seed --durations=0
 
   estimation_mode:

From ffb18be03374783d27d74e5a016f8546266cb68f Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 11:14:59 -0700
Subject: [PATCH 16/29] Added explicit call for pytest to run
 test_random_seed.py

---
 .github/workflows/core_tests.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/core_tests.yml b/.github/workflows/core_tests.yml
index d48e9db049..16867e4066 100644
--- a/.github/workflows/core_tests.yml
+++ b/.github/workflows/core_tests.yml
@@ -302,7 +302,7 @@ jobs:
 
       - name: Test Random Seed Generation
         run: |
-          python -m pytest test/random_seed --durations=0
+          python -m pytest test/random_seed/test_random_seed.py --durations=0
 
   estimation_mode:
     needs: foundation

From 666f389749014a70250ed6a69e5a005275b5a967 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 11:37:38 -0700
Subject: [PATCH 17/29] Added print statement at start of script for debugging

---
 test/random_seed/test_random_seed.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/test/random_seed/test_random_seed.py b/test/random_seed/test_random_seed.py
index 053949bab1..85a5e2d60c 100644
--- a/test/random_seed/test_random_seed.py
+++ b/test/random_seed/test_random_seed.py
@@ -1,3 +1,5 @@
+print('Running Test')
+
 # ActivitySim
 # See full license in LICENSE.txt.
 import os

From 2b81d9dcafec2d06e790412e84df149678aa9d07 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 11:38:37 -0700
Subject: [PATCH 18/29] Replaced single quotes with double quotes in print
 statement

---
 test/random_seed/test_random_seed.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/random_seed/test_random_seed.py b/test/random_seed/test_random_seed.py
index 85a5e2d60c..8e6523c459 100644
--- a/test/random_seed/test_random_seed.py
+++ b/test/random_seed/test_random_seed.py
@@ -1,4 +1,4 @@
-print('Running Test')
+print("Running Test")
 
 # ActivitySim
 # See full license in LICENSE.txt.

From 810d52549218e7411f215402a7be901e871cf6a8 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 14:06:51 -0700
Subject: [PATCH 19/29] Moved random seed test into activitysim folder

---
 {test => activitysim/tests}/random_seed/configs/network_los.yaml | 0
 {test => activitysim/tests}/random_seed/configs/settings.yaml    | 0
 {test => activitysim/tests}/random_seed/simulation.py            | 0
 {test => activitysim/tests}/random_seed/test_random_seed.py      | 0
 4 files changed, 0 insertions(+), 0 deletions(-)
 rename {test => activitysim/tests}/random_seed/configs/network_los.yaml (100%)
 rename {test => activitysim/tests}/random_seed/configs/settings.yaml (100%)
 rename {test => activitysim/tests}/random_seed/simulation.py (100%)
 rename {test => activitysim/tests}/random_seed/test_random_seed.py (100%)

diff --git a/test/random_seed/configs/network_los.yaml b/activitysim/tests/random_seed/configs/network_los.yaml
similarity index 100%
rename from test/random_seed/configs/network_los.yaml
rename to activitysim/tests/random_seed/configs/network_los.yaml
diff --git a/test/random_seed/configs/settings.yaml b/activitysim/tests/random_seed/configs/settings.yaml
similarity index 100%
rename from test/random_seed/configs/settings.yaml
rename to activitysim/tests/random_seed/configs/settings.yaml
diff --git a/test/random_seed/simulation.py b/activitysim/tests/random_seed/simulation.py
similarity index 100%
rename from test/random_seed/simulation.py
rename to activitysim/tests/random_seed/simulation.py
diff --git a/test/random_seed/test_random_seed.py b/activitysim/tests/random_seed/test_random_seed.py
similarity index 100%
rename from test/random_seed/test_random_seed.py
rename to activitysim/tests/random_seed/test_random_seed.py

From 3c3a8bee5b4e0530b99cea3be1a1407281db73c0 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 14:17:09 -0700
Subject: [PATCH 20/29] Edited core_tests.yml to reflect change in location of
 random seed test

---
 .github/workflows/core_tests.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/core_tests.yml b/.github/workflows/core_tests.yml
index 16867e4066..55f9f85105 100644
--- a/.github/workflows/core_tests.yml
+++ b/.github/workflows/core_tests.yml
@@ -302,7 +302,7 @@ jobs:
 
       - name: Test Random Seed Generation
         run: |
-          python -m pytest test/random_seed/test_random_seed.py --durations=0
+          python -m pytest activitysim/test/random_seed/test_random_seed.py --durations=0
 
   estimation_mode:
     needs: foundation

From f85a70cc9a2b2dfee759fb9282241a6fe7d34bd7 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 14:39:22 -0700
Subject: [PATCH 21/29] Renamed tests folder to test to reflect call in
 core-tests.yml

---
 activitysim/{tests => test}/random_seed/configs/network_los.yaml | 0
 activitysim/{tests => test}/random_seed/configs/settings.yaml    | 0
 activitysim/{tests => test}/random_seed/simulation.py            | 0
 activitysim/{tests => test}/random_seed/test_random_seed.py      | 0
 4 files changed, 0 insertions(+), 0 deletions(-)
 rename activitysim/{tests => test}/random_seed/configs/network_los.yaml (100%)
 rename activitysim/{tests => test}/random_seed/configs/settings.yaml (100%)
 rename activitysim/{tests => test}/random_seed/simulation.py (100%)
 rename activitysim/{tests => test}/random_seed/test_random_seed.py (100%)

diff --git a/activitysim/tests/random_seed/configs/network_los.yaml b/activitysim/test/random_seed/configs/network_los.yaml
similarity index 100%
rename from activitysim/tests/random_seed/configs/network_los.yaml
rename to activitysim/test/random_seed/configs/network_los.yaml
diff --git a/activitysim/tests/random_seed/configs/settings.yaml b/activitysim/test/random_seed/configs/settings.yaml
similarity index 100%
rename from activitysim/tests/random_seed/configs/settings.yaml
rename to activitysim/test/random_seed/configs/settings.yaml
diff --git a/activitysim/tests/random_seed/simulation.py b/activitysim/test/random_seed/simulation.py
similarity index 100%
rename from activitysim/tests/random_seed/simulation.py
rename to activitysim/test/random_seed/simulation.py
diff --git a/activitysim/tests/random_seed/test_random_seed.py b/activitysim/test/random_seed/test_random_seed.py
similarity index 100%
rename from activitysim/tests/random_seed/test_random_seed.py
rename to activitysim/test/random_seed/test_random_seed.py

From 00ad0aa161aaf00cbb31a8a9d302effc9e9e28e4 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 16:13:00 -0700
Subject: [PATCH 22/29] Added test_random_seed function for PyTest to read

---
 activitysim/test/random_seed/test_random_seed.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/activitysim/test/random_seed/test_random_seed.py b/activitysim/test/random_seed/test_random_seed.py
index 8e6523c459..d777fabace 100644
--- a/activitysim/test/random_seed/test_random_seed.py
+++ b/activitysim/test/random_seed/test_random_seed.py
@@ -116,5 +116,9 @@ def check_outputs(df1, df2, should_be_equal=True):
     check_outputs("None-b", "1-a", False)
 
 
+def test_random_seed():
+    run_test_random_seed()
+
+
 if __name__ == "__main__":
     run_test_random_seed()

From 730c5f9b4b40eace2156b4c292f6b0c848da9ebe Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 16:36:32 -0700
Subject: [PATCH 23/29] Fixed defition of seeds list and removed print
 statements

---
 activitysim/test/random_seed/test_random_seed.py | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/activitysim/test/random_seed/test_random_seed.py b/activitysim/test/random_seed/test_random_seed.py
index d777fabace..adf84e0867 100644
--- a/activitysim/test/random_seed/test_random_seed.py
+++ b/activitysim/test/random_seed/test_random_seed.py
@@ -1,5 +1,3 @@
-print("Running Test")
-
 # ActivitySim
 # See full license in LICENSE.txt.
 import os
@@ -64,9 +62,8 @@ def create_rng_configs(rng_base_seed=None):
         ("Undefined", ""),
     ]
 
-    seeds = list(set(runs.values()))
+    seeds = list(set([run[1] for run in runs]))
     for seed in seeds:
-        print("Creating config files for random seed = {}".format(seed))
         create_rng_configs(seed)
 
     outputs = {}

From 0fbb38c32ac55fef135e680fe09ebd08b850c1b7 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 17:01:58 -0700
Subject: [PATCH 24/29] Added line to create output folder as that's not being
 committed

---
 activitysim/test/random_seed/test_random_seed.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/activitysim/test/random_seed/test_random_seed.py b/activitysim/test/random_seed/test_random_seed.py
index adf84e0867..7c7e005836 100644
--- a/activitysim/test/random_seed/test_random_seed.py
+++ b/activitysim/test/random_seed/test_random_seed.py
@@ -95,6 +95,7 @@ def check_outputs(df1, df2, should_be_equal=True):
             test_path("output"),
         ]
 
+        os.mkdir(test_path("output"))
         subprocess.run(["coverage", "run", "-a", file_path] + run_args, check=True)
 
         # Read in output to memory to compare later

From 053da39679f70623c3e57d1c3a7d794fb1e70be5 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 17:16:30 -0700
Subject: [PATCH 25/29] create_rng_configs() now copies entire example configs
 directory instead of just settings.yaml

---
 activitysim/test/random_seed/test_random_seed.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/activitysim/test/random_seed/test_random_seed.py b/activitysim/test/random_seed/test_random_seed.py
index 7c7e005836..544d9f1649 100644
--- a/activitysim/test/random_seed/test_random_seed.py
+++ b/activitysim/test/random_seed/test_random_seed.py
@@ -45,7 +45,8 @@ def create_rng_configs(rng_base_seed=None):
         new_configs_dir = test_path("configs_random_seed_=_{}".format(rng_base_seed))
         new_settings_file = os.path.join(new_configs_dir, "settings.yaml")
         os.mkdir(new_configs_dir)
-        copy(os.path.join(example_path("configs"), "settings.yaml"), new_configs_dir)
+        for f in os.listdir(example_path("configs")):
+            copy(os.path.join(example_path("configs"), f), new_configs_dir)
 
         update_settings(new_settings_file, "models", steps_to_run)
         if rng_base_seed != "":  # Undefined

From b6ed4f8c4166bc1a0cedf557948586e9f681ebe8 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 27 Oct 2022 17:30:04 -0700
Subject: [PATCH 26/29] Added try except statement when creating the output
 directory in case it already exists

---
 activitysim/test/random_seed/test_random_seed.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/activitysim/test/random_seed/test_random_seed.py b/activitysim/test/random_seed/test_random_seed.py
index 544d9f1649..fc9d17fe60 100644
--- a/activitysim/test/random_seed/test_random_seed.py
+++ b/activitysim/test/random_seed/test_random_seed.py
@@ -96,7 +96,11 @@ def check_outputs(df1, df2, should_be_equal=True):
             test_path("output"),
         ]
 
-        os.mkdir(test_path("output"))
+        try:
+            os.mkdir(test_path("output"))
+        except FileExistsError:
+            pass
+
         subprocess.run(["coverage", "run", "-a", file_path] + run_args, check=True)
 
         # Read in output to memory to compare later

From 8b2c0286e38c4272cc6685658068f5d925fb423b Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 17 Nov 2022 16:20:31 -0800
Subject: [PATCH 27/29] Applied suggested changes

---
 HOW_TO_RELEASE.md                             | 112 ++++++++++--------
 activitysim/test/random_seed/.gitignore       |   2 +
 .../test/random_seed/configs/settings.yaml    |  25 ----
 .../test/random_seed/test_random_seed.py      |   2 +
 4 files changed, 69 insertions(+), 72 deletions(-)
 create mode 100644 activitysim/test/random_seed/.gitignore
 delete mode 100644 activitysim/test/random_seed/configs/settings.yaml

diff --git a/HOW_TO_RELEASE.md b/HOW_TO_RELEASE.md
index edbfee7c22..e8f2a42634 100644
--- a/HOW_TO_RELEASE.md
+++ b/HOW_TO_RELEASE.md
@@ -5,9 +5,9 @@
 01. Check that the branch you intend to release is passing tests on Travis.
     If it's not passing there you should not release it.
 
-00. Start from a completely clean conda environment
-    and git repository.  Assuming you have `conda` installed, you can do so
-    by starting where ActivitySim is not yet cloned (e.g. in an empty
+00. Start from a completely clean conda environment 
+    and git repository.  Assuming you have `conda` installed, you can do so 
+    by starting where ActivitySim is not yet cloned (e.g. in an empty 
     directory) and running:
     ```sh
     conda create -n TEMP-ASIM-DEV python=3.9 git gh -c conda-forge --override-channels
@@ -19,7 +19,7 @@
 
 00. Per project policy, code on the main branch should have been released,
     but if you are *preparing* a release then the code should be on the `develop`
-    branch.  Switch to that branch now, and make sure it is synced to the
+    branch.  Switch to that branch now, and make sure it is synced to the 
     version on GitHub:
     ```sh
     git switch develop
@@ -28,46 +28,45 @@
 
 00. Update your Conda environment for testing.  We do not want to use an
     existing environment on your machine, as it may be out-of-date
-    and we want to make sure everything passes muster using the
+    and we want to make sure everything passes muster using the 
     most up-to-date dependencies available.  The following command
-    will update the active environment (we made this to be `TEMP-ASIM-DEV`
+    will update the active environment (we made this to be `TEMP-ASIM-DEV` 
     if you followed the directions above).
     ```sh
     conda env update --file=conda-environments/activitysim-dev.yml
     ```
-    If you add to the ActivitySim dependencies, make sure to also update
-    the environments in `conda-environments`, which are used for testing
-    and development.  If they are not updated, these environments will end
+    If you add to the ActivitySim dependencies, make sure to also update 
+    the environments in `conda-environments`, which are used for testing 
+    and development.  If they are not updated, these environments will end 
     up with dependencies loaded from *pip* instead of *conda-forge*.
 
-00. Run `black` to ensure that the codebase passes minimal style checks.
+00. Run pycodestyle to ensure that the codebase passes all style checks.
     This check should only take a few seconds.  These checks are also done on
-    GitHub Actions and are platform independent, so they should not be necessary to
+    Travis and are platform independent, so they should not be necessary to
     replicate locally, but are listed here for completeness.
     ```sh
-    black --check --diff .
+    pycodestyle .
     ```
 
-00. Run the regular test suite on Windows. Most GitHub Actions tests are done on Linux,
-    Linux (it's faster to start up and run a new clean VM for testing) but most
-    users are on Windows, and the test suite should also be run on Windows to
-    ensure that it works on that platform as well.  If you
+00. Run the regular test suite on Windows. Travis tests are done on Linux,
+    but most users are on Windows, and the test suite should also be run
+    on Windows to ensure that it works on that platform as well.  If you
     are not preparing this release on Windows, you should be sure to run
-    at least through this step on a Windows machine before finalizing a
-    release.
-
-    A few of the tests require pre-created data that is not included in the
-    repository directly, but rather recreated on the fly before testing. The
-    regular test suite takes some time to run, between about half an hour and
+    at least through this step on a Windows machine before finalizing a 
+    release.  
+    
+    A few of the tests require pre-created data that is not included in the 
+    repository directly, but rather recreated on the fly before testing. The 
+    regular test suite takes some time to run, between about half an hour and 
     two hours depending on the specs of your machine.
     ```sh
     python activitysim/examples/placeholder_multiple_zone/scripts/two_zone_example_data.py
     python activitysim/examples/placeholder_multiple_zone/scripts/three_zone_example_data.py
     pytest .
     ```
-
+ 
 00. Test the full-scale regional examples. These examples are big, too
-    large to run on GitHub Actions, and will take a lot of time (many hours) to
+    large to run on Travis, and will take a lot of time (many hours) to 
     download and run.
     ```sh
     mkdir tmp-asim
@@ -77,7 +76,7 @@
     call run_all_examples.bat
     ```
     These tests will run through the gamut even if some of them crash, so
-    if you don't sit and watch them go (please don't do this) you'll need
+    if you don't sit and watch them go (please don't do this) you'll need 
     to scan through the results to make sure there are no errors after the
     fact.
     ```sh
@@ -85,33 +84,60 @@
     ```
 
 00. Test the notebooks in `activitysim/examples/prototype_mtc/notebooks`.
-    There are also demo notebooks for estimation, but their functionality
+    There are also demo notebooks for estimation, but their functionality  
     is completely tested in the unit tests run previously.
 
-00. For non-development releases, open a pull request to merge the proposed
-    release into main. The following command will open a web browser for
+00. Use bump2version to tag the release commit and update the 
+    version number.  The following code will generate a "patch" release,
+    incrementing the third value in the version number (i.e. "1.2.3" 
+    becomes "1.2.4").  Alternatively, make a "minor" or "major" release. 
+    The `--list` command will generate output to your console to confirm 
+    that the old and new version numbers are what you expect, before you 
+    push the commit (with the changed version in the code) and tags to 
+    GitHub.
+    ```sh
+    bump2version patch --list
+    ```
+    
+    It is also possible to make a development pre-release. To do so, 
+    explicitly set the version number to the next patch plus a ".devN" 
+    suffix:  
+    
+    ```sh
+    bump2version patch --new-version 1.2.3.dev0 --list
+    ```
+    
+    Then, when ready to make a "final" release, set the version by 
+    explicitly removing the suffix:  
+    ```sh
+    bump2version patch --new-version 1.2.3 --list
+    ```
+
+00. Push the tagged commit to GitHub.
+    ```sh
+    git push --tags
+    ```
+
+00. For non-development releases, open a pull request to merge the proposed 
+    release into main. The following command will open a web browser for 
     you to create the pull request.
     ```sh
     gh pr create --web
     ```
     After creating the PR, confirm with the ActivitySim PMC that the release
     is ready before actually merging it.
-
+    
     Once final approval is granted, merge the PR into main.  The presence
     of the git tags added earlier will trigger automated build steps to
     prepare and deploy the release to pypi and conda-forge.
-
+    
 00. Create a "release" on GitHub.
     ```sh
     gh release create v1.2.3
     ```
-    The process of creating and tagging a release will automatically
-    trigger various GitHub Actions scripts to build, test, and publish the
-    new release to PyPI and conda forge, assuming there are no errors.
-
     For a development pre-release, include the `--prerelease` argument.
     As the project's policy is that only formally released code is merged
-    to the main branch, any pre-release should also be built against a
+    to the main branch, any pre-release should also be built against a 
     non-default branch.  For example, to pre-release from the `develop`
     branch:
     ```sh
@@ -121,19 +147,11 @@
         --notes "this pre-release is for a cool new feature" \
         --title "Development Pre-Release"
     ```
-
-00. Clean up your workspace, including removing the Conda environment used for
-    testing (which will prevent you from accidentally using an old
+    
+00. Clean up your workspace, including removing the Conda environment used for 
+    testing (which will prevent you from accidentally using an old 
     environment when you should have a fresh up-to-date one next time).
     ```sh
     conda deactivate
     conda env remove -n TEMP-ASIM-DEV
-    ```
-
-00. Change the default redirect page for the ActivitySim documentation to point
-    to the newly released documentation.  The redirect page can be edited
-    [here](https://github.com/ActivitySim/activitysim/blob/gh-pages/index.html).
-
-00. Add the new release to the `switch.json` file. Don't delete the references
-    for existing old documentation.  The switcher can be edited
-    [here](https://github.com/ActivitySim/activitysim/blob/gh-pages/switcher.json).
+    ```
\ No newline at end of file
diff --git a/activitysim/test/random_seed/.gitignore b/activitysim/test/random_seed/.gitignore
new file mode 100644
index 0000000000..4d692c93ae
--- /dev/null
+++ b/activitysim/test/random_seed/.gitignore
@@ -0,0 +1,2 @@
+# Configuration files created by test
+configs_random_seed_*/
\ No newline at end of file
diff --git a/activitysim/test/random_seed/configs/settings.yaml b/activitysim/test/random_seed/configs/settings.yaml
deleted file mode 100644
index abaebfa7b6..0000000000
--- a/activitysim/test/random_seed/configs/settings.yaml
+++ /dev/null
@@ -1,25 +0,0 @@
-inherit_settings: True
-
-# treat warnings as errors
-strict: True
-
-# number of households to simulate
-households_sample_size:  10
-chunk_size: 0
-
-# - shadow pricing global switches
-use_shadow_pricing: False
-
-# turn writing of sample_tables on and off for all models
-# (if True, tables will be written if DEST_CHOICE_SAMPLE_TABLE_NAME is specified in individual model settings)
-want_dest_choice_sample_tables: False
-
-cleanup_pipeline_after_run: True
-
-output_tables:
-  h5_store: False
-  action: include
-  prefix: final_
-  sort: True
-  tables:
-    - trips
diff --git a/activitysim/test/random_seed/test_random_seed.py b/activitysim/test/random_seed/test_random_seed.py
index fc9d17fe60..d0d39a43f4 100644
--- a/activitysim/test/random_seed/test_random_seed.py
+++ b/activitysim/test/random_seed/test_random_seed.py
@@ -85,6 +85,8 @@ def check_outputs(df1, df2, should_be_equal=True):
 
     file_path = os.path.join(os.path.dirname(__file__), "simulation.py")
 
+    # running prototype mtc model through workplace_location with 3 random seed settings, 0, 1, None, and undefined.
+    # final_persons.csv is compared to ensure the same setting returns the same variables and a different setting returns something different.
     for name, seed in runs:
 
         run_args = [

From 5d543fd6ade2945de2feee3c9cf580411deddb21 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 17 Nov 2022 16:48:00 -0800
Subject: [PATCH 28/29] Removed activitysim\test\random_seed\configs directory
 to reduce confusion

---
 activitysim/test/random_seed/configs/network_los.yaml | 6 ------
 1 file changed, 6 deletions(-)
 delete mode 100644 activitysim/test/random_seed/configs/network_los.yaml

diff --git a/activitysim/test/random_seed/configs/network_los.yaml b/activitysim/test/random_seed/configs/network_los.yaml
deleted file mode 100644
index 1c4cd79daf..0000000000
--- a/activitysim/test/random_seed/configs/network_los.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-inherit_settings: True
-
-# read cached skims (using numpy memmap) from output directory (memmap is faster than omx )
-read_skim_cache: False
-# write memmapped cached skims to output directory after reading from omx, for use in subsequent runs
-write_skim_cache: False

From 067c717bf3474bf6d52ce325714d1d1208a93018 Mon Sep 17 00:00:00 2001
From: JoeJimFlood <joejimflood@gmail.com>
Date: Thu, 17 Nov 2022 17:10:23 -0800
Subject: [PATCH 29/29] Moved random seed test directory to  folder that had
 already existed

---
 .github/workflows/core_tests.yml                           | 2 +-
 {activitysim/test => test}/random_seed/.gitignore          | 0
 {activitysim/test => test}/random_seed/simulation.py       | 0
 {activitysim/test => test}/random_seed/test_random_seed.py | 0
 4 files changed, 1 insertion(+), 1 deletion(-)
 rename {activitysim/test => test}/random_seed/.gitignore (100%)
 rename {activitysim/test => test}/random_seed/simulation.py (100%)
 rename {activitysim/test => test}/random_seed/test_random_seed.py (100%)

diff --git a/.github/workflows/core_tests.yml b/.github/workflows/core_tests.yml
index 55f9f85105..16867e4066 100644
--- a/.github/workflows/core_tests.yml
+++ b/.github/workflows/core_tests.yml
@@ -302,7 +302,7 @@ jobs:
 
       - name: Test Random Seed Generation
         run: |
-          python -m pytest activitysim/test/random_seed/test_random_seed.py --durations=0
+          python -m pytest test/random_seed/test_random_seed.py --durations=0
 
   estimation_mode:
     needs: foundation
diff --git a/activitysim/test/random_seed/.gitignore b/test/random_seed/.gitignore
similarity index 100%
rename from activitysim/test/random_seed/.gitignore
rename to test/random_seed/.gitignore
diff --git a/activitysim/test/random_seed/simulation.py b/test/random_seed/simulation.py
similarity index 100%
rename from activitysim/test/random_seed/simulation.py
rename to test/random_seed/simulation.py
diff --git a/activitysim/test/random_seed/test_random_seed.py b/test/random_seed/test_random_seed.py
similarity index 100%
rename from activitysim/test/random_seed/test_random_seed.py
rename to test/random_seed/test_random_seed.py