Separate environments and upgraded submission requirements

[lacava]

This PR introduces separate environments for each method in SRBench, much like what was done in the 2022 competition. This wil l make it much easier to test and maintain new methods as the project evolves.

Method submission are now structured to follow the competition 2022 format, which includes requiring methods to output sympy compatible models.

Methods are contained within folders in the algorithms directory (e.g., algorithms/feat). This folder should contain:

1. metadata.yml (required): A file describing your submission, following the descriptions in [submission/feat-example/metadata.yml][metadata].
2. regressor.py (required): a Python file that defines your method, named appropriately. See [submission/feat-example/regressor.py][regressor] for complete documentation.
   It should contain:
   • est: a sklearn-compatible Regressor object.
   • model(est, X=None): a function that returns a sympy-compatible string specifying the final model. It can optionally take the training data as an input argument. See guidance below.
   • eval_kwargs (optional): a dictionary that can specify method-specific arguments to evaluate_model.py.
3. LICENSE (optional) A license file
4. environment.yml (optional): a conda environment file that specifies dependencies for your submission.
   It will be used to update the baseline environment (environment.yml in the root directory).
   To the extent possible, conda should be used to specify the dependencies you need.
   If your method is part of conda, great! You can just put that in here and leave install.sh blank.
5. requirements.txt (option): a pypi requirements file. The script will run pip install -r requirements.txt if this file is found, before proceeding.
6. install.sh (optional): a bash script that installs your method.
   **Note: scripts should not require sudo permissions. The library and include paths should be directed to conda environment; the environmental variable $CONDA_PREFIX specifies the path to the environment.
7. additional files (optional): you may include a folder containing the code for your method in the submission. However it is much preferred to pull your code from a repo within install.sh.

TODO

• Since the requirements have been updated and the tests have been rewritten, at the moment, many methods are not passing the tests. Will merge once all methods are passing.
• updated docs on submitting algorithms.

Algorithm status

If you're tagged below, please take a look and see if you can help get your algorithm to install and test correctly. Thank you!

• afp
• geneticengine
• operon @foolnotion
• bingo
• gpgomea @marcovirgolin
• ps-tree @hengzhe-zhang
• dsr @brendenpetersen
• gplearn
• e2et
• gpzgd
• pysr @MilesCranmer (maybe just needs the test params to be set?)
• eplex
• gsgp
• qlattice
• eql
• itea
• sbp-gp
• feat
• mrgp
• taylorgp
• ffx
• nsga-dcgp
• uDSR @brendenpetersen
• TIR @folivetti
• DAG search

[lacava]

@marcovirgolin put the furniture instructions down and save your baby

[lacava]

thanks @foolnotion for the updates!

[MilesCranmer]

Is there anything I can help with? I keep seeing SR papers that exclude PySR as one of the baseline models simply because it's not in SRBench 🙁 ... Would be great to get this all working

[lacava]

Is there anything I can help with? I keep seeing SR papers that exclude PySR as one of the baseline models simply because it's not in SRBench 🙁 ... Would be great to get this all working

i'll plan to get it merged this week.

[lacava]

for some reason pysr is failing again @MilesCranmer

[MilesCranmer]

My guess is that the base conda env is not fixing all the versions, and was just updated to Python 3.12, which e.g., the conda-forge version of PySR was not yet packaged for.

Could not solve for environment specs
The following packages are incompatible
└─ pysr 0.16  is installable with the potential options
   ├─ pysr 0.16.0 would require
   │  └─ python >=3.10,<3.11.0a0 , which can be installed;
   ├─ pysr 0.16.0 would require
   │  └─ python >=3.11,<3.12.0a0 , which can be installed;
   ├─ pysr 0.16.0 would require
   │  └─ python >=3.8,<3.9.0a0 , which can be installed;
   └─ pysr 0.16.0 would require
      └─ python >=3.9,<3.10.0a0 , which can be installed.

[MilesCranmer]

Suggested change

	use-mamba: true
	use-mamba: true
	python-version: 3.11

Looks like the Python version was not set here, so recently updated to 3.12 which will break a bunch of packages that haven't yet released their 3.12 versions (PyTorch usually takes a while to do so, for example).

[MilesCranmer]

Suggested change

	use-mamba: true
	use-mamba: true
	python-version: 3.11