Merged
Conversation
… the prior solve only pushed it to 0.20 which is incompatible with Apple's M1/M2 processors. Updating lock files accordingly.
…Base class. This makes it consistent with the current argparse config
…e name as a commandline parameter. The default filename remains environment.yml
…eating environments for yet-unsupported versions of Python
…nning format consistent
…onment.yml files and for updating lockfiles from a given environment file.
The rebuild-locks command now runs a vulnerability scan on the packages reported in the new lockfiles. A make-archive command has been added to allow for making offline archives of an existing environment. The archives are os-specific. It's a simple wrapper for conda-pack.
…at are necessary for functioning environments.
…ng syntax for exact vs. fuzzy matching for conda-forge packages since it appears conda-lock doesn't properly parse these specifications when the channel scope is also provided
Most importantly, in-place updates to the tinyRNA environment are no longer supported. The environment must be removed and recreated. When this functionality was originally added, dependency updates were taking place within the same Python version. Now they are across Python versions, and this can cause many, many problems. I tried several approaches to preserve this functionality, and while some got close, they were not stable or reliable. This is the safest way. Additionally: - Updated the version of miniconda that the script installs (if necessary) - The Apple Silicon version of Miniconda is installed, if appropriate and necessary - The Miniconda installer is executed with bash as it appears the latest version doesn't work with zsh - If Miniconda was installed, the user is instructed to source their shell's .rc file to finalize installation - It is now possible to run the script from outside of the project directory
Collaborator
|
Test installation and run on MacBook-M2 and server. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The tinyRNA environment and its dependencies have been upgraded to use Python 3.10. This PR also improves the developer experience for managing dependencies and updating lockfiles.
Environment Management
The
update-lockfiles.shscript has been replaced withenv.py. The previous script only converted an environment.yml file (manually created) to lockfiles for osx and linux. The new script offers the following commands:new-env: Automated creation of new environment.yml files for a specified version of Python and a template file. The file environment_template.yml has been provided. If additional dependencies or pinned specifications are required, they should be manually added to the template before passing it to the command. All unpinned specifications are solved with mamba before the new file is written.rebuild-locks: Rebuilds the (OS-specific) lockfiles from the (OS-agnostic) environment file using conda-lock. Once complete, the new lockfiles are scanned for vulnerable Conda packages using jake. Lockfiles produced using this command will overwrite any existing lockfiles so that they are immediately available for use withsetup.sh(i.e. unlike the environment.yml files produced above, their filenames do not indicate the version of Python)make-archive: Produces a compressed archive of an existing Conda environment using conda-pack. The archive can be used to perform a fully offline installation of the environment on another machine, even if the requested packages don't exist in the target machine's local package cache. The archive is OS-specific, however.Installation Script Changes and Improvements
The installation script no longer supports in-place upgrades for existing tinyRNA environments. The environment, and any packages the user may have installed, must now be removed by the script during an update. This functionality had been introduced when dependency updates were frequent but the Python version remained unchanged. Unfortunately it seems that there isn't a reliable way to change an environment's Python version without re-solving the environment, which brings back the unacceptable 20-40 minute wait times. If users really want to do an in-place upgrade, they can always use
conda update --filewith the desired environment*.yml file to initiate the re-solve but this is advanced usage and likely to cause issues.Additionally:
Security Improvements
This PR improves the handling of configuration file fields to avoid potential shell injection vulnerabilities.
Closes #261