Move requirements-dev.txt dependencies to environment.yml

[seisman]

Description of proposed changes

Delete the 'requirements-dev.txt' file, and move two of the dependencies (make and codecov) to the conda 'environment.yml' file. This allows the "Install GMT and required dependencies" step in '.github/workflows/ci_tests.yaml' to be simplified.

All these packages/dependencies can be installed via pip, so the comments in
these two files are not accurate.

Reminders

• Run make format and make check to make sure the code follows the style guide.
• Add tests for new features or tests that would have caught the bug that you're fixing.
• Add new public functions/methods/classes to doc/api/index.rst.
• Write detailed docstrings for all functions/methods.
• If adding new functionality, add an example to docstrings or tutorials.

Notes

• You can write /format in the first line of a comment to lint the code automatically

[weiji14]

Since we're on this, can we think about consolidating PyGMT's dependency files:

• requirements.txt
• requirements-dev.txt
• environment.yml

This mess seems to be a hangover from the Fatiando a Terra days (#187, sorry Leo). We could probably move the requirements*.txt files to pyproject.toml and keep the environment.yml for conda only.

[seisman]

Since we're on this, can we think about consolidating PyGMT's dependency files:

• requirements.txt
• requirements-dev.txt
• environment.yml

This mess seems to be a hangover from the Fatiando a Terra days (#187, sorry Leo). We could probably move the requirements*.txt files to pyproject.toml and keep the environment.yml for conda only.

Do you mean poetry? It doesn't support dynamic versioning from setuptools_scm, see #667 (comment).

[weiji14]

Do you mean poetry? It doesn't support dynamic versioning from setuptools_scm, see #667 (comment).

Yes I'm thinking of poetry. The dynamic versioning is still defined in setup.py at this line

pygmt/setup.py

Line 38 in 4ed63f8

SETUP_REQUIRES = ["setuptools_scm"]

right? I still think we can move the dependency list to the pyproject.toml file (but keep the setup.py intact).

Let's keep this PR on hold for now, I'll try and see if I can move requirements*.txt to pyproject.toml after sorting out other higher priority feature PRs.

[willschlitzer]

@seisman and @weiji14 , you are definitely more experienced developers than I am, but I feel like it is common practice in Python repos to have a requirements.txt. While I understand wanting to consolidate files for the sake of organization, I think requirements.txt and requirements-dev.txt are pretty standard files, and are recognizable to someone looking to get involved in the project and needing to install the necessary packages.

[weiji14]

@seisman and @weiji14 , you are definitely more experienced developers than I am, but I feel like it is common practice in Python repos to have a requirements.txt. While I understand wanting to consolidate files for the sake of organization, I think requirements.txt and requirements-dev.txt are pretty standard files, and are recognizable to someone looking to get involved in the project and needing to install the necessary packages.

Traditionally yes, requirements.txt is a 'standard', but have a read about the blog post at https://snarky.ca/what-the-heck-is-pyproject-toml/ which mentions PEP518 (see also the original discussion at #667). There's actually a way to pip install --use-pep517, and using pyproject.toml will likely be more common in the future (though it will take time).

As a compromise, I'm happy to keep the requirements.txt file, but move requirements-dev.txt to pyproject.toml. And/or we can improve the page at https://www.pygmt.org/dev/install.html#installing-pygmt to make it easier for new developers to get started.

[willschlitzer]

@GenericMappingTools/python Do you have any inputs on whether we are going to move to requirements-dev.txt to pyproject.toml? I read the blog post that @weiji14 left and I don't see the advantages of switching to use a .toml file, but I'm also not too familiar with the work involved in Python packaging. I'm fine with whatever we go with, but think that our contributing guide should include how to use pyproject.toml if we do go with that option.

[weiji14]

@GenericMappingTools/python Do you have any inputs on whether we are going to move to requirements-dev.txt to pyproject.toml? I read the blog post that @weiji14 left and I don't see the advantages of switching to use a .toml file, but I'm also not too familiar with the work involved in Python packaging. I'm fine with whatever we go with, but think that our contributing guide should include how to use pyproject.toml if we do go with that option.

Let's keep requirements.txt (the compromise I mentioned in #812 (comment)), but delete requirement-dev.txt and move it to environment.yml (see 5612a68). The pyproject.toml shift will require some more work and documentation and can happen at a later date.

[weiji14]

So I tried moving the dependencies from requirements-dev.txt to environment.yml in commits 17613f2 and f822ab3, and now the CI tests can't find the GMT library 😅 Feel free to revert those changes, or I'll do more troubleshooting/edits after release v0.3.0.

[seisman]

So I tried moving the dependencies from requirements-dev.txt to environment.yml in commits 17613f2 and f822ab3, and now the CI tests can't find the GMT library 😅 Feel free to revert those changes, or I'll do more troubleshooting/edits after release v0.3.0.

From the build log, it seems that GMT is installed in the pygmt environment, but PyGMT is installed in the test environment.

#
# To activate this environment, use
#
#     $ conda activate pygmt
#
# To deactivate an active environment, use
#
#     $ conda deactivate

and

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/share/miniconda3/envs/test/lib/python3.7/site-packages/pygmt/__init__.py", line 39, in <module>
    _begin()
  File "/usr/share/miniconda3/envs/test/lib/python3.7/site-packages/pygmt/session_management.py", line 16, in begin
    with Session() as lib:
  File "/usr/share/miniconda3/envs/test/lib/python3.7/site-packages/pygmt/clib/session.py", line 183, in __enter__
    self.create("pygmt-session")
  File "/usr/share/miniconda3/envs/test/lib/python3.7/site-packages/pygmt/clib/session.py", line 333, in create
    restype=ctp.c_void_p,
  File "/usr/share/miniconda3/envs/test/lib/python3.7/site-packages/pygmt/clib/session.py", line 282, in get_libgmt_func
    self._libgmt = load_libgmt()
  File "/usr/share/miniconda3/envs/test/lib/python3.7/site-packages/pygmt/clib/loading.py", line 48, in load_libgmt
    "Error loading the GMT shared library "
pygmt.exceptions.GMTCLibNotFoundError: Error loading the GMT shared library libgmt.so.
 libgmt.so: cannot open shared object file: No such file or directory.
make: *** [Makefile:33: test] Error 1

[weiji14]

From the build log, it seems that GMT is installed in the pygmt environment, but PyGMT is installed in the test environment.

Ok, fixed in 6958959. Just needed to use conda env update --name test --file environment.yml.

[seisman]

From the build log, it seems that GMT is installed in the pygmt environment, but PyGMT is installed in the test environment.

Ok, fixed in 6958959. Just needed to use conda env update --name test --file environment.yml.

Not sure if it works, but I feel it's better to change the setup-miniconda default environment name to pygmt, and remove --name pygmt. See https://github.com/conda-incubator/setup-miniconda#use-a-different-environment-name-or-path

[seisman]

What about adding a few comment lines to group the dependencies into:

1. required dependencies
2. dev dependencies
3. CI-specific dependencies (codecov, and make?)

[weiji14]

Ok. But should we consider make a CI dependency, or a dev dependency? I feel as though make here might help on Windows, see #914 (comment).

[weiji14]

Perhaps just do required and dev dependencies (see the Suggested Changes I made below). codecov is a tiny 16.4kb wheel. make is 6.0MB on Windows but could be considered a dev dependency too.

[seisman]

Looks good.

[seisman]

Looks good to me, but I can't approve it because I opened this PR.

[weiji14]

Oops, almost forgot you opened this. Good to merge then!