Switch to setup.cfg for metadata specification?

[pfmoore]

See pypa/sampleproject#56 - should we be recommending that new projects put their metadata in setup.cfg rather than in setup.py?

[theacodes]

I can't think of a good reason why we shouldn't.

[dstufft]

Main reason against it would be that it's a relatively new feature and doesn't fail gracefully if someone has an older setuptools.

[theacodes]

Our guides generally recommend people install the latest versions, or did you mean for people installing the package from sdist?

[dstufft]

Yea, it affects both the author and random people installing the package from sdist. This also affects services like RTD (which just came up in #pypa yesterday).

[theacodes]

Hrm. that makes things a bit more tricky.

[pfmoore]

Good point. I guess it depends on the role of the PUG, to an extent. If we're intending to be a discussion of the options available, I strongly believe this should be mentioned (as it's far nicer to use than the setup() approach), although we'd probably mention both approaches in that case. If we're intending to be an opinionated guide, then I'd prefer that we emphasised "modern" practice, rather than older conventions, although we should note the compatibility issues. If we're looking to be a guide to how to best navigate all of the nasty pitfalls that people might hit when packaging Python projects, then we need to mention the new form, explain why it can cause problems, and recommend the older form "until a sufficiently large proportion of users have caught up".

Looks like setuptools has had this feature since Dec 2016. If we don't mention it now, when should we start to do so? (If anyone says "when it's the default setuptools in RHEL" I'll scream 😄) If there was a clear timescale (once RTD switch, or once pip 18 comes out with PEP 518 support for requiring a specific version of setuptools, or something) then I'd be OK with waiting.

One of the things that prompted me to flag this up is that as far as I know, this feature was introduced with very little publicity - which I'm frustrated about because it's such an improvement in usability (IMO). Functionally it makes little difference, but by emphasising declarative metadata it's a major shift.

[theacodes]

If we're intending to be a discussion of the options available, I strongly believe this should be mentioned (as it's far nicer to use than the setup() approach), although we'd probably mention both approaches in that case.

Yes, I think this is very appropriate for something that goes into the guides section.

If we're intending to be an opinionated guide, then I'd prefer that we emphasised "modern" practice, rather than older conventions, although we should note the compatibility issues.

The "tutorials" are where we get opinionated. I plan to use flit for the new tutorial for packaging items, so this particular discussion is non-applicable. I do think that the guides on setuptools can and should say "do it this way if you can, but you can also do this way if you must".

If we're looking to be a guide to how to best navigate all of the nasty pitfalls that people might hit when packaging Python projects, then we need to mention the new form, explain why it can cause problems, and recommend the older form "until a sufficiently large proportion of users have caught up".

This is good stuff for a discussion article, but it can also be accommodated in a guide on setuptools.

Looks like setuptools has had this feature since Dec 2016. If we don't mention it now, when should we start to do so? (If anyone says "when it's the default setuptools in RHEL" I'll scream 😄) If there was a clear timescale (once RTD switch, or once pip 18 comes out with PEP 518 support for requiring a specific version of setuptools, or something) then I'd be OK with waiting.

I look to the rest of the PyPA to figure this out. :)

[pfmoore]

I plan to use flit for the new tutorial for packaging items

I don't want to divert the subject here, but given that we're talking about when it's reasonable to recommend a new feature, is it really consistent to be hesitant about recommending a setuptools feature that's been out for 9 months which can't be used when installing from source with an older setuptools, and yet be willing to recommend flit as the build tool when you can't install from source with that unless you use the as-yet unreleased pip 10 with PEP 517 support?

[dstufft]

I don't think we should recommend flit until pip has PEP 517 support, and even then I'm not sure what level of uptake makes sense.

There are some differences of course, flit restricts itself to stuff that can ship universal wheels, so for most people pip install is going to get a wheel and if you're installing from a sdist and your pip is too old, you get a hard failure instead of with setuptools feature you just get an UNKNOWN project with version UNKNOWN and no files.

[pfmoore]

There's also some questions we'd have to consider in terms of our overall position on project layout. Flit is taking the view that it supports a single "best practice" approach to project layout, so we'd need to be sure the guide is consistent with that - specifically flit doesn't currently support the layout of having code in a src subdirectory (currently under discussion in #320 and pypa/flit#115). I suspect (as @takluyver noted on the flit issue) that this a wider question than flit or the PUG can decide unilaterally.

[theacodes]

(summing @ncoghlan for his take as well)

As part of the new tutorials we wanted to experiment with using some newer, more opinionated tools as the first ones that users interact with. The idea is that while they aren't perfect for every use case, they are simple and present less friction. For example, we've gotten good feedback on the new installing packages tutorial that uses pipenv instead of pip & virtualenv. None of this is to say that we want to not document the other tools - for example, I won't replace the current installing packages tutorial with the pipenv one until I'm certain that the pip & virtualenv instructions have a new home under guides.

That said, flit still may or may not be as ready as pipenv is to be that entrypoint. Perhaps this feature of setuptools is a viable alternative for having a simple introduction. I'm personally open to writing both and seeing what everyone thinks - but if y'all feel strongly right now that flit isn't the right choice I'll focus on a simplified and opinionated tutorial using setuptools.

[ncoghlan]

(Tangent: for RHEL, my own guidance is already:

1. Use OpenShift, RHSCL, or softwarecollections.org to get a newer base Python & pip
2. Always runs pip install --upgrade pip setuptools wheel as the first command in a new virtual environment

/tangent)

As far as publishing guidance goes, I think we need to split that into two distinct categories:

1. Advice for newcomers to Python who just want to get their project onto PyPI, so they and their close peer group can download it from there. For this use case, we should be recommending tools that offer the simplest and most straightforward publisher experience, even if we have to put a caveat on the advice that says "Note: end users running older installation tools that also choose not to use your uploaded wheel files may not be able to install source packages published this way. If this is something that matters to you, please see this alternate guide that provides guidance on maximising your ability to reach users of older clients and platforms"

2. Advice specifically for professional software developers that have a target audience of "All the Pythonistas, with correct software, no matter what platform they're running".

I see this distinction as the packaging equivalent of getting newcomers to Python to start with Python 3.6, and then only later giving them the bad news about all the features they're going to have to stop using if they want to make their libraries support older Python versions.

On the topic of src directories, I think that falls victim to the same kind of distinction in the learning curve: it's a code correctness issue that should NOT be inflicted on beginner programmers, just like unit testing, type checkers, etc. Beginner code is tested by running it and asking yourself "Did that do what I wanted it to do?". Professionals need to back the hell off and stop trying to teach people to run marathons at a point where they're still trying to grasp the basics of walking.