Setuptools does not pass config_settings through backend

[di]

The build project now has a --config-setting flag to pass options to the backend:

$ python -m build --help
usage: python -m build [-h] [--version] [--sdist] [--wheel] [--outdir dir] [--skip-dependencies]
                       [--no-isolation] [--config-setting CONFIG_SETTING]
                       [srcdir]

- - - >8 - - -

  --config-setting CONFIG_SETTING, -C CONFIG_SETTING
                        pass option to the backend

I wanted to use this to set wheel's --plat-name argument like so:

$ python -m build --wheel --config-setting plat-name=foo

I found that this worked as expected until the settings got to setuptool's _build_with_temp_dir function:

setuptools/setuptools/build_meta.py

Lines 190 to 200 in d368be2

	def _build_with_temp_dir(self, setup_command, result_extension,
	result_directory, config_settings):
	config_settings = self._fix_config(config_settings)
	result_directory = os.path.abspath(result_directory)
	# Build in a temporary directory, then copy to the target.
	os.makedirs(result_directory, exist_ok=True)
	with tempfile.TemporaryDirectory(dir=result_directory) as tmp_dist_dir:
	sys.argv = (sys.argv[:1] + setup_command +
	['--dist-dir', tmp_dist_dir] +
	config_settings["--global-option"])

In this example, _build_with_temp_dir gets called with config_settings={'plat-name': 'foo'}, which then gets 'fixed' into config_settings={'plat-name': 'foo', '--global-option': []}, and then setuptools ignores everything in config_settings and only considers --global-option.

It almost seems like --global-option itself could be used for this, but unfortunately it doesn't work as it's value is passed to lots of things that don't accept the arguments I'm hoping to pass:

$ python -m build --wheel --config-setting="--global-option=--plat-name" --config-setting="--global-option=foo" -n
usage: _in_process.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: _in_process.py --help [cmd1 cmd2 ...]
   or: _in_process.py --help-commands
   or: _in_process.py cmd --help

error: option --plat-name not recognized
...

I created a commit that "fixes" this for me as a proof of concept: fc95b3b

Possibly related issues:

• this is very similar to the request in build meta - allow moving the build folder #1816, but @gaborbernat is asking for the ability to pass options before the setup command, rather than after.
• build_meta:__legacy__ config_settings not compatible with pip options #1928 may also be related

[jaraco]

I could have used something like this too for Setuptools itself to disable post-release tags for #2500 whereas currently it's using this hack. I suspect Setuptools needs to devise a way to map "config settings" to tweak knobs at different scopes in the build process (global, per-command, ...).

[abitrolly]

@di why not put this into PR?

[di]

@abitrolly Because I'm not sure it's right. The means by which a frontend passes these arguments to the backend is not well defined.

[abitrolly]

@di what is missing here https://www.python.org/dev/peps/pep-0517/#config-settings ?

[di]

@abitrolly All these phrases from that section indicate that this is not well defined:

Build backends MAY assign any semantics they like to this dictionary.

Build frontends SHOULD provide some mechanism for users to specify arbitrary string-key/string-value pairs to be placed in this dictionary.

Build frontends MAY also provide arbitrary other mechanisms for users to place entries in this dictionary

[abitrolly]

@di I proposed some clarifications in python/peps#1766 Not sure where it should go from there.

Normalizing parameters to be passed as "option": "value" without dashes seems to be a good compromise. Because interface to build system is defined as callback functions and not as CLI API, keeping dashes would mean that backend should somehow pass the names through argparse.

[FFY00]

FYI I am also in favor of removing the dashes, I don't think they are needed in PEP 517.

[layday]

Incidentally, global options in setuptools seem to mean something different than they do in pip where they are placed in front of the setuptools command. Here, they are appended to the command. pip users might find the discrepancy surprising.

I'm not sure why arguments to sdist and bdist_wheel should not be funneled through config_settings verbatim - why is --global-option needed at all?

[jaraco]

I'm sure the reason why the PEP is non-specific about the format is because it would be difficult for a protocol like that to be specific without being over-constrained for a particular use-case.

I'm not sure why arguments to sdist and bdist_wheel should not be funneled through config_settings verbatim - why is --global-option needed at all?

I imagine that the sdist and bdist_wheel commands have settings that are unique to the command. I also expect there could be settings that are applied globally, independent of any command.

Regardless, what I'd like to see happen is for someone to examine the code and determine all the command-line parameters setuptools currently solicits that would be relevant to a build backend... and determine at what scopes those settings are relevant (globally, per command, other?). From there, we can create a scheme, something simple, maybe namespaced, to accept those parameters through config_settings.