GH-142950: Process format specifiers before colourization in argparse help

[savannahostrowski]

• Issue: Cannot use format specifiers in argparse help #142950

[alexprengere]

Thanks for working on this and fixing the bug I introduced 😭 (I was planning to fix it today but saw your PR).
One minor nit: I tested it locally and this will raise when %% is encountered in the help text:

# parser.add_argument("--str", default="test", help="100%%")
#
Traceback (most recent call last):
  File "/Users/aprengere/Dev/cpython/Lib/argparse.py", line 1810, in _check_help
    formatter._expand_help(action)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/Users/aprengere/Dev/cpython/Lib/argparse.py", line 712, in _expand_help
    raise ValueError(f"invalid format specifier in: {help_string!r}")
ValueError: invalid format specifier in: '100%%'

This is due to the final check:

       if '%' in result:
            raise ValueError(f"invalid format specifier in: {help_string!r}")

Removing this check it makes test_invalid_help fail though.

[hamdanal]

I also hit the issue linked here in my nightly tests.
From a look at this fix, it looks like it would not handle all edge cases of % formatting. In addition to the case mentioned above #142960 (comment), the regex used here doesn't handle length modifiers. Here is a modified patch that handles both issues and uses a regex that emulated Python's % formatting as described here

Suggested change

	if not t.reset:
	return help_string % params
	# Format first to preserve types for specifiers, like %x that require int.
	def colorize(match):
	spec, name = match.group(0, 1)
	if spec == '%%':
	return '%'
	if name in params:
	formatted = spec % {name: params[name]}
	return f'{t.interpolated_value}{formatted}{t.reset}'
	return spec
	# Match %% or %(name)... format specifiers
	result = _re.sub(r'%%|%\((\w+)\)[^a-z]*[a-z]', colorize,
	help_string, flags=_re.IGNORECASE)
	if '%' in result:
	raise ValueError(f"invalid format specifier in: {help_string!r}")
	return result
	result = help_string % params # always format the whole string to raise on invalid input
	if not t.reset:
	return result
	# Format first to preserve types for specifiers, like %x that require int.
	def colorize(match):
	spec, name = match.group(0, 'mapping')
	if spec == '%%':
	return '%'
	if name in params:
	formatted = spec % {name: params[name]}
	return f'{t.interpolated_value}{formatted}{t.reset}'
	return spec
	# Match %% or %(name)... format specifiers
	result = _re.sub(
	r"""
	% # Percent character
	(?:\((?P<mapping>[^)]*)\))? # Mapping key
	(?P<flag>[#0\-+ ])? # Conversion Flags
	(?P<width>\*|\d+)? # Minimum field width
	(?P<precision>\.(?:\*?|\d*))? # Precision
	[hlL]? # Length modifier (ignored)
	(?P<format>[diouxXeEfFgGcrsa%]) # Conversion type
	""",
	colorize,
	help_string,
	flags=_re.VERBOSE,
	)
	return result

[serhiy-storchaka]

Any characters can occur in the key name, not only \w. It is even more complicated, because nested ( and ) should be balanced, but this is impossible to check with a regular expression.

There are formatting codes denoted by the upper case: "X", "E", "F" and "G".

Some letters before the format specifier should be ignored: "h", "l" and "L".