Implement CLI options to set truncation thresholds

[zhukoff-pavel]

closes #12765

Example implementation of truncation limit setting with CLI flags

[nicoddemus]

Thanks @zhukoff-pavel, we appreciate the PR!

I would like to see what other maintainers think of this feature, overall I'm +1 on it, the only downside I see is the cognitive load of extra options... but perhaps others have different opinions here.

I also wonder if this is better implemented as an ini option instead... seems like it would be more common to set this in the config file in test suites where this feature is often wanted (and of course it is always pass this using the -o option in the command line).

[zhukoff-pavel]

@nicoddemus, thank you for your reply!

I also wonder if this is better implemented as an ini option instead... seems like it would be more common to set this in the config file in test suites where this feature is often wanted (and of course it is always pass this using the -o option in the command line).

Maybe it is better to implement an .ini option and CLI flag to e.g. temporarily override limits in a singular launch?

Is it possible to implement such behaviour with current option reading machinery (with some combination of addoption and addini methods)?

[nicoddemus]

Maybe it is better to implement an .ini option and CLI flag to e.g. temporarily override limits in a singular launch?

Your suggestion makes sense, however we have for awhile moved away from this because:

1. It involves a lot of code duplication
2. We have mechanisms today where users can use them interchangeably (use addopts to hard-code command-line options in the ini file, and use -o to pass config options in the command line).

For this reason nowadays we prefer to have either a command-line option (for features where users are likely to change between pytest invocations) or an ini option (where users are most likely to set once per repository).

Is it possible to implement such behaviour with current option reading machinery (with some combination of addoption and addini methods)?

Unfortunately no, it requires duplicating the code.

[zhukoff-pavel]

I see.
I can then rewrite this PR to implement .ini option instead, as it definitely looks more like a one-time config setting rather than per launch.

Thank you for a quick reply!

[nicoddemus]

I can then rewrite this PR to implement .ini option instead

Thanks! I suggest to wait a bit in case some other maintainer has comments or reservations about the feature though.

[nicoddemus]

Been awhile and nobody seems to be against the idea, so feel free to go ahead.

[nicoddemus]

Thanks @zhukoff-pavel!

Please take a look at my comments.

[zhukoff-pavel]

@nicoddemus , thank you for a thorough review!

Please have a look at the changes.

I implemented some logic regarding value of 0 being set in one mode or both at a time.

[nicoddemus]

Thanks @zhukoff-pavel!

Two minor things still missing, but after that it LGTM! 😁

Thanks for the patience.

[zhukoff-pavel]

@nicoddemus, thank you for review and ideas!

[nicoddemus]

Thanks a lot @zhukoff-pavel, appreciate the patience! 👍

Leaving this open for awhile in case anybody else wants to review it, otherwise I will merge it soon.