Test Optimization Proposal (SC-736)

[holmanb]

Reduce template rendering test runtime

Additional Context

Pytest consistently shows the parametrized template rendering tests as the most time-consuming in the unit test suite (0.5s per test). Profile analysis shows that nearly all of the time spent is in one-time setup tasks (importing modules). Currently each test invokes a subprocess to execute the test.

This change should be non-functional. It simply moves the rendering code into the templater module and uses function calls in place of spawning a new process per parametrized test (function calls share the setup cost of module import). A single test that calls the tool as a subprocess was added to maintain test coverage.

This saves ~7s on my machine (which is 13% of the total unittest runtime).

[TheRealFalcon]

Good idea. These tests have bugged me in the past too.

My only concern is when it comes to separating concerns, I'm not a big fan of having functions that combine taking cli/argparse args with additional logic, because those arguments can pretty hard to construct outside of calling from the CLI. That makes reuse and testing harder.

Can we instead have the render_cloudcfg function take arguments like variant, template, and output_file (or whatever), and leave all of the argument parsing where it was?

[holmanb]

Good feedback, I agree. Fixed in the latest commit.

[TheRealFalcon]

Looks good, thanks!