testing: simplify test_upgrade.py (SC-185)

[TheRealFalcon]

Proposed Commit Message

testing: simplify test_upgrade.py

test_upgrade.py was outputting a ton of stuff that had to be manually
collected and verified. This commit adds more assertions to the test
and outputs directly to the logs rather than separate files.

Additional Context

The current "automated" way of running upgrade tests is actually slower than running things manually. All of the info was output to files that had to be manually collected and side-by-side compared. Verifying the upgrade will still require finding the output in the logs and doing a quick compare, but that will be a lot faster than dealing with the external files.

Here's an example of what the log looks like inline with the test output:

2021-06-29 18:11:38 INFO      integration_testing.test_upgrade:test_upgrade.py:131 
=== `systemd-analyze` before:
Startup finished in 5.795s (userspace) = 2d 18h 12min 59.459s
graphical.target reached after 4.653s in userspace
=== `systemd-analyze` after:
Startup finished in 3.819s (userspace) = 2d 18h 13min 23.847s
graphical.target reached after 2.624s in userspace

=== `systemd-analyze blame` before (first 10 lines):
          1.574s snapd.seeded.service
           854ms cloud-init.service
           731ms cloud-init-local.service
           655ms cloud-config.service
           563ms systemd-networkd-wait-online.service
           485ms cloud-final.service
           428ms apparmor.service
           300ms snapd.service
           129ms systemd-udev-trigger.service
           110ms networkd-dispatcher.service
=== `systemd-analyze blame` after (first 10 lines):
           769ms systemd-networkd-wait-online.service
           732ms cloud-init.service
           687ms cloud-init-local.service
           619ms cloud-config.service
           571ms cloud-final.service
           200ms snapd.service
           120ms systemd-udev-trigger.service
           102ms networkd-dispatcher.service
            63ms apparmor.service
            57ms accounts-daemon.service

=== `cloud-init analyze show` before:')
Finished stage: (init-local) 00.28600 seconds
Finished stage: (init-network) 00.43300 seconds
Finished stage: (modules-config) 00.19000 seconds
Finished stage: (modules-final) 00.04900 seconds
Total Time: 0.95800 seconds
=== `cloud-init analyze show` after:')
Finished stage: (init-local) 00.23300 seconds
Finished stage: (init-network) 00.31500 seconds
Finished stage: (modules-config) 00.17600 seconds
Finished stage: (modules-final) 00.06200 seconds
Total Time: 0.78600 seconds

=== `cloud-init analyze blame` before (first 10 lines): ')
-- Boot Record 01 --
     00.17400s (init-network/config-ssh)
     00.11100s (init-network/config-users-groups)
     00.09700s (modules-config/config-grub-dpkg)
     00.06600s (modules-config/config-apt-configure)
     00.02800s (init-local/search-NoCloud)
     00.01300s (modules-final/config-keys-to-console)
     00.00700s (init-network/config-resizefs)
     00.00500s (modules-final/config-final-message)
     00.00500s (modules-config/config-locale)
=== `cloud-init analyze blame` after (first 10 lines): ')
-- Boot Record 01 --
     00.14300s (init-network/config-ssh)
     00.09300s (modules-config/config-grub-dpkg)
     00.06100s (modules-config/config-apt-configure)
     00.02700s (init-local/search-NoCloud)
     00.02200s (init-network/config-users-groups)
     00.02100s (modules-final/config-keys-to-console)
     00.00600s (modules-final/config-final-message)
     00.00500s (init-network/config-resizefs)
     00.00500s (init-network/config-growpart)

Test Steps

pytest tests/integration_tests/test_upgrade.py

Checklist:

• My code follows the process laid out in the documentation
• I have updated or added any unit tests accordingly
• I have updated or added any documentation accordingly

[blackboxsw]

Was this omission from output or comparisons intentional? It seems like this check could be part of # Ensure important things stayed the same

[TheRealFalcon]

It was intentional. I think the original intent of this check was to show that networking stayed the same, even when upgrading from ENI to netplan. In a post-xenial world, everything is netplan.

I can add the check back though. Checking that networking hasn't changed at all can still be valuable.