Ubuntu Focal to Jammy migration support

[seunghun1ee]

Add playbook and shell scripts to support automated Ubuntu migration.

[technowhizz]

Overall, Good job guys @seunghun1ee @Alex-Welsh

[technowhizz]

LGTM 👍

Thank you for the changes @seunghun1ee

[markgoddard]

Would be nice to see some docs for this, but they can come later.

[markgoddard]

I think it would be sensible to set os_release to jammy at the beginning of the procedure, as we do for RL9. It's generally used as the 'target' release. This would avoid needing to pass in additional variables to pulp sync, host configure, etc.

[Alex-Welsh]

Would be nice to see some docs for this, but they can come later.

We have a wiki page for now. Docs are a bit awkward to update quickly. I'd like to have something more official in skc docs once it's stable though
https://wiki.stackhpc.com/doc/migrating-to-ubuntu-jammy-qtQNt9yw1m

[technowhizz]

@Alex-Welsh When approved and ready to merge, I think it would be a good idea to squash merge this PR so that it can easily be cherry picked

[Alex-Welsh]

Error with latest compute node:

TASK [Do release upgrade] ***********************************************************************************************************************************************************************************************************│···································································································································································································
Monday 05 February 2024  11:46:20 +0000 (0:00:00.531)       0:00:58.421 *******                                                                                                                                                      │···································································································································································································
fatal: [ts-hcomp01]: FAILED! => changed=true                                                                                                                                                                                         │···································································································································································································
  cmd:                                                                                                                                                                                                                               │···································································································································································································
  - do-release-upgrade                                                                                                                                                                                                               │···································································································································································································
  - -f                                                                                                                                                                                                                               │···································································································································································································
  - DistUpgradeViewNonInteractive                                                                                                                                                                                                    │···································································································································································································
  delta: '0:00:02.624216'                                                                                                                                                                                                            │···································································································································································································
  end: '2024-02-05 11:46:23.248756'                                                                                                                                                                                                  │···································································································································································································
  msg: non-zero return code                                                                                                                                                                                                          │···································································································································································································
  rc: 1                                                                                                                                                                                                                              │···································································································································································································
  start: '2024-02-05 11:46:20.624540'                                                                                                                                                                                                │···································································································································································································
  stderr: ''                                                                                                                                                                                                                         │···································································································································································································
  stderr_lines: <omitted>                                                                                                                                                                                                            │···································································································································································································
  stdout: |-                                                                                                                                                                                                                         │···································································································································································································
    Checking for a new Ubuntu release                                                                                                                                                                                                │···································································································································································································
    You have not rebooted after updating a package which requires a reboot. Please reboot before upgrading.                                                                                                                          │···································································································································································································
  stdout_lines: <omitted> 

Not sure if it's the node or the change to the ordering

[seunghun1ee]

Error with updated ordering:

TASK [Do release upgrade] ***********************************************************************************************************************************************************************************************************│···································································································································································································
Monday 05 February 2024  11:46:20 +0000 (0:00:00.531)       0:00:58.421 *******                                                                                                                                                      │···································································································································································································
fatal: [ts-hcomp01]: FAILED! => changed=true                                                                                                                                                                                         │···································································································································································································
  cmd:                                                                                                                                                                                                                               │···································································································································································································
  - do-release-upgrade                                                                                                                                                                                                               │···································································································································································································
  - -f                                                                                                                                                                                                                               │···································································································································································································
  - DistUpgradeViewNonInteractive                                                                                                                                                                                                    │···································································································································································································
  delta: '0:00:02.624216'                                                                                                                                                                                                            │···································································································································································································
  end: '2024-02-05 11:46:23.248756'                                                                                                                                                                                                  │···································································································································································································
  msg: non-zero return code                                                                                                                                                                                                          │···································································································································································································
  rc: 1                                                                                                                                                                                                                              │···································································································································································································
  start: '2024-02-05 11:46:20.624540'                                                                                                                                                                                                │···································································································································································································
  stderr: ''                                                                                                                                                                                                                         │···································································································································································································
  stderr_lines: <omitted>                                                                                                                                                                                                            │···································································································································································································
  stdout: |-                                                                                                                                                                                                                         │···································································································································································································
    Checking for a new Ubuntu release                                                                                                                                                                                                │···································································································································································································
    You have not rebooted after updating a package which requires a reboot. Please reboot before upgrading.                                                                                                                          │···································································································································································································
  stdout_lines: <omitted> 

This probably means that the host got newer version of kernel in current release.
As far as I know, rebooting with newly installed kernel will clear this issue.