Schema users

[blackboxsw]

Proposed Commit Message

schema: add json defs for module cc_users_groups

Additional support for markdown docs of nested $refs:
- Support types as list for doc rendering versus single {"type": "string"}
- Add _sort_property_order so 'array' and 'object' types are sorted last
  when multiple types are listed via oneOf or type lists
- Decorate enum values with double back-ticks and replace "string" type
- Add _flatten_schema_refs support for extracting $defs into inline schema
  definition

LP: #1858930

Additional Context

Test Steps

# To see general doc changes due to refactors in schema.py it'd be good to review before and after for modules unrelated to cc_users_groups
git checkout upstream/main
PYTHONPATH=. python3 -m cloudinit.cmd.main devel schema --docs all > orig-docs
git checkout <this_branch>
PYTHONPATH=. python3 -m cloudinit.cmd.main devel schema --docs all > new-docs
diff -urN orig-docs new-docs

Also review full rendered docs for typos, omissions,decoration,grammar etc.
tox -e doc; xdg-open 

Checklist:

• My code follows the process laid out in the documentation
• I have updated or added any unit tests accordingly
• I have updated or added any documentation accordingly

[TheRealFalcon]

Some really nice changes here. Thanks!

Unfortunately, there's some cruft in this module that we need to deal with. A user key is valid as a means to replace the default user. Also, the users, groups, individual user, and individual group definitions can all technically be a list, dict, or string.

E.g., the following is (somehow) valid:

#cloud-config
user: foo
users: foobar

Also, user definition is missing the following keys:

• plain_text_passwd, hashed_passwd, create_groups

Unrelated, we should probably slip in removing line 636 of the schema.

              "partition": {
                "type": ["string", "integer"],
                "oneOf": [
                  {
                    "type": "string",
                    "enum": ["auto", "any", "none"]
                  },
                  {"type": "integer"}
                ],

type is redundant here and now renders the docs as partition: (string/integer/string/integer) .

Everything else looks really good though!

[TheRealFalcon]

I think it'd be worthwhile to add the default user to the example and make it the first example. Something like this is probably the most common use case, so I think it makes sense to have it be first. We could remove default from example 3.

[TheRealFalcon]

Not a problem but just wondering Is there a reason you went with refs here rather than keeping the definition inline? Do you think it's worth moving the refs to be directly above this definition so they're easier to find?

[blackboxsw]

I think this makes sense to avoid the reference, I had originally thought that items might be more complex than it ended up being. I think we should reserve $defs for things that are duplicated, or where the size of the schema is so large in oneOfs that it makes it hard to see the alternative schemas defined within a certain scope. So I'll drop /users_groups.group_with_users and make it inline, but I'd lean toward keeping users_groups.user so we can easily see that funky "default" vs <users_groups.user> schema

[blackboxsw]

#cloud-config
user: foo
users: foobar

Strange works for me

$ PYTHONPATH=. python3 -m cloudinit.cmd.main devel schema --annotate -c ug.yaml
#cloud-config
user: foo
users: foobar		# E1

# Errors: -------------
# E1: 'foobar' is not of type 'array'

[TheRealFalcon]

Ooooh, sorry, I meant valid as in it's valid to the code, so it needs to be valid to the schema too.

[blackboxsw]

Unfortunately, there's some cruft in this module that we need to deal with. A user key is valid as a means to replace the default user.

Added a DEPRECATED warning as we should be able to replace this fully with users: directives. Will carve out schema docs for this key though all sub-keys are passed through as supported 'users` object keys, so the schema docs can cross-reference instead of repeat the content.

Also, user definition is missing the following keys:

* plain_text_passwd, hashed_passwd, create_groups

Added to schema

Unrelated, we should probably slip in removing line 636 of the schema.

agreed/done

[TheRealFalcon]

Top level users is still defined as array, but should support string, object, array

[TheRealFalcon]

Adding some more context to my last comment, I think users should be string/object/array of (string/object/array). These are all valid configs:

users: "aaa,bbb"

users:
  aaa:
    shell: /bin/bash
  bbb: true

users:
  - ccc
  - name: ddd
    shell: /bin/bash
  - [aaa, bbb]

[blackboxsw]

Adding some more context to my last comment, I think users should be string/object/array of (string/object/array). These are all valid configs:

users: "aaa,bbb"

users:
  aaa:
    shell: /bin/bash
  bbb: true

users:
  - ccc
  - name: ddd
    shell: /bin/bash
  - [aaa, bbb]

Fixed those missing cases here too and validated each schema suggested with unittests and PYTHONPATH=. python3 -m cloudinit.cmd.main devel schema -c <schema>

[TheRealFalcon]

This looks good to me now, and the unit tests (including lowest support) pass for me now. If you didn't have anything else left here, feel free to merge!