[WIP] new config system, 1.2 tagset support

[nitzmahone]

Here's a quick and dirty crack at a more broadly-encompassing dynamic config system like we've talked about before...

• builds on @perlpunk 's excellent work getting 1.2 support into PyYAML by:
  • adding a TagSet type with some pre-defined instances for common 1.2 schemas
  • allowing users to create their own tagsets
• leaves existing class hierarchy intact (no changes to instancing behavior)
• introduces new Config mixins that:
  • creates dynamic Loader/Dumper subclasses to isolate custom configuration- no need to manually subclass or monkeypatch/twiddle config in global base classes. Note that this explicitly does not provide a way to do truly global config changes- I think that's pretty sane behavior for a library. The user is required to hang onto their customized Loader/Dumper types and explicitly pass them to yaml.load()/yaml.dump() et al to get the customized behavior, and the customized subclasses are GC'd once dropped on the floor.
  • allow complete inherit/add/replace of Loader/Dumper tagsets with any combo of builtin or user-defined
  • support config override of all existing Dumper config options (and arbitrary kwargs to Loader and Dumper to support any new config options).
• provides new FastestBaseLoader and FastestBaseDumper helper classes backed by libyaml if available, and the pure-Python version if not
• uses modern Python type-hinting for self-documentation, type/arg validation, and IDE auto-completion

We can also wrap up the existing individual types that make up the tagsets so that users can pick and choose, and combine partial schemas at will without having to redefine everything. Once that's all done, we should actually be able to completely redefine all the existing currently unrelated Loader and Dumper subclasses using this, and they'll actually be related in the class hierarchy instead of just sharing mixins.

A few examples of what's in and working:

json tagset on fastest available loader

import yaml
from yaml.tagset import json

# I want the fastest possible loader using only the 1.2 json tagset...
json_loader = yaml.FastestBaseLoader.config(tagset=json)
print(f'custom loader type is: {json_loader}')
print(f"result was {yaml.load('hi: mom', Loader=json_loader)}")

minimal custom tagset

import yaml
from yaml.tagset import TagSet

# I want the fastest possible loader using a custom minimal tagset
my_fallback_tagset = TagSet(name='my_fallback',
                            constructors={},
                            resolvers=[],
                            representers={})

yaml_to_load = """
    not_a_bool: true
    not_an_int: 123
"""

fallback_loader = yaml.FastestBaseLoader.config(tagset=my_fallback_tagset)
print(f'custom loader type is: {fallback_loader}')
print(f"result was {yaml.load(yaml_to_load, Loader=fallback_loader)}")

override dumper behavior so it doesn't have to be specified every call to dump

import yaml
from yaml.tagset import core

data = {
    'key': {
        'subkey1': 'subvalue1'
    }
}

# override dumper formatting on fastest available dumper
pedantic_documents_with_single_quotes_dumper = yaml.FastestBaseDumper.config(tagset=core, explicit_start=True, explicit_end=True, default_style="'", indent=2, canonical=True)

print(yaml.dump(data, Dumper=pedantic_documents_with_single_quotes_dumper))

Still TODO:

• implement partial tagsets for Python types
• wrap existing type definitions in a new dataclass
• replace all existing Loader/Dumper subclasses with dynamically-created ones (eating our own dogfood)
• fire deprecation warnings if direct modification of base classes is detected
• tests
• docs

[nitzmahone]

this would probably be better done with inspect.signature(); then it's 100% dynamic off whatever init args the current class accepts

[ingydotnet]

This is a great start, Matt!

I really like the approach overall of generating a configured subclass around the existing architecture.
We get custom configured subclasses (which we can instantiate into objects if we want).
No existing code breaks.
With configuration we'll be able to support all the things I'm on the fence about supporting by just making them config options.

The new .config(...) method returns a new generated subclass. A more precise method name would be .subclass_using_config(...).
I really like the idea of calling this method .new(...). My second favorite would be .subclass(...).

While we can add this method to the the Loader and Dumper classes I think that: yaml.new(...config..) should return a YamlClass class containing both a generated Loader class and a generated Dumper class. Then you can cleanly do:

import yaml
myaml = yaml.new(tagset=yaml.tagset.json, indent=3)
text = 'hi: mom'
data = myaml.load(text)
text = myaml.dump(data)

I think that's a super clean top level way of using the new ideas you've added.

myaml.load(text)
# is a lot simpler and more intuitive than using the old:
yaml.load(text, Loader=myaml.loader)

Also, the .new() method would default to .new(base=FastestBase) which is same .new(base_loader=FastestBaseLoader, base_dumper=FastestBaseDumper). This way when we add libfyaml support soon, we people can fyaml = yaml.new(base=FYaml). Note: fyaml is fast like libyaml but also passes 100% of the test suite and is very actively maintained.
PyYAML and libyaml only pass ~82% of the suite but they share a lot of the same failures, which is to say they are quite bugwards compatible with each other. So I the the FastestBase... or whatever we want to call it should just select between those 2, and newer (even if better) backends should be explicitly requested.

I assume you can do this:

ayaml = yaml.new(...config-stuff...)
byaml = ayaml.new(...more-config-stuff...)

All this is to say, I really like where you have gone so far, at least as I understand it. I'd just like to see the common usage idioms be even cleaner. I'll try to write up a file of all the possible usages so we can discuss them among the release team.

[jtmedley]

Any updates on this?