Implement easy collection unstructuring

HISTORY.rst

@@ -6,6 +6,8 @@ History
 ------------------
 * Fix an issue with ``GenConverter`` unstructuring ``attrs`` classes and dataclasses with generic fields.
   (`#65 <https://github.com/Tinche/cattrs/issues/65>`_)
+* ``GenConverter`` has support for easy overriding of collection unstructuring types (for example, unstructure all sets to lists) through its ``unstruct_collection_overrides`` argument.
+  (`#137 <https://github.com/Tinche/cattrs/pull/137>`_)
 
 1.4.0 (2021-03-21)
 ------------------

docs/converters.rst

@@ -51,7 +51,7 @@ classes.
 * structuring and unstructuring can be customized
 * support for ``attrs`` classes with PEP563 (postponed) annotations
 * support for generic ``attrs`` classes
+* support for easy overriding collection unstructuring
 
 
-
-The ``GenConverter`` will become the default converter type in a later release.
+The ``GenConverter`` will become the default converter type in a later release.

docs/unstructuring.rst

@@ -35,6 +35,60 @@ a complex or recursive collection.
     >>> data is copy
     False
 
+Customizing collection unstructuring
+------------------------------------
+
+.. important::
+   This feature is supported for Python 3.9 and later.
+
+Sometimes it's useful to be able to override collection unstructuring in a
+generic way. A common example is using a JSON library that doesn't support
+sets, but expects lists and tuples instead.
+
+Using ordinary unstructuring hooks for this is unwieldy due to the semantics of
+``singledispatch``; in other words, you'd need to register hooks for all
+specific types of set you're using (``set[int]``, ``set[float]``,
+``set[str]``...), which is not useful.
+
+Function-based hooks can be used instead, but come with their own set of
+challenges - they're complicated to write efficiently.
+
+The ``GenConverter`` supports easy customizations of collection unstructuring
+using its ``unstruct_collection_overrides`` parameter. For example, to
+unstructure all sets into lists, try the following:
+
+.. doctest::
+
+  >>> from collections.abc import Set
+  >>> converter = cattr.GenConverter(unstruct_collection_overrides={Set: list})
+  >>>
+  >>> converter.unstructure({1, 2, 3})
+  [1, 2, 3]
+
+Going even further, the ``GenConverter`` contains heuristics to support the
+following Python types, in order of decreasing generality:
+
+    * ``Sequence``, ``MutableSequence``, ``list``, ``tuple``
+    * ``Set``, ``MutableSet``, ``set``
+
+For example, if you override the unstructure type for ``Sequence``, but not for
+``MutableSequence``, ``list`` or ``tuple``, the override will also affect those
+types. An easy way to remember the rule:
+
+    * all ``MutableSequence`` s are ``Sequence`` s, so the override will apply
+    * all ``list`` s are ``MutableSequence`` s, so the override will apply
+    * all ``tuple`` s are ``Sequence`` s, so the override will apply
+
+If, however, you override only ``MutableSequence``, fields annotated as
+``Sequence`` will not be affected (since not all sequences are mutable
+sequences), and fields annotated as tuples will not be affected (since tuples
+are not mutable sequences in the first place).
+
+Similar logic applies to the set hierarchy.
+
+Make sure you're using the types from ``collections.abc`` on Python 3.9+, and
+from ``typing`` on older Python versions.
+
 ``typing.Annotated``
 --------------------
 

src/cattr/_compat.py

@@ -15,12 +15,12 @@
     Dict,
     FrozenSet,
     List,
-    Mapping,
-    MutableMapping,
-    MutableSequence,
-    MutableSet,
-    Sequence,
-    Set,
+    Mapping as TypingMapping,
+    MutableMapping as TypingMutableMapping,
+    MutableSequence as TypingMutableSequence,
+    MutableSet as TypingMutableSet,
+    Sequence as TypingSequence,
+    Set as TypingSet,
     Tuple,
 )
 
@@ -91,6 +91,11 @@ def adapted_fields(type) -> List[Attribute]:
 
 
 if is_py37 or is_py38:
+    Set = TypingSet
+    MutableSet = TypingMutableSet
+    Sequence = TypingSequence
+    MutableSequence = TypingMutableSequence
+
     from typing import Union, _GenericAlias
 
     def is_annotated(_):
@@ -113,7 +118,7 @@ def is_sequence(type: Any) -> bool:
         return type in (List, list, Tuple, tuple) or (
             type.__class__ is _GenericAlias
             and type.__origin__ is not Union
-            and issubclass(type.__origin__, Sequence)
+            and issubclass(type.__origin__, TypingSequence)
         )
 
     def is_mutable_set(type):
@@ -129,16 +134,16 @@ def is_frozenset(type):
         )
 
     def is_mapping(type):
-        return type in (Mapping, dict) or (
+        return type in (TypingMapping, dict) or (
             type.__class__ is _GenericAlias
-            and issubclass(type.__origin__, Mapping)
+            and issubclass(type.__origin__, TypingMapping)
         )
 
     bare_list_args = List.__args__
-    bare_seq_args = Sequence.__args__
-    bare_mapping_args = Mapping.__args__
+    bare_seq_args = TypingSequence.__args__
+    bare_mapping_args = TypingMapping.__args__
     bare_dict_args = Dict.__args__
-    bare_mutable_seq_args = MutableSequence.__args__
+    bare_mutable_seq_args = TypingMutableSequence.__args__
 
     def is_bare(type):
         args = type.__args__
@@ -167,6 +172,11 @@ def is_bare(type):
         Set as AbcSet,
     )
 
+    Set = AbcSet
+    MutableSet = AbcMutableSet
+    Sequence = AbcSequence
+    MutableSequence = AbcMutableSequence
+
     def is_annotated(type) -> bool:
         return getattr(type, "__class__", None) is _AnnotatedAlias
 
@@ -194,8 +204,8 @@ def is_sequence(type: Any) -> bool:
             in (
                 List,
                 list,
-                Sequence,
-                MutableSequence,
+                TypingSequence,
+                TypingMutableSequence,
                 AbcMutableSequence,
                 tuple,
             )
@@ -205,7 +215,7 @@ def is_sequence(type: Any) -> bool:
                     (origin is not tuple)
                     and issubclass(
                         origin,
-                        Sequence,
+                        TypingSequence,
                     )
                     or origin is tuple
                     and type.__args__[1] is ...
@@ -217,10 +227,10 @@ def is_sequence(type: Any) -> bool:
 
     def is_mutable_set(type):
         return (
-            type in (Set, MutableSet, set)
+            type in (TypingSet, TypingMutableSet, set)
             or (
                 type.__class__ is _GenericAlias
-                and issubclass(type.__origin__, MutableSet)
+                and issubclass(type.__origin__, TypingMutableSet)
             )
             or (
                 getattr(type, "__origin__", None)
@@ -245,10 +255,10 @@ def is_bare(type):
 
     def is_mapping(type):
         return (
-            type in (Mapping, Dict, MutableMapping, dict)
+            type in (TypingMapping, Dict, TypingMutableMapping, dict)
             or (
                 type.__class__ is _GenericAlias
-                and issubclass(type.__origin__, Mapping)
+                and issubclass(type.__origin__, TypingMapping)
             )
             or (getattr(type, "__origin__", None) is dict)
         )

src/cattr/converters.py

@@ -1,6 +1,15 @@
 from enum import Enum
 from functools import lru_cache
-from typing import Any, Callable, Dict, Mapping, Optional, Tuple, Type, TypeVar
+from typing import (
+    Any,
+    Callable,
+    Dict,
+    Mapping,
+    Optional,
+    Tuple,
+    Type,
+    TypeVar,
+)
 
 from attr import resolve_types, has as attrs_has
 
@@ -18,6 +27,10 @@
     has,
     fields,
     has_with_generic,
+    Set,
+    MutableSet,
+    Sequence,
+    MutableSequence,
 )
 from .disambiguators import create_uniq_field_dis_func
 from .dispatch import MultiStrategyDispatch
@@ -471,20 +484,56 @@ def _get_dis_func(union):
 class GenConverter(Converter):
     """A converter which generates specialized un/structuring functions."""
 
-    __slots__ = ("omit_if_default", "type_overrides")
+    __slots__ = (
+        "omit_if_default",
+        "type_overrides",
+        "_unstruct_collection_overrides",
+    )
 
     def __init__(
         self,
         dict_factory: Callable[[], Any] = dict,
         unstruct_strat: UnstructureStrategy = UnstructureStrategy.AS_DICT,
         omit_if_default: bool = False,
         type_overrides: Mapping[Type, AttributeOverride] = {},
+        unstruct_collection_overrides: Mapping[Type, Callable] = {},
     ):
         super().__init__(
             dict_factory=dict_factory, unstruct_strat=unstruct_strat
         )
         self.omit_if_default = omit_if_default
-        self.type_overrides = type_overrides
+        self.type_overrides = dict(type_overrides)
+
+        self._unstruct_collection_overrides = unstruct_collection_overrides
+
+        # Do a little post-processing magic to make things easier for users.
+        co = unstruct_collection_overrides
+
+        # abc.MutableSet overrrides, if defined, apply to sets
+        if MutableSet in co:
+            if set not in co:
+                co[set] = co[MutableSet]
+
+        # abc.Set overrides, if defined, apply to abc.MutableSets and sets
+        if Set in co:
+            if MutableSet not in co:
+                co[MutableSet] = co[Set]
+            if set not in co:
+                co[set] = co[Set]
+
+        # abc.MutableSequence overrides, if defined, can apply to lists
+        if MutableSequence in co:
+            if list not in co:
+                co[list] = co[MutableSequence]
+
+        # abc.Sequence overrides, if defined, can apply to MutableSequences, lists and tuples
+        if Sequence in co:
+            if MutableSequence not in co:
+                co[MutableSequence] = co[Sequence]
+            if list not in co:
+                co[list] = co[Sequence]
+            if tuple not in co:
+                co[tuple] = co[Sequence]
 
         if unstruct_strat is UnstructureStrategy.AS_DICT:
             # Override the attrs handler.
@@ -576,7 +625,10 @@ def gen_structure_attrs_fromdict(self, cl: Type[T]) -> T:
         # only direct dispatch so that subclasses get separately generated
         return h
 
-    def gen_unstructure_iterable(self, cl: Any, unstructure_to=list):
+    def gen_unstructure_iterable(self, cl: Any, unstructure_to=None):
+        unstructure_to = self._unstruct_collection_overrides.get(
+            get_origin(cl) or cl, unstructure_to or list
+        )
         h = make_iterable_unstructure_fn(
             cl, self, unstructure_to=unstructure_to
         )