Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,11 @@ Our backwards-compatibility policy can be found [here](https://github.com/python

## 25.2.0 (unreleased)

- Add a `use_alias` parameter to {class}`cattrs.Converter`.
- **Potentially breaking**: Sequences are now structured into tuples.
This allows hashability, better immutability and is more consistent with the [`collections.abc.Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence) type.
See [Migrations](https://catt.rs/en/latest/migrations.html#sequences-structuring-into-tuples) for steps to restore legacy behavior.
([#663](https://github.com/python-attrs/cattrs/pull/663))
- Add a `use_alias` parameter to {class}`cattrs.Converter`.
{func}`cattrs.gen.make_dict_unstructure_fn_from_attrs`, {func}`cattrs.gen.make_dict_unstructure_fn`,
{func}`cattrs.gen.make_dict_structure_fn_from_attrs`, {func}`cattrs.gen.make_dict_structure_fn`
and {func}`cattrs.gen.typeddicts.make_dict_structure_fn` will use the value for the `use_alias` parameter from the given converter by default now.
Expand Down
13 changes: 9 additions & 4 deletions docs/customizing.md
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,7 @@ Available predicates are:
- {meth}`is_any_set`
- {meth}`is_frozenset`
- {meth}`is_set`
- {meth}`is_mutable_sequence`
- {meth}`is_sequence`
- {meth}`is_mapping`
- {meth}`is_namedtuple`
Expand All @@ -432,6 +433,7 @@ Available hook factories are:

- {meth}`iterable_unstructure_factory`
- {meth}`list_structure_factory`
- {meth}`homogenous_tuple_structure_factory`
- {meth}`namedtuple_structure_factory`
- {meth}`namedtuple_unstructure_factory`
- {meth}`namedtuple_dict_structure_factory`
Expand All @@ -442,15 +444,15 @@ Available hook factories are:

Additional predicates and hook factories will be added as requested.

For example, by default sequences are structured from any iterable into lists.
For example, by default mutable sequences are structured from any iterable into lists.
This may be too lax, and additional validation may be applied by wrapping the default list structuring hook factory.

```{testcode} list-customization
from cattrs.cols import is_sequence, list_structure_factory
from cattrs.cols import is_mutable_sequence, list_structure_factory

c = Converter()

@c.register_structure_hook_factory(is_sequence)
@c.register_structure_hook_factory(is_mutable_sequence)
def strict_list_hook_factory(type, converter):

# First, we generate the default hook...
Expand All @@ -466,7 +468,7 @@ def strict_list_hook_factory(type, converter):
return strict_list_hook
```

Now, all sequence structuring will be stricter:
Now, all mutable sequence structuring will be stricter:

```{doctest} list-customization
>>> c.structure({"a", "b", "c"}, list[str])
Expand All @@ -477,6 +479,9 @@ ValueError: Not a list!

```{versionadded} 24.1.0

```
```{versionchanged} 25.2.0
Added the {meth}`is_mutable_sequence` predicate and {meth}`homogenous_tuple_structure_factory` hook factory.
```

### Customizing Named Tuples
Expand Down
27 changes: 17 additions & 10 deletions docs/defaulthooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,8 +137,7 @@ None
Lists can be structured from any iterable object.
Types converting to lists are:

- `typing.Sequence[T]`
- `typing.MutableSequence[T]`
- `collections.abc.MutableSequence[T]`
- `typing.List[T]`
- `list[T]`

Expand All @@ -154,6 +153,10 @@ A bare type, for example `Sequence` instead of `Sequence[int]`, is equivalent to
When unstructuring, lists are copied and their contents are handled according to their inner type.
A useful use case for unstructuring collections is to create a deep copy of a complex or recursive collection.

```{versionchanged} 25.2.0
Sequences are no longer structured into lists by default, but tuples.
```

### Dictionaries

Dictionaries can be produced from other mapping objects.
Expand All @@ -162,8 +165,8 @@ and be able to be passed to the `dict` constructor as an argument.
Types converting to dictionaries are:

- `dict[K, V]` and `typing.Dict[K, V]`
- `collections.abc.MutableMapping[K, V]` and `typing.MutableMapping[K, V]`
- `collections.abc.Mapping[K, V]` and `typing.Mapping[K, V]`
- `collections.abc.MutableMapping[K, V]`
- `collections.abc.Mapping[K, V]`

In all cases, a new dict will be returned, so this operation can be used to copy a mapping into a dict.
Any type parameters set to `typing.Any` will be passed through unconverted.
Expand Down Expand Up @@ -234,14 +237,15 @@ _cattrs_ will be able to structure it by default.
Homogeneous and heterogeneous tuples can be structured from iterable objects.
Heterogeneous tuples require an iterable with the number of elements matching the number of type parameters exactly.

Use:
Heterogenous tuples use:

- `Tuple[A, B, C, D]`
- `typing.Tuple[A, B, C, D]`
- `tuple[A, B, C, D]`

Homogeneous tuples use:

- `Tuple[T, ...]`
- `collections.abc.Sequence[T]`
- `typing.Tuple[T, ...]`
- `tuple[T, ...]`

In all cases a tuple will be produced.
Expand All @@ -263,6 +267,10 @@ When unstructuring, heterogeneous tuples unstructure into tuples since it's fast
Structuring heterogenous tuples are not supported by the BaseConverter.
```

```{versionchanged} 25.2.0
Sequences are now structured into tuples.
```

### Deques

Deques can be structured from any iterable object.
Expand Down Expand Up @@ -293,13 +301,12 @@ Deques are unstructured into lists, or into deques when using the {class}`BaseCo
Sets and frozensets can be structured from any iterable object.
Types converting to sets are:

- `typing.Set[T]`
- `typing.MutableSet[T]`
- `collections.abc.Set[T]`
- `collections.abc.MutableSet[T]`
- `set[T]`

Types converting to frozensets are:

- `typing.FrozenSet[T]`
- `frozenset[T]`

In all cases, a new set or frozenset will be returned.
Expand Down
16 changes: 15 additions & 1 deletion docs/migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,20 @@
_cattrs_ sometimes changes in backwards-incompatible ways.
This page contains guidance for changes and workarounds for restoring legacy behavior.

## 25.2.0

### Sequences structuring into tuples

Sequences were changed to structure into tuples instead of lists.

The old behavior can be restored by registering the `list_structure_factory` using the `is_sequence` predicate on a converter.

```python
>>> from cattrs.cols import is_sequence, list_structure_factory

>>> converter.register_structure_hook_factory(is_sequence, list_structure_factory)
```

## 24.2.0

### The default structure hook fallback factory
Expand All @@ -24,4 +38,4 @@ The old behavior can be restored by explicitly passing in the old hook fallback

The internal `cattrs.gen.MappingStructureFn` and `cattrs.gen.DictStructureFn` types were replaced by a more general type, `cattrs.SimpleStructureHook[In, T]`.
If you were using `MappingStructureFn`, use `SimpleStructureHook[Mapping[Any, Any], T]` instead.
If you were using `DictStructureFn`, use `SimpleStructureHook[Mapping[str, Any], T]` instead.
If you were using `DictStructureFn`, use `SimpleStructureHook[Mapping[str, Any], T]` instead.
36 changes: 22 additions & 14 deletions src/cattrs/_compat.py
Original file line number Diff line number Diff line change
Expand Up @@ -306,34 +306,42 @@ def get_notrequired_base(type) -> Union[Any, NothingType]:
return NOTHING


def is_mutable_sequence(type: Any) -> bool:
"""A predicate function for mutable sequences.

Matches lists, mutable sequences, and deques.
"""
origin = getattr(type, "__origin__", None)
return (
type in (List, list, TypingMutableSequence, AbcMutableSequence, deque, Deque)
or (
type.__class__ is _GenericAlias
and (
((origin is not tuple) and is_subclass(origin, TypingMutableSequence))
or (origin is tuple and type.__args__[1] is ...)
)
)
or (origin in (list, deque, AbcMutableSequence))
)


def is_sequence(type: Any) -> bool:
"""A predicate function for sequences.

Matches lists, sequences, mutable sequences, deques and homogenous
tuples.
"""
origin = getattr(type, "__origin__", None)
return (
type
in (
List,
list,
TypingSequence,
TypingMutableSequence,
AbcMutableSequence,
tuple,
Tuple,
deque,
Deque,
)
return is_mutable_sequence(type) or (
type in (TypingSequence, tuple, Tuple)
or (
type.__class__ is _GenericAlias
and (
((origin is not tuple) and is_subclass(origin, TypingSequence))
or (origin is tuple and type.__args__[1] is ...)
)
)
or (origin in (list, deque, AbcMutableSequence, AbcSequence))
or (origin is AbcSequence)
or (origin is tuple and type.__args__[1] is ...)
)

Expand Down
44 changes: 44 additions & 0 deletions src/cattrs/cols.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
is_bare,
is_frozenset,
is_mapping,
is_mutable_sequence,
is_sequence,
is_subclass,
)
Expand All @@ -47,10 +48,12 @@

__all__ = [
"defaultdict_structure_factory",
"homogenous_tuple_structure_factory",
"is_any_set",
"is_defaultdict",
"is_frozenset",
"is_mapping",
"is_mutable_sequence",
"is_namedtuple",
"is_sequence",
"is_set",
Expand Down Expand Up @@ -151,6 +154,47 @@ def structure_list(
return structure_list


def homogenous_tuple_structure_factory(
type: type, converter: BaseConverter
) -> StructureHook:
"""A hook factory for homogenous (all elements the same, indeterminate length) tuples.

Converts any given iterable into a tuple.
"""

if is_bare(type) or type.__args__[0] in ANIES:

def structure_tuple(obj: Iterable[T], _: type = type) -> tuple[T, ...]:
return tuple(obj)

return structure_tuple

elem_type = type.__args__[0]

try:
handler = converter.get_structure_hook(elem_type)
except RecursionError:
# Break the cycle by using late binding.
handler = converter.structure

if converter.detailed_validation:

# We have to structure into a list first anyway.
list_structure = list_structure_factory(type, converter)

def structure_tuple(obj: Iterable[T], _: type = type) -> tuple[T, ...]:
return tuple(list_structure(obj, _))

else:

def structure_tuple(
obj: Iterable[T], _: type = type, _handler=handler, _elem_type=elem_type
) -> tuple[T, ...]:
return tuple([_handler(e, _elem_type) for e in obj])

return structure_tuple


def namedtuple_unstructure_factory(
cl: type[tuple], converter: BaseConverter, unstructure_to: Any = None
) -> UnstructureHook:
Expand Down
7 changes: 5 additions & 2 deletions src/cattrs/converters.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,19 +43,21 @@
is_hetero_tuple,
is_literal,
is_mapping,
is_mutable_sequence,
is_mutable_set,
is_optional,
is_protocol,
is_sequence,
is_tuple,
is_typeddict,
is_union_type,
signature,
)
from .cols import (
defaultdict_structure_factory,
homogenous_tuple_structure_factory,
is_defaultdict,
is_namedtuple,
is_sequence,
iterable_unstructure_factory,
list_structure_factory,
mapping_structure_factory,
Expand Down Expand Up @@ -271,7 +273,8 @@ def __init__(
),
(is_literal, self._structure_simple_literal),
(is_literal_containing_enums, self._structure_enum_literal),
(is_sequence, list_structure_factory, "extended"),
(is_sequence, homogenous_tuple_structure_factory, "extended"),
(is_mutable_sequence, list_structure_factory, "extended"),
(is_deque, self._structure_deque),
(is_mutable_set, self._structure_set),
(is_frozenset, self._structure_frozenset),
Expand Down
24 changes: 23 additions & 1 deletion tests/test_cols.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""Tests for the `cattrs.cols` module."""

from collections.abc import Set
from collections.abc import MutableSequence, Sequence, Set
from typing import Dict

from immutables import Map
Expand All @@ -9,7 +9,9 @@
from cattrs._compat import FrozenSet
from cattrs.cols import (
is_any_set,
is_sequence,
iterable_unstructure_factory,
list_structure_factory,
mapping_unstructure_factory,
)

Expand Down Expand Up @@ -53,3 +55,23 @@ def test_mapping_unstructure_to(genconverter: Converter):
"""`unstructure_to` works."""
hook = mapping_unstructure_factory(Dict[str, str], genconverter, unstructure_to=Map)
assert hook({"a": "a"}).__class__ is Map


def test_structure_sequences(converter: BaseConverter):
"""Sequences are structured to tuples."""

assert converter.structure(["1", 2, 3.0], Sequence[int]) == (1, 2, 3)


def test_structure_sequences_override(converter: BaseConverter):
"""Sequences can be overriden to structure to lists, as previously."""

converter.register_structure_hook_factory(is_sequence, list_structure_factory)

assert converter.structure(["1", 2, 3.0], Sequence[int]) == [1, 2, 3]


def test_structure_mut_sequences(converter: BaseConverter):
"""Mutable sequences are structured to lists."""

assert converter.structure(["1", 2, 3.0], MutableSequence[int]) == [1, 2, 3]
8 changes: 8 additions & 0 deletions tests/test_generics.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from collections import deque
from collections.abc import Sequence
from typing import Deque, Dict, Generic, List, Optional, TypeVar, Union

import pytest
Expand Down Expand Up @@ -109,9 +110,16 @@ class GenericCols(Generic[T]):
(
(TClass[int, int, int], str, int, TClass(TClass(1, 2), "a")),
(List[TClass[int, int, int]], str, int, TClass([TClass(1, 2)], "a")),
(
Sequence[TClass[str, str, str]],
str,
str,
TClass((TClass("a", "b", "c"),), "b", "c"),
),
),
)
def test_structure_nested_generics(converter: BaseConverter, t, t2, t3, result):
"""Structuring nested generics works."""
res = converter.structure(asdict(result), TClass[t, t2, t3])

assert res == result
Expand Down
Loading