marshals ¶

Support for marshalling Python data structures into primitive equivalents.

Notes

"Marshalling" your data structure prepares it to be serialized into binary and sent over the wire, but does not serialize it. We keep these stages isolated to ensure maximum flexibility and simplicity.

We ensure that your marshalled data is compatible with Python's built-in json module. This provides maximum compatibility with most serialization protocols by limiting the output to simple Python builtin types:

Tip

You may use this package directly, but we encourage you to work with our high-level API provided by the top-level typelib module.

Typical Usage

>>> import dataclasses
>>> import decimal
>>> from typelib import marshals
>>>
>>> @dataclasses.dataclass(slots=True, weakref_slot=True, kw_only=True)
... class Struct:
...     key: str
...     number: decimal.Decimal
...
>>>
>>> data = Struct(key="some-key", number=decimal.Decimal("1.0"))
>>> marshals.marshal(data)
{'key': 'some-key', 'number': '1.0'}
>>> marshaller = marshals.marshaller(Struct)
>>> marshaller(data)
{'key': 'some-key', 'number': '1.0'}

See Also

Modules:

api –

The API for marshalling higher-order Python objects into simple, encodable Python objects.
routines –

Type-specific logic for marshalling Python objects in to simple types for representation over-the-wire.

Classes:

AbstractMarshaller –

Abstract base class defining the common interface for marshallers.
DelayedMarshaller –

Delayed proxy for a given type's marshaller, used when we encounter a typing.ForwardRef.
EnumMarshaller –

A marshaller that converts an enum.Enum instance to its assigned value.
FixedTupleMarshaller –

A marshaller for dumping a "fixed" tuple to a simple list.
IterableMarshaller –

A marshaller for dumping any iterable into a simple list.
LiteralMarshaller –

A marshaller that enforces the given value be one of the values in the defined typing.Literal
MappingMarshaller –

A marshaller for dumping any mapping into a simple dict.
PatternMarshaller –

A marshaller that converts a re.Pattern to a string.
StructuredTypeMarshaller –

A marshaller for dumping a structured (user-defined) type to a simple dict.
SubscriptedIterableMarshaller –

A marshaller for dumping a subscripted iterable into a simple list.
SubscriptedMappingMarshaller –

A marshaller for dumping a subscripted mapping into a simple dict.
UnionMarshaller –

A marshaller for dumping a given value via one of the types in the defined bound union.

Functions:

marshal –

Marshal value from typ into MarshalledValueT.
marshaller –

Get a marshaller routine for a given type.

AbstractMarshaller ¶

AbstractMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: ABC, Generic[T]

Abstract base class defining the common interface for marshallers.

Marshallers are custom callables which maintain type-specific information. They use this information to provide robust, performant logic reducing Python objects into their primitive representations for over-the-wire encoding.

Marshallers support contextual serialization, which enables the marshalling of nested types.

Attributes:

t (type[T]) –

The root type of this marshaller.
origin (type[T]) –

If t is a generic, this will be an actionable runtime type related to t, otherwise it is the same as t.
context (ContextT) –

The complete type context for this unmarshaller.
var (str | None) –

If this unmarshaller is used in a nested context, this will reference the field/parameter/index at which this unmarshaller should be used.

Parameters:

t ¶
(type[T]) –

The root type of this marshaller.
context ¶
(ContextT) –

The complete type context for this marshaller.
var ¶
(str | None, default: None ) –

The associated field or parameter name for this unmarshaller (optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[T], context: ContextT, *, var: str | None = None):
    """Construct a marshaller instance.

    Args:
        t: The root type of this marshaller.
        context: The complete type context for this marshaller.
        var: The associated field or parameter name for this unmarshaller (optional).
    """
    self.t = t
    self.origin = inspection.origin(self.t)
    self.context = context
    self.var = var

DelayedMarshaller ¶

DelayedMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[T]

Delayed proxy for a given type's marshaller, used when we encounter a typing.ForwardRef.

Notes

This allows us to delay the resolution of the given type reference until call-time, enabling support for cyclic and recursive types.

Attributes:

resolved (AbstractMarshaller[T]) –

The resolved marshaller.

Source code in src/typelib/marshals/api.py

def __init__(
    self, t: type[T], context: routines.ContextT, *, var: str | None = None
):
    super().__init__(t, context, var=var)
    self._resolved: routines.AbstractMarshaller[T] | None = None

resolved `property` ¶

resolved: AbstractMarshaller[T]

The resolved marshaller.

EnumMarshaller ¶

EnumMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[EnumT], Generic[EnumT]

A marshaller that converts an enum.Enum instance to its assigned value.

Parameters:

t ¶
(type[T]) –

The root type of this marshaller.
context ¶
(ContextT) –

The complete type context for this marshaller.
var ¶
(str | None, default: None ) –

The associated field or parameter name for this unmarshaller (optional).

Methods:

__call__ –

Marshal an enum.Enum instance into a [serdes.MarshalledValueT][].

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[T], context: ContextT, *, var: str | None = None):
    """Construct a marshaller instance.

    Args:
        t: The root type of this marshaller.
        context: The complete type context for this marshaller.
        var: The associated field or parameter name for this unmarshaller (optional).
    """
    self.t = t
    self.origin = inspection.origin(self.t)
    self.context = context
    self.var = var

call ¶

__call__(val: EnumT) -> MarshalledValueT

Marshal an enum.Enum instance into a [serdes.MarshalledValueT][].

Parameters:

val ¶
(EnumT) –

The enum instance to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: EnumT) -> serdes.MarshalledValueT:
    """Marshal an [`enum.Enum`][] instance into a [`serdes.MarshalledValueT`][].

    Args:
        val: The enum instance to marshal.
    """
    return val.value

FixedTupleMarshaller ¶

FixedTupleMarshaller(t: type[TupleT], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[TupleT]

A marshaller for dumping a "fixed" tuple to a simple list.

Values are marshalled according to the value-type in the order they are defined.

See Also

FixedTupleUnmarshaller

Parameters:

t ¶
(type[TupleT]) –

The type to unmarshal from.
context ¶
(ContextT) –

Any nested type context. Used to resolve the member marshallers.
var ¶
(str | None, default: None ) –

A variable name for the indicated type annotation (unused, optional).

Methods:

__call__ –

Marshal a tuple into a simple list.

Source code in src/typelib/marshals/routines.py

def __init__(
    self, t: type[compat.TupleT], context: ContextT, *, var: str | None = None
):
    """Constructor.

    Args:
        t: The type to unmarshal from.
        context: Any nested type context. Used to resolve the member marshallers.
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t, context, var=var)
    self.stack = inspection.args(t, evaluate=True)
    self.ordered_routines = [self.context[vt] for vt in self.stack]

call ¶

__call__(val: TupleT) -> MarshalledIterableT

Marshal a tuple into a simple list.

Parameters:

val ¶
(TupleT) –

The tuple to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: compat.TupleT) -> MarshalledIterableT:
    """Marshal a tuple into a simple [`list`][].

    Args:
        val: The tuple to marshal.
    """
    return [
        routine(v)
        for routine, v in zip(
            self.ordered_routines, serdes.itervalues(val), strict=False
        )
    ]

IterableMarshaller ¶

IterableMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[IterableT], Generic[IterableT]

A marshaller for dumping any iterable into a simple list.

Parameters:

t ¶
(type[T]) –

The root type of this marshaller.
context ¶
(ContextT) –

The complete type context for this marshaller.
var ¶
(str | None, default: None ) –

The associated field or parameter name for this unmarshaller (optional).

Methods:

__call__ –

Marshal an iterable into a simple list.

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[T], context: ContextT, *, var: str | None = None):
    """Construct a marshaller instance.

    Args:
        t: The root type of this marshaller.
        context: The complete type context for this marshaller.
        var: The associated field or parameter name for this unmarshaller (optional).
    """
    self.t = t
    self.origin = inspection.origin(self.t)
    self.context = context
    self.var = var

call ¶

__call__(val: IterableT) -> MarshalledIterableT

Marshal an iterable into a simple list.

Parameters:

val ¶
(IterableT) –

The iterable to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: IterableT) -> MarshalledIterableT:
    """Marshal an iterable into a simple [`list`][].

    Args:
        val: The iterable to marshal.
    """
    return [*val]

LiteralMarshaller ¶

LiteralMarshaller(t: type[LiteralT], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[LiteralT], Generic[LiteralT]

A marshaller that enforces the given value be one of the values in the defined typing.Literal

Parameters:

t ¶
(type[LiteralT]) –

The Literal type to enforce.
context ¶
(ContextT) –

Nested type context (unused).
var ¶
(str | None, default: None ) –

A variable name for the indicated type annotation (unused, optional).

Methods:

__call__ –

Enforce the given value is a member of the bound Literal type.

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[LiteralT], context: ContextT, *, var: str | None = None):
    """Constructor.

    Args:
        t: The Literal type to enforce.
        context: Nested type context (unused).
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t, context, var=var)
    self.values = inspection.args(t)

call ¶

__call__(val: LiteralT) -> MarshalledValueT

Enforce the given value is a member of the bound Literal type.

Parameters:

val ¶
(LiteralT) –

The value to enforce.

Raises:

ValueError –

If val is not a member of the bound Literal type.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: LiteralT) -> serdes.MarshalledValueT:
    """Enforce the given value is a member of the bound `Literal` type.

    Args:
        val: The value to enforce.

    Raises:
        ValueError: If `val` is not a member of the bound `Literal` type.
    """
    if val in self.values:
        return val  # type: ignore[return-value]

    raise ValueError(f"{val!r} is not one of {self.values!r}")

MappingMarshaller ¶

MappingMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[MappingT], Generic[MappingT]

A marshaller for dumping any mapping into a simple dict.

Parameters:

t ¶
(type[T]) –

The root type of this marshaller.
context ¶
(ContextT) –

The complete type context for this marshaller.
var ¶
(str | None, default: None ) –

The associated field or parameter name for this unmarshaller (optional).

Methods:

__call__ –

Marshal a mapping into a simple dict.

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[T], context: ContextT, *, var: str | None = None):
    """Construct a marshaller instance.

    Args:
        t: The root type of this marshaller.
        context: The complete type context for this marshaller.
        var: The associated field or parameter name for this unmarshaller (optional).
    """
    self.t = t
    self.origin = inspection.origin(self.t)
    self.context = context
    self.var = var

call ¶

__call__(val: MappingT) -> MarshalledMappingT

Marshal a mapping into a simple dict.

Parameters:

val ¶
(MappingT) –

The mapping object to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: MappingT) -> MarshalledMappingT:
    """Marshal a mapping into a simple [`dict`][].

    Args:
        val: The mapping object to marshal.
    """
    return {**val}

PatternMarshaller ¶

PatternMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[PatternT]

A marshaller that converts a re.Pattern to a string.

Parameters:

t ¶
(type[T]) –

The root type of this marshaller.
context ¶
(ContextT) –

The complete type context for this marshaller.
var ¶
(str | None, default: None ) –

The associated field or parameter name for this unmarshaller (optional).

Methods:

__call__ –

Marshal a compiled regex pattern into a string.

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[T], context: ContextT, *, var: str | None = None):
    """Construct a marshaller instance.

    Args:
        t: The root type of this marshaller.
        context: The complete type context for this marshaller.
        var: The associated field or parameter name for this unmarshaller (optional).
    """
    self.t = t
    self.origin = inspection.origin(self.t)
    self.context = context
    self.var = var

call ¶

__call__(val: PatternT) -> str

Marshal a compiled regex pattern into a string.

Parameters:

val ¶
(PatternT) –

The pattern to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: PatternT) -> str:
    """Marshal a compiled regex pattern into a string.

    Args:
        val: The pattern to marshal.
    """
    return val.pattern

StructuredTypeMarshaller ¶

StructuredTypeMarshaller(t: type[_ST], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[_ST]

A marshaller for dumping a structured (user-defined) type to a simple dict.

See Also

StructuredTypeUnmarshaller

Parameters:

t ¶
(type[_ST]) –

The type to unmarshals from.
context ¶
(ContextT) –

Any nested type context. Used to resolve the member marshallers.
var ¶
(str | None, default: None ) –

A variable name for the indicated type annotation (unused, optional).

Methods:

__call__ –

Marshal a structured type into a simple dict.

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[_ST], context: ContextT, *, var: str | None = None):
    """Constructor.

    Args:
        t: The type to unmarshals from.
        context: Any nested type context. Used to resolve the member marshallers.
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t, context, var=var)
    self.fields_by_var = self._fields_by_var()

call ¶

__call__(val: _ST) -> MarshalledMappingT

Marshal a structured type into a simple dict.

Parameters:

val ¶
(_ST) –

The structured type to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: _ST) -> MarshalledMappingT:
    """Marshal a structured type into a simple [`dict`][].

    Args:
        val: The structured type to marshal.
    """
    fields = self.fields_by_var
    return {f: fields[f](v) for f, v in serdes.iteritems(val) if f in fields}

SubscriptedIterableMarshaller ¶

SubscriptedIterableMarshaller(t: type[IterableT], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[IterableT], Generic[IterableT]

A marshaller for dumping a subscripted iterable into a simple list.

Values are marshalled according to the defined value-type.

See Also

SubscriptedIterableUnmarshaller

Parameters:

t ¶
(type[IterableT]) –

The type to unmarshals from.
context ¶
(ContextT) –

Any nested type context. Used to resolve the member marshallers.
var ¶
(str | None, default: None ) –

A variable name for the indicated type annotation (unused, optional).

Methods:

__call__ –

Marshal an iterable into a simple list.

Source code in src/typelib/marshals/routines.py

def __init__(
    self, t: type[IterableT], context: ContextT, *, var: str | None = None
):
    """Constructor.

    Args:
        t: The type to unmarshals from.
        context: Any nested type context. Used to resolve the member marshallers.
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t=t, context=context, var=var)
    # supporting tuple[str, ...]
    (value_t, *_) = inspection.args(t, evaluate=True)
    self.values = context[value_t]

call ¶

__call__(val: IterableT) -> MarshalledIterableT

Marshal an iterable into a simple list.

Parameters:

val ¶
(IterableT) –

The iterable to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: IterableT) -> MarshalledIterableT:
    """Marshal an iterable into a simple [`list`][].

    Args:
        val: The iterable to marshal.
    """
    # Always decode bytes.
    values = self.values
    return [values(v) for v in serdes.itervalues(val)]

SubscriptedMappingMarshaller ¶

SubscriptedMappingMarshaller(t: type[MappingT], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[MappingT], Generic[MappingT]

A marshaller for dumping a subscripted mapping into a simple dict.

Keys are marshalled according to the defined key-type, values according to the defined value-type.

See Also

SubscriptedMappingUnmarshaller

Parameters:

t ¶
(type[MappingT]) –

The type to unmarshals from.
context ¶
(ContextT) –

Any nested type context. Used to resolve the member marshallers.
var ¶
(str | None, default: None ) –

A variable name for the indicated type annotation (unused, optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[MappingT], context: ContextT, *, var: str | None = None):
    """Constructor.

    Args:
        t: The type to unmarshals from.
        context: Any nested type context. Used to resolve the member marshallers.
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t, context, var=var)
    key_t, value_t = inspection.args(t, evaluate=True)
    self.keys = context[key_t]
    self.values = context[value_t]

UnionMarshaller ¶

UnionMarshaller(t: type[UnionT], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[UnionT], Generic[UnionT]

A marshaller for dumping a given value via one of the types in the defined bound union.

See Also

UnionUnmarshaller

Parameters:

t ¶
(type[UnionT]) –

The type to unmarshal into.
context ¶
(ContextT) –

Any nested type context. Used to resolve the member marshallers.
var ¶
(str | None, default: None ) –

A variable name for the indicated type annotation (unused, optional).

Methods:

__call__ –

Marshal a value into the bound UnionT.

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[UnionT], context: ContextT, *, var: str | None = None):
    """Constructor.

    Args:
        t: The type to unmarshal into.
        context: Any nested type context. Used to resolve the member marshallers.
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t, context, var=var)
    self.stack = inspection.args(t, evaluate=True)
    self.nullable = inspection.isoptionaltype(t)
    self.ordered_routines = [self.context[typ] for typ in self.stack]

call ¶

__call__(val: UnionT) -> MarshalledValueT

Marshal a value into the bound UnionT.

Parameters:

val ¶
(UnionT) –

The input value to unmarshal.

Raises:

ValueError –

If val cannot be marshalled via any member type.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: UnionT) -> serdes.MarshalledValueT:
    """Marshal a value into the bound `UnionT`.

    Args:
        val: The input value to unmarshal.

    Raises:
        ValueError: If `val` cannot be marshalled via any member type.
    """
    if self.nullable and val is None:
        return val

    for routine in self.ordered_routines:
        with contextlib.suppress(
            ValueError, TypeError, SyntaxError, AttributeError
        ):
            unmarshalled = routine(val)
            return unmarshalled

    raise ValueError(f"{val!r} is not one of types {self.stack!r}")

marshal ¶

marshal(value: Any, *, t: type[T] | ForwardRef | str | None = None) -> MarshalledValueT

Marshal value from typ into MarshalledValueT.

Parameters:

value ¶
(Any) –

The value to reduce to a simple, encode-able type.
t ¶
(type[T] | ForwardRef | str | None, default: None ) –

The type to use for building the marshaller (optional). If not provided, we'll default to the type of the input value.

Source code in src/typelib/marshals/api.py

def marshal(
    value: tp.Any, *, t: type[T] | refs.ForwardRef | str | None = None
) -> serdes.MarshalledValueT:
    """Marshal `value` from `typ` into [`MarshalledValueT`][typelib.serdes.MarshalledValueT].

    Args:
        value: The value to reduce to a simple, encode-able type.
        t:
            The type to use for building the marshaller (optional).
            If not provided, we'll default to the type of the input value.
    """
    typ = value.__class__ if t is None else t
    routine: routines.AbstractMarshaller[T] = marshaller(typ)
    unmarshalled = routine(value)
    return unmarshalled

marshaller ¶

marshaller(t: type[T] | ForwardRef | TypeAliasType | str) -> AbstractMarshaller[T]

Get a marshaller routine for a given type.

Parameters:

t ¶
(type[T] | ForwardRef | TypeAliasType | str) –

The type annotation to generate a marshaller for. Can be a type, type alias, typing.ForwardRef, or string reference.

Source code in src/typelib/marshals/api.py

@compat.cache
def marshaller(
    t: type[T] | refs.ForwardRef | compat.TypeAliasType | str,
) -> routines.AbstractMarshaller[T]:
    """Get a marshaller routine for a given type.

    Args:
        t:
            The type annotation to generate a marshaller for. Can be a type, type alias,
            [`typing.ForwardRef`][], or string reference.
    """
    nodes = graph.static_order(t)
    context: ctx.TypeContext[routines.AbstractMarshaller] = ctx.TypeContext()
    if not nodes:
        return routines.NoOpMarshaller(t=t, context=context, var=None)  # type: ignore[arg-type]

    # "root" type will always be the final node in the sequence.
    root = nodes[-1]
    for node in nodes:
        context[node.type] = _get_unmarshaller(node, context=context)
        context[node.unwrapped] = context[node.type]

    return context[root.type]

marshals ¶

AbstractMarshaller ¶

t ¶

context ¶

var ¶

DelayedMarshaller ¶

resolved property ¶

EnumMarshaller ¶

t ¶

context ¶

var ¶

__call__ ¶

val ¶

FixedTupleMarshaller ¶

t ¶

context ¶

var ¶

__call__ ¶

val ¶

IterableMarshaller ¶

t ¶

context ¶

var ¶

__call__ ¶

val ¶

LiteralMarshaller ¶

t ¶

context ¶

var ¶

__call__ ¶

val ¶

MappingMarshaller ¶

t ¶

context ¶

var ¶

__call__ ¶

val ¶

PatternMarshaller ¶

t ¶

context ¶

var ¶

__call__ ¶

val ¶

StructuredTypeMarshaller ¶

t ¶

context ¶

var ¶

__call__ ¶

val ¶

SubscriptedIterableMarshaller ¶

t ¶

context ¶

var ¶

__call__ ¶

val ¶

SubscriptedMappingMarshaller ¶

t ¶

context ¶

var ¶

UnionMarshaller ¶

t ¶

context ¶

var ¶

__call__ ¶

val ¶

marshal ¶

value ¶

t ¶

marshaller ¶

t ¶

`t` ¶

`context` ¶

`var` ¶

resolved `property` ¶

`t` ¶

`context` ¶

`var` ¶

call ¶

`val` ¶

`t` ¶

`context` ¶

`var` ¶

call ¶

`val` ¶

`t` ¶

`context` ¶

`var` ¶

call ¶

`val` ¶

`t` ¶

`context` ¶

`var` ¶

call ¶

`val` ¶

`t` ¶

`context` ¶

`var` ¶

call ¶

`val` ¶

`t` ¶

`context` ¶

`var` ¶

call ¶

`val` ¶

`t` ¶

`context` ¶

`var` ¶

call ¶

`val` ¶

`t` ¶

`context` ¶

`var` ¶

call ¶

`val` ¶

`t` ¶

`context` ¶

`var` ¶

`t` ¶

`context` ¶

`var` ¶

call ¶

`val` ¶

`value` ¶

`t` ¶

`t` ¶