marshals

Support for marshalling Python data structures into primitive equivalents.

Notes

"Marshalling" your data structure prepares it to be serialized into binary and sent over the wire, but does not serialize it. We keep these stages isolated to ensure maximum flexibility and simplicity.

We ensure that your marshalled data is compatible with Python's built-in json module. This provides maximum compatibility with most serialization protocols by limiting the output to simple Python builtin types:

• bool
• int
• float
• str
• None
• list
• dict

Tip

You may use this package directly, but we encourage you to work with our high-level API provided by the top-level typelib module.

Typical Usage

>>> import dataclasses
>>> import decimal
>>> from typelib import marshals
>>>
>>> @dataclasses.dataclass(slots=True, weakref_slot=True, kw_only=True)
... class Struct:
...     key: str
...     number: decimal.Decimal
...
>>>
>>> data = Struct(key="some-key", number=decimal.Decimal("1.0"))
>>> marshals.marshal(data)
{'key': 'some-key', 'number': '1.0'}
>>> marshaller = marshals.marshaller(Struct)
>>> marshaller(data)
{'key': 'some-key', 'number': '1.0'}

See Also

• marshal
• marshaller
• typelib.codec

AbstractMarshaller

AbstractMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: ABC, Generic[T]

Abstract base class defining the common interface for marshallers.

Marshallers are custom callables which maintain type-specific information. They use this information to provide robust, performant logic reducing Python objects into their primitive representations for over-the-wire encoding.

Marshallers support contextual serialization, which enables the marshalling of nested types.

Attributes:

• t (type[T]) –

  The root type of this marshaller.

• origin (type[T]) –

  If t is a generic, this will be an actionable runtime type related to t, otherwise it is the same as t.

• context (ContextT) –

  The complete type context for this unmarshaller.

• var (str | None) –

  If this unmarshaller is used in a nested context, this will reference the field/parameter/index at which this unmarshaller should be used.

Parameters:

• t (type[T]) –

  The root type of this marshaller.

• context (ContextT) –

  The complete type context for this marshaller.

• var (str | None, default: None ) –

  The associated field or parameter name for this unmarshaller (optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[T], context: ContextT, *, var: str | None = None):
    """Construct a marshaller instance.

    Args:
        t: The root type of this marshaller.
        context: The complete type context for this marshaller.
        var: The associated field or parameter name for this unmarshaller (optional).
    """
    self.t = t
    self.origin = inspection.origin(self.t)
    self.context = context
    self.var = var

DelayedMarshaller

DelayedMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[T]

Delayed proxy for a given type's marshaller, used when we encounter a typing.ForwardRef.

Notes

This allows us to delay the resolution of the given type reference until call-time, enabling support for cyclic and recursive types.

Source code in src/typelib/marshals/api.py

def __init__(
    self, t: type[T], context: routines.ContextT, *, var: str | None = None
):
    super().__init__(t, context, var=var)
    self._resolved: routines.AbstractMarshaller[T] | None = None

resolved property

resolved: AbstractMarshaller[T]

The resolved marshaller.

EnumMarshaller

EnumMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[EnumT], Generic[EnumT]

A marshaller that converts an enum.Enum instance to its assigned value.

Parameters:

• t (type[T]) –

  The root type of this marshaller.

• context (ContextT) –

  The complete type context for this marshaller.

• var (str | None, default: None ) –

  The associated field or parameter name for this unmarshaller (optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[T], context: ContextT, *, var: str | None = None):
    """Construct a marshaller instance.

    Args:
        t: The root type of this marshaller.
        context: The complete type context for this marshaller.
        var: The associated field or parameter name for this unmarshaller (optional).
    """
    self.t = t
    self.origin = inspection.origin(self.t)
    self.context = context
    self.var = var

__call__

__call__(val: EnumT) -> MarshalledValueT

Marshal an enum.Enum instance into a [serdes.MarshalledValueT][serdes.MarshalledValueT].

Parameters:

• val (EnumT) –

  The enum instance to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: EnumT) -> serdes.MarshalledValueT:
    """Marshal an [`enum.Enum`][] instance into a [`serdes.MarshalledValueT`][].

    Args:
        val: The enum instance to marshal.
    """
    return val.value

FixedTupleMarshaller

FixedTupleMarshaller(t: type[TupleT], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[TupleT]

A marshaller for dumping a "fixed" tuple to a simple list.

Values are marshalled according to the value-type in the order they are defined.

See Also

• FixedTupleUnmarshaller

Parameters:

• t (type[TupleT]) –

  The type to unmarshal from.

• context (ContextT) –

  Any nested type context. Used to resolve the member marshallers.

• var (str | None, default: None ) –

  A variable name for the indicated type annotation (unused, optional).

Source code in src/typelib/marshals/routines.py

def __init__(
    self, t: type[compat.TupleT], context: ContextT, *, var: str | None = None
):
    """Constructor.

    Args:
        t: The type to unmarshal from.
        context: Any nested type context. Used to resolve the member marshallers.
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t, context, var=var)
    self.stack = inspection.args(t, evaluate=True)
    self.ordered_routines = [self.context[vt] for vt in self.stack]

__call__

__call__(val: TupleT) -> MarshalledIterableT

Marshal a tuple into a simple list.

Parameters:

• val (TupleT) –

  The tuple to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: compat.TupleT) -> MarshalledIterableT:
    """Marshal a tuple into a simple [`list`][].

    Args:
        val: The tuple to marshal.
    """
    return [
        routine(v)
        for routine, v in zip(self.ordered_routines, serdes.itervalues(val))
    ]

IterableMarshaller

IterableMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[IterableT], Generic[IterableT]

A marshaller for dumping any iterable into a simple list.

Parameters:

• t (type[T]) –

  The root type of this marshaller.

• context (ContextT) –

  The complete type context for this marshaller.

• var (str | None, default: None ) –

  The associated field or parameter name for this unmarshaller (optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[T], context: ContextT, *, var: str | None = None):
    """Construct a marshaller instance.

    Args:
        t: The root type of this marshaller.
        context: The complete type context for this marshaller.
        var: The associated field or parameter name for this unmarshaller (optional).
    """
    self.t = t
    self.origin = inspection.origin(self.t)
    self.context = context
    self.var = var

__call__

__call__(val: IterableT) -> MarshalledIterableT

Marshal an iterable into a simple list.

Parameters:

• val (IterableT) –

  The iterable to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: IterableT) -> MarshalledIterableT:
    """Marshal an iterable into a simple [`list`][].

    Args:
        val: The iterable to marshal.
    """
    return [*val]

LiteralMarshaller

LiteralMarshaller(t: type[LiteralT], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[LiteralT], Generic[LiteralT]

A marshaller that enforces the given value be one of the values in the defined typing.Literal

Parameters:

• t (type[LiteralT]) –

  The Literal type to enforce.

• context (ContextT) –

  Nested type context (unused).

• var (str | None, default: None ) –

  A variable name for the indicated type annotation (unused, optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[LiteralT], context: ContextT, *, var: str | None = None):
    """Constructor.

    Args:
        t: The Literal type to enforce.
        context: Nested type context (unused).
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t, context, var=var)
    self.values = inspection.args(t)

__call__

__call__(val: LiteralT) -> MarshalledValueT

Enforce the given value is a member of the bound Literal type.

Parameters:

• val (LiteralT) –

  The value to enforce.

Raises:

• ValueError –

  If val is not a member of the bound Literal type.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: LiteralT) -> serdes.MarshalledValueT:
    """Enforce the given value is a member of the bound `Literal` type.

    Args:
        val: The value to enforce.

    Raises:
        ValueError: If `val` is not a member of the bound `Literal` type.
    """
    if val in self.values:
        return val  # type: ignore[return-value]

    raise ValueError(f"{val!r} is not one of {self.values!r}")

MappingMarshaller

MappingMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[MappingT], Generic[MappingT]

A marshaller for dumping any mapping into a simple dict.

Parameters:

• t (type[T]) –

  The root type of this marshaller.

• context (ContextT) –

  The complete type context for this marshaller.

• var (str | None, default: None ) –

  The associated field or parameter name for this unmarshaller (optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[T], context: ContextT, *, var: str | None = None):
    """Construct a marshaller instance.

    Args:
        t: The root type of this marshaller.
        context: The complete type context for this marshaller.
        var: The associated field or parameter name for this unmarshaller (optional).
    """
    self.t = t
    self.origin = inspection.origin(self.t)
    self.context = context
    self.var = var

__call__

__call__(val: MappingT) -> MarshalledMappingT

Marshal a mapping into a simple dict.

Parameters:

• val (MappingT) –

  The mapping object to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: MappingT) -> MarshalledMappingT:
    """Marshal a mapping into a simple [`dict`][].

    Args:
        val: The mapping object to marshal.
    """
    return {**val}

PatternMarshaller

PatternMarshaller(t: type[T], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[PatternT]

A marshaller that converts a re.Pattern to a string.

Parameters:

• t (type[T]) –

  The root type of this marshaller.

• context (ContextT) –

  The complete type context for this marshaller.

• var (str | None, default: None ) –

  The associated field or parameter name for this unmarshaller (optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[T], context: ContextT, *, var: str | None = None):
    """Construct a marshaller instance.

    Args:
        t: The root type of this marshaller.
        context: The complete type context for this marshaller.
        var: The associated field or parameter name for this unmarshaller (optional).
    """
    self.t = t
    self.origin = inspection.origin(self.t)
    self.context = context
    self.var = var

__call__

__call__(val: PatternT) -> str

Marshal a compiled regex pattern into a string.

Parameters:

• val (PatternT) –

  The pattern to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: PatternT) -> str:
    """Marshal a compiled regex pattern into a string.

    Args:
        val: The pattern to marshal.
    """
    return val.pattern

StructuredTypeMarshaller

StructuredTypeMarshaller(t: type[_ST], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[_ST]

A marshaller for dumping a structured (user-defined) type to a simple dict.

See Also

• StructuredTypeUnmarshaller

Parameters:

• t (type[_ST]) –

  The type to unmarshals from.

• context (ContextT) –

  Any nested type context. Used to resolve the member marshallers.

• var (str | None, default: None ) –

  A variable name for the indicated type annotation (unused, optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[_ST], context: ContextT, *, var: str | None = None):
    """Constructor.

    Args:
        t: The type to unmarshals from.
        context: Any nested type context. Used to resolve the member marshallers.
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t, context, var=var)
    self.fields_by_var = self._fields_by_var()

__call__

__call__(val: _ST) -> MarshalledMappingT

Marshal a structured type into a simple dict.

Parameters:

• val (_ST) –

  The structured type to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: _ST) -> MarshalledMappingT:
    """Marshal a structured type into a simple [`dict`][].

    Args:
        val: The structured type to marshal.
    """
    fields = self.fields_by_var
    return {f: fields[f](v) for f, v in serdes.iteritems(val) if f in fields}

SubscriptedIterableMarshaller

SubscriptedIterableMarshaller(t: type[IterableT], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[IterableT], Generic[IterableT]

A marshaller for dumping a subscripted iterable into a simple list.

Values are marshalled according to the defined value-type.

See Also

• SubscriptedIterableUnmarshaller

Parameters:

• t (type[IterableT]) –

  The type to unmarshals from.

• context (ContextT) –

  Any nested type context. Used to resolve the member marshallers.

• var (str | None, default: None ) –

  A variable name for the indicated type annotation (unused, optional).

Source code in src/typelib/marshals/routines.py

def __init__(
    self, t: type[IterableT], context: ContextT, *, var: str | None = None
):
    """Constructor.

    Args:
        t: The type to unmarshals from.
        context: Any nested type context. Used to resolve the member marshallers.
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t=t, context=context, var=var)
    # supporting tuple[str, ...]
    (value_t, *_) = inspection.args(t, evaluate=True)
    self.values = context[value_t]

__call__

__call__(val: IterableT) -> MarshalledIterableT

Marshal an iterable into a simple list.

Parameters:

• val (IterableT) –

  The iterable to marshal.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: IterableT) -> MarshalledIterableT:
    """Marshal an iterable into a simple [`list`][].

    Args:
        val: The iterable to marshal.
    """
    # Always decode bytes.
    values = self.values
    return [values(v) for v in serdes.itervalues(val)]

SubscriptedMappingMarshaller

SubscriptedMappingMarshaller(t: type[MappingT], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[MappingT], Generic[MappingT]

A marshaller for dumping a subscripted mapping into a simple dict.

Keys are marshalled according to the defined key-type, values according to the defined value-type.

See Also

• SubscriptedMappingUnmarshaller

Parameters:

• t (type[MappingT]) –

  The type to unmarshals from.

• context (ContextT) –

  Any nested type context. Used to resolve the member marshallers.

• var (str | None, default: None ) –

  A variable name for the indicated type annotation (unused, optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[MappingT], context: ContextT, *, var: str | None = None):
    """Constructor.

    Args:
        t: The type to unmarshals from.
        context: Any nested type context. Used to resolve the member marshallers.
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t, context, var=var)
    key_t, value_t = inspection.args(t, evaluate=True)
    self.keys = context[key_t]
    self.values = context[value_t]

UnionMarshaller

UnionMarshaller(t: type[UnionT], context: ContextT, *, var: str | None = None)

Bases: AbstractMarshaller[UnionT], Generic[UnionT]

A marshaller for dumping a given value via one of the types in the defined bound union.

See Also

• UnionUnmarshaller

Parameters:

• t (type[UnionT]) –

  The type to unmarshal into.

• context (ContextT) –

  Any nested type context. Used to resolve the member marshallers.

• var (str | None, default: None ) –

  A variable name for the indicated type annotation (unused, optional).

Source code in src/typelib/marshals/routines.py

def __init__(self, t: type[UnionT], context: ContextT, *, var: str | None = None):
    """Constructor.

    Args:
        t: The type to unmarshal into.
        context: Any nested type context. Used to resolve the member marshallers.
        var: A variable name for the indicated type annotation (unused, optional).
    """
    super().__init__(t, context, var=var)
    self.stack = inspection.args(t, evaluate=True)
    self.nullable = inspection.isoptionaltype(t)
    self.ordered_routines = [self.context[typ] for typ in self.stack]

__call__

__call__(val: UnionT) -> MarshalledValueT

Marshal a value into the bound UnionT.

Parameters:

• val (UnionT) –

  The input value to unmarshal.

Raises:

• ValueError –

  If val cannot be marshalled via any member type.

Source code in src/typelib/marshals/routines.py

def __call__(self, val: UnionT) -> serdes.MarshalledValueT:
    """Marshal a value into the bound `UnionT`.

    Args:
        val: The input value to unmarshal.

    Raises:
        ValueError: If `val` cannot be marshalled via any member type.
    """
    if self.nullable and val is None:
        return val

    for routine in self.ordered_routines:
        with contextlib.suppress(
            ValueError, TypeError, SyntaxError, AttributeError
        ):
            unmarshalled = routine(val)
            return unmarshalled

    raise ValueError(f"{val!r} is not one of types {self.stack!r}")

marshal

marshal(value: Any, *, t: type[T] | ForwardRef | str | None = None) -> MarshalledValueT

Marshal value from typ into MarshalledValueT.

Parameters:

• value (Any) –

  The value to reduce to a simple, encode-able type.

• t (type[T] | ForwardRef | str | None, default: None ) –

  The type to use for building the marshaller (optional). If not provided, we'll default to the type of the input value.

Source code in src/typelib/marshals/api.py

def marshal(
    value: tp.Any, *, t: type[T] | refs.ForwardRef | str | None = None
) -> serdes.MarshalledValueT:
    """Marshal `value` from `typ` into [`MarshalledValueT`][typelib.serdes.MarshalledValueT].

    Args:
        value: The value to reduce to a simple, encode-able type.
        t:
            The type to use for building the marshaller (optional).
            If not provided, we'll default to the type of the input value.
    """
    typ = value.__class__ if t is None else t
    routine: routines.AbstractMarshaller[T] = marshaller(typ)
    unmarshalled = routine(value)
    return unmarshalled

marshaller

marshaller(t: type[T] | ForwardRef | TypeAliasType | str) -> AbstractMarshaller[T]

Get a marshaller routine for a given type.

Parameters:

• t (type[T] | ForwardRef | TypeAliasType | str) –

  The type annotation to generate a marshaller for. Can be a type, type alias, typing.ForwardRef, or string reference.

Source code in src/typelib/marshals/api.py

@compat.cache
def marshaller(
    t: type[T] | refs.ForwardRef | compat.TypeAliasType | str,
) -> routines.AbstractMarshaller[T]:
    """Get a marshaller routine for a given type.

    Args:
        t:
            The type annotation to generate a marshaller for. Can be a type, type alias,
            [`typing.ForwardRef`][], or string reference.
    """
    nodes = graph.static_order(t)
    context: ctx.TypeContext[routines.AbstractMarshaller] = ctx.TypeContext()
    if not nodes:
        return routines.NoOpMarshaller(t=t, context=context, var=None)  # type: ignore[arg-type]

    # "root" type will always be the final node in the sequence.
    root = nodes[-1]
    for node in nodes:
        context[node.type] = _get_unmarshaller(node, context=context)
        context[node.unwrapped] = context[node.type]

    return context[root.type]