IterUtils: Iteration Utilities#

class my.utils.IterUtils.IterUtils#

Utility functions for working with iterators, sequences, mappings, and other containers.

Examples

Every method is reachable through the combined ut facade:

>>> from my import ut
>>> ut.partition([1, 2, 3, 4], lambda v: v % 2 == 0)
([1, 3], [2, 4])

I Construction#

classmethod IterUtils.build(val: V, *functions: Callable[[V], V]) → V#

Apply a sequence of functions to a value using reduce.

Parameters:

  • val – Initial value.

  • *functions – Functions to apply in sequence.

Returns:

Final transformed value after applying all functions.

Examples

Pipe a value through successive transforms:

>>> from my import ut
>>> ut.build('  Hi  ', str.strip, str.lower)
'hi'
classmethod IterUtils.map_items(value: MapT[K, V]) → list[tuple[K, V]]#
classmethod IterUtils.map_items(value: object) → list[tuple[Any, Any]]

Extract key-value pairs from mapping-like or tuple sequence objects.

Parameters:

value – Object to extract items from (dict, mapping, or sequence of 2-tuples).

Returns:

List of (key, value) tuples, or empty list if extraction fails.

Examples

Accept both mappings and pair sequences:

>>> from my import ut
>>> ut.map_items({'a': 1, 'b': 2})
[('a', 1), ('b', 2)]
>>> ut.map_items([('a', 1), ('b', 2)])
[('a', 1), ('b', 2)]
classmethod IterUtils.partition(items: Iterable, pred: Callable[[V], bool]) → tuple[list[V], list[V]]#

Partition items into two lists based on a predicate.

Parameters:

  • items – Iterable to partition.

  • pred – Predicate function (True items go to second list).

Returns:

Tuple of (fails, passes) (NOTE that fails come first!).

Examples

Partition odds from evens:

>>> from my import ut
>>> ut.partition([1, 2, 3, 4], lambda x: x % 2 == 0)
([1, 3], [2, 4])
classmethod IterUtils.multi_partition(items: Iterable, **preds: Callable[[V], object]) → dict[str, list[V]]#

Partition items into multiple named buckets based on predicates.

Parameters:

  • items – Iterable to partition.

  • **preds – Named predicates (keys become bucket names).

Returns:

Dict with predicate names as keys, plus ‘rest’ for unmatched items.

Raises:

AssertionError – If ‘rest’ is used as a predicate key name.

Examples

Route items into named buckets, with leftovers under ‘rest’:

>>> from my import ut
>>> ut.multi_partition(range(6), even=lambda x: x % 2 == 0, big=lambda x: x > 3)
{'even': [0, 2, 4], 'big': [5], 'rest': [1, 3]}
classmethod IterUtils.type_partition(container: Iterable[T0 | T1], t0: type[T0], t1: type[T1]) → tuple[list[T0], list[T1]]#

Partition a container into two lists based on type.

Parameters:

  • container – Iterable of mixed-type items.

  • t0 – Type collected into the first list.

  • t1 – Type collected into the second list.

Returns:

Tuple of (items of type t0, items of type t1); items of neither type are dropped.

Examples

Separate ints from strings:

>>> from my import ut
>>> ut.type_partition([1, 'a', 2, 'b'], int, str)
([1, 2], ['a', 'b'])
classmethod IterUtils.bucket(items: Iterable, pred: Callable[[T], K]) → dict[K, list[T]]#

Group items into buckets based on a key function.

Parameters:

  • items – Iterable to bucket.

  • pred – Function returning bucket key for each item.

Returns:

Defaultdict mapping bucket keys to lists of items.

Examples

Group words by first letter:

>>> from my import ut
>>> dict(ut.bucket(['apple', 'avocado', 'banana'], lambda w: w[0]))
{'a': ['apple', 'avocado'], 'b': ['banana']}

II Selection#

classmethod IterUtils.find(container: ~collections.abc.Sequence, predicate: ~collections.abc.Callable[[V], bool] | V = <class 'bool'>) → int#

Find index of first item matching predicate or value.

Parameters:

  • container – Sequence to search.

  • predicate – Predicate function or value to match (default: bool for truthiness).

Returns:

Index of first match, or -1 if not found.

Examples

Locate by truthiness, predicate, or literal value:

>>> from my import ut
>>> ut.find([0, 0, 3, 5])
2
>>> ut.find(['x', 'y', 'z'], 'y')
1
>>> ut.find([0, 0])
-1
classmethod IterUtils.find_key(items: ~my.infra.types.MapT, predicate: ~collections.abc.Callable[[V], bool] | V = <class 'bool'>, default: K | None = None) → K | None#

Find the first key in a map whose value matches the provided predicate.

Parameters:

  • items – Mapping or iterable of (key, value) pairs to search.

  • predicate – Predicate function or value to match (default: truthiness check).

  • default – Default value to return if no match found (default: None).

Returns:

First matching key, or default if none found.

Examples

Find the key of the first matching value:

>>> from my import ut
>>> ut.find_key({'a': 0, 'b': 2, 'c': 3}, lambda v: v > 1)
'b'
>>> ut.find_key({'a': 0}, lambda v: v > 1, default='n/a')
'n/a'
classmethod IterUtils.next_in(container: Container, items: Iterable) → V | None#

Find first item from iterable that exists in container.

Parameters:

  • container – Container to check membership in.

  • items – Items to check.

Returns:

First item found in container, or None.

Examples

Take the first item the container admits:

>>> from my import ut
>>> ut.next_in({'b', 'c'}, ['a', 'b', 'c'])
'b'
classmethod IterUtils.condense(items: Iterable[V | False | None], pred: Pred[V] = bool) → list[V]#
classmethod IterUtils.condense(items: Iterable[V], pred: Pred[V] = bool) → list[V]

Filter items by predicate, returning list of matches.

Parameters:

  • items – Iterable to filter.

  • pred – Predicate function (default: bool for truthiness).

Returns:

List of items matching predicate.

Examples

Drop falsy values, or filter by an explicit predicate:

>>> from my import ut
>>> ut.condense([0, 1, '', 'x', None])
[1, 'x']
>>> ut.condense([1, 2, 3, 4], lambda x: x % 2)
[1, 3]
classmethod IterUtils.normalize_predicate(pred: Pred) → Callable[[P], bool]#

Convert a predicate that may be a value, iterable, or function into a standard function.

Parameters:

pred – Predicate to normalize (value, iterable, or function).

Returns:

A one-argument function returning a boolean.

Examples

Turn a container into a membership test, and a value into an equality test:

>>> from my import ut
>>> fn = ut.normalize_predicate([1, 2])
>>> fn(1), fn(5)
(True, False)
>>> eq = ut.normalize_predicate(3)
>>> eq(3), eq(4)
(True, False)
classmethod IterUtils.predicate(items: Iterable, *preds: Pred) → Iterator#

Filter items by one or more predicates, yielding matches.

Parameters:

  • items – Iterable to filter.

  • *preds – Predicates to apply (value, iterable, or function).

Yields:

Items satisfying every given predicate.

Examples

Keep only items that pass all predicates:

>>> from my import ut
>>> list(ut.predicate([1, 2, 3, 4], lambda x: x > 1, lambda x: x < 4))
[2, 3]
classmethod IterUtils.map_condense(items: MapT, pred: Pred = <class 'bool'>) → Iterator[tuple[K, V]]#

Filter a mapping by a predicate function on values.

Parameters:

  • items – Mapping or iterable of (key, value) pairs to filter.

  • pred – Predicate function applied to values (default: bool for truthiness).

Yields:

(key, value) tuples where value satisfies the predicate.

Examples

Keep entries with truthy values:

>>> from my import ut
>>> list(ut.map_condense({'a': 0, 'b': 2, 'c': ''}))
[('b', 2)]
classmethod IterUtils.get_all(data: dict[K, V], *args: Pred[K], mandatory: bool = True) → dict[K, V]#
classmethod IterUtils.get_all(data: BaseModel | object, *args: Pred[K], mandatory: bool = True) → dict[str, Any]

Extract multiple keys from a dictionary or model.

Parameters:

  • data – Source of key-value pairs, to be extracted.

  • *args – Keys or predicates to extract.

  • mandatory – If True, only return values if all supplied arguments are satisfied/present; otherwise return whatever partial matches were found (default: True).

Returns:

Dict with requested keys that exist, or {} if mandatory and any key is missing.

Examples

All-or-nothing extraction, unless mandatory is disabled:

>>> from my import ut
>>> ut.get_all({'a': 1, 'b': 2, 'c': 3}, 'a', 'b')
{'a': 1, 'b': 2}
>>> ut.get_all({'a': 1}, 'a', 'z')
{}
>>> ut.get_all({'a': 1}, 'a', 'z', mandatory=False)
{'a': 1}
classmethod IterUtils.get_any(data: MapT[K, V], *args: Pred[K]) → dict[K, V]#
classmethod IterUtils.get_any(data: BaseModel | object, *args: Pred[K]) → dict[str, Any]

Extract multiple keys from a dictionary or model.

Note

Remember, a predicate is: V | Iterable[V] | Callable[[V], R]. In this case, R is locked to bool.

Parameters:

  • data – Source of key-value pairs, to be extracted.

  • *args – Keys or predicates to extract.

Returns:

Dict with requested keys that exist.

Examples

Collect whichever keys are present:

>>> from my import ut
>>> ut.get_any({'a': 1, 'b': 2}, 'a', 'z')
{'a': 1}
classmethod IterUtils.get_first(data: MapT[K, V], *args: K | Callable[[K], bool], default: V, unique: bool = False) → V#
classmethod IterUtils.get_first(data: MapT[K, V], *args: K | Callable[[K], bool], unique: bool = False) → V | None
classmethod IterUtils.get_first(data: Iterable[V], *args: V | Callable[[V], bool], default: V, unique: bool = False) → V
classmethod IterUtils.get_first(data: Iterable[V], *args: V | Callable[[V], bool], unique: bool = False) → V | None

Get value for first matching value from the container.

Note

data must be a mapping or a live iterator (e.g. iter([...])); despite the Iterable overloads, a plain non-iterator sequence falls through every branch and returns None.

Parameters:

  • data – Structure to search.

  • *args – Keys to try in order.

  • default – Default value if no keys found (default: None).

  • unique – If True, raise error if multiple keys found (default: False).

Returns:

Value of first matching key, or default.

Raises:

ValueError – If unique=True and multiple keys match.

Examples

Match keys by predicate, falling back to a default:

>>> from my import ut
>>> ut.get_first({'alpha': 1, 'beta': 2}, lambda k: k.startswith('b'))
2
>>> ut.get_first({'a': 1}, 'z', default=0)
0
classmethod IterUtils.normalize(data: str | bytes | bytearray | memoryview | IO) → str#
classmethod IterUtils.normalize(data: MapT[K, V]) → dict[K, V]
classmethod IterUtils.normalize(data: Iterable[V] | StructT[V]) → list[V]
classmethod IterUtils.normalize(data: V) → V

Normalize the input data into a more workable form for casting.

Parameters:

data – Value to normalize; containers are normalized recursively.

Returns:

A str/dict/list-shaped equivalent, with byte-strings decoded and times cast to UTC; atoms (None, types, enums, models, …) pass through untouched.

Examples

Decode byte-strings and rebuild containers:

>>> from my import ut
>>> ut.normalize({'a': (1, 2), 'b': b'x'})
{'a': [1, 2], 'b': 'x'}
classmethod IterUtils.safe(container: MapT[K, V | Mapping[Hashable, Any] | Iterable[tuple[Hashable, Any]] | ItemsView | None], *keys: K) → V | None#

Safely access nested map values with multiple keys.

Parameters:

  • container – Map to access.

  • *keys – Sequence of keys to traverse.

Returns:

Value at nested location if all keys exist, else None.

Examples

Traverse nested keys without raising:

>>> from my import ut
>>> ut.safe({'a': {'b': 1}}, 'a', 'b')
1
>>> ut.safe({'a': {'b': 1}}, 'a', 'z') is None
True

III Application#

classmethod IterUtils.val_map(func: Callable[[T0], T1], data: Mapping[K, T0] | Iterable[tuple[K, T0]], drop: bool = False) → dict[K, T1]#
classmethod IterUtils.val_map(func: Callable[[K], T1], data: Iterable[K], drop: bool = False) → dict[K, T1]

Map a function over values in a mapping or iterable, returning new dictionary.

Parameters:

  • func – Function to apply to each value.

  • data – Mapping, iterable of (key, value) pairs, or iterable of keys.

  • drop – If True, drop falsy values from result (default: False).

Returns:

Dictionary with function applied to values (or to items if data is simple iterable).

Examples

Map over a mapping’s values, or build a dict from bare keys:

>>> from my import ut
>>> ut.val_map(str.upper, {'a': 'x', 'b': 'y'})
{'a': 'X', 'b': 'Y'}
>>> ut.val_map(len, ['hi', 'there'])
{'hi': 2, 'there': 5}
classmethod IterUtils.attr_map(obj: object, fields: Iterable[str], drop: bool = False) → dict[str, Any]#

Extract attributes from object into dictionary.

Parameters:

  • obj – Object to extract attributes from.

  • fields – Attribute names to extract.

  • drop – If True, drop falsy values and use default=’’ (default: False).

Returns:

Dict mapping field names to attribute values.

Examples

Snapshot attributes into a dict:

>>> from my import ut
>>> ut.attr_map(complex(3, 4), ['real', 'imag'])
{'real': 3.0, 'imag': 4.0}
classmethod IterUtils.apply(functions: Callable[[P], R] | Iterable[Callable[[P], R]], *args: P, **kwargs: P) → Iterator#

Apply multiple functions to an item, yielding non-falsy results.

Parameters:

  • functions – Functions to apply.

  • *args – Positional arguments to pass to each function.

  • **kwargs – Keyword arguments to pass to each function.

Yields:

Non-falsy results from function applications.

Examples

Fan one input out over several functions:

>>> from my import ut
>>> list(ut.apply([str.upper, str.title], 'hi there'))
['HI THERE', 'Hi There']
classmethod IterUtils.inverse_map(functions: Iterable[Callable[[P], R]], predicate: Callable[[R], Any] | None, *args: P, **kwargs: P) → Iterator#

Apply multiple functions to a single set of arguments, returning filtered results.

Parameters:

  • functions – Functions to apply.

  • predicate – Predicate with which to filter results (pass None for plain truthiness).

  • *args – Positional arguments to pass to each function.

  • **kwargs – Keyword arguments to pass to each function.

Yields:

Results for which the predicate holds.

Examples

Apply many functions to one argument list:

>>> from my import ut
>>> list(ut.inverse_map([min, max], None, [3, 1, 4]))
[1, 4]
classmethod IterUtils.indexof(iterable: Sequence, *preds: Callable[[T], bool] | T) → int#

Return the first index in the given iterable that satisfies the predicate.

Parameters:

  • iterable – Sequence to search.

  • *preds – Plain functions (e.g. lambdas) used as predicates; any other value – including builtin or bound callables – is instead compared by equality.

Returns:

Index of the first item matching any predicate, or -1 if none match.

Examples

Search by predicate or by literal value:

>>> from my import ut
>>> ut.indexof(['a', 'bb', 'ccc'], lambda s: len(s) > 1)
1
>>> ut.indexof(['a', 'bb'], 'bb')
1
classmethod IterUtils.sorted_insert(array: ~collections.abc.MutableSequence, item: T, key: ~collections.abc.Callable[[T], ~typing.Any] = <function IterUtils.<lambda>>) → int#

Insert an item into a sorted sequence, keeping it sorted by key.

Parameters:

  • array – A mutable sequence, already sorted ascending by key. Mutated in place.

  • item – The item to insert.

  • key – Sort key function (default: the item itself).

Returns:

The index the item was inserted at.

Examples

Keep a running sorted list without a full re-sort:

>>> from my import ut
>>> array = [1, 3, 5]
>>> ut.sorted_insert(array, 4)
2
>>> array
[1, 3, 4, 5]
classmethod IterUtils.groupby(iterable: Iterable, key: int | str | Callable[[T], Hashable], drop: bool = False, keys: type[Hashable] | None = None) → Iterable[tuple[Hashable, list[T]]]#

Group the items in the given iterable by the given key.

Unlike itertools.groupby, the input does not need to already be sorted by key – this sorts internally first, so every group is returned complete and contiguous.

The key type is left as the broad Hashable (rather than inferred through a generic type parameter) because key’s three accepted shapes – an int index, an attribute name, or a callable – don’t let a type checker relate the argument to a single concrete key type; narrow the result at the call site if a caller needs more.

Parameters:

  • iterable – Items to group.

  • key – An int index (for tuple/sequence items), an attribute name (str), or a callable extracting the group key from an item.

  • drop – If True, drop items whose key is falsy before grouping.

  • keys – Unused; reserved for a future explicit key-type annotation/validation.

Yields:

(key, items) pairs, one per distinct key, each with every matching item.

Examples

Group by an attribute name, dropping items with a falsy key:

>>> from my import ut
>>> class Item:
...     def __init__(self, tag):
...         self.tag = tag
>>> items = [Item('x'), Item(''), Item('y'), Item('x')]
>>> {k: len(v) for k, v in ut.groupby(items, 'tag', drop=True)}
{'x': 2, 'y': 1}
classmethod IterUtils.locate(seq: Sequence, pred: Callable[[T], bool]) → tuple[int, T]#

Return the (index, item) pair for the first item in seq matching pred.

Parameters:

  • seq – Sequence to search.

  • pred – Predicate function to apply to each item.

Returns:

A (index, item) tuple for the first match.

Raises:

AssertionError – If no item in seq satisfies pred.

Examples

>>> from my import ut
>>> ut.locate([10, 20, 30], lambda x: x > 15)
(1, 20)

IV Execution#

classmethod IterUtils.repeat_until_complete(func: Callable[[C, V], tuple[int, V]]) → Callable#

Decorator to repeatedly apply function until it returns 0 changes.

Parameters:

func – Function returning (num_changes, transformed_value).

Returns:

Wrapped function that repeats until num_changes is 0.

Examples

Halve a number until it goes odd, counting the passes:

>>> from my import ut
>>> @ut.repeat_until_complete
... def halve_evens(_, n):
...     return (1, n // 2) if n % 2 == 0 else (0, n)
>>> halve_evens(None, 40)
(3, 5)

V Presence#

classmethod IterUtils.has_all(container: Container, *args: V) → bool#

Check if container contains all specified items.

Parameters:

  • container – Container to check.

  • *args – Items that must all be present.

Returns:

True if all items present, False otherwise or if container empty.

Examples

Require every item:

>>> from my import ut
>>> ut.has_all({'a', 'b', 'c'}, 'a', 'b')
True
>>> ut.has_all({'a'}, 'a', 'z')
False
classmethod IterUtils.has_any(container: Container, *args: V) → bool#

Check if container contains any of the specified items.

Parameters:

  • container – Container to check.

  • *args – Items to check for (any match succeeds).

Returns:

True if any item present, False otherwise or if container empty.

Examples

Any one match suffices:

>>> from my import ut
>>> ut.has_any(['a', 'b'], 'b', 'z')
True
classmethod IterUtils.has_only(container: Collection, *args: V) → bool#

Check if container contains exactly the specified items, no more no less.

Parameters:

  • container – Collection to check.

  • *args – Items that should comprise the entire collection.

Returns:

True if container contains exactly these items.

Examples

Exact membership, order-independent:

>>> from my import ut
>>> ut.has_only(['b', 'a'], 'a', 'b')
True
classmethod IterUtils.has_none(container: Container, *args: V) → bool#

Check if container contains none of the specified items.

Parameters:

  • container – Container to check.

  • *args – Items that must all be absent.

Returns:

True if no items present, False otherwise.

Examples

Require total absence:

>>> from my import ut
>>> ut.has_none({'a'}, 'x', 'y')
True
classmethod IterUtils.all_has_all(containers: Iterable[Container], *args: V) → bool#

Check if all containers contain all specified items.

Parameters:

  • containers – Containers to check.

  • *args – Items that must be in all containers.

Returns:

True if every container has all items, False otherwise or if empty.

Examples

Every container must contain every item:

>>> from my import ut
>>> ut.all_has_all([{'a', 'b'}, {'a', 'b', 'c'}], 'a', 'b')
True
classmethod IterUtils.any_has_all(containers: Iterable[Container], *args: V) → bool#

Check if any container contains all specified items.

Parameters:

  • containers – Containers to check.

  • *args – Items that must all be in at least one container.

Returns:

True if at least one container has all items, False otherwise or if empty.

Examples

One container with every item suffices:

>>> from my import ut
>>> ut.any_has_all([{'a'}, {'a', 'b'}], 'a', 'b')
True
classmethod IterUtils.all_has_any(containers: Iterable[Container], *args: V) → bool#

Check if all containers contain at least one of the specified items.

Parameters:

  • containers – Containers to check.

  • *args – Items (at least one must be in each container).

Returns:

True if every container has at least one item, False otherwise or if empty.

Examples

Every container needs at least one of the items:

>>> from my import ut
>>> ut.all_has_any([{'a'}, {'b'}], 'a', 'b')
True
classmethod IterUtils.any_has_any(containers: Iterable[Container], *args: V) → bool#

Check if any container contains any of the specified items.

Parameters:

  • containers – Containers to check.

  • *args – Items to look for.

Returns:

True if at least one container has at least one item, False otherwise or if empty.

Examples

Any overlap at all suffices:

>>> from my import ut
>>> ut.any_has_any([{'x'}, {'y'}], 'y', 'z')
True

VI Comparison#

classmethod IterUtils.shared_prefix(*strings: str) → str#

Find longest common prefix of all strings.

Parameters:

*strings – Strings to compare.

Returns:

Longest common prefix string.

Examples

Extract the common start:

>>> from my import ut
>>> ut.shared_prefix('flowchart', 'flow', 'flower')
'flow'
classmethod IterUtils.shared_suffix(*strings: str) → str#

Find longest common suffix of all strings.

Parameters:

*strings – Strings to compare.

Returns:

Longest common suffix string.

Examples

Extract the common ending:

>>> from my import ut
>>> ut.shared_suffix('walking', 'running')
'ing'
classmethod IterUtils.common_elements(lhs: Sequence | set[V], rhs: Sequence | set[V]) → list[V]#

Return a version of the first sequence with only values found in the second.

Treats repeated elements according to their counts in each sequence.

Parameters:

  • lhs – First sequence or set.

  • rhs – Second sequence or set.

Returns:

List of common elements. For sequences, includes duplicates.

Examples

Duplicates survive only as often as both sides carry them:

>>> from my import ut
>>> ut.common_elements([9, 1, 3, 9, 9], [1, 9, 9])
[9, 1, 9]
classmethod IterUtils.exclusive_elements(lhs: S, rhs: Iterable) → S#

Return a version of the first sequence with the second’s values subtracted from it.

The complement of common_elements(): an ordered multiset difference, where each occurrence in rhs cancels exactly one matching occurrence in lhs.

Parameters:

  • lhs – First sequence.

  • rhs – Iterable of values to exclude, one occurrence at a time.

Returns:

A sequence of the same type as lhs, containing only unexcluded occurrences.

Examples

Subtract per-occurrence – one of the three nines survives:

>>> from my import ut
>>> ut.exclusive_elements([9, 1, 3, 9, 9], [1, 9, 9])
[3, 9]

VII Modification#

classmethod IterUtils.drop_at(data: Sequence, mask: Iterable[int]) → list[V]#

Remove elements at specified indices from sequence.

Parameters:

  • data – Sequence to filter.

  • mask – Indices to drop.

Returns:

List with elements at masked indices removed.

Examples

Remove by index:

>>> from my import ut
>>> ut.drop_at(['a', 'b', 'c', 'd'], [1, 3])
['a', 'c']
classmethod IterUtils.drop_duplicates(data: MutableSequence) → None#

Remove duplicate elements from a list, preserving order.

Parameters:

data – Mutable sequence to deduplicate in place.

Examples

Deduplicate a list in place:

>>> from my import ut
>>> items = [1, 2, 1, 3, 2]
>>> ut.drop_duplicates(items)
>>> items
[1, 2, 3]